4a9cf878eae24fba3fac1a515c7b6fa0d90e510e
23 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
fabi
|
bac30404e3 |
fix(export): stop a caption bricking the viewer, and ship readable archives
Two defects in the keepsake, both silent server-side and both only visible by
extracting the real artifact and trying to use it.
1. A CAPTION COULD BRICK THE VIEWER.
The viewer's data is inlined as `<script>window.__EXPORT_DATA__={…}</script>` -- it has
to be, since guests open index.html over file:// where fetching a sibling data.json is
blocked. The escape was `</` -> `<\/`. Against XSS that holds; I fired
`</script><img src=x onerror=…>` through a real Chromium parser and it round-trips
inert.
It does not stop the caption steering the HTML TOKENIZER. `<!--<script` with no later
`-->` drives the parser into script-data-double-escaped state, where the template's own
`</script>` only steps back to script-data-escaped instead of closing the element.
Everything after it -- including the viewer bundle -- is swallowed as script data.
Nothing executes and nothing leaks: `__EXPORT_DATA__` is simply never assigned and the
keepsake opens blank. A denial of the deliverable, not an XSS.
Reproduced in Chromium before changing anything, and the near-miss is worth recording:
`<!--<script>alert(1)</script>-->` comes back CLEAN, because the trailing `-->` returns
the parser to script-data state. A probe using the terminated form quietly repairs the
thing it is testing for.
Fix: escape every `<` as `<`, not just `</`. `<` never appears in JSON structural
syntax -- only inside string values -- so a global replace is sound, and one rule covers
`</script`, `<!--` and `<script` together. That is the point: the old escape was named
for the single case it handled. Only the INLINED copy is escaped; data.json is written
separately, in no HTML context, and stays literal.
2. EVERY ENTRY IN BOTH ARCHIVES WAS STORED MODE 0000.
`ZipEntryBuilder::new` leaves the external file attribute at zero and async_zip's host
compatibility defaults to Unix, so `unzip -Z` showed `?---------` on every line of both
Gallery.zip and Memories.zip. Windows Explorer ignores Unix modes, which is why this
survived; on Linux and macOS `unzip` faithfully applies what the archive asks for and
the guest gets a folder of photos none of which they can open.
Unconditional -- every keepsake ever produced, no hostile input required -- and
invisible server-side: the export succeeds, the ZIP is well-formed, the job writes
`done`, /export/status is green.
Found by accident. The browser test for defect 1 failed with ERR_ACCESS_DENIED on
file://, which looked exactly like a Playwright sandbox quirk; I twice "worked around"
it (fresh context, then a separately launched browser) before checking the extracted
files and finding mode 000. The workaround was suppressing a real bug. Both workarounds
are gone -- the ordinary `page` fixture loads the archive fine now.
Fix: all six ZipEntryBuilder sites route through one `keepsake_entry` helper stamping
`S_IFREG | 0644`, so the mode cannot be forgotten at a call site.
Tests: 3 unit (no `<` survives; the payload still decodes to the original value, because
this is a transport encoding and not a sanitiser; a clean payload is untouched) and 2
e2e that release for real, download the real archives, and check them from outside the
app -- one opening index.html over file:// in Chromium and asserting the viewer booted,
the captions came back verbatim and nothing executed; one asserting every stored mode
and every extracted file is readable. Both assertions verified to FAIL against the
pre-fix artifacts.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
fabi
|
a20b96d893 |
refactor(export): share the visibility filter between the row query and the estimate
`query_uploads` selects the rows the archives are built from; `estimate_export_bytes` sizes them for the disk preflight. They stated the same WHERE clause separately, and the direction of drift matters: an estimate that MISSES rows the archive writes under-reserves, which is precisely the ENOSPC the preflight exists to prevent. The integration test claimed to guard this and cannot. Both sides of `the_estimate_sums_exactly_the_rows_the_archive_will_contain` are `SRC:`-marked hand-copies in tests/common/mod.rs -- neither is production code -- so drift means production moved while both copies sat still, and the test goes on passing. The convention is sound for pinning behaviour; it is structurally incapable of detecting divergence from the thing it copies. So fix it where it can be fixed. One `export_visibility_where!()` fragment, `concat!`-ed into both queries at compile time (still `&'static str`, no allocation), with the `u`/`usr` alias contract stated. Divergence is now impossible by construction rather than watched for. The tests keep their value and lose the overclaim: the docstrings now say they pin WHICH uploads may be counted -- each excluded row in the fixture is excluded by a different predicate, so weakening any one of them still fails here -- and say plainly that they do not detect drift, with a pointer to what does. No behaviour change. The filters were verified identical before the hoist (`u.event_id = $1 AND u.deleted_at IS NULL AND usr.uploads_hidden = FALSE AND usr.is_banned = FALSE`); 99 backend tests still pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
fabi
|
43c2a0d09c |
feat(host): warn about low disk before it becomes unrecoverable
Storage visibility existed in exactly one place: a passive Speicherauslastung widget on the ADMIN dashboard. A host who isn't the admin had no view of it, and nothing warned anyone. README carried "Low-disk alert (< 10 GB free)" under Planned since v1. Two things make this a safety net rather than a nice-to-have. postgres_data, media_data and exports_data are all Docker named volumes on ONE filesystem, so running out doesn't degrade a subsystem -- Postgres stops being able to write and the whole event goes down. And the keepsake needs room for two gallery-sized archives, which the export preflight can only ever refuse AFTER the release, when the event is over and every remedy is harder. So the threshold is not a fixed number alone. It fires on the 10 GB floor the README always named, OR on "you could not build the keepsake right now" -- the trigger a host can still act on, computed with the same arithmetic the preflight uses. Unknown free space is NOT low: it fails open like the upload quota and the preflight do, because a banner that cries wolf on an unreadable mount is a banner nobody reads. Carried on GET /host/event, which the dashboard already fetches on load and on every reload -- no new endpoint, no new poll. Rendered above everything else including the PIN-reset queue, and it names the consequence (the event, not just the download) rather than only the number. Also fixes the host page's formatBytes, which topped out at MB: 30 GB free would have rendered as "30720.0 MB", and a guest with 2 GB of uploads was already being shown that way in the user list. Tests: 5 unit on the threshold (including that plenty of free space is still low when the keepsake wouldn't fit -- the case a fixed threshold misses entirely), 3 e2e. The e2e drives it through `original_size_bytes` rather than a genuinely full disk: the estimate is pure SQL over that column, so overstating one row moves the accounting without touching a byte on disk. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
fabi
|
281eb3bec7 |
fix(export): refuse an export that cannot fit, and stop peaking at two generations
Nothing in export.rs ever asked whether the keepsake would fit. Both archives write
their media `Compression::Stored`, so each is essentially a byte-for-byte second copy
of the originals -- Gallery.zip always, and Memories.zip for every video and every
image at or under 5 MB. On the documented CX33 (80 GB, all three volumes on one
filesystem) the upload quota's fixed point leaves ~40 GB free, and a release spawns
BOTH halves concurrently against it.
The failure is not "the export failed", it is "the deliverable is stuck":
1. ENOSPC lands partway through a multi-GB write.
2. The epoch has already moved, so the job row is `failed` at the CURRENT
generation and readiness (epoch = event.export_epoch AND status = 'done') is
false -- GET /export/zip 404s.
3. The last good archive sits on disk, unreferenced and unreachable.
4. POST /host/export/rebuild, the only escape, re-arms the same doomed write.
Three changes.
Reclaim before building. `prune_stale_export_files` ran only after the new archive
was written, renamed and finalised. That reads as durability but buys nothing: the
moment `invalidate_and_arm` bumps the epoch the old archive is ALREADY unreachable,
so keeping it reserves gigabytes for a download nobody can perform -- and for a
takedown it is content someone explicitly asked to have removed. Peak usage is now
one generation. Narrower than the post-finalize prune on purpose: final archives
only, never a `.tmp` or a `viewer_tmp_` dir, since a superseded worker can still be
streaming into those and at build START is far more likely to be alive.
Preflight the space. SUM(original_size_bytes) over exactly `query_uploads`'
visibility filter, +10% for ZIP overhead, multiplied by the number of armed jobs --
without that multiplier each of the two concurrent halves independently sees "it
fits" and together they don't. Runs AFTER claim_job, not before as reported: bailing
before the claim leaves the row `pending` with no worker and no error, the
spinner-forever state `mark_failed`'s status guard exists to prevent. Fails open when
the mount can't be read, exactly as the upload quota does.
Show the host the reason. /export/status returned {status, progress_pct} and nothing
else, so the host dashboard could only render "fehlgeschlagen" next to the retry
button. The message was written to the row and surfaced solely in the ADMIN job list
-- a different screen, possibly a different person. It now travels with the status,
and only on a failure, so a message left on a since-succeeded row can't appear beside
a green "ist bereit".
Tests: 10 unit (the u128 clamp caught a real bug in the first draft -- saturating_mul
then /100 turns an overflow into a number ~100x too small, the one direction that
authorises the write being guarded against; the carried-forward archive must survive
its own older epoch in the filename), 4 DB-backed (the estimate is asserted against
the row set the archive actually contains, not against a restatement of the WHERE
clause, so the two queries cannot drift), 3 e2e over the four-hop plumbing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
fabi
|
3c984e2932 |
fix(export): apply EXIF orientation in the keepsake too
Round 1 fixed EXIF orientation in the compression worker, which corrected the live app — feed preview and diashow display. The export worker was missed, and it does not reuse those derivatives: it re-decodes the originals itself with `image::open`, which ignores the orientation tag, then re-encodes to JPEG, which drops the tag — so the viewer has no way to recover it. The damage was oddly shaped, which is exactly why it reads as a viewer bug: Gallery.zip originals correct (byte-copied, EXIF intact) Memories viewer grid thumbnails SIDEWAYS (always) Memories viewer full image >5 MB SIDEWAYS (re-encoded at 2000px) Memories viewer full image ≤5 MB correct (streamed byte-for-byte) So in the keepsake people actually keep, every portrait photo in the grid was on its side, and clicking through silently "fixed" small photos but not large ones. Rather than paste the decoder dance a third time, extract `services::imaging:: decode_oriented` and route both workers through it, so there is exactly one way to turn a file on disk into a DynamicImage. It carries a second invariant that had also drifted: `image::open` applies NO decode limits, so the export path was decoding arbitrary user-supplied images unbounded — the decompression-bomb cap existed only in the compression worker. Both now come as a pair, which is the point of having one function. Not done: switching export to consume the existing `display` derivative. It would fix orientation and drop a redundant full-resolution decode per photo, but it would also replace the pristine ≤5 MB originals in the keepsake with 2048px re-encodes — a real quality regression in the one artefact people keep forever. Test uploads the round-1 fixture (40x20 landscape tagged Orientation=6), runs a real export, pulls the thumbnail out of Memories.zip and asserts it came back portrait — with a sanity check that the source really is stored landscape, so the test can't pass against a pipeline that does nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
fabi
|
c14ccd2df1 |
fix(compression): reclaim failed originals instead of leaking them
Round 1 stopped the compression worker deleting an upload's original on failure — a transient ENOSPC or a codec panic must never destroy the only copy of a photo a guest cannot retake. But it left `Upload::soft_delete`'s quota refund in place, so the bytes stayed on disk while the uploader was charged nothing for them. That is worse than it first looks. The row is soft-deleted, so the file is invisible and unowned; a guest hitting a reproducible codec failure can accumulate orphans indefinitely at zero personal cost. And `active_uploaders` counts only users with non-deleted uploads, so dropping out of that count RAISES everyone's per-user ceiling — the leak loosens the very quota meant to contain it. Keep the refund: the uploader didn't cause the failure and shouldn't silently lose quota to it. Bound the leak instead, with an hourly sweep alongside the existing session cleanup in `spawn_periodic_tasks`, reclaiming failed originals older than 14 days — comfortably longer than any single event, so an operator investigating a failed upload still has the file. The selection predicate is the entire safety argument, so it is deliberately narrow: `compression_status = 'failed'` AND soft-deleted AND past the window AND `original_path <> ''`. That is exactly the state the give-up path leaves behind, and it cannot reach a live upload, an owner-deleted one, or a failure still inside its recovery window. `original_path` is cleared after a successful reclaim, which makes the sweep idempotent — otherwise a row whose file is already gone is re-selected on every tick forever. The row itself is kept as the audit trail. Tests reproduce the selection verbatim (same pattern as upload_concurrency) and assert it against five near-misses that must survive, both sides of the retention boundary, and the idempotence property. Also fixes two comments in export.rs still claiming "the compression worker hard-deletes an original when its transcode fails" — no longer true, and the defensive handling they justify is now justified by this sweep and by ordinary deletes instead. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
MechaCat02
|
3fb1b5d80d |
feat(comments): hide every comment mention when COMMENTS_ENABLED=false
The kill-switch previously left comment UI/text visible in several places. Sweep the whole surface so a comments-off instance shows no trace: Frontend (main app): - VirtualFeed grid tile: gate the comment button/count on $commentsEnabled (was ungated — the only feed surface still showing it). - admin stats: hide the "Kommentare" count card. - UploadSheet "Uploads geschlossen" notice, host ban-modal description, and host/admin unban confirmations: drop the "…und kommentieren" wording. - export page: drop "Kommentaren" from the keepsake description. Keepsake export (had no concept of the flag): - export.rs: thread comments_enabled into the exported data (ViewerEvent), wired through spawn_export_jobs/recover_exports and their call sites (host.rs, main.rs). - export-viewer: gate comment counts (list + grid) and the lightbox comments section; older exports without the field default to enabled (?? true). Backend still 403s comment writes when disabled (unchanged) — this is the UI half so stale clients and archives match. Verified on the running stack (COMMENTS_ENABLED=false): /event reports comments_enabled=false, a regenerated keepsake embeds "comments_enabled": false with no comment UI, and the uploads-closed notice renders "…ansehen und liken." Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
57a907eca5 |
feat(theme+comments): runtime colour theme and COMMENTS_ENABLED switch
Colour theme is configurable at runtime from two seed colours (brand + accent); neutrals stay fixed for contrast safety. Tailwind v4 var()-based tokens let a :root:root override recolour everything with no rebuild; the 50->950 ramps are derived via a color-mix ladder. Config lives in the DB config table (admin UI: Config > Farbschema, presets + custom pickers + live preview), served on the public /event endpoint with env defaults (THEME_PRESET/PRIMARY/ACCENT), propagated live via event-updated SSE, and cached in localStorage for a no-flash boot. The keepsake export mirrors the same ladder in Rust so offline archives match the event theme. COMMENTS_ENABLED (env, default true) is a boot-time kill-switch: the backend rejects new comments with 403 and the frontend hides the comment button (feed card) and panel/composer (lightbox). Existing comments stay in the DB, hidden, and return when re-enabled. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
3654aca18b |
feat(export): single-file offline keepsake viewer + guest download
Rebuild the guest "keepsake" (offline HTML gallery) so it renders when the extracted index.html is opened over file://. Browsers block external ES-module scripts and fetch() at origin null, so a normal multi-file SvelteKit build shows a blank window. Build the viewer as one self-contained index.html (inlined JS/CSS via vite-plugin-singlefile, standalone non-SvelteKit entry) and inject the data as window.__EXPORT_DATA__; the backend embeds the built viewer via include_dir! and writes the data global into index.html when zipping. Also surface the guest-facing download: a feed banner and the BottomNav Export tab, both shown once the host releases the export. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
fabi
|
ee554e7f38 |
style(backend): rustfmt the whole tree; gate cargo fmt --check in CI
The backend had never been run through rustfmt. Doing it in one mechanical pass (134 files) so no future functional diff is buried under formatting churn, then gating `cargo fmt --check` in checks.yml so it stays clean. Formatting only — no logic, SQL, or behaviour changed. Verified after the reformat: cargo test 56 passed, clippy --all-targets -D warnings clean, cargo fmt --check clean. This is the deferred cleanup noted when CI's Format step was first left out. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
fabi
|
e5201a9889 |
test(backend): DB-backed tests for the risky SQL; test isolation; clippy cleanup
All 40 backend tests were pure-function tests — not one line of SQL ran under `cargo test`,
even though the riskiest code in the repo is SQL. Add 12 DB-backed `#[sqlx::test]` tests
(fresh throwaway DB per test, real migrations) pinning the invariants that code is
load-bearing for, each mutation-verified to fail on broken code:
export_epoch.rs (8): epoch is post-increment on release (a worker born pre-increment is
inert); epoch strictly monotonic across reopen; a retired-epoch worker's writes are
no-ops; export_current is EXACTLY the invariant (8-case table); the ViewerOnly
carry-forward carries a done ZIP forward AND matches nothing when the ZIP is unfinished
(the
|
||
|
fabi
|
2bef6e19ef |
fix(export): deleting a comment mid-build no longer strands the ZIP forever
Found while auditing the test suite: the ViewerOnly carry-forward (added by me in
|
||
|
fabi
|
9666d74a46 |
fix(export): close all 13 findings from the multi-agent review
A 6-lens multi-agent review (103 agents; every finding adversarially verified by three
refute-by-default skeptics) confirmed the epoch redesign is sound — the core invariant
holds — and found 13 real defects around it. All are fixed here.
The redesign's invariant was re-verified and stands: an accepted upload is always in the
keepsake, and a downloadable keepsake always reflects the most recent release.
But a WEAKER adjacent invariant did NOT hold — "a downloadable keepsake reflects the
current content" — and that is the theme of the biggest fixes.
CONTENT REMOVAL NOW ALWAYS REACHES THE KEEPSAKE
Regeneration was wired only to host deletes. Ban, unban, guest self-delete and guest
comment-delete did not trigger it — yet the export query filters `is_banned = FALSE`,
so the pipeline already agreed that content must not be there. Ban someone for abusive
content after release and every guest kept downloading an archive containing it.
All five removal paths now invalidate and rebuild. `query_comments` also gained the
banned/hidden filter it was missing entirely (the ZIP had it; the viewer did not).
Each removal is now ATOMIC with its invalidation (one tx, models made executor-generic).
Previously the delete committed in its own tx and the regeneration in a second: a dropped
handler future between them left the taken-down photo permanently downloadable, and
nothing could notice — the keepsake still looked complete and the host could no longer
even find the upload to retry.
NO MORE TAKEDOWN STORM
Every removal spawned two uncancellable full exports. A superseded worker was INERT, not
STOPPED: it still ran every ffmpeg spawn and image resize and wrote a whole archive before
discovering it had lost. Five deletes left ten workers alive, each holding a
full-gallery-sized temp, in a 1 GB container.
Now: a debounce before `claim_job` (a superseded worker fails its claim and does ZERO
work, collapsing a burst into one export), plus `update_progress` — which already ran
exactly the right liveness predicate and threw the answer away — returning it so both file
loops bail the instant they are retired. Moderating a comment no longer rebuilds the
multi-GB ZIP either: the ZIP is media-only, so it is carried forward, with prune taught to
protect any file a live row still references.
AN ESCAPE HATCH
A failed export was terminal at runtime: release_gallery refuses an already-released event,
recovery only runs at boot, and there was no retry route — the only outs were restarting the
container or reopening uploads to every guest. Added POST /host/export/rebuild. Also widened
mark_failed's guard to `status IN ('running','pending')`: when claim_job itself ERRORED, the
row was still `pending`, so the failure was never recorded and the job sat at 0% forever.
SILENT INVALIDATION
Retiring the epoch 404s the download instantly, but nothing told the clients — guests kept
seeing an enabled download button through the whole rebuild. Now broadcasts an invalidation.
And the download was a top-level <a href>: a 404 (or a 429 from the per-IP export limit that
everyone on the venue WiFi shares) NAVIGATED THE USER OUT OF THE PWA onto raw JSON. It now
targets a hidden iframe, so an error response is harmless.
ALSO
- The HTML export still had the exists()-then-open TOCTOU the ZIP path was fixed to remove,
at three sites, two with a hard `?` that failed the ENTIRE viewer. It is reachable: the
compression worker hard-deletes an original when a transcode fails and can still be running
when the gallery is released.
- Crash-orphaned .tmp files were immortal (prune deliberately skips .tmp). Swept at boot,
where it is unambiguously safe.
- export_status read the epoch in one statement and used it in the next; now one query.
- DiskCache::snapshot IGNORED its path argument on a cache hit, returning whatever filesystem
was measured last. Latent only because both callers pass media_path — it would have silently
misreported the moment anyone measured the exports volume. Now keyed by path.
- Corrected two comments that asserted safety properties the code does not have (claim_job
does NOT prevent a retired-epoch claim — retirement is enforced at READ time; and a
multi-replica reap is NOT contained, it can corrupt the keepsake). Added a rollback runbook
to migration 014, the repo's first destructive migration.
TESTS
The regression guard for the headline FOR SHARE fix was probabilistic — it could pass on
broken code if the interleaving fell the wrong way. It is now STRUCTURAL: the upload is held
open mid-body with its pre-flight check already passed, so the release provably lands inside
the exact window. Verified 5/5 FAILING (201 instead of 403) with the fix reverted, 5/5
passing with it.
Verified: backend 40 tests, svelte-check 0 errors, e2e typecheck clean,
e2e 160 passed / 1 skipped on chromium-desktop.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
fabi
|
8e906d7866 |
refactor(export): single epoch authority; fix upload-path TOCTOU losing photos
A deep investigative review of the export concurrency (four parallel adversarial
reviewers) found that three rounds of race fixes had been chasing the wrong bug, and
that the model itself was the root cause. This replaces the model and fixes the real
data loss.
THE ACTUAL BUG (upload path, not the state machine)
`upload` checked `uploads_locked_at` BEFORE streaming the body — minutes, for a large
video — then committed the row without re-checking. A release landing mid-upload let the
row commit AFTER the export snapshot: the photo appeared in the live feed and was
PERMANENTLY missing from the keepsake, with nothing to regenerate it. release_gallery's
comment claims to prevent exactly this; it never did. Every prior round fixed the DB
state machine, and the leak was upstream of it.
Fix: re-assert the lock under `SELECT ... FOR SHARE` INSIDE the commit tx. That row lock
conflicts with release_gallery's `UPDATE event`, so either the upload commits first (and
the release — hence the snapshot — is ordered after it) or the release wins and the
upload is rejected as `uploads_locked` (reversible; the client keeps the blob).
Regression test verified NON-VACUOUS: with the fix reverted it fails, reporting accepted
uploads missing from the keepsake.
THE MODEL (migration 014)
"Which generation is current?" had no single answer: it was assembled from three
separately-written pieces across two tables (`export_released_at`, a per-job
`release_seq`, and cached `export_{zip,html}_ready` flags). Keeping them in agreement
took ~10 hand-written guards; each review round found one more path that slipped through.
Now: ONE monotonic `event.export_epoch`, bumped in the same UPDATE as any change to
`export_released_at`. Each job carries a copy. Downloadable IFF
`released AND job.epoch = event.export_epoch AND status = 'done'` — readiness DERIVED,
never stored. A retired-epoch worker is INERT BY CONSTRUCTION: losing a race can no
longer corrupt state, only waste work.
Deleted: both ready-flag columns, both ready-flip blocks, `if ready { continue }`, the
reopen seq-bump, open_event's transaction, and the cross-table claim guard.
That last one was also UNSOUND: under READ COMMITTED, a blocked UPDATE re-evaluates its
WHERE against the updated target row but answers subqueries on OTHER tables from the
original snapshot — so the previous `EXISTS (SELECT ... FROM event ...)` claim guard
could admit a claim against an already-reopened event while RETURNING the post-bump seq.
Every guard is now same-row (`epoch = $n`), which EPQ re-evaluates correctly.
OTHER REAL DEFECTS FIXED
- release_gallery committed the release and THEN awaited the enqueue inside the handler
future. Axum drops that future on client disconnect → event released, uploads locked,
ZERO export_job rows, host unable to retry ("bereits freigegeben"), restart-only
recovery. Now one tx; workers spawn (detached) after commit.
- No flush/fsync before the tmp→final rename. async_zip's close() never flushes the inner
file and tokio's File drop DISCARDS write errors — so a full disk silently produced a
truncated archive that was then marked done and published, after which the last good
generation was pruned. Now flush + sync_all, which also surfaces ENOSPC.
- Download read the ready flag and the file path in TWO queries; a reopen between them
served a keepsake that should 404. Now one read through the `export_current` view.
- `if !src.exists() { continue }` then `open()?` — a TOCTOU that turned a tolerated gap
into a hard failure of the ENTIRE keepsake (unretryable while released). Now open-first,
warn, and skip.
- Recovery was flag-driven and never checked the disk: a `done` row whose archive was gone
was a permanent dead end needing manual DB surgery. Now verifies the file and re-arms.
- Temp files/dirs leaked on every error path (the leak is what fills the disk that
corrupts the next archive). Now cleaned up.
- Export archives were not event-scoped (`viewer_tmp_` already was), so a second event
would collide on paths and one event's prune would delete another's live keepsake.
- Host deletes after release never regenerated the keepsake — a photo removed on request
lived on in the archive forever. Now bumps the epoch and rebuilds without it.
- claim_job swallowed DB errors as "someone else won", stranding a job pending at 0%.
- export_status reported retired-epoch rows (a progress bar that never moves).
- The startup sweep's single-instance assumption is now documented, not hidden.
Verified: backend 40 tests, svelte-check 0 errors, e2e typecheck clean,
e2e 160 passed / 1 skipped on chromium-desktop (incl. 2 new regressions; the
upload-acceptance one confirmed to fail without the fix).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
fabi
|
9f3712894d |
fix(export): pin export workers to a live release epoch; make reopen atomic
Adversarial re-review of |
||
|
fabi
|
d643256f36 |
fix(rereview): close residual export finalize↔flip double-race at reopen
Adversarial re-review of
|
||
|
fabi
|
df275bbefa |
fix(rereview): close export-flip race + queue-dedup reload regression
Adversarial re-review of the persona-audit + audit-followup rounds (6411747..)
found one HIGH and one MED regression plus LOW gaps. All fixed with coverage.
HIGH — export stale-keepsake resurrected by an open_event race
A reopen landing in the window between a *current* export worker's finalize_job
and its ready-flag flip cleared export_released_at + the ready flags but left the
export_job row `done` at the same release_seq. The seq-guarded flip then still
matched and re-set export_{zip,html}_ready=TRUE on a pre-reopen snapshot; the next
re-release read that stale TRUE and skipped regeneration (`if ready { continue }`),
serving a keepsake missing every upload from the reopen window — the exact data
loss migration 012 exists to prevent. Both ready-flip UPDATEs are now additionally
anchored on `export_released_at IS NOT NULL`, so a landed reopen makes the flip a
no-op and the re-release regenerates cleanly.
MED — queue dedup broke for reloaded items
loadQueue rebuilt QueueItems from IndexedDB without copying lastModified, which the
new addToQueue dedup keys on. A file re-selected after a page reload / PWA relaunch
missed the duplicate check and uploaded twice. Rehydration now carries lastModified
(extracted to a pure, tested entryToQueueItem helper).
LOW
- diashow: clear the upload-processed debounce timer in onDestroy (no stray
post-unmount /feed fetch).
- USER_JOURNEYS §9.5: document the reconnect-delta ban replay (hidden_user_ids /
uploads_hidden_at, migration 013), not just the live user-hidden SSE.
- e2e api-client: drop the misleading hide_uploads param from banUser — the backend
takes no body and always hides; strip the dead boolean at all call sites.
Tests
- Extract isReversibleLock (the terminal-403 KEEP-vs-PURGE-blob discriminator) into a
pure exported helper + unit tests, so the data-loss-critical branch is covered
without an XHR harness.
- entryToQueueItem unit tests lock the lastModified-carry regression.
- Document the export flip-race guard in the reopen/re-release spec (the sub-ms
finalize↔flip interleave isn't deterministically forceable with fast fixtures;
covered by the SQL guard + the end-to-end completeness test).
Verified: backend 40 tests, frontend 44 unit tests, svelte-check 0 errors,
e2e 156 passed / 1 skipped on chromium-desktop.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
fabi
|
36fe59caa5 |
fix(user-flow): persona-audit fixes — ban replay, locked-upload data loss, host UX, admin session
Follows the perf + security + user-flow work with a role/persona audit (guest, host, admin, projector) and fixes across three review rounds. Highlights: HIGH - Ban now replays on reconnect. A ban isn't a soft-delete, and the `user-hidden` SSE has no replay, so a client that missed it (esp. the unattended diashow) kept cycling a banned user's slides. New `uploads_hidden_at` (migration 013) + `hidden_user_ids` in /feed/delta; feed + diashow evict those users. Applied even on a truncated delta. MEDIUM - Locked-upload data loss: a photo staged offline during a lock/release was purged as a terminal 4xx and lost when the host reopened. New reversible `uploads_locked` error code; the queue keeps the blob and auto-resumes on the `event-opened` SSE. - Reopen after release now warns (ConfirmSheet) that it revokes the published keepsake. - Host "forgotten-PIN" badge updates live (`pin-reset-requested` was broadcast but never in KNOWN_EVENTS / subscribed); host page also refetches on `pin-reset` so a two-host race can't hand out a conflicting PIN. - Ban modal copy fixed (read-only ban, not "session ended"); Degradieren/Sperren/Entsperren hidden on peer-host rows for non-admins (they always 403'd). - Host dashboard shows live keepsake generation progress / ready state + link to /export. - Admin JWT moved to sessionStorage (§11.1) to bound exposure on shared devices. Export generation guard (H1 from the prior round, hardened): per-(event,type) `release_seq` (migration 012) with seq-guarded claim/finalize/mark_failed/update_progress, per-generation temp/final paths, download follows `file_path`, prune only strictly-older generations. LOW: diashow coalesces upload-processed (avoids self-rate-limit); event-closed reconciles galleryReleased; /recover gains a forgot-PIN request + drops a stale cached PIN on 401; delta `>=` tie-break + 429 retry; misc copy/labels. Adds e2e: ban-replay, upload-lock-code, and rewrites export-reopen-rerelease with a data-completeness test. Reconciles USER_JOURNEYS §9/§11. Verified: cargo build clean, 40 unit tests, svelte-check 0 errors, 33 frontend unit tests, 155 e2e passing on chromium-desktop. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
641174717c |
fix(user-flow): close offline-queue, export, session, ban & realtime gaps
Comprehensive user-flow review across guest/host/admin roles, then fixes with
e2e regression guards. Highlights:
- Offline upload queue auto-resumes on reconnect: network errors keep items
pending (not error), 4xx are terminal (no infinite retry), quota exhaustion
returns a distinct 413; queue cap + dedup.
- Export lifecycle: release atomically locks uploads; reopen invalidates and
re-release regenerates the keepsake; workers claim their job atomically so a
reopen->re-release can't corrupt the ZIP; startup re-spawns interrupted
exports.
- Sessions slide on activity (no 30-day cliff); JWT expiry deferred to the
revocable session row; sign-out-everywhere + revoke-on-PIN-reset.
- Ban always hides content (v_feed / find_visible_media / export filter
is_banned) but stays a read-only ban per USER_JOURNEYS §10 — sessions are
not revoked, read access + keepsake download preserved.
- Realtime: server-clock SSE delta cursor; event-closed/opened drive the UI
live; feed_delta rate-limited; like returns {liked, like_count} to fix
multi-device drift; lightbox live comments; diashow delta backfill.
- Forgotten-PIN in-app request flow; simultaneous same-name join returns 409;
quota increment is transactional; operator floor.
Adds e2e/specs/10-flow-review/ (offline resume, export integrity, deterministic
anti-race guard) and updates existing specs for the new contracts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
fabi
|
d6c91974eb |
perf(backend): close config/upload/export hot paths + stability fixes
Performance: - Cache the runtime `config` table in-memory (ConfigCache) with synchronous invalidation on every write (admin PATCH + test reseed). Was re-reading each key from Postgres on every request (~8 round-trips per upload). - Stream uploads chunk-by-chunk to a temp file instead of buffering the whole body in RAM (peak was up to the per-class cap, e.g. 500 MB/video); only 512 sniff-bytes are kept for magic-byte detection, then atomic rename into place. - Cache the media-filesystem disk snapshot (DiskCache, 15s TTL) shared by the quota check and admin stats; drop the discarded System::refresh_all(). - HTML export streams video (and small-image) originals straight into the ZIP via a manifest instead of copying them to a temp dir first (removed the transient 2x disk usage) and drops the double directory scan. - Auth extractor resolves session -> live user in one JOIN (was two queries), touching last_seen_at by token hash. Stability: - SSE: on broadcast lag, emit a `resync` event so the client runs a delta fetch instead of silently losing events; frontend reconciles adds, deletions, and (via an in-place refresh) like/comment counts on visible cards. - Storage quota fails OPEN when the disk can't be read (was a 0-byte limit that locked out all uploads). - Graceful shutdown drains in-flight requests on SIGTERM/SIGINT, bounded by a 10s backstop so open SSE streams can't stall a deploy. - Upload removes the persisted file if the DB transaction fails (no orphaned bytes with no row to reclaim them). Tests: - New pure select_disk() with 5 unit tests (longest-prefix, fallbacks, fail-open). - New e2e export-video spec covering the HTML export's video-streaming branch. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
fabi
|
dba4d3f932 |
fix(review-2): critical — repair comment posting + close export data leak
CR1: the lightbox comment button POSTed to /upload/{id}/comment (singular);
no such route exists, so every UI-submitted comment 404'd and was lost.
Fixed to /comments (plural). The prior e2e passed because its seed helper
POSTs the API directly — added comment-ui.spec.ts which drives the real
component so this can't regress silently again.
CR2: export archives (Gallery.zip / Memories.zip / HTML viewer) were written
under media_path, which is a public ServeDir — so GET /media/exports/
Gallery.zip served the entire event (every photo, caption, comment,
uploader name) to any anonymous visitor at a guessable URL, bypassing the
ticket + release gate. Moved exports to a separate EXPORT_PATH (=/exports)
on its own volume (Dockerfile chown, compose volume, gated handler reads
the new path). export-leak.spec.ts asserts /media/exports/Gallery.zip → 404.
Riders (git can't split hunks): LightboxModal also gains H3 live-eviction on
comment-deleted/upload-deleted; config.rs also wires compression_concurrency
from boot config (medium).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
MechaCat02
|
ffc926bf4d |
feat: replace HTML export with SvelteKit viewer
Replace the minijinja-based HTML export with a full SvelteKit static viewer app. The new export produces a ZIP with: - Pre-built viewer assets (index.html + JS/CSS bundle) - data.json with all posts, comments, tags, and like counts - Processed media: 400px thumbnails for grid, full images (2000px cap if >5MB), video thumbnails via ffmpeg Remove minijinja dependency, add include_dir to embed viewer assets at compile time. Update Dockerfile to copy static/ for builds. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
MechaCat02
|
258e2bd84d |
feat: implement export engine
Add async ZIP and HTML offline viewer export workers, download endpoints,
and a guest-facing /export page.
Backend — export workers (tokio::spawn, run after gallery release):
- ZIP worker: streams all non-deleted originals into Gallery.zip via
async_zip (Stored compression), organised into Photos/ and Videos/
with {date}_{uploader}_{id}.{ext} filenames; updates progress_pct in DB
- HTML worker: renders Memories.html via minijinja template (self-contained:
inlined CSS + JS, relative media paths); packs it with README.txt and
all media into Memories.zip (Deflate for text, Stored for media)
- Both workers mark export_job status (running → done/failed), update
export_zip_ready / export_html_ready on the event, and broadcast SSE
export-progress + export-available when both complete
Backend — new endpoints (AuthUser):
- GET /export/zip → streams Gallery.zip if export_zip_ready
- GET /export/html → streams Memories.zip if export_html_ready
- GET /export/status → released flag + per-type status/progress (moved from admin)
Memories.html features: warm keepsake aesthetic, responsive grid, fullscreen
lightbox with captions/comments/likes, client-side hashtag filter chips,
XSS-safe JS, fully offline (no external deps)
Frontend — /export page:
- Locked state: padlock illustration + message
- Released state: ZIP and HTML cards with progress bars (SSE-driven),
download buttons enabled only when ready
- HTML guide modal (unzip instructions + Wi-Fi tip) before download begins
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|