EventSnap

Author SHA1 Message Date

Author	SHA1	Message	Date
fabi	fae12bd7ec	fix(upload): refuse undecodable images at the door, and stop retrying them Two halves of the same complaint: an oversized photo was accepted with a 201 and then silently soft-deleted minutes later, after the worker had burned six seconds of backoff re-reaching a conclusion it could not change. Admission. The compression budget now runs at upload time, against the header only, so a guest is told immediately and told why: "Bild hat zu viele Bildpunkte (ca. 99 Megapixel) und kann nicht verarbeitet werden. Bitte verkleinere es und lade es erneut hoch." instead of watching the photo vanish behind a vague "could not be processed" — which arrived only if they happened to still be on the feed with that card loaded. Nothing is stored, so there is no row to soft-delete and no orphan for the sweep to reclaim. Admission and the worker share ONE function (`decoder_within_budget`), so they cannot drift apart and start disagreeing about what is acceptable — a photo accepted at the door and rejected by the worker would be worse than either behaviour alone. The worker keeps its own check: the backfill decodes files that predate this check, and defence in depth is the whole reason the budget exists. Retries. The loop retried every failure, including ones that are a property of the input. An image over the budget, a corrupt file, an unsupported format: each fails identically on all three attempts, so the only effect was 2s + 4s of sleep and three near-identical warnings before the same outcome. `is_permanent_image_error` classifies the `ImageError` variants that cannot change between attempts — Limits, Unsupported, Decoding — and the loop gives up on those at once. `IoError` is deliberately excluded: an ENOSPC while writing a derivative is exactly the transient case the retry exists for, and misclassifying it would turn a blip back into the data loss round 1 fixed. Measured: retry log lines went from 3 per oversized upload to 0. Tests: unit tests for both sides of the classifier (a Limits error is permanent, a missing file is not) and for admission agreeing with the decoder on accept AND reject. The e2e spec is rewritten for the new contract — 400 with an actionable message, nothing stored, backend alive after a burst of four — plus a mirror asserting an ordinary photo still uploads and processes, since a budget that rejected everything would satisfy the other two. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-29 07:29:21 +02:00
fabi	f0d69f1cda	fix(imaging): restore the decode allocation guard I removed in round 1 This is a regression I introduced, not a pre-existing gap. Before `05948d8` the compression worker used `ImageReader::decode()`, which does: let mut decoder = Self::make_decoder(format, self.inner, limits.clone())?; limits.reserve(decoder.total_bytes())?; // enforces max_alloc decoder.set_limits(limits)?; Reading the EXIF orientation tag needs `into_decoder()` instead, and that skips the reserve entirely — the crate's own FIXME concedes `from_decoder` doesn't compensate. Nothing else enforces `max_alloc`: the JPEG decoder's `set_limits` only checks support and dimensions. So the 256 MiB budget has been inert since that commit, and round 2 then propagated the weakened path into export.rs through the shared helper, in a commit whose message claimed the helper "carries" the decompression-bomb cap. It didn't, and the comment saying max_alloc "hard-caps the decode allocation" was simply false. What was left was only the per-axis cap, which permits 12000x12000 — 412 MiB decoded, 824 MiB for the two concurrent decodes the worker runs by default, against a 1 GiB container. Deploy-blocking right now because bumping DERIVATIVES_REV makes the first boot after a deploy re-decode the entire gallery two at a time: an OOM kill there restarts the container, which re-runs the backfill. A boot loop, on the first deploy of these fixes. Re-add the reserve exactly as `decode()` does it. Per the budget decision it stays at 256 MiB (~89 MP for RGB8, above any mainstream phone's real output); two concurrent decodes now peak at 512 MiB. Oversized images take the graceful path from round 1 — original retained, quota refunded, upload-error toast — and fail after the header parse but BEFORE any pixels are read, so they cost a header read rather than an allocation. Measured peak during a concurrent oversized burst: 3.0 MiB. Test parity is the other half, and the reason this was invisible: the e2e app container had NO memory limit while production is capped at 1 GiB, so a decode that would OOM-kill production simply succeeded in CI. Mirror the 1 GiB cap in docker-compose.test.yml. That is the third divergence of this shape, after WebKit missing from CI and /health existing only in Caddyfile.test. Tests: a fixture that is 568 KiB on disk and 283 MiB decoded (11000x9000 = 99 MP, deliberately UNDER the per-axis cap so the axis check cannot be what rejects it). A unit test asserts the refusal — it fails against the old code, which decoded it into an 11000x9000 buffer — with a companion asserting an ordinary photo still decodes AND still gets its orientation applied, so the guard didn't become a blanket refusal. An e2e test uploads it singly and as a concurrent pair, asserting compression lands in 'failed' and the backend is still serving and still processing afterwards. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-28 22:31:09 +02:00
fabi	3c984e2932	fix(export): apply EXIF orientation in the keepsake too Round 1 fixed EXIF orientation in the compression worker, which corrected the live app — feed preview and diashow display. The export worker was missed, and it does not reuse those derivatives: it re-decodes the originals itself with `image::open`, which ignores the orientation tag, then re-encodes to JPEG, which drops the tag — so the viewer has no way to recover it. The damage was oddly shaped, which is exactly why it reads as a viewer bug: Gallery.zip originals correct (byte-copied, EXIF intact) Memories viewer grid thumbnails SIDEWAYS (always) Memories viewer full image >5 MB SIDEWAYS (re-encoded at 2000px) Memories viewer full image ≤5 MB correct (streamed byte-for-byte) So in the keepsake people actually keep, every portrait photo in the grid was on its side, and clicking through silently "fixed" small photos but not large ones. Rather than paste the decoder dance a third time, extract `services::imaging:: decode_oriented` and route both workers through it, so there is exactly one way to turn a file on disk into a DynamicImage. It carries a second invariant that had also drifted: `image::open` applies NO decode limits, so the export path was decoding arbitrary user-supplied images unbounded — the decompression-bomb cap existed only in the compression worker. Both now come as a pair, which is the point of having one function. Not done: switching export to consume the existing `display` derivative. It would fix orientation and drop a redundant full-resolution decode per photo, but it would also replace the pristine ≤5 MB originals in the keepsake with 2048px re-encodes — a real quality regression in the one artefact people keep forever. Test uploads the round-1 fixture (40x20 landscape tagged Orientation=6), runs a real export, pulls the thumbnail out of Memories.zip and asserts it came back portrait — with a sanity check that the source really is stored landscape, so the test can't pass against a pipeline that does nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-28 20:46:38 +02:00

fabi

fae12bd7ec

fix(upload): refuse undecodable images at the door, and stop retrying them

Two halves of the same complaint: an oversized photo was accepted with a 201 and
then silently soft-deleted minutes later, after the worker had burned six seconds
of backoff re-reaching a conclusion it could not change.

Admission. The compression budget now runs at upload time, against the header
only, so a guest is told immediately and told why:

  "Bild hat zu viele Bildpunkte (ca. 99 Megapixel) und kann nicht verarbeitet
   werden. Bitte verkleinere es und lade es erneut hoch."

instead of watching the photo vanish behind a vague "could not be processed" —
which arrived only if they happened to still be on the feed with that card
loaded. Nothing is stored, so there is no row to soft-delete and no orphan for
the sweep to reclaim.

Admission and the worker share ONE function (`decoder_within_budget`), so they
cannot drift apart and start disagreeing about what is acceptable — a photo
accepted at the door and rejected by the worker would be worse than either
behaviour alone. The worker keeps its own check: the backfill decodes files that
predate this check, and defence in depth is the whole reason the budget exists.

Retries. The loop retried every failure, including ones that are a property of
the input. An image over the budget, a corrupt file, an unsupported format: each
fails identically on all three attempts, so the only effect was 2s + 4s of sleep
and three near-identical warnings before the same outcome. `is_permanent_image_error`
classifies the `ImageError` variants that cannot change between attempts — Limits,
Unsupported, Decoding — and the loop gives up on those at once. `IoError` is
deliberately excluded: an ENOSPC while writing a derivative is exactly the
transient case the retry exists for, and misclassifying it would turn a blip back
into the data loss round 1 fixed. Measured: retry log lines went from 3 per
oversized upload to 0.

Tests: unit tests for both sides of the classifier (a Limits error is permanent, a
missing file is not) and for admission agreeing with the decoder on accept AND
reject. The e2e spec is rewritten for the new contract — 400 with an actionable
message, nothing stored, backend alive after a burst of four — plus a mirror
asserting an ordinary photo still uploads and processes, since a budget that
rejected everything would satisfy the other two.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

2026-07-29 07:29:21 +02:00

fabi

f0d69f1cda

fix(imaging): restore the decode allocation guard I removed in round 1

This is a regression I introduced, not a pre-existing gap. Before 05948d8 the
compression worker used `ImageReader::decode()`, which does:

    let mut decoder = Self::make_decoder(format, self.inner, limits.clone())?;
    limits.reserve(decoder.total_bytes())?;   // enforces max_alloc
    decoder.set_limits(limits)?;

Reading the EXIF orientation tag needs `into_decoder()` instead, and that skips
the reserve entirely — the crate's own FIXME concedes `from_decoder` doesn't
compensate. Nothing else enforces `max_alloc`: the JPEG decoder's `set_limits`
only checks support and dimensions. So the 256 MiB budget has been inert since
that commit, and round 2 then propagated the weakened path into export.rs through
the shared helper, in a commit whose message claimed the helper "carries" the
decompression-bomb cap. It didn't, and the comment saying max_alloc "hard-caps
the decode allocation" was simply false.

What was left was only the per-axis cap, which permits 12000x12000 — 412 MiB
decoded, 824 MiB for the two concurrent decodes the worker runs by default,
against a 1 GiB container. Deploy-blocking right now because bumping
DERIVATIVES_REV makes the first boot after a deploy re-decode the entire gallery
two at a time: an OOM kill there restarts the container, which re-runs the
backfill. A boot loop, on the first deploy of these fixes.

Re-add the reserve exactly as `decode()` does it. Per the budget decision it stays
at 256 MiB (~89 MP for RGB8, above any mainstream phone's real output); two
concurrent decodes now peak at 512 MiB. Oversized images take the graceful path
from round 1 — original retained, quota refunded, upload-error toast — and fail
after the header parse but BEFORE any pixels are read, so they cost a header read
rather than an allocation. Measured peak during a concurrent oversized burst: 3.0
MiB.

Test parity is the other half, and the reason this was invisible: the e2e app
container had NO memory limit while production is capped at 1 GiB, so a decode
that would OOM-kill production simply succeeded in CI. Mirror the 1 GiB cap in
docker-compose.test.yml. That is the third divergence of this shape, after WebKit
missing from CI and /health existing only in Caddyfile.test.

Tests: a fixture that is 568 KiB on disk and 283 MiB decoded (11000x9000 = 99 MP,
deliberately UNDER the per-axis cap so the axis check cannot be what rejects it).
A unit test asserts the refusal — it fails against the old code, which decoded it
into an 11000x9000 buffer — with a companion asserting an ordinary photo still
decodes AND still gets its orientation applied, so the guard didn't become a
blanket refusal. An e2e test uploads it singly and as a concurrent pair, asserting
compression lands in 'failed' and the backend is still serving and still
processing afterwards.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

2026-07-28 22:31:09 +02:00

fabi

3c984e2932

fix(export): apply EXIF orientation in the keepsake too

Round 1 fixed EXIF orientation in the compression worker, which corrected the live
app — feed preview and diashow display. The export worker was missed, and it does
not reuse those derivatives: it re-decodes the originals itself with `image::open`,
which ignores the orientation tag, then re-encodes to JPEG, which drops the tag —
so the viewer has no way to recover it.

The damage was oddly shaped, which is exactly why it reads as a viewer bug:

  Gallery.zip originals              correct  (byte-copied, EXIF intact)
  Memories viewer grid thumbnails    SIDEWAYS (always)
  Memories viewer full image >5 MB   SIDEWAYS (re-encoded at 2000px)
  Memories viewer full image ≤5 MB   correct  (streamed byte-for-byte)

So in the keepsake people actually keep, every portrait photo in the grid was on
its side, and clicking through silently "fixed" small photos but not large ones.

Rather than paste the decoder dance a third time, extract `services::imaging::
decode_oriented` and route both workers through it, so there is exactly one way to
turn a file on disk into a DynamicImage. It carries a second invariant that had
also drifted: `image::open` applies NO decode limits, so the export path was
decoding arbitrary user-supplied images unbounded — the decompression-bomb cap
existed only in the compression worker. Both now come as a pair, which is the
point of having one function.

Not done: switching export to consume the existing `display` derivative. It would
fix orientation and drop a redundant full-resolution decode per photo, but it
would also replace the pristine ≤5 MB originals in the keepsake with 2048px
re-encodes — a real quality regression in the one artefact people keep forever.

Test uploads the round-1 fixture (40x20 landscape tagged Orientation=6), runs a
real export, pulls the thumbnail out of Memories.zip and asserts it came back
portrait — with a sanity check that the source really is stored landscape, so the
test can't pass against a pipeline that does nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

2026-07-28 20:46:38 +02:00

3 Commits