Commit Graph

4 Commits

Author SHA1 Message Date
fabi
ceb68939a7 fix(upload): narrow the admission check to the memory budget only
The admission check I just added rejected ANY image the decoder couldn't build —
corrupt, truncated, or unsupported, not only over-budget. That broke two
adversarial tests, and they were right to break.

07-adversarial/file-upload-attacks pins, deliberately, that acceptance follows the
MAGIC BYTES: a payload whose first three bytes are a JPEG header is accepted
regardless of what follows, because the security property under test is that the
client-declared Content-Type has no influence. Both failing cases upload 1024
bytes of JPEG magic followed by zeros. Rejecting those at admission is a
different, broader contract than the one asked for, and rewriting an adversarial
test to match new behaviour is precisely the thing that needs justifying rather
than doing quietly.

So admission now checks only what it was meant to: `exceeds_decode_budget`
returns true solely for `ImageError::Limits`. A corrupt file goes to the
compression worker exactly as before — which handles it gracefully and, since the
retry classifier in the previous commit, no longer burns backoff on it. The
resource guard is the part that had to move earlier; nothing else did.

Tests: the size agreement between admission and the worker is still asserted in
both directions, plus a new one writing a magic-bytes-only stub and asserting
admission accepts it WHILE the worker still rejects it — pinning the boundary
between the two checks so a future widening fails here rather than in the
adversarial suite.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 07:55:56 +02:00
fabi
fae12bd7ec fix(upload): refuse undecodable images at the door, and stop retrying them
Two halves of the same complaint: an oversized photo was accepted with a 201 and
then silently soft-deleted minutes later, after the worker had burned six seconds
of backoff re-reaching a conclusion it could not change.

Admission. The compression budget now runs at upload time, against the header
only, so a guest is told immediately and told why:

  "Bild hat zu viele Bildpunkte (ca. 99 Megapixel) und kann nicht verarbeitet
   werden. Bitte verkleinere es und lade es erneut hoch."

instead of watching the photo vanish behind a vague "could not be processed" —
which arrived only if they happened to still be on the feed with that card
loaded. Nothing is stored, so there is no row to soft-delete and no orphan for
the sweep to reclaim.

Admission and the worker share ONE function (`decoder_within_budget`), so they
cannot drift apart and start disagreeing about what is acceptable — a photo
accepted at the door and rejected by the worker would be worse than either
behaviour alone. The worker keeps its own check: the backfill decodes files that
predate this check, and defence in depth is the whole reason the budget exists.

Retries. The loop retried every failure, including ones that are a property of
the input. An image over the budget, a corrupt file, an unsupported format: each
fails identically on all three attempts, so the only effect was 2s + 4s of sleep
and three near-identical warnings before the same outcome. `is_permanent_image_error`
classifies the `ImageError` variants that cannot change between attempts — Limits,
Unsupported, Decoding — and the loop gives up on those at once. `IoError` is
deliberately excluded: an ENOSPC while writing a derivative is exactly the
transient case the retry exists for, and misclassifying it would turn a blip back
into the data loss round 1 fixed. Measured: retry log lines went from 3 per
oversized upload to 0.

Tests: unit tests for both sides of the classifier (a Limits error is permanent, a
missing file is not) and for admission agreeing with the decoder on accept AND
reject. The e2e spec is rewritten for the new contract — 400 with an actionable
message, nothing stored, backend alive after a burst of four — plus a mirror
asserting an ordinary photo still uploads and processes, since a budget that
rejected everything would satisfy the other two.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 07:29:21 +02:00
fabi
f0d69f1cda fix(imaging): restore the decode allocation guard I removed in round 1
This is a regression I introduced, not a pre-existing gap. Before 05948d8 the
compression worker used `ImageReader::decode()`, which does:

    let mut decoder = Self::make_decoder(format, self.inner, limits.clone())?;
    limits.reserve(decoder.total_bytes())?;   // enforces max_alloc
    decoder.set_limits(limits)?;

Reading the EXIF orientation tag needs `into_decoder()` instead, and that skips
the reserve entirely — the crate's own FIXME concedes `from_decoder` doesn't
compensate. Nothing else enforces `max_alloc`: the JPEG decoder's `set_limits`
only checks support and dimensions. So the 256 MiB budget has been inert since
that commit, and round 2 then propagated the weakened path into export.rs through
the shared helper, in a commit whose message claimed the helper "carries" the
decompression-bomb cap. It didn't, and the comment saying max_alloc "hard-caps
the decode allocation" was simply false.

What was left was only the per-axis cap, which permits 12000x12000 — 412 MiB
decoded, 824 MiB for the two concurrent decodes the worker runs by default,
against a 1 GiB container. Deploy-blocking right now because bumping
DERIVATIVES_REV makes the first boot after a deploy re-decode the entire gallery
two at a time: an OOM kill there restarts the container, which re-runs the
backfill. A boot loop, on the first deploy of these fixes.

Re-add the reserve exactly as `decode()` does it. Per the budget decision it stays
at 256 MiB (~89 MP for RGB8, above any mainstream phone's real output); two
concurrent decodes now peak at 512 MiB. Oversized images take the graceful path
from round 1 — original retained, quota refunded, upload-error toast — and fail
after the header parse but BEFORE any pixels are read, so they cost a header read
rather than an allocation. Measured peak during a concurrent oversized burst: 3.0
MiB.

Test parity is the other half, and the reason this was invisible: the e2e app
container had NO memory limit while production is capped at 1 GiB, so a decode
that would OOM-kill production simply succeeded in CI. Mirror the 1 GiB cap in
docker-compose.test.yml. That is the third divergence of this shape, after WebKit
missing from CI and /health existing only in Caddyfile.test.

Tests: a fixture that is 568 KiB on disk and 283 MiB decoded (11000x9000 = 99 MP,
deliberately UNDER the per-axis cap so the axis check cannot be what rejects it).
A unit test asserts the refusal — it fails against the old code, which decoded it
into an 11000x9000 buffer — with a companion asserting an ordinary photo still
decodes AND still gets its orientation applied, so the guard didn't become a
blanket refusal. An e2e test uploads it singly and as a concurrent pair, asserting
compression lands in 'failed' and the backend is still serving and still
processing afterwards.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 22:31:09 +02:00
fabi
3c984e2932 fix(export): apply EXIF orientation in the keepsake too
Round 1 fixed EXIF orientation in the compression worker, which corrected the live
app — feed preview and diashow display. The export worker was missed, and it does
not reuse those derivatives: it re-decodes the originals itself with `image::open`,
which ignores the orientation tag, then re-encodes to JPEG, which drops the tag —
so the viewer has no way to recover it.

The damage was oddly shaped, which is exactly why it reads as a viewer bug:

  Gallery.zip originals              correct  (byte-copied, EXIF intact)
  Memories viewer grid thumbnails    SIDEWAYS (always)
  Memories viewer full image >5 MB   SIDEWAYS (re-encoded at 2000px)
  Memories viewer full image ≤5 MB   correct  (streamed byte-for-byte)

So in the keepsake people actually keep, every portrait photo in the grid was on
its side, and clicking through silently "fixed" small photos but not large ones.

Rather than paste the decoder dance a third time, extract `services::imaging::
decode_oriented` and route both workers through it, so there is exactly one way to
turn a file on disk into a DynamicImage. It carries a second invariant that had
also drifted: `image::open` applies NO decode limits, so the export path was
decoding arbitrary user-supplied images unbounded — the decompression-bomb cap
existed only in the compression worker. Both now come as a pair, which is the
point of having one function.

Not done: switching export to consume the existing `display` derivative. It would
fix orientation and drop a redundant full-resolution decode per photo, but it
would also replace the pristine ≤5 MB originals in the keepsake with 2048px
re-encodes — a real quality regression in the one artefact people keep forever.

Test uploads the round-1 fixture (40x20 landscape tagged Orientation=6), runs a
real export, pulls the thumbnail out of Memories.zip and asserts it came back
portrait — with a sanity check that the source really is stored landscape, so the
test can't pass against a pipeline that does nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 20:46:38 +02:00