fix(compression): stop a single large PNG from OOM-killing the container

An 8000x8000 RGBA PNG passes admission — 256,000,000 bytes is just under the
256 MiB max_alloc, and smooth content is under 3 MB on disk, far below any size
cap. Processing it peaked at ~1250 MiB inside a 1 GiB cgroup. Measured, not
argued: the new (ignored) test builds exactly that image and reads VmHWM around
the pipeline, resetting the watermark via /proc/self/clear_refs so the number
covers only the code under test.

Three independent causes, all of which had to go:

1. The decode outlived everything. `resize` takes &self, and the no-downscale
   arm bound `img` into `display`, so the ~244 MiB buffer was still alive when
   oxipng ran — and oxipng decodes the PNG *again*, holding a full-size buffer
   per filter trial. The decode now lives in a block that yields the display
   derivative; the else arm moves `img` out, which is what makes "the block's
   value is the only survivor" true in both arms.

2. oxipng was unbounded in every dimension: preset 2 with timeout: None, and the
   default features pull in rayon, which evaluates filter trials concurrently
   with a full-size buffer each and has no Options knob to cap it. Now gated at
   8 MP, given a 20 s timeout, and built with default-features = false so
   oxipng's own sequential shim is used. "filetime" is kept — without it
   preserve_attrs silently no-ops. Dropping "binary" also stops compiling
   clap/glob/env_logger (a CLI's deps) into the server image.

3. Even with those fixed it still measured 516 MiB, and compression_concurrency
   defaults to 2 — so two guests uploading big photos at once was another OOM,
   1032 MiB against a 1 GiB limit. The cost is dominated by image's Lanczos3
   resize, which accumulates in f32: the intermediate is new_width * old_height
   * 16 bytes, i.e. 262 MiB for this image — larger than the decode itself, and
   invisible to max_alloc. Two changes: the preview now derives from the 2048px
   display instead of re-resizing the original (one full-size pass, not two),
   and a job whose header-estimated peak exceeds 150 MiB takes an exclusive
   permit so two giants can never overlap. Ordinary photos (a 12 MP JPEG
   estimates ~50 MiB) never touch that permit, so throughput is unchanged for
   everything except the case that must not run in parallel.

Chaining 8000 -> 2048 -> 800 for the preview is not a quality trade: a staged
Lanczos3 downscale is standard for large ratios and is visually
indistinguishable at 800px.

The blocking half is now a free function so its memory behaviour is testable —
the fix is a scoping property a future edit could silently undo.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
MechaCat02
2026-07-31 22:42:03 +02:00
parent 6fd75adb27
commit a4bac03628
4 changed files with 348 additions and 238 deletions

View File

@@ -27,7 +27,17 @@ tracing-subscriber = { version = "0.3", features = ["env-filter"] }
dotenvy = "0.15"
sysinfo = "0.32"
image = "0.25"
oxipng = "9"
# default-features = false drops "parallel", which is what actually bounds oxipng's memory:
# with rayon it evaluates row filters concurrently, each trial holding its own full-size
# buffer, and there is no Options knob to cap that. Without the feature, lib.rs swaps in a
# sequential shim (oxipng's own supported path) so peak scales with ONE trial, not N.
# PNG optimisation gets slower; it is a background, best-effort, lossless size saving.
#
# "filetime" must be KEPT: without it OutFile::Path { preserve_attrs: true } silently no-ops.
# Dropping "binary" also removes clap/glob/env_logger — a CLI's dependencies that were being
# compiled into a server image — and "zopfli", which preset 2 does not use (it selects
# Deflaters::Libdeflater, which is not feature-gated).
oxipng = { version = "9", default-features = false, features = ["filetime"] }
async_zip = { version = "0.0.17", features = ["tokio", "deflate"] }
include_dir = "0.7"
infer = "0.15"