fix(compression): stop a single large PNG from OOM-killing the container
An 8000x8000 RGBA PNG passes admission — 256,000,000 bytes is just under the 256 MiB max_alloc, and smooth content is under 3 MB on disk, far below any size cap. Processing it peaked at ~1250 MiB inside a 1 GiB cgroup. Measured, not argued: the new (ignored) test builds exactly that image and reads VmHWM around the pipeline, resetting the watermark via /proc/self/clear_refs so the number covers only the code under test. Three independent causes, all of which had to go: 1. The decode outlived everything. `resize` takes &self, and the no-downscale arm bound `img` into `display`, so the ~244 MiB buffer was still alive when oxipng ran — and oxipng decodes the PNG *again*, holding a full-size buffer per filter trial. The decode now lives in a block that yields the display derivative; the else arm moves `img` out, which is what makes "the block's value is the only survivor" true in both arms. 2. oxipng was unbounded in every dimension: preset 2 with timeout: None, and the default features pull in rayon, which evaluates filter trials concurrently with a full-size buffer each and has no Options knob to cap it. Now gated at 8 MP, given a 20 s timeout, and built with default-features = false so oxipng's own sequential shim is used. "filetime" is kept — without it preserve_attrs silently no-ops. Dropping "binary" also stops compiling clap/glob/env_logger (a CLI's deps) into the server image. 3. Even with those fixed it still measured 516 MiB, and compression_concurrency defaults to 2 — so two guests uploading big photos at once was another OOM, 1032 MiB against a 1 GiB limit. The cost is dominated by image's Lanczos3 resize, which accumulates in f32: the intermediate is new_width * old_height * 16 bytes, i.e. 262 MiB for this image — larger than the decode itself, and invisible to max_alloc. Two changes: the preview now derives from the 2048px display instead of re-resizing the original (one full-size pass, not two), and a job whose header-estimated peak exceeds 150 MiB takes an exclusive permit so two giants can never overlap. Ordinary photos (a 12 MP JPEG estimates ~50 MiB) never touch that permit, so throughput is unchanged for everything except the case that must not run in parallel. Chaining 8000 -> 2048 -> 800 for the preview is not a quality trade: a staged Lanczos3 downscale is standard for large ratios and is visually indistinguishable at 800px. The blocking half is now a free function so its memory behaviour is testable — the fix is a scoping property a future edit could silently undo. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -88,6 +88,40 @@ fn decoder_within_budget(path: &Path) -> Result<impl image::ImageDecoder> {
|
||||
Ok(decoder)
|
||||
}
|
||||
|
||||
/// Rough peak heap an image will cost to turn into derivatives, read from the HEADER only —
|
||||
/// no pixels are decoded. `None` when the header can't be read or the image is over budget
|
||||
/// (the caller is about to fail on it anyway).
|
||||
///
|
||||
/// Two terms, and the second is the one that surprises:
|
||||
///
|
||||
/// - the decoded buffer, `width * height * channels`; and
|
||||
/// - the resize intermediate. `image`'s Lanczos3 path accumulates in `f32`, so the buffer
|
||||
/// between the horizontal and vertical passes is `new_width * old_height * 4 channels * 4
|
||||
/// bytes` — 16 bytes per pixel-row-slot, not the 4 the output uses. For an 8000x8000
|
||||
/// original that is 262 MiB on top of a 244 MiB decode, measured. It is bigger than the
|
||||
/// decode for any tall image, which is why "the decode is bounded by max_alloc" was never
|
||||
/// the whole story.
|
||||
///
|
||||
/// Used to decide whether an image is heavy enough to need exclusive use of the box's memory
|
||||
/// headroom, NOT to reject anything.
|
||||
pub fn estimated_processing_peak_bytes(path: &Path, display_edge: u32) -> Option<u64> {
|
||||
let decoder = decoder_within_budget(path).ok()?;
|
||||
let (width, height) = decoder.dimensions();
|
||||
let decoded = decoder.total_bytes();
|
||||
|
||||
// Aspect-preserving fit into `display_edge`, matching DynamicImage::resize. No downscale
|
||||
// means no intermediate at all.
|
||||
let intermediate = if width > display_edge || height > display_edge {
|
||||
let ratio = f64::from(display_edge) / f64::from(width.max(height));
|
||||
let new_width = (f64::from(width) * ratio).round().max(1.0) as u64;
|
||||
new_width * u64::from(height) * 16
|
||||
} else {
|
||||
0
|
||||
};
|
||||
|
||||
Some(decoded.saturating_add(intermediate))
|
||||
}
|
||||
|
||||
/// Megapixels an image would decode to, or `None` if its header can't be read. Used only
|
||||
/// to put a concrete number in the message the guest sees.
|
||||
pub fn megapixels(path: &Path) -> Option<f64> {
|
||||
|
||||
Reference in New Issue
Block a user