MechaCat02 592747f1e0 fix(crawler): budget image tail against actual sniff-prefix length ...

The per-image download cap subtracted the constant SNIFF_PREFIX_BYTES (64)
from the budget instead of the bytes actually drained into the prefix.
Because the prefix loop appends whole chunks, a single ~16 KiB first chunk
fills the 64-byte sniff window in one drain, so the tail could then add
another near-full cap — storing up to ~2× max_image_bytes per page.

Capture prefix.len() before the prefix is moved into the stream and budget
the tail as `max_image_bytes - prefix_len` via a small `remaining_after_prefix`
helper (unit-tested for the overshoot and saturation cases).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

2026-07-03 20:57:29 +02:00

862 B

Raw Blame History

View Raw