fix(ops): bound logs, drain ffmpeg's stderr, and stop three silent hangs

Six independent operational defects, none of which needed a new feature to fix.

Log rotation. Docker's json-file driver is unbounded by default, and those files
land on the HOST filesystem — outside every deploy.resources.limits in the compose
file, and on the same disk as postgres_data and media_data. A full disk stops
Postgres writing WAL, which takes the event down. Capped at 10m x 3 per service.

Log level. RUST_LOG was set in neither .env.example nor docker-compose.yml, so the
code fallback WAS the production level — and it was `debug`, with tower_http=debug
emitting a line per request and per response into that unrotated file. Now info,
with tower_http=warn to state that those spans are diagnostics, not an access log.

ffmpeg pipe deadlock. run_ffmpeg piped stdout and stderr and then called wait(),
which drains neither. Once the ~64 KiB pipe buffer filled, ffmpeg blocked writing
and wait() never returned — burning the full 120s timeout, twice per seek position,
three times per compression attempt. And the timeout is an Err, so the end state was
a soft-deleted upload: a guest's playable video destroyed by a poster-frame failure.
Now stdout is null (nothing ever read it) and stderr is drained by wait_with_output,
whose tail is logged on a non-zero exit. Note wait_with_output consumes the child, so
the old kill-on-timeout is gone; kill_on_drop(true) already covers it.

Readiness probe. /health never touched the pool, so the disk-full endgame above
stayed green all the way down. Adds /health/ready (SELECT 1 under 2s) as a SECOND
route — the compose healthcheck deliberately keeps pointing at /health, because
caddy gates its startup on it and a DB-dependent probe would turn a Postgres blip
into the reverse proxy refusing to start.

api.ts request timeout. The abort timer was cleared in a finally around fetch(),
which resolves on the response HEAD — leaving res.text() uncovered and no longer
abortable. An upstream that sends headers then stalls the body hung the call
forever. The timer now lives until the body is read, including the 204 path (which
otherwise leaked a live 20s timer per no-content request).

Upload XHR watchdog. The XHR had no timeout while processQueue held the
isProcessing latch across it; on a half-open socket neither error nor abort ever
fires, so the latch pinned and the queue wedged. Bounds SILENCE rather than total
duration — a 500 MB video over a venue uplink legitimately runs 30+ minutes while
making steady progress. Rejects as NetworkError, which is already the retryable
branch, so a stalled upload now recovers like any network blip.

IndexedDB failures. addToQueue called getDb() unguarded and handleSubmit had no
catch, so a private-mode refusal or a QuotaExceededError on a large blob left a
permanent "Wird hochgeladen…" spinner, no toast, and — for an in-app camera
capture — the only copy of the photo gone. Now reported as 'failed', which keeps
the staged files on screen and stays on the page.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
MechaCat02
2026-07-31 22:25:23 +02:00
parent 7d0334bf22
commit 6fd75adb27
11 changed files with 394 additions and 32 deletions

View File

@@ -1,5 +1,10 @@
import { describe, it, expect } from 'vitest';
import { classifyUploadStatus, isReversibleLock, entryToQueueItem } from './upload-queue';
import {
classifyUploadStatus,
isReversibleLock,
entryToQueueItem,
shouldAbortForStall
} from './upload-queue';
/**
* Regression guard for the upload-queue retry policy (H2 + M1). The bug being locked out:
@@ -107,3 +112,32 @@ describe('entryToQueueItem', () => {
expect(item.hashtags).toBe('');
});
});
/**
* The upload XHR had no timeout of any kind while `processQueue` held the `isProcessing`
* latch across it. On a half-open socket neither `error` nor `abort` ever fires, so the
* latch was pinned forever and the whole queue wedged with no recovery but a reload.
*
* The policy that matters: bound SILENCE, not total duration. A 500 MB video over a venue
* uplink legitimately runs 30+ minutes while making steady progress, and a flat total cap
* would kill exactly the uploads worth keeping.
*/
describe('shouldAbortForStall', () => {
const now = 1_000_000;
it('lets a long upload run as long as progress keeps arriving', () => {
// Two hours in, but progress landed a second ago.
expect(shouldAbortForStall(now - 1_000, now, false)).toBe(false);
});
it('aborts once the body stalls past the no-progress ceiling', () => {
expect(shouldAbortForStall(now - 89_000, now, false)).toBe(false);
expect(shouldAbortForStall(now - 91_000, now, false)).toBe(true);
});
it('applies the wider ceiling once the body is sent and progress goes quiet', () => {
// Silence that would abort mid-body is normal while waiting for the response.
expect(shouldAbortForStall(now - 91_000, now, true)).toBe(false);
expect(shouldAbortForStall(now - 121_000, now, true)).toBe(true);
});
});