A full crawl made the instance answer 4 of 26 course pages with an nginx
503 "temporarily unavailable" front-page — it pushes back when several
hundred requests arrive quickly. Nothing retried, so the index was
quietly incomplete: 22 courses and 153 files rather than 26 and 205,
with the failures recorded per course rather than surfaced as a problem.
The client now retries 429/500/502/503/504 and transient network errors
with exponential backoff plus jitter (so parallel crawl workers do not
retry in lockstep), honouring Retry-After when sent. Every call here is
an idempotent GET, so retrying is safe. 401 and 404 are deliberately not
retried: an expired token will not recover, and neither will a bad id.
Re-crawled after the fix: 26 courses, 205 files, zero failures.
Also: `schulcloud --help` printed 'Unknown command "--help"' because a
leading flag was parsed as the command name.
63 tests.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
MechaCat02
e4d087682d