Retry transient upstream failures; fix schulcloud --help

A full crawl made the instance answer 4 of 26 course pages with an nginx
503 "temporarily unavailable" front-page — it pushes back when several
hundred requests arrive quickly. Nothing retried, so the index was
quietly incomplete: 22 courses and 153 files rather than 26 and 205,
with the failures recorded per course rather than surfaced as a problem.

The client now retries 429/500/502/503/504 and transient network errors
with exponential backoff plus jitter (so parallel crawl workers do not
retry in lockstep), honouring Retry-After when sent. Every call here is
an idempotent GET, so retrying is safe. 401 and 404 are deliberately not
retried: an expired token will not recover, and neither will a bad id.

Re-crawled after the fix: 26 courses, 205 files, zero failures.

Also: `schulcloud --help` printed 'Unknown command "--help"' because a
leading flag was parsed as the command name.

63 tests.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-09-12 22:07:02 +02:00
parent e6f2258df9
commit e4d087682d
3 changed files with 138 additions and 11 deletions

View File

@@ -31,10 +31,13 @@ SCHULCLOUD_SYNC_DIR. Config file: ${configPath()}
`;
async function main(argv: string[]): Promise<number> {
const [command, ...rest] = argv;
const flags = parseFlags(rest);
// A leading flag means no command was given: `schulcloud --help` must not be
// read as a command called "--help".
const hasCommand = argv[0] !== undefined && !argv[0].startsWith('-');
const command = hasCommand ? argv[0] : undefined;
const flags = parseFlags(hasCommand ? argv.slice(1) : argv);
if (!command || command === 'help' || flags.help) {
if (!command || command === 'help' || flags.help || flags.h) {
process.stdout.write(USAGE);
return 0;
}