Add Postgres store, path safety, and the crawl indexer

Store: crawl generations as the sync cursor. Diffs compare generations
on entity identity plus a content digest, never on upstream timestamps —
GET /course-rooms/{id}/board returns request time as updatedAt for most
elements, so a timestamp cursor would report every board as changed on
every crawl. Identity diffing also yields deletions, which no timestamp
scheme can. A per-course crawl carries the other courses' rows forward
so every completed generation is a complete picture and any two diff
directly; without that a partial crawl reads as a mass deletion.

FTS uses the german dictionary with weighted title/body, plus a pg_trgm
arm because stemming will not match "Datenschutz" inside
"Datenschutzgrundverordnung" and German compounds make that the common
case. file_texts is keyed by file record id and deliberately outlives
generations: records are immutable upstream, so text extracted once is
valid forever and a re-crawl of unchanged content costs nothing.

Store.open returns undefined instead of throwing when Postgres is
unreachable — the index is an accelerator, and a Pi that loses its
database should get slower, not broken.

core/paths.ts is the security boundary for the mirror. Course titles,
card titles and filenames are all user-supplied upstream, so this is
where a hostile name stops being text and becomes a path. Two bugs found
by its own tests: "///" produced "---" instead of falling back, and dot
runs survived mid-component. Now no ".." can survive anywhere, which
makes the invariant checkable rather than a claim about ordering.

Indexer coalesces concurrent refreshes onto one run and enforces a
minimum interval, since a full crawl is ~270 requests from an account
that looks like a student.

9 store tests against a real Postgres (mocks would test nothing here)
and 13 path tests; 47 total.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-09-12 21:10:53 +02:00
parent 81dd633863
commit a18b526267
11 changed files with 1547 additions and 1 deletions

View File

@@ -27,6 +27,7 @@
"exceljs": "^4.4.0",
"express": "^5.1.0",
"mammoth": "^1.11.0",
"pg": "^8.23.0",
"unpdf": "^1.3.2",
"unzipper": "^0.12.3",
"zod": "^3.25.76"
@@ -34,6 +35,7 @@
"devDependencies": {
"@types/express": "^5.0.3",
"@types/node": "^22.15.0",
"@types/pg": "^8.23.1",
"@types/unzipper": "^0.10.11",
"typescript": "^5.9.2"
}