Add Postgres store, path safety, and the crawl indexer
Store: crawl generations as the sync cursor. Diffs compare generations
on entity identity plus a content digest, never on upstream timestamps —
GET /course-rooms/{id}/board returns request time as updatedAt for most
elements, so a timestamp cursor would report every board as changed on
every crawl. Identity diffing also yields deletions, which no timestamp
scheme can. A per-course crawl carries the other courses' rows forward
so every completed generation is a complete picture and any two diff
directly; without that a partial crawl reads as a mass deletion.
FTS uses the german dictionary with weighted title/body, plus a pg_trgm
arm because stemming will not match "Datenschutz" inside
"Datenschutzgrundverordnung" and German compounds make that the common
case. file_texts is keyed by file record id and deliberately outlives
generations: records are immutable upstream, so text extracted once is
valid forever and a re-crawl of unchanged content costs nothing.
Store.open returns undefined instead of throwing when Postgres is
unreachable — the index is an accelerator, and a Pi that loses its
database should get slower, not broken.
core/paths.ts is the security boundary for the mirror. Course titles,
card titles and filenames are all user-supplied upstream, so this is
where a hostile name stops being text and becomes a path. Two bugs found
by its own tests: "///" produced "---" instead of falling back, and dot
runs survived mid-component. Now no ".." can survive anywhere, which
makes the invariant checkable rather than a claim about ordering.
Indexer coalesces concurrent refreshes onto one run and enforces a
minimum interval, since a full crawl is ~270 requests from an account
that looks like a student.
9 store tests against a real Postgres (mocks would test nothing here)
and 13 path tests; 47 total.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -6,6 +6,27 @@
|
||||
# `networks` block at the bottom and deploy/Caddyfile.snippet.
|
||||
|
||||
services:
|
||||
# Dev/standalone Postgres. On the Pi, point DATABASE_URL at the existing
|
||||
# instance instead and remove this service — the schema lives in its own
|
||||
# database and user, so it coexists with whatever else is already there.
|
||||
postgres:
|
||||
image: postgres:17-alpine
|
||||
container_name: schulcloud-mcp-db
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
POSTGRES_DB: schulcloud
|
||||
POSTGRES_USER: schulcloud
|
||||
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-schulcloud}
|
||||
volumes:
|
||||
- pgdata:/var/lib/postgresql/data
|
||||
networks:
|
||||
- caddy
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U schulcloud -d schulcloud"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
|
||||
schulcloud-mcp:
|
||||
build: .
|
||||
image: schulcloud-mcp:latest
|
||||
@@ -15,6 +36,14 @@ services:
|
||||
environment:
|
||||
PORT: 8080
|
||||
BIND_HOST: 0.0.0.0
|
||||
MIRROR_DIR: /data/mirror
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
volumes:
|
||||
# The mirror is the one thing this server writes; everything else stays
|
||||
# read-only, so it gets its own volume rather than loosening read_only.
|
||||
- mirror:/data/mirror
|
||||
# No ports are published to the host: Caddy reaches the container over the
|
||||
# shared Docker network, so the only way in from the internet is through
|
||||
# Caddy's TLS and this server's bearer check.
|
||||
@@ -35,6 +64,10 @@ services:
|
||||
cap_drop:
|
||||
- ALL
|
||||
|
||||
volumes:
|
||||
pgdata:
|
||||
mirror:
|
||||
|
||||
networks:
|
||||
caddy:
|
||||
# Set to true once this joins the network your existing Caddy already uses,
|
||||
|
||||
Reference in New Issue
Block a user