Add the schulcloud CLI, and document the split

The CLI talks only to the Pi's /api surface and holds no Schulcloud
credential — only the same bearer token the Claude connector uses. That
is not layering for its own sake: a Schulcloud session dies after two
hours idle and a CLI process lives for seconds, so a CLI with its own
token would be dead most times you reached for it. Routing through the
Pi means one session, one keepalive, one monthly cookie paste.

sync is a one-way mirror, which follows from the data rather than from
scope-cutting: file records are immutable upstream, so there is no
versioning, no conflict resolution and no merge. State is keyed by file
record id with the path as derived output, so an upstream rename moves
the local file instead of duplicating it — verified against the live
server. Verification is size-only because the download endpoint exposes
no ETag and Schulcloud publishes no hash; size still catches the failure
that happens, a truncated download. Downloads land on a .part neighbour
and are renamed, so an interrupted run leaves no half-file that a later
run mistakes for complete. Deletions are reported but not propagated —
a teacher removing a worksheet is no reason to destroy the student's
copy — with --prune to opt in.

what_changed now clamps to the oldest stored generation instead of
refusing, and says it did: "what's new this week" is a reasonable
question to ask a two-day-old index.

Two build bugs caught by the checks rather than by luck: the smoke
harness constructed the app without services, so the index-backed tools
were never exercised; and the Docker build could not see
scripts/copy-assets.mjs, so the image would have shipped without
migrations and silently degraded to live-only.

67 unit tests (9 needing Postgres), smoke green both ways — 34 checks
with an index, 32 without, because graceful degradation is a supported
mode and not a fallback nobody runs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

CLAUDE.md

@@ -4,60 +4,90 @@ Guidance for Claude Code when working in this repository.
 
 ## What this is
 
-An MCP server exposing a Schulcloud (HPI Schul-Cloud / Schulcloud-Verbund-Software)
-account to Claude, read-only: courses, column boards, lessons, tasks, and file
-downloads with text extraction. TypeScript, Node 22+, `@modelcontextprotocol/sdk`.
+Read-only access to a Schulcloud (HPI Schul-Cloud / Schulcloud-Verbund-Software)
+account: courses, column boards, lessons, tasks, files with text extraction, and
+a Postgres-backed full-text index. TypeScript, Node 22+,
+`@modelcontextprotocol/sdk`.
 
-Two entry points, one server definition:
-- `src/bin/http.ts` — Streamable HTTP, the deployed form, behind Caddy on a Pi.
+Three entry points over one core:
+- `src/bin/http.ts` — Streamable HTTP + `/api`, the deployed form, behind Caddy on a Pi.
 - `src/bin/stdio.ts` — stdio, for local Claude Code / Desktop use.
+- `src/bin/cli.ts` — the `schulcloud` CLI, which talks to the HTTP server, never
+  to Schulcloud.
 
 ## Commands
 
 ```bash
-npm run build       # tsc → dist/
+npm run build       # tsc → dist/ (also copies store/migrations/*.sql)
 npm run dev         # watch mode, runs src/ directly via type stripping
 npm test            # unit tests (node:test), no network
 npm run typecheck
 npm run probe       # verify token + API assumptions against the LIVE instance
 npm run smoke       # full end-to-end: real server + real MCP client + real data
+npm run keepalive-status   # is the deployed container holding its session?
 npm run session-diagnose   # ~2.5h: measure what actually ends the session
 ```
 
 `probe` and `smoke` hit the live Schulcloud and need a valid `.env`. Both are
-read-only. Run `smoke` after touching anything in `src/tools/` or
-`src/schulcloud/` — the unit tests cover only pure functions.
+read-only with respect to Schulcloud. Run `smoke` after touching `src/core/`,
+`src/mcp/` or `src/http/` — the unit tests cover only pure functions.
+
+Run smoke **both ways**: with `DATABASE_URL` set (34 checks, index-backed) and
+without (32 checks, live-only). The degradation path is a supported mode, not a
+fallback nobody exercises.
+
+Store tests need a database and skip without one:
+`TEST_DATABASE_URL=postgresql://… npm test`. They use a real Postgres on
+purpose — the generation/diff semantics are entirely SQL, so a mock would test
+nothing.
 
 ## Architecture
 
 ```
-bin/{http,stdio}.ts → server.ts (createServer)
-                        └─ tools/{overview,content,files,search,raw}.ts
-                             └─ context.ts (caches /me → school id)
-                                  └─ schulcloud/client.ts  (all GET, no writes)
-                                     schulcloud/board.ts    (assembles boards)
-                                     extract.ts             (documents → text)
-                                     render.ts              (→ Markdown)
+bin/{http,stdio}.ts ─┬─ mcp/server.ts ── mcp/tools/*
+                     └─ http/{server,api,auth}.ts        /mcp and /api
+                          │
+bin/cli.ts ──HTTP──────────┘   cli/{config,client,sync}.ts
+
+           services.ts  (process-wide: client, Store, Indexer)
+                │
+   indexer/indexer.ts ── store/store.ts ── Postgres
+                │
+            core/{client,board,crawl,extract,text,paths,types}
 ```
 
-- **`schulcloud/client.ts`** — every upstream call. Methods are `GET`-only by
-  design; see "Invariants" below.
-- **`schulcloud/board.ts`** — the non-obvious part. A column board needs three
-  kinds of call to reconstruct; this hides that.
-- **`tools/*.ts`** — each registers a group of tools and formats results as
-  Markdown. Tool descriptions are prompts: they are how Claude decides which
-  tool to reach for, so they carry the German domain terms (Kurse, Themen,
-  Aufgaben) and say when *not* to use the tool.
+- **`core/`** knows nothing of MCP, HTTP or the CLI.
+  - `client.ts` — every upstream call; `GET`-only except `extendSession`.
+  - `board.ts` — a column board needs three kinds of call to reconstruct.
+  - **`crawl.ts`** — the one traversal. Search, the indexer, the what-changed
+    diff and the file mirror all need it; keep it here, not in a tool.
+  - `paths.ts` — the security boundary for mirrored filenames. See Invariants.
+- **`store/`** — crawl generations, identity diffs, `german` + `pg_trgm` FTS.
+  `Store.open` returns `undefined` when Postgres is down; callers degrade.
+- **`indexer/`** — crawl → persist → mirror bytes → extract text → index.
+  Coalesces concurrent refreshes; enforces a minimum interval.
+- **`mcp/tools/*.ts`** — tool descriptions are prompts: they are how Claude picks
+  a tool, so they carry the German domain terms (Kurse, Themen, Aufgaben) and say
+  when *not* to use the tool.
 - **`context.ts`** — per-session state. Only `/me` is cached, because the school
-  id is required on every files-storage path and cannot change for a token.
+  id is on every files-storage path and cannot change for a token.
 
 ## Invariants
 
-**Everything that touches user data is read-only.** Every client method is a
+**Everything that touches Schulcloud is read-only.** Every client method is a
 `GET` except `extendSession` (the keepalive's `refresh-session` call, which
 touches only our own session and is not exposed as a tool, so no model-driven
-call can be a POST). `api_get` rejects non-`/api/` paths and anything carrying
-a scheme or host. The endpoint
+call can be a POST). `api_get` rejects non-`/api/` paths and anything carrying a
+scheme or host. `refresh_index` and `POST /api/refresh` write only to the Pi's
+own index and mirror — every upstream call they make is still a GET.
+
+**Filenames from Schulcloud are untrusted paths.** Course titles, card titles
+and filenames are all user-supplied upstream, and both the server's mirror and
+the CLI's sync turn them into filesystem paths. Everything goes through
+`core/paths.ts`: `safeComponent` reduces one string to one safe component, and
+`resolveWithin` refuses anything that escapes the root. Do not bypass them with
+`path.join`, and keep the property that no `..` survives anywhere in a
+component — it is what makes the invariant checkable. The endpoint
 is internet-facing by necessity, so "a leaked token cannot act as the user" is
 the property that makes that acceptable. Do not add a write tool without the
 user explicitly asking for one and understanding this.
@@ -88,6 +118,13 @@ These cost real time to discover; `docs/API.md` has the full list with evidence.
   its own OpenAPI document. It is not in the main `docs-json`.
 - Legacy lesson responses return ids as `{buffer:{data:[...]}}`; use
   `normalizeObjectId`.
+- **`updatedAt` on the course-board projection is the request time**, not a
+  modification time — two reads seconds apart differ. Never build change
+  detection on it; the store diffs crawl generations by identity instead. The
+  dedicated endpoints (`/boards/{id}`, `/cards`, file records) are stable.
+- Many course PDFs are **image-only scans with no text layer** (3 of 4 sampled),
+  so extraction legitimately yields nothing. `extract.ts` detects this and says
+  so; do not "fix" it by retrying.
 - **`exp` (30 days) is not the session lifetime.** The binding limit is a Valkey
   whitelist entry with a `JWT_TIMEOUT_SECONDS` TTL (7200s; live value at
   `GET /api/v3/config/public`) that every authenticated request re-sets.
@@ -112,16 +149,18 @@ These cost real time to discover; `docs/API.md` has the full list with evidence.
   what the line does. Several such comments encode findings that are expensive
   to rediscover; do not strip them.
 - Tool failures return `isError: true` with an actionable message via
-  `tools/result.ts`. `toToolError` separates 401 (token expired — the user must
+  `mcp/tools/result.ts`. `toToolError` separates 401 (token expired — the user must
   act) from 403 (no access) from 404 (bad id) deliberately; keep that split.
 
 ## Adding a tool
 
-1. Add the client method in `schulcloud/client.ts` (`GET` only).
-2. Register the tool in the relevant `tools/*.ts`, with a description that says
-   when to use it *and when not to*.
+1. Add the client method in `core/client.ts` (`GET` only).
+2. Register the tool in the relevant `mcp/tools/*.ts`, with a description that
+   says when to use it *and when not to*.
 3. Format output as Markdown, keeping ids visible for follow-up calls.
-4. Add a check to `scripts/smoke.mjs` and run `npm run smoke`.
+4. If it reads the index, handle `context.store === undefined` with a message
+   saying what is unavailable and what still works.
+5. Add a check to `scripts/smoke.mjs` and run `npm run smoke` both ways.
 
 ## Environment