af4464decb
Schulcloud says what was uploaded and WebUntis says what was scheduled. Neither says what was *taught* — which point the teacher laboured, which example landed, what "will definitely come up". That lives in two places this server could not reach: the notes the user takes in the lesson, and WebUntis' class register. Notes are a directory of Markdown files (NOTES_DIR), not a table. They have to be writable from a phone in a classroom, readable when Postgres is down, and outlive this project, and files are the only shape that is all three — so the files are the truth and the index is a view of them, the same split as file_texts and the mirror. list_notes and get_note read disk, so they answer before the first crawl; search, what_changed and all three German prompts read them alongside the Schulcloud material. add_note writes one, and is the only thing in this server that writes anything. That is not a hole in the read-only invariant but a different store: it is bounded to NOTES_DIR by the same safeComponent/resolveWithin pair that stops a hostile Schulcloud filename escaping the mirror, so a note titled ../../.ssh/authorized_keys becomes a filename. Schulcloud and WebUntis stay GET-only and allowlisted respectively. NOTES_READONLY refuses writes outright. Appending targets the *lesson*, not the title: "halt das auch noch fest" mid-lesson carries a new title, and deriving the path from it would start a second note every time, which is the one thing append exists to prevent. Notes.app has no export — its bodies are compressed protobuf and the iCloud copy is encrypted — so scripting the app is not the clumsy route to the notes but the only one. scripts/export-apple-notes.js reads them through AppleScript into one JSON object per line, and `schulcloud note import` converts the HTML to Markdown, takes the Notes folder as the subject and the *creation* date as the lesson's date. Attachments cannot come across; a note that was a photo of the board imports as a line saying so, because importing it empty would hide the loss. The class register needed one API property to become cheap: getLessonTopic2017 answers per *series*, not per period, so a term is reconstructed by asking about the latest period of each lesson series and merging back by id — a few dozen calls for a school year rather than one per lesson. untis_lesson_topics now takes a subject as well as a period id, and UNTIS_HISTORY_DAYS of register goes into the index under a kind of its own, so "what did we actually do before the test" is searchable. Sharing the snapshot rather than duplicating it caught one thing on the way: the search tool's live path had to learn notes too, or fresh=true would have quietly disagreed with the index. 305 tests; 88/89 smoke against the local instance, the one failure being the H5P service that instance does not run. The live smoke could not be retaken: that session has lapsed and needs a fresh jwt cookie. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
8.1 KiB
Extensions: built and possible
Items 1–3 are now implemented — see docs/CLI.md and the store/,
indexer/ and cli/ modules. What remains below is the rationale (worth
keeping) and the items still open.
Decisions already taken
| Decision | Choice | Rejected |
|---|---|---|
| Endpoint auth | Static bearer token (MCP_AUTH_TOKEN, constant-time check) |
OAuth 2.1; secret URL path |
| State | Stateless — every tool call hits the API live | Postgres cache; cache + full-text search |
| File delivery | Inline extracted text; images inline | Raw base64 only |
| If persistence is added | The Pi's existing PostgreSQL, with its own database and user | A dedicated container; SQLite |
Two of the rejected options need no revisiting. OAuth 2.1 is disproportionate
machinery for a single-user endpoint a bearer token already protects. "Raw bytes
as well as extracted text" got built anyway — download_file takes raw: true.
1. Full-text search over file contents — BUILT
The gap: search covers course/board/card/lesson/task titles and text, and
file names — never file contents. The 93 PDFs in this account are opaque to
it. "Where does it explain the Caesar cipher?" only hits if those words are in a
filename.
The cost: one search walks ~26 course pages, ~30 board skeletons plus card
fetches, and ~150 per-element file listings — roughly 270 upstream requests,
several seconds, every time.
Design sketch
- Postgres
tsvector+ GIN. Use thegermandictionary, so Verschlüsselung matches Verschlüsselungsverfahren; the defaultenglishconfig stems these wrongly. - Add
pg_trgmalongside. German compounds defeat stemming — Datenschutzgrundverordnung will not match Datenschutz — and trigram similarity also absorbs typos. - Index: titles and text of courses, boards, cards, lessons, tasks; file names; and extracted file text.
- Later, optionally: embeddings in
pgvector, so "the thing about encryption" finds Verschlüsselung without sharing a word. Needs an embedding model in the loop, so it is a separate step, not part of this.
2. Cache, with bypass — BUILT (search fresh=true, refresh_index)
The split that makes staleness tolerable — index is discovery, live API is detail. Search the index to find where something is; always re-fetch it to read it. A stale index then costs a missed or spurious hit, never wrong content. That is a far better failure mode than a cache that can serve last week's homework list as current.
fresh: trueonsearch(re-crawl the relevant course first) and on the detail tools.- State freshness in every cached result ("indexed 2h ago"), regardless of the flag — a bypass only helps if the agent knows it needs one.
- Extracted file text is cacheable indefinitely. A
fileRecordid maps to immutable bytes; an edit produces a new record. PDF parsing happens once, ever. Cheapest large win here, and free of staleness risk. - Sync: nightly full crawl (~270 requests, trivial) plus on-demand per course.
Consequence to weigh: coursework text would then live on the Pi at rest, outside Schulcloud. It does not weaken the read-only property, but it is new — consider disk encryption and whether it lands in a backup.
3. "What's new since …" — BUILT (what_changed)
Falls out of having an index with history: diff successive crawls to surface new boards, cards, files and tasks. Impossible today at any speed — the API has no changed-since filter anywhere. Arguably the most useful item on this list for a student, and nearly free once the sync job exists.
4. Remaining gaps, measured
Scanned the instance's 98 GET routes against what the tools cover. Almost everything student-facing is now covered; what is left, with live checks:
| Area | State on this account |
|---|---|
GET /groups/class |
3 real classes — the only uncovered endpoint with data |
GET /rooms |
empty — the feature is unused here |
GET /media-boards/me |
exists, no content |
GET /course-info |
403 — teacher/admin only |
GET /alert |
empty |
tools, oauth2, school, registrations, systems, user-login-migrations |
admin/auth plumbing, not student-facing |
Genuinely unavailable, not merely uncovered:
- Collaborative text editor contents — the endpoint returns an editor URL, never the document text.
- Calendar — a separate service, absent from the v3 document.
- Image-only PDFs — 22 of 255 indexed files are scans with no text layer, so their contents cannot be searched without OCR.
- Numeric grades — the API's
gradewas null on every graded submission here, so that path stays unverified against real data.
4b. A third source: the user's own notes — BUILT
Measured after a term of use: Schulcloud says what was uploaded and WebUntis says what was scheduled. Neither says what was taught — which point the teacher laboured, which example landed, what "will definitely come up". That is only ever in what the student wrote down, and it was sitting in Apple Notes where nothing could read it.
Built as a directory of Markdown files (NOTES_DIR), read by list_notes /
get_note, indexed as kind: 'note', searched by both the index and the live
path, diffed by what_changed, and consulted by all three German prompts.
add_note writes one — the only write in this server, and bounded to that
directory by the same safeComponent/resolveWithin pair that guards the file
mirror. scripts/export-apple-notes.js plus schulcloud note import is the
migration path out of Notes.app, which has no export of its own. See
docs/NOTES.md.
Files rather than a table, deliberately: they have to be writable from a classroom, readable when Postgres is down, and outlive this project.
4c. The class register into the index — BUILT
untis_lesson_topics could already answer "where did we get to" for one series
from one period id. What it could not do was answer "what have we done in
Deutsch this term", and none of it was searchable.
Both fall out of one API property: getLessonTopic2017 answers per series, so
a term costs one call per lesson series (core/untis-history.ts). The tool now
takes a subject as well as a period id, and UNTIS_HISTORY_DAYS of register —
topics, the notes teachers leave on a period, announced tests, homework — go
into the index as kind: 'untis'.
5. Still open
- Video/audio transcription — this account has 5 MP4s and a WebM that are currently just "here is a file you cannot read".
- MCP resources and prompts — partly built. Courses and rooms are resources,
and there are two German prompts (
zusammenfassung,pruefungsvorbereitung). Still open: files as resources — about a thousand of them do not fit a picker that lists every entry, so they need a browsing UI (an MCP App) rather than a longer list — and claude.ai, which cannot reach a server on localhost and does not send a static bearer token. - Calendar — a separate
schulcloud-calendarservice, not part of the v3 API mapped inAPI.md.
5. Ruled out
- Collaborative text editor contents. Confirmed unavailable:
GET /api/v3/collaborative-text-editor/{parentType}/{parentId}returns a URL to the editor, never the document text. - OCR — partially wrong, revised. For reading, it remains unnecessary:
images go to Claude inline and it reads them. For indexing it is a real gap.
Measured after building the indexer: 3 of 4 sampled course PDFs have no
embedded fonts at all — they are scans, so extraction legitimately yields
nothing and they are unsearchable by content.
extract.tsnow reports these as image-only rather than as an empty result. Making them searchable would need OCR (or page rasterisation plus a vision pass), which is the largest remaining gap in search coverage. - Write tools (submitting homework, marking tasks done). Technically easy,
but this forfeits the property that makes an internet-facing endpoint
acceptable: that a leaked token cannot act as the user. A deliberate, separate
decision — see the invariant in
CLAUDE.md.