diff --git a/README.md b/README.md index 61ca239..ed3e9c1 100644 --- a/README.md +++ b/README.md @@ -83,6 +83,10 @@ per file element — because a model asking "what's on this board" wants the answer, not a traversal plan. Output is Markdown with ids preserved for follow-up calls, not raw JSON. +Possible extensions — full-text search over file contents, a cache with a +bypass, "what's new since…" — are sketched with their trade-offs in +[docs/ROADMAP.md](docs/ROADMAP.md). None are built. + ## Layout ``` diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md new file mode 100644 index 0000000..f4fcbb8 --- /dev/null +++ b/docs/ROADMAP.md @@ -0,0 +1,92 @@ +# Possible extensions + +**Nothing here is built.** This records options considered during setup, the +decisions already taken, and the trade-offs found while building — so none of it +has to be rediscovered. + +## Decisions already taken + +| Decision | Choice | Rejected | +|---|---|---| +| Endpoint auth | Static bearer token (`MCP_AUTH_TOKEN`, constant-time check) | OAuth 2.1; secret URL path | +| State | Stateless — every tool call hits the API live | Postgres cache; cache + full-text search | +| File delivery | Inline extracted text; images inline | Raw base64 only | +| **If** persistence is added | **The Pi's existing PostgreSQL, with its own database and user** | A dedicated container; SQLite | + +Two of the rejected options need no revisiting. OAuth 2.1 is disproportionate +machinery for a single-user endpoint a bearer token already protects. "Raw bytes +as well as extracted text" got built anyway — `download_file` takes `raw: true`. + +## 1. Full-text search over file contents + +**The gap:** `search` covers course/board/card/lesson/task titles and text, and +file *names* — never file *contents*. The 93 PDFs in this account are opaque to +it. "Where does it explain the Caesar cipher?" only hits if those words are in a +filename. + +**The cost:** one `search` walks ~26 course pages, ~30 board skeletons plus card +fetches, and ~150 per-element file listings — roughly **270 upstream requests**, +several seconds, every time. + +**Design sketch** + +- Postgres `tsvector` + GIN. Use the **`german`** dictionary, so *Verschlüsselung* + matches *Verschlüsselungsverfahren*; the default `english` config stems these + wrongly. +- Add **`pg_trgm`** alongside. German compounds defeat stemming — + *Datenschutzgrundverordnung* will not match *Datenschutz* — and trigram + similarity also absorbs typos. +- Index: titles and text of courses, boards, cards, lessons, tasks; file names; + and **extracted file text**. +- Later, optionally: embeddings in `pgvector`, so "the thing about encryption" + finds *Verschlüsselung* without sharing a word. Needs an embedding model in the + loop, so it is a separate step, not part of this. + +## 2. Cache, with bypass + +**The split that makes staleness tolerable** — index is *discovery*, live API is +*detail*. Search the index to find where something is; always re-fetch it to read +it. A stale index then costs a missed or spurious hit, never wrong content. That +is a far better failure mode than a cache that can serve last week's homework +list as current. + +- `fresh: true` on `search` (re-crawl the relevant course first) and on the + detail tools. +- **State freshness in every cached result** ("indexed 2h ago"), regardless of + the flag — a bypass only helps if the agent knows it needs one. +- **Extracted file text is cacheable indefinitely.** A `fileRecord` id maps to + immutable bytes; an edit produces a new record. PDF parsing happens once, ever. + Cheapest large win here, and free of staleness risk. +- Sync: nightly full crawl (~270 requests, trivial) plus on-demand per course. + +**Consequence to weigh:** coursework text would then live on the Pi at rest, +outside Schulcloud. It does not weaken the read-only property, but it is new — +consider disk encryption and whether it lands in a backup. + +## 3. "What's new since …" + +Falls out of having an index with history: diff successive crawls to surface new +boards, cards, files and tasks. **Impossible today at any speed** — the API has no +changed-since filter anywhere. Arguably the most useful item on this list for a +student, and nearly free once the sync job exists. + +## 4. Smaller items + +- **Video/audio transcription** — this account has 5 MP4s and a WebM that are + currently just "here is a file you cannot read". +- **MCP resources and prompts** — expose courses/boards as attachable resources; + canned prompts such as "summarise this week's homework". +- **Calendar** — a separate `schulcloud-calendar` service, not part of the v3 API + mapped in `API.md`. + +## 5. Ruled out + +- **Collaborative text editor contents.** Confirmed unavailable: + `GET /api/v3/collaborative-text-editor/{parentType}/{parentId}` returns a URL + to the editor, never the document text. +- **OCR.** Unnecessary — images are returned inline and Claude reads them + directly. +- **Write tools** (submitting homework, marking tasks done). Technically easy, + but this forfeits the property that makes an internet-facing endpoint + acceptable: that a leaked token cannot act as the user. A deliberate, separate + decision — see the invariant in `CLAUDE.md`.