Read the user's own lesson notes, and the class register behind them

Schulcloud says what was uploaded and WebUntis says what was scheduled.
Neither says what was *taught* — which point the teacher laboured, which
example landed, what "will definitely come up". That lives in two places
this server could not reach: the notes the user takes in the lesson, and
WebUntis' class register.

Notes are a directory of Markdown files (NOTES_DIR), not a table. They
have to be writable from a phone in a classroom, readable when Postgres
is down, and outlive this project, and files are the only shape that is
all three — so the files are the truth and the index is a view of them,
the same split as file_texts and the mirror. list_notes and get_note read
disk, so they answer before the first crawl; search, what_changed and all
three German prompts read them alongside the Schulcloud material.

add_note writes one, and is the only thing in this server that writes
anything. That is not a hole in the read-only invariant but a different
store: it is bounded to NOTES_DIR by the same safeComponent/resolveWithin
pair that stops a hostile Schulcloud filename escaping the mirror, so a
note titled ../../.ssh/authorized_keys becomes a filename. Schulcloud and
WebUntis stay GET-only and allowlisted respectively. NOTES_READONLY
refuses writes outright.

Appending targets the *lesson*, not the title: "halt das auch noch fest"
mid-lesson carries a new title, and deriving the path from it would start
a second note every time, which is the one thing append exists to prevent.

Notes.app has no export — its bodies are compressed protobuf and the
iCloud copy is encrypted — so scripting the app is not the clumsy route
to the notes but the only one. scripts/export-apple-notes.js reads them
through AppleScript into one JSON object per line, and `schulcloud note
import` converts the HTML to Markdown, takes the Notes folder as the
subject and the *creation* date as the lesson's date. Attachments cannot
come across; a note that was a photo of the board imports as a line
saying so, because importing it empty would hide the loss.

The class register needed one API property to become cheap:
getLessonTopic2017 answers per *series*, not per period, so a term is
reconstructed by asking about the latest period of each lesson series and
merging back by id — a few dozen calls for a school year rather than one
per lesson. untis_lesson_topics now takes a subject as well as a period
id, and UNTIS_HISTORY_DAYS of register goes into the index under a kind
of its own, so "what did we actually do before the test" is searchable.

Sharing the snapshot rather than duplicating it caught one thing on the
way: the search tool's live path had to learn notes too, or fresh=true
would have quietly disagreed with the index.

305 tests; 88/89 smoke against the local instance, the one failure being
the H5P service that instance does not run. The live smoke could not be
retaken: that session has lapsed and needs a fresh jwt cookie.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
MechaCat02
2026-09-18 21:45:39 +02:00
parent ad8ba28313
commit af4464decb
34 changed files with 3078 additions and 61 deletions

View File

@@ -390,6 +390,15 @@ uses is the API the Untis Mobile app uses, verified against
`previousTopics`: what the earlier lessons of that series actually covered,
from the class register. A `periodIds` array is rejected as "period 0 not
found".
- **It answers per *series*, so one call covers a term.** The entries come
back with their own `periodId` and date, which is what lets a range of
lessons be reconstructed from a handful of calls rather than one per period:
take the distinct `lessonId`s in the range, ask about the **latest**
`periodId` of each, and merge the answers back onto the periods by id.
Asking about the earliest period of a series instead reaches none of its
history, because "previous" is relative to the period given.
`core/untis-history.ts` is that walk, and it is what puts the class register
into the search index.
- **The exam module is unused at this school**, so `getExams2017` is empty and
`period.exam` is null. Announced tests are typed into the period's **info
text** instead ("LF10: Leistungskontrolle agile Softwareentwicklung …"), which

View File

@@ -39,6 +39,7 @@ schulcloud ls [--course <id>] [--long]
schulcloud get <fileId> [--out <path>]
schulcloud sync [--dry-run] [--full] [--prune] [--dir <path>] [--jobs <n>]
schulcloud refresh [--course <id>] [--force]
schulcloud note ... the notes you take in class — see below
```
`ls --long` prints file ids, which is what `get` takes.
@@ -114,6 +115,39 @@ while.
`Persönliche Dateien/…`, `Team-Dateien/<team>/…` and `Geteilte Dateien/`, once
the server's index includes them (`INDEX_FILE_MANAGER`, on by default).
### Your own lesson notes (`note`)
The notes you take in class, which Claude reads as context. The full story is in
[NOTES.md](NOTES.md); these are the commands.
```
schulcloud note ls [--subject <name>] [--since <date>] [--until <date>] [--long]
schulcloud note show <path>
schulcloud note add --title <title> [--subject <name>] [--date <date>]
[--tags a,b] [--append] text on stdin, or --text
schulcloud note import <export.ndjson> [--subject <name>] [--out <dir>] [--dry-run]
```
```console
$ pbpaste | schulcloud note add --title "Subnetting" --subject LF07
Saved LF07/2026-09-16 Subnetting.md
$ schulcloud note ls --subject Deutsch --since 2026-09-01
2026-09-22 [Deutsch] Sprachanalyse
2026-09-15 [Deutsch] Erörterung — Aufbau
$ schulcloud note show "Deutsch/2026-09-15 Erörterung.md"
```
`note add` reads the note from stdin, so it comes just as easily from a
clipboard, an editor or another command; `--append` adds to the note already
written for that subject and day rather than starting a second one.
`note import` takes the file `scripts/export-apple-notes.js` writes on a Mac.
`--dry-run` shows what it would do and `--out <dir>` writes the Markdown
locally instead of sending it to the server — the only note command that needs
no server at all.
## How sync works
It is a **one-way mirror, not a two-way sync**, and that follows from the data

205
docs/NOTES.md Normal file
View File

@@ -0,0 +1,205 @@
# Your own lesson notes
Schulcloud holds the material and WebUntis holds the schedule. Neither holds
what was actually said in the room — which teacher stressed what, the example
that finally made it click, the aside that turns up in the test. That is in
whatever you write down during the lesson, and until now it lived somewhere no
agent could read.
This is the third source: a directory of Markdown files the server reads,
indexes and searches alongside everything else.
```
NOTES_DIR/
Deutsch/
2026-09-15 Erörterung.md
2026-09-22 Sprachanalyse.md
LF07/
2026-09-16 Subnetting.md
Allgemein/
2026-09-18 Elternabend.md
```
## Why files
Three properties, in this order:
- **They have to be writable from a classroom.** Whatever you take notes in on a
phone or a laptop, it can produce text files; nothing can produce rows in the
Pi's Postgres.
- **They have to be readable when the index is down.** `list_notes` and
`get_note` read the disk, so they answer before the first crawl and while
Postgres is unreachable — the index is a view of the notes, never the notes
themselves. That is the same split as extracted file text and the mirror.
- **They have to survive this project.** A directory of dated Markdown is
greppable, diffable, syncable and still yours if the server is thrown away.
## What a note looks like
Frontmatter, then the note. Every field is optional, and a plain `.md` file with
no frontmatter at all is a perfectly good note.
```markdown
---
title: Erörterung — Aufbau
date: 2026-09-15
subject: Deutsch
tags: [klausur, aufsatz]
---
Dreischritt: These, Argument mit Beleg, Fazit.
Frau Meier betont: das **Gegenargument** darf nicht fehlen, sonst gibt es
Abzug — kam letztes Jahr in der Arbeit dran.
```
What the server reads out of it:
| Field | Falls back to |
|---|---|
| `title` | the first `# heading`, else the filename without its date |
| `date` | a `YYYY-MM-DD` the **filename** starts with, else nothing |
| `subject` | the first folder name (`Deutsch/…`) |
| `tags` | none |
| `courseId` | none — set it to tie a note to a Schulcloud course in `search` |
`date` also accepts `15.09.2026`, and `fach:` works as a German spelling of
`subject:`. Anything else in the block is kept but not interpreted.
**The file's modification time is never used as the date.** An import writes
every note today; dating a year of lessons "today" would make the whole store
useless for revision, which is most of what it is for.
## Reading them
| Tool | Answers |
|---|---|
| `list_notes` | "what did I write down in Deutsch before the test" — filter by subject or date |
| `get_note` | one note in full, by the path everything else prints |
| `search` | notes by their contents, next to board text and the inside of PDFs |
| `what_changed` | notes that appeared or were edited since a date |
The `pruefungsvorbereitung`, `zusammenfassung` and `tagesvorbereitung` prompts
consult them on their own, and are told to say when a note disagrees with the
uploaded material rather than quietly preferring one.
## Writing them
Four ways in, all landing in the same files:
```bash
# from the command line, text piped in
pbpaste | schulcloud note add --title "Subnetting" --subject LF07
# from an editor, or anything else that writes files
$EDITOR "$NOTES_DIR/LF07/2026-09-16 Subnetting.md"
# by asking Claude, during or after the lesson
# "halt fest: Gegenargument nicht vergessen, kam letztes Jahr dran"
# → add_note, subject Deutsch, today's date
# by syncing a folder you already write in — see below
```
`add_note` and `schulcloud note add` take `append`, which adds to the note
already written for that subject and day instead of starting a second one. That
is the right choice while a lesson is running: notes accumulate in one file the
way they do on paper.
**This is the only thing in this server that writes anything.** It writes into
the notes directory and nowhere else: every path component goes through
`safeComponent` and the result through `resolveWithin`, the same two functions
that stop a hostile filename escaping the file mirror, so a note titled
`../../.ssh/authorized_keys` becomes a filename. Schulcloud and WebUntis stay
strictly read-only — see the invariants in `CLAUDE.md`. Set `NOTES_READONLY=1`
to refuse writes entirely, which is right when the notes are synced in from
somewhere else and should have exactly one writer.
## Migrating out of Apple Notes
Notes.app has no export. Its database is a Core Data store whose bodies are
compressed protobuf and whose iCloud copy is encrypted, so scripting the app is
not the clumsy route to your notes — it is the only one.
**On the Mac**, from a checkout of this repo:
```bash
osascript -l JavaScript scripts/export-apple-notes.js > notes.ndjson
```
The first run raises a macOS permission dialog ("Terminal wants access to
Notes"); without it every note comes back empty. `--folder Deutsch` exports one
Notes folder.
Then convert and import. Look at it first:
```bash
schulcloud note import notes.ndjson --dry-run
```
```
would import: 2026-09-15 · Deutsch · Erörterung
would import: 2026-09-16 · LF07 · Subnetting
Would import 84 of 91 note(s), skipped 5 with no text, 2 unreadable in Notes.
```
and then for real, either straight into the server:
```bash
schulcloud note import notes.ndjson
```
or into a local directory, to read through before anything is sent anywhere:
```bash
schulcloud note import notes.ndjson --out ~/Notizen
```
What the import does with each note:
- **The Notes folder becomes the subject** — its last segment, so `Schule/Deutsch`
is `Deutsch`. Notes' own default folders (`Notizen`, `Recently Deleted`) are
ignored rather than becoming a subject. `--subject LF07` overrides all of it.
- **The creation date becomes the note's date**, because that is the day of the
lesson. The modification date is whenever you last tidied it up, which is not
a school day at all.
- **The HTML becomes Markdown** — headings, lists, checklists, bold, italics and
links survive; anything else keeps its words and loses its tag.
- **Attachments do not come across.** A note that was a photo of the board
imports as a line saying an attachment was there. Better than importing empty:
you can see which notes still need the picture.
- **Locked notes cannot be read at all** and are listed by name at the end.
Unlock them in Notes and export again.
Re-running the import creates second copies rather than overwriting, since the
importer cannot tell an edited note from a new one with the same title. Import
once, then keep writing in the new place.
## Writing them from a phone, afterwards
The notes are files, so any sync tool will do and the server does not need to
know which. Bind-mount the folder instead of using the volume:
```yaml
# docker-compose.yml, under schulcloud-mcp:
volumes:
- /home/pi/Notizen:/data/notes
```
Then point Syncthing, Nextcloud, an Obsidian vault or `git` at
`/home/pi/Notizen` on one side and your phone on the other. New files are picked
up by the next crawl; nothing has to be told about them.
Without any of that, `schulcloud note add` and `add_note` still work — they go
over the same authenticated `/api` as everything else the CLI does.
## What the crawl does with them
A **full** crawl reads the directory and indexes every note: title, subject,
tags and body, under the kind `note`, with its path as its id. A **per-course**
refresh leaves them alone — notes belong to the account, not to a course — and
the store carries the previous generation's rows forward, so a per-course crawl
never looks like the notes were deleted.
Notes are diffed by content, not by modification time, so a sync tool that
rewrites a file byte-for-byte does not show up in `what_changed` as an edit.

View File

@@ -94,6 +94,38 @@ Genuinely unavailable, not merely uncovered:
- **Numeric grades** — the API's `grade` was null on every graded submission
here, so that path stays unverified against real data.
## 4b. A third source: the user's own notes — BUILT
Measured after a term of use: Schulcloud says what was *uploaded* and WebUntis
says what was *scheduled*. Neither says what was *taught* — which point the
teacher laboured, which example landed, what "will definitely come up". That is
only ever in what the student wrote down, and it was sitting in Apple Notes
where nothing could read it.
Built as a directory of Markdown files (`NOTES_DIR`), read by `list_notes` /
`get_note`, indexed as `kind: 'note'`, searched by both the index and the live
path, diffed by `what_changed`, and consulted by all three German prompts.
`add_note` writes one — the only write in this server, and bounded to that
directory by the same `safeComponent`/`resolveWithin` pair that guards the file
mirror. `scripts/export-apple-notes.js` plus `schulcloud note import` is the
migration path out of Notes.app, which has no export of its own. See
`docs/NOTES.md`.
**Files rather than a table**, deliberately: they have to be writable from a
classroom, readable when Postgres is down, and outlive this project.
## 4c. The class register into the index — BUILT
`untis_lesson_topics` could already answer "where did we get to" for one series
from one period id. What it could not do was answer "what have we done in
Deutsch this term", and none of it was searchable.
Both fall out of one API property: `getLessonTopic2017` answers per *series*, so
a term costs one call per lesson series (`core/untis-history.ts`). The tool now
takes a subject as well as a period id, and `UNTIS_HISTORY_DAYS` of register —
topics, the notes teachers leave on a period, announced tests, homework — go
into the index as `kind: 'untis'`.
## 5. Still open
- **Video/audio transcription** — this account has 5 MP4s and a WebM that are