Commit Graph

49 Commits

Author SHA1 Message Date
MechaCat02
c259d97f04 Do not let a table be the end of a note
A table, code block, quote or list as the last element of a
contenteditable element is a dead end: there is no node after it to put
the caret in, and no key that makes one, so the note simply cannot be
continued below it. Every engine behaves this way. The editor now keeps
an empty paragraph after such a block — it serializes to nothing, so it
never reaches the file, which a test pins down.

Tables got the rest of what they were missing while the cause was in
view: Tab walks the cells and a Tab out of the last one adds a row, the
toolbar button adds a row when the cursor is already in a table (a phone
has no Tab key, and it renames itself so it says which it will do), and
Ctrl/Cmd+Enter opens a paragraph after whatever block the cursor is in.
Empty cells are given a break, because a `<td></td>` with nothing in it
cannot be clicked into in Gecko — a blank cell was uneditable.

Driven in Firefox against a page that reproduces the dead end first.
2026-09-20 13:06:25 +02:00
MechaCat02
534b1b0f58 Write notes as formatted text, store them as Markdown
The editor was a textarea holding raw Markdown, which is the wrong thing
to hand someone taking notes during a lesson: nobody types `##` and `**`
while a teacher is talking. It now shows the note formatted and puts a
toolbar above it — headings, bold, lists, tick boxes, quotes, links,
tables — while the file on disk stays exactly what it was, because that
is what the indexer reads and what outlives this app.

`markdown.js` is the whole translation: `markdownToHtml` on the way in,
`markdownFromDom` on the way out. The property that matters is that the
round trip settles — one pass may tidy a note, a second must change
nothing — because these notes are the only record of what was said in
the room and there is nothing to restore a lossy save from. `editor.js`
checks exactly that before opening a note formatted, and a note it
cannot hold unchanged opens in the Markdown view and says so instead of
being quietly reduced.

No editor library: the content security policy allows no outside script
and the app has no bundler, so this is `contenteditable` and
`execCommand` with a tolerant serializer behind it — an element it does
not model keeps its words and loses its tag. Pasted HTML is converted to
Markdown before it reaches the document, which is the one place where
sanitising and formatting are the same operation.

Tested against `test/mini-dom.ts`, sixty lines of read-only DOM, rather
than a headless browser or a DOM dependency; the toolbar itself was
driven by hand in Firefox. WebKit has still never run it.
2026-09-19 21:24:42 +02:00
MechaCat02
5c0b658855 Merge: the user's own lesson notes, the class register, and an app to write them in
Three sources instead of two. Schulcloud has the material, WebUntis has
the schedule and now the class register, and the notes have what the
teacher actually stressed — which was the half nothing here could reach.

- core/notes.ts, the notes as Markdown files; a note with ## headings is
  a school day and is indexed per lesson, not whole.
- core/untis-history.ts, the class register read backwards — one call per
  lesson series covers a term, because getLessonTopic2017 answers per
  series rather than per period.
- core/day-note.ts and http/app*, the app at /app: a login, a day editor
  whose headings come from WebUntis, and a settings page for the
  Schulcloud token.
- scripts/export-apple-notes.js and `schulcloud note import`, the way out
  of Notes.app, which has no export of its own.

docs/NOTES.md is the guide, docs/DEPLOY-NOTES.md the rollout runbook.
2026-09-19 17:32:38 +02:00
MechaCat02
ac61aea870 Create /data/notes in the image, and write the rollout runbook
The Dockerfile creates /data/mirror and /data/state with the right owner,
and the comment above it says exactly why: Docker initialises a new named
volume from the image directory, so a mount point the image does not have
lands root-owned and the unprivileged user gets EACCES on every write.
/data/notes was added to docker-compose.yml without being added here, so
every save on a fresh deployment would have failed that way — verified
both directions before fixing it.

docs/DEPLOY-NOTES.md is the runbook for putting this on a server that is
already running: publish, decide where the notes live *before* anything
writes one, set WEB_PASSWORD, verify, migrate, index. Plus rollback,
which is uneventful — no migration, and the old image simply ignores the
new settings and leaves the notes volume alone.

PI.md's backup table needed the bigger change. Everything else this
server stores is a copy of something upstream and a crawl rebuilds it;
the notes are not, and nothing can. They are now the one entry in that
table marked irreplaceable, and the EACCES row says to fix the volume's
ownership rather than delete it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-19 17:32:30 +02:00
MechaCat02
dc50b4bcd5 Write the notes in an app, a school day at a time
The notes existed but there was nowhere to write them: a CLI command on a
laptop, a tool call through Claude, or a file in a Docker volume. None of
those is reachable from a phone in a lesson, which is where notes are
actually taken.

So: `/app`, served only when WEB_PASSWORD is set. A login, the day's
notes, and a settings page for the Schulcloud token — the one surface
here meant for a person rather than a program.

The shape follows how the notes are written: one note per school day,
one `##` heading per lesson, prose and lists and tables beneath. That
turns out to be the design decision that matters, twice over.

First, it is what lets WebUntis earn its keep. Opening a day with no
note fills in that day's lessons — numbered, with times, teacher and
room, cancellations dropped and substitutions marked. Retyping the
timetable is exactly the work the second upstream exists to avoid, and
"Stunden ergänzen" tops up a note started before the day ended without
touching what is already written.

Second, it changes how notes are indexed. A day note is indexed per
lesson, not whole: search answers "my own note, Deutsch, 18.09.2026"
rather than "my own note, Friday", and `list_notes subject=Deutsch`
finds a day whose frontmatter names no subject at all. Indexed whole,
every hit would read as a weekday and "what did we do in Deutsch" would
match notes whose other five lessons were something else. `lessonHeading`
and `subjectFromHeading` are a loop — the app writes the heading, the
indexer reads the subject back out — and a test holds them to it.

Notes taken in a lesson cannot be retaken, so the editor is built
around not losing them: autosave, every keystroke mirrored to local
storage, a save when the phone locks, and a fallback to the local copy
when the request never arrives. A save that would overwrite a version
the editor never saw is refused and the choice handed back — the notes
folder is synced and open in more than one place, and a phone must not
silently win over a laptop. `replaceNote` is separate from `writeNote`
for that reason: never-overwrite is right for `add_note` and exactly
wrong for an editor.

WEB_PASSWORD is the first credential here a human types, so it is the
first that can be guessed: scrypt at startup, never stored or compared
in the clear, per-address rate limiting — which is not decoration, since
the scrypt cost is itself a denial-of-service vector without it. The
session is a signed HttpOnly SameSite=Strict cookie whose key is derived
from the password, so changing it logs everyone out and there is no
second secret to keep. It opens /api, because a session is the user, and
never /mcp, because nothing in a browser speaks MCP.

Also here, because the app made them matter: frontmatter now reads the
indented `- item` list form editors write, so an Obsidian vault
round-trips its tags; and a four-digit folder is a filing scheme, not a
subject, so `2026/` does not file a school year under one.

357 tests; 106/107 smoke against the local instance, the one failure
being the H5P service that instance does not run.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-19 17:21:17 +02:00
MechaCat02
af4464decb Read the user's own lesson notes, and the class register behind them
Schulcloud says what was uploaded and WebUntis says what was scheduled.
Neither says what was *taught* — which point the teacher laboured, which
example landed, what "will definitely come up". That lives in two places
this server could not reach: the notes the user takes in the lesson, and
WebUntis' class register.

Notes are a directory of Markdown files (NOTES_DIR), not a table. They
have to be writable from a phone in a classroom, readable when Postgres
is down, and outlive this project, and files are the only shape that is
all three — so the files are the truth and the index is a view of them,
the same split as file_texts and the mirror. list_notes and get_note read
disk, so they answer before the first crawl; search, what_changed and all
three German prompts read them alongside the Schulcloud material.

add_note writes one, and is the only thing in this server that writes
anything. That is not a hole in the read-only invariant but a different
store: it is bounded to NOTES_DIR by the same safeComponent/resolveWithin
pair that stops a hostile Schulcloud filename escaping the mirror, so a
note titled ../../.ssh/authorized_keys becomes a filename. Schulcloud and
WebUntis stay GET-only and allowlisted respectively. NOTES_READONLY
refuses writes outright.

Appending targets the *lesson*, not the title: "halt das auch noch fest"
mid-lesson carries a new title, and deriving the path from it would start
a second note every time, which is the one thing append exists to prevent.

Notes.app has no export — its bodies are compressed protobuf and the
iCloud copy is encrypted — so scripting the app is not the clumsy route
to the notes but the only one. scripts/export-apple-notes.js reads them
through AppleScript into one JSON object per line, and `schulcloud note
import` converts the HTML to Markdown, takes the Notes folder as the
subject and the *creation* date as the lesson's date. Attachments cannot
come across; a note that was a photo of the board imports as a line
saying so, because importing it empty would hide the loss.

The class register needed one API property to become cheap:
getLessonTopic2017 answers per *series*, not per period, so a term is
reconstructed by asking about the latest period of each lesson series and
merging back by id — a few dozen calls for a school year rather than one
per lesson. untis_lesson_topics now takes a subject as well as a period
id, and UNTIS_HISTORY_DAYS of register goes into the index under a kind
of its own, so "what did we actually do before the test" is searchable.

Sharing the snapshot rather than duplicating it caught one thing on the
way: the search tool's live path had to learn notes too, or fresh=true
would have quietly disagreed with the index.

305 tests; 88/89 smoke against the local instance, the one failure being
the H5P service that instance does not run. The live smoke could not be
retaken: that session has lapsed and needs a fresh jwt cookie.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-18 21:46:26 +02:00
MechaCat02
ad8ba28313 Document the H5P findings
docs/API.md gains the endpoint and what it returns, why `play` is the bigger
and lesser of the two, the element-by-id route found while probing, and the
per-library shapes with the traps in them — the string "true", the inline cloze
markers, the correct-option-first convention, and the UI subtrees that drown an
exercise in button labels.

CLAUDE.md's "no quiz of its own" note said a contentId was the only handle onto
the content. That was the reason nobody looked further, so it now says where to
look instead.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 21:39:19 +02:00
MechaCat02
a0cef532c6 Read the quizzes behind H5P elements
A quiz in Schulcloud is an H5P element, and a board hands over nothing but a
contentId — so a teacher's exercise was until now a line saying one exists.
The player shows a single question at a time, which makes it look like
something to step through or scrape. It is not:
`GET /api/v3/h5p-editor/params/{contentId}` returns the JSON the player is fed,
so one request holds every question, every option and which of them are
correct. (`play/{id}` is the same content plus the player's script lists: 74 kB
against 51 kB for the live quiz. Neither docs-json describes the service.)

get_h5p prints the exercise, and solutions=false keeps the options while
dropping the answers, so it can be used to ask the questions instead of
answering them. get_board names the exercise — title, question count, kinds —
rather than printing a bare id, and the crawl indexes its text, so a phrase
that exists only inside a quiz is now findable. That is the treatment pads
already get, for the same reason: it is course material and nothing else
surfaces it.

What varies is the shape inside `params`, which belongs to whichever H5P
library the teacher used. Modelled: MultiChoice, whose `behaviour.singleAnswer`
is the only honest source for "tick exactly one"; TrueFalse, whose `correct` is
the string "true"; the cloze libraries, which mark solutions inline as
`*answer:tip*`; SingleChoiceSet and Summary, which put the correct option first
and let the player shuffle; and Column. Anything else has its text harvested
and labelled unmodelled — an exercise reported as "0 questions" would be worse
than a clumsy rendering of one. The harvest skips the UI and l10n subtrees, or
a quiz reads as "Überprüfen, Wiederholen, Absenden".

Verified against this account's quiz, an H5P.QuestionSet of 20 MultiChoice
questions on a room's board: 239 tests, smoke 91/91 live-only and 93/93 with
the index.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 21:39:19 +02:00
MechaCat02
19d9bce2f7 Fix a flaky arm64 check in publish-image
`! docker buildx inspect | grep -q linux/arm64` under `set -o pipefail` fails
when grep exits on its first match before buildx has finished writing: buildx
dies of SIGPIPE, the pipeline reports 141, and the script tells the operator to
register emulation that is already registered. It refused to publish twice in a
row that way, with arm64 present each time — and the same race is why it ever
worked. Matching against the captured output has no pipe to break.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 20:46:04 +02:00
MechaCat02
ccbf3ad3e9 Document WebUntis
docs/API.md gains the findings: the endpoint and its required version
parameter, the one-time code and the two error codes worth naming, the Z that
means local time, a substitution being two periods, the unused exam module
that puts announced tests in the period notes, and a day without lessons that
is not a holiday. Those cost an afternoon of probing to learn and nothing
upstream states them.

docs/AUTH.md sets the key against the Schulcloud token it sits beside: no
password, nothing to keep alive, revocable on its own, and not read-only in
itself — which is why the allowlist exists. PI.md and DEPLOYMENT.md add the
four values, the container recreate a changed key needs, and the clock
requirement; the Pi's troubleshooting table gains both failure messages.

README and CLAUDE.md say what the server now is: Schulcloud for the material,
WebUntis for the day. Smoke is 89 checks with the index and a key, 87
live-only, 9 fewer without one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 20:43:24 +02:00
MechaCat02
eaf9c7aa38 Add a German prompt that prepares a school day
Tagesvorbereitung is where the two systems meet. It attaches the day's
timetable and then asks for the rest: match each lesson to its Schulcloud
course, read what is new there, look at what the previous lesson covered,
check both lists of due work, and read the notes on the periods, where the
announced tests are. Written for the morning before school, so it asks for
something short.

The argument takes heute, morgen, übermorgen or a date in either notation,
because evening preparation is the normal case and a German keyboard writes
21.09.2026. Registered only with WebUntis configured: a Tagesvorbereitung
that has to ask which lessons exist is not one.

readTimetable is shared with untis_timetable, the way readCourse is shared
with get_course, so an attached day reads exactly like a fetched one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 20:43:24 +02:00
MechaCat02
74b97bc4dc Read the timetable from WebUntis
Schulcloud holds the material for a lesson but not the lesson: this school's
course `times` are empty and it publishes the schedule in WebUntis. So "what
do I have today, and has anything been cancelled" was unanswerable, and the
timetable cannot be typed into a prompt either — it changes daily.

core/untis.ts talks to the API the Untis Mobile app uses, and three tools sit
on it: untis_timetable (a day or a range, Entfall, Vertretung, room changes,
the notes on each period, inline homework, the period id), untis_homework (the
class register's list, which is not Schulcloud's tasks) and
untis_lesson_topics (what earlier lessons of a series actually covered, which
is what says where a subject got to).

Read-only, but not by the Schulcloud client's rule: this API is JSON-RPC, so
every call is a POST, reads included. READ_METHODS is the guarantee instead,
enforced at the single choke point and asserted by a test. It matters because
the key can do what the app can — the live account holds W_OWN_ABSENCE, so
the same key could report the user absent.

What the live instance taught us, all recorded in docs/API.md:

- `startDateTime` ends in Z and is local time. The 08:00 lesson reports
  08:00Z, so new Date() would move every lesson by an hour or two.
- A substitution is two periods, the original CANCELLED and the replacement
  IRREGULAR beside it, not one period with a changed teacher.
- Announced tests live in the period's info text. The exam module is unused
  here, so getExams2017 is always empty and that field carries the tests.
- A day with no lessons is not a holiday: the weeks this account spends in the
  company simply have no periods.
- `?v=i3.2` is required, or the call fails with a Java NPE reported as -8998.
  Errors arrive with HTTP 200 and an error member. -8504 is a rejected key and
  -8524 a drifting clock; the tools name both, because no retry fixes either.

Configuration is all four UNTIS_* values or none — three are identifiers and
the fourth is a credential, so a half-filled block is a paste that went wrong.
Without them the tools are not registered at all, since a tool that can only
fail is worse than a missing one. whoami reports the WebUntis identity and
survives a dead Schulcloud session, so "is the server reachable" no longer
gets a misleadingly total no. mcp-env.sh switches WebUntis off for a fixture
run: that key belongs to the real school.

224 tests. All 10 WebUntis smoke checks pass, with a key and without one; the
Schulcloud checks in those runs answer 401 because this machine's session is
logged out.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 20:43:24 +02:00
MechaCat02
87c5cedf3c Add RFC 6238 one-time codes
WebUntis authenticates the mobile app not with a password but with a static
base32 key, from which every request derives a fresh code. That is the
credential an always-on server wants: it works under the school's SSO, needs
no session, and expires only when a new key is generated.

Implemented rather than pulled in. It is HMAC-SHA1 plus a truncation,
node:crypto has the hard part, and a dependency that handles a credential is
one worth not having. The tests are the RFC's own vectors, which validate the
base32 table as much as the arithmetic.

The code comes back zero-padded, as a string. One in ten begins with a zero
and a JSON number would drop it, which is a login that fails a tenth of the
time and looks like a server fault.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 20:43:24 +02:00
MechaCat02
196e10eacc Keep school dates in the school's timezone
The container runs on UTC and the school does not: at 00:30 in Erfurt the
process clock still says yesterday, so anything deriving "today" from a Date
would prepare the wrong school day twice a night.

core/dates.ts holds calendar dates as plain YYYY-MM-DD strings, which is what
a school day is — today in Europe/Berlin, whole-day arithmetic anchored at
noon UTC so no daylight-saving change can shift a date, the German weekday and
day formats, and the compact form the timetable API takes.

isCalendarDate round-trips rather than only matching a shape: Date.parse turns
2026-02-30 into March 2nd instead of refusing it, so a shape check alone would
let a caller read a different day than it asked for.

germanDate moves here from mcp/prompts.ts, its only previous home, so the
timezone is stated in one place.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 20:43:24 +02:00
MechaCat02
ab581aa5ca Give claude.ai a token of its own, sent as a request header
claude.ai's connector dialog does offer request headers, on its second step,
after the URL has been probed, so the connector no longer needs the secret
path. MCP_AUTH_TOKEN already worked there as a bearer or X-Api-Key, but it
also opens /api, which can replace the Schulcloud token and stream the file
mirror, and claude.ai stores the header's value.

MCP_CONNECTOR_TOKEN is a second token, accepted on /mcp only and refused on
/api, and rotated without touching Claude Code or the CLI. The config refuses
one shorter than 32 characters, equal to MCP_AUTH_TOKEN, or set without it,
and never echoes a value. Every accepted token is compared in full, so the
timing does not tell which one matched.

The gate also takes a bare Authorization value, because claude.ai sends a
header exactly as typed and its docs warn that most servers reject a token
entered without "Bearer ". It takes X-Auth-Token too, the other name its
dialog offers.

The docs now set up the header; the secret path stays as a fallback for
clients that cannot send one. 184 tests. Smoke 79/79 and 77/77 on the local
instance.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 22:03:56 +02:00
MechaCat02
bfccb3f343 Pull the server image on the Pi instead of building it
The Pi now runs registry.mc02.dev/schulcloud-mcp. Its compose file `!reset`s
the build section docker-compose.yml declares, so no `docker compose` command
there can fall back to building (Compose 2.24 or later). SCHULCLOUD_MCP_TAG
pins a commit's image; unset, the Pi follows `latest`.

`npm run publish-image` publishes from a development machine. It builds only a
clean working tree, so a commit tag names exactly that commit's code, for amd64
and arm64, tagged `latest` and with the short commit id. On an x86 machine
arm64 builds under QEMU, which the script checks for and explains rather than
registering unasked.

PI.md logs in to the registry, pulls and starts instead of building, and
updates by publishing and pulling. Rehearsed in a scratch compose project: the
image came from the registry, `up -d --build` built nothing, and the server
came up healthy with its database and no published ports.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 21:26:44 +02:00
MechaCat02
bac91303bf Add a setup guide for the Pi, and a compose file its .env selects
docs/PI.md goes from a Pi with Docker to a working claude.ai connector:
configuration, the first token, Caddy, DNS, the VPS forwarding raw TCP,
checks from outside, connecting the clients, the monthly token, updates,
backups and troubleshooting.

docker-compose.override.yml is tracked, so Compose would have merged it on
the Pi as well — publishing ports and switching the crawl timer off. The
Pi's .env sets COMPOSE_FILE to add deploy/docker-compose.pi.yml instead,
which joins the existing Caddy network by name and builds DATABASE_URL, so
nothing tracked needs editing there. Postgres moves to a private network
shared only with the server.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 20:19:17 +02:00
MechaCat02
ab265b5b0c Serve MCP at a secret path, so claude.ai can connect
claude.ai's connector dialog takes a name and a URL. Sending a bearer token
needs a "Request headers" beta most accounts lack, and OAuth is not built
yet, so with MCP_PATH_SECRET set the endpoint is also served at
/<secret>/mcp without the bearer token — a trial until OAuth replaces it.

The path is the credential there. It is compared in constant time, and a
wrong one answers 404 like any unknown path. The config refuses fewer than 32
URL-safe characters and never echoes the value, nothing in the server logs
request paths, and the Caddy snippet rewrites the segment before an access
log entry is written (verified against Caddy 2.11). Claude Code and the CLI
keep the bearer token; DEPLOYMENT.md says what the path trades away.

178 tests. Smoke 76/76 and 74/74 on the local instance.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 20:19:17 +02:00
MechaCat02
973b82ebf5 Replace the Schulcloud token without a restart
A token lasts 30 days and only a browser login yields one — the account is
federated, so the server cannot mint it. Replacing it meant editing .env and
recreating the container, every month.

`schulcloud token set` (a hidden prompt, or piped input) and a /token page
both send it to PUT /api/token. The server checks it with Schulcloud first —
well-formed, unexpired, still logged in, the same account — then swaps it
into the config every request reads, restarts the keepalive and saves it in
STATE_DIR, a new volume, with mode 0600. At startup the newer of the saved
token and TSC_JWT_COOKIE wins, unless they belong to different accounts. A
refused paste changes nothing, and the token is never logged.

The keepalive's pings carry a generation, so a 401 for the old token that
arrives after a swap cannot stop the new cycle. `schulcloud token`, whoami
and the log report the expiry and warn a week ahead.

Found on the way: a host that is off for more than two hours loses the
session however long the token has left — this machine lost it overnight —
which is what the always-on Pi is for.

174 tests. Smoke 72/72 on the local instance, and a real swap verified end to
end there.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 20:19:16 +02:00
MechaCat02
9d0272c622 Offer courses and rooms as MCP resources, with two German prompts
A course or room can now be attached to a message rather than fetched:
schulcloud://courses/<id> and schulcloud://rooms/<id> carry exactly what
get_course and get_room return. Deliberately coarse — a picker lists every
resource at once, which suits some twenty courses and not a thousand files.

Two prompts, in German because the school is: zusammenfassung summarises a
course or room, and pruefungsvorbereitung prepares for an exam with practice
questions and a study plan. Each embeds the overview and says where material
hides and what cannot be read.

Claude Code shaped the details, read from its bundle rather than its docs.
It splits prompt arguments on whitespace and drops extra words, so words
arrive joined with "_", and courses match by fragments, whole words first,
so LF1 is not ambiguous with LF10. Its @ autocomplete shows a resource's
description, so the description carries the name. Errors are ProtocolError,
because McpError's message prefix is doubled by the client.

Verified in interactive Claude Code: @-mention, autocomplete and the prompt
commands. 157 tests. Smoke 67/67 live; 69/69 and 67/67 on the local instance.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 20:19:16 +02:00
MechaCat02
3e44e66dde Survive a first full crawl: poll, time out on silence, retry downloads
The first full crawl with the file manager ran 14 minutes, downloading every
file once, and broke in three ways:

- `schulcloud refresh` reported "fetch failed" for a crawl that was
  succeeding: Node's fetch abandons a response without headers after five
  minutes. POST /api/refresh takes wait:false and the CLI polls /api/status;
  refresh_index answers after 50 s and leaves the crawl running, and
  index_status says when a first crawl is under way.
- Downloads were bounded by the 30 s request timeout, which cut 11 MB scans
  off mid-transfer. They now time out on 30 s of silence instead.
- Failures were recorded once and never retried. A download failure is now
  retried on the next crawl while an extraction failure stays final, and PDF
  text containing NUL, which Postgres refuses, is stripped.

On the re-crawl all six failed files succeeded; only two videos above the
mirror cap stay metadata-only, by design. 137 tests.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 20:19:16 +02:00
MechaCat02
bed3923902 Browse the file manager ("Dateien") as a filesystem
Many teachers never use topics or boards; their material sits in the
course's file area, and the tools answered "0 files" for courses holding
dozens of worksheets — 21 of 26 courses on the live account. Persönliche,
Kurs-, Team- and Geteilte Dateien live in the legacy file store, not in
files-storage, and its service is not in the public ingress. The only way in
is the legacy client: HTML listings, and GET /files/signedurl for a
pre-signed download.

core/legacy-files.ts turns that into one path tree — /my, /courses/<course>,
/teams/<team>, /shared — resolving names that contain "/", ids anywhere in a
path, and wrong or ambiguous names with a message saying what is there. A
listing that does not parse throws; it never reads as an empty folder.

Some of the legacy client's GET routes write (GET /files/share/ mints a
share token), so getFileManagerPage allows only the listing routes, by
pattern. Signed URLs are fetched with no credentials and must be https.

- MCP: fs_list, fs_tree, fs_find and fs_read; get_course lists course files.
- CLI: schulcloud fs ls, tree, find and get, recursive and resumable.
- API: /api/fs/list, tree, find and file.
- Index: the crawl walks the file manager (INDEX_FILE_MANAGER, on by
  default), so search covers the text inside those files and sync mirrors
  them under <course>/Kurs-Dateien.

The local instance gains a fixture for all four areas. It needed a loopback,
so signed URLs open from the host, and a pre-created bucket, since MinIO
does not implement PutBucketCors.

135 tests. Smoke 55/55 live; 57/57 and 55/55 on the local instance.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 20:19:16 +02:00
MechaCat02
10c6544579 Prepare for a live account: separate the indexes, and probe new routes live
With .env pointing at the real account, testing against the local instance
became dangerous: process env beats --env-file, so a local smoke run would
crawl fixtures straight into the live index, where a per-course refresh
carries them forward indefinitely. mcp-env.sh now pins its own database
(schulcloud_local) and mirror as well as the instance.

The override publishes the server on MCP_HOST_PORT and takes
CRAWL_INTERVAL_MS from .env, since what_changed can only report what
happened between crawls. The build context leaves out tmp/, which holds a
mirror of the account's files, and local-instance/.

probe checks live what the gap fixes established locally: the /api/v1 course
and user routes, classes, room allowedOperations as an object, and the
preview enums on a real file. A failure there means a feature degrades
rather than breaks.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 20:19:16 +02:00
MechaCat02
5ae2210459 Close the gaps an audit of courses, tasks, files and grades turned up
Every area — courses, rooms, boards, topics, tasks, files, quizzes, teams,
groups, submissions, grades — was checked for data the instance has and the
tools did not show.

Grades and feedback. A teacher's /homework page is a different page from a
student's: grade and comment live in the grading form, one block per
submission, so a teacher account reported every graded submission as having
neither. parseTeacherGrading reads the form, and list_submissions can now
include the written feedback and who handed the work in.

Names. /api/v1 is partly served: courses, users and classes survive in the
deployment's ingress table, and users/{id} is the only route from an id to a
name. Submitters, file creators and course teachers resolve through it, and
degrade to "not visible to this account" where a student may not read them.

Courses, rooms and classes. get_course adds the description, teachers,
member count and weekly timetable from /api/v1/courses. list_classes is new.
get_room reports what the account may do — allowedOperations is an object of
booleans, not the list it was typed as — and applicants and invitation links
where it may manage them.

Board and topic content. Link descriptions, image alt text, drawing and
video-conference titles, the ids behind external tools and H5P content (the
only thing resembling a quiz), and what a deleted element used to be. Topic
Etherpad pads are read like board pads, and htmlToText keeps table columns
apart and drops template indentation.

Files. A scan with no text layer falls back to the preview endpoint, whose
width and outputFormat are undocumented enums, so Claude gets a picture of
the page; list_files reports counts and sizes. Teams stay documented as
unreadable at any API version; their files come later.

What the crawl missed. Tasks attached to topics (18 of 60 on the live
account), each course's own file area, and — behind INDEX_PERSONAL_FILES —
personal files and submissions with their grade comments, so search and
what_changed cover grading. A submission hit points at get_task.

The local instance's preview profile gets an ImageMagick policy that allows
the coders its 7.1.2 build needs; the image's own denies them all.

110 tests.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 20:19:16 +02:00
MechaCat02
a3b17a680c local-instance: raise the proxy's header buffers, or topic pages 502
A topic page makes the legacy client set several large cookies at once —
jwt plus the Etherpad session it asks for — which overflows nginx's default
4k header buffer. The upstream answered 200 and the proxy still returned 502
with "upstream sent too big header".

It went unnoticed because the MCP server's topic-page scrape degrades to an
empty list by design: the task ids it recovers from that page silently
vanished instead of failing. Changed in the generator and in the generated
config.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 20:19:16 +02:00
MechaCat02
237395e4f4 Run the tests on Node 22.16, and fix what a fresh setup tripped over
Node 22.16 runs TypeScript only behind --experimental-strip-types (on by
default from 22.18), so `npm test` failed on every file with
ERR_UNKNOWN_FILE_EXTENSION. The flag is harmless on newer versions.

probe and session-diagnose still imported dist/schulcloud/client.js, which
the move to core/ renamed, so both scripts died at import.

A relative MIRROR_DIR broke file serving: res.sendFile refuses a relative
path, and resolveWithin returns an absolute path only when its root is one.
The config resolves it once, at load.

package-lock.json is what `npm install` records today: the `schulcloud`
bin, and no stale peer flags. 101 tests.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 20:19:15 +02:00
MechaCat02
1de026ca43 Serve rooms ("Räume"), which are not courses however the urls read
The account this was built against is in no rooms, so the whole space was
invisible and easy to dismiss as an empty endpoint. It is not empty in
general — the user had rooms until a teacher removed access — and the
UI's naming actively hides the distinction: the sidebar's *Kurse* entry
links to `/rooms/courses-overview` and lists courses, while *Räume* links
to `/rooms` and lists rooms. A url containing `/rooms` identifies neither.

list_rooms and get_room cover the latter. A room holds boards and nothing
else, so get_room lists boards for get_board (which already reports "in
room" from the board context) plus who else is in it. Room boards report
`isVisible`, which the course-page projection does not, so a draft is
named as a draft instead of being offered and then answering 403.

Rooms also go through the crawl, or they would have become the next
blind spot: their boards are indexed, searchable by both the index and
the live-crawl path, diffed by what_changed, and mirrored by the CLI
under the room's name. The board traversal and the snapshot matcher are
now shared between courses and rooms rather than duplicated, which also
fixed the live-crawl path silently not searching pad contents.

The CLI needed no new command — it is file-centric and inherits rooms
through the manifest — but `--course` now accepts a room id, and says so.

`kind` gains 'room'; the column is plain TEXT, so no migration. 112 tests.
Smoke: 42/42 and 44/44 local, 41/41 and 43/43 live.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-13 19:24:53 +02:00
MechaCat02
3a168e37e5 local-instance: simulate a teacher, and run the files-storage consumer
`scripts/simulate-teacher.mjs` creates, edits and deletes what teachers
create — course, room, topic, task, board, columns, cards, rich text,
link, Etherpad pad, folder, files — so the MCP server can be exercised
against content the real account has never held. It is the only thing
here that writes to a Schulcloud and refuses any non-localhost address.
`scripts/mcp-env.sh` points the server and CLI at the instance.

Two gaps it exposed in the stack itself:

The files-storage AMQP consumer is a separate entrypoint, and we were
running only the HTTP one. Nothing was bound to the `files-storage`
exchange, so `TaskService.delete` — which awaits deleteFilesOfParent
over AMQP before touching the task — hung until the request timeout.
Deleting any task or topic answered 408 with the entity still there.

The demo data is dated 2017-2018 and the v3 endpoints filter on those
dates, so a student saw no tasks at all. seed.sh now brings courses and
homework into the present, which is the difference between a fixture
that exercises the student-facing surface and one that looks empty.

Teams turn out to be uncreatable through the API (the legacy service
registers no `create`, v3 has no route), and a new board is unpublished
and 403s for students, so the simulation adopts a seeded team and leaves
one board a draft on purpose.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-13 15:40:46 +02:00
MechaCat02
521c21f7ae Reach tasks attached to topics, and read Etherpad pads
Testing against a local instance turned up four things the server was
getting wrong, all of them invisible against the live account because the
data that exposes them had never been produced there.

`GET /lessons/{id}/tasks` returns a bare array, not the `{data,total}`
envelope every sibling endpoint uses, so `.data` was undefined and a
topic's tasks silently vanished. Its items also carry no id at all —
`LessonLinkedTaskResponse` has no id property — which leaves a
topic-attached task unidentifiable: it is not a task element on the
course page, and once past due it is in neither task list. So its
submission, and its grade, could not be reached by any route. That is 18
of 60 tasks on the real account, now reachable: the ids come off the
legacy topic page, where each task is linked as `/homework/{id}`.

The types said `id: string` and `status: TaskStatus` on something that
has neither, which is what let this stay quiet; `LessonLinkedTask` and
`ResolvedTask` now say what is actually there.

Collaborative text editor elements come back with `content: {}`, and the
tool said their contents were unavailable. They are available: the
content-element endpoint returns the pad url *and* an Etherpad session
cookie, and the pad exports itself as text to whoever holds it. No API
key needed. Pads are now shown by get_board and indexed for search.

The store's file digest covered id and size on the grounds that file
records are immutable. `PATCH /file/rename/{id}` renames one in place,
so a rename was reported as nothing at all.

Finally, get_board reported an unpublished board as "no permission",
which sends the reader hunting for an access problem that is not there.

smoke gains checks for topic tasks and for pads, and no longer assumes a
populated index or a search term that happens to match. 39/39 live-only
and 41/41 index-backed, against both the live instance and a local one.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-13 15:39:51 +02:00
MechaCat02
0ae9198428 Run Etherpad by default: without it every topic page 500s
The legacy client requests an Etherpad session on every topic page whose
lesson has contents, without ever checking whether the lesson contains a
pad — controllers/topics.js collects `etherpadPads` and then ignores it.
With Etherpad unreachable the request fails, `validUntil` comes back
undefined, and `new Date(undefined * 1000)` makes Express reject the
session cookie: "option expires is invalid", rendered as a 500.

So Etherpad was only nominally optional. Moving it out of the `tools`
profile also matches the live deployment, which always runs it.

The topic pages that still 500 are courses the signed-in user is not a
member of; the same page returns 200 for its own teacher. That is the
legacy client rendering a 403 as a 500, upstream behaviour we don't own.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-13 14:34:12 +02:00
MechaCat02
a3aded110c Add a local Schulcloud instance modelled on the live target
A Docker Compose stack that runs the deployed images
(quay.io/schulcloudverbund/*, thr theme, tag 33.40 — the versions
schulcloud-thueringen.de reports) rather than a rebuild of main, so what we
develop against is the deployed artefact. It exists to produce the states we
can otherwise only observe read-only: log in as the teacher, grade, then read
it back the way the MCP server does.

Faithful where it matters and honest where it isn't:

- Feature flags in env/api.env are a replay of GET /api/v3/config/public from
  the live instance, not a hand-picked set; instance identity mirrors the thr
  group_vars from dof_app_deploy.
- The proxy is generated from the deployment's own ingress table
  (scripts/gen-proxy-conf.py) so the legacy-client / SPA / API path split
  matches production; getting it wrong tests a different application.
- Valkey runs in `single` mode so the JWT whitelist expires sessions the way
  production does, rather than the in-memory shortcut that hides it.
- No external OAuth / Schulportal login (excluded by request and not
  reproducible locally), no BigBlueButton; each divergence is marked at the
  line it affects. Everything binds to 127.0.0.1 and uses the upstream dev
  credentials, which are public.

Profiles keep the heavy pieces opt-in: `tools` adds Etherpad/H5P/tldraw/
Collabora, `av` adds ClamAV, `preview` adds thumbnailing.

seed.sh loads the upstream demo school (the same call the deployment's init job
makes) and registers MinIO as the legacy storage provider, which has no seed
data on purpose. The demo data already contains the grading states that are
hard to obtain from the real account — a feedback-only grade and a 100% one —
which is what surfaced the past-due submitted-text scrape gap.

One config finding baked in: file-storage and h5p validate a token's
issuer/audience against JWT_DOMAIN (default "localhost"), while the API stamps
SC_DOMAIN; without keeping them equal, the homework page's file lookups 401.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-13 14:17:30 +02:00
MechaCat02
290b352b07 Scrape submitted text for past-due submissions, not just editable ones
Once a submission's due date passes, the legacy homework page renders the
student's text read-only in a `<div class="comment">` that is a sibling
*after* `</section id="submission">`, which then holds only the file list.
`parseHomeworkPage` searched for that div *inside* the submission section, so
for every past-due submission it silently returned no submitted text while
still reporting the grade and feedback — the reader would conclude the student
handed in nothing.

Search the whole page instead. `class="comment"` (quote right after the word)
stays specific: the teacher's feedback is `class="comment ckcontent"` and does
not match, and the editable-textarea branch is still tried first.

Found by standing up a local instance from the deployed images and reading a
real 33.40 page for a seeded past-due submission; the earlier test fixture had
nested the div inside the section, which is why the gap was invisible. The
fixtures now match the real DOM.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-13 14:17:14 +02:00
ab10bbcd14 Stop calling a feedback-only grade "no numeric grade recorded"
You were right that the output was wrong, though not quite for the
reason given: the schema has no textual grade. It is
grade: { type: Number, min: 0, max: 100 } with gradeComment: String, so
"OK" is the comment, not the grade.

What the model does allow is exactly your case — teachers grade with the
comment alone and leave grade unset. Rendering that as "graded (no
numeric grade recorded)" reads as missing or broken data when in fact
the written verdict is the whole grade. It now says "graded by feedback,
with no percentage given", and reserves the it-is-absent wording for
when there is genuinely neither a percentage nor a comment.

Also fixes a real misrepresentation next to it: grade is a percentage,
and both the detail and list views printed it bare, so an 85 could be
read as a mark out of 100, 15 or 6. Now rendered as 85%.

formatGradeState is pure and covered by seven cases, including 0% staying
distinct from "no grade" — the bug that an `if (grade)` test would have
introduced.

85 tests, 38/38 smoke.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 13:32:08 +02:00
a8badc2fd1 Read submitted text and teacher feedback from the homework page
You were right that this data never reaches the browser as an API call.
The legacy front end calls the Feathers API server-side for
submission.comment, submission.grade and submission.gradeComment and
renders them into GET /homework/{taskId}. That Feathers API is not
exposed publicly — /api/v1/* 404s — so the rendered page is the only way
to reach these fields from outside.

core/homework-page.ts parses it, hooked on the data-testid attributes
the project's own e2e tests use rather than incidental markup. The page
authenticates by jwt *cookie*; an Authorization header is ignored and
redirects to the identity provider. Every field is optional and parse
failures return undefined, so a markup change degrades to "not found"
and cannot break get_task. The wording distinguishes the two: absent
feedback is reported as not found, never as none given.

Measured on one course: 4 of 7 graded submissions carry feedback no API
call can return — "vollständig und nachvollziehbar", "Feedback siehe
Zettel", and so on.

This exposed a bug in a shared utility: htmlToText decoded only six
entities, so any named entity passed through raw. German content makes
that routine — "vollst&auml;ndig" would have reached the model verbatim
from boards and task descriptions too, not just here. It now decodes
named, decimal and hex references in one pass, so &amp;auml; stays
literal instead of decoding twice, and leaves unknown names alone rather
than mangling them.

Also fixes a documented-recovery bug found while restoring the session:
`docker compose restart` does not re-read env_file, so it silently kept
serving the dead token. `up -d` is correct and the docs said the wrong
thing.

78 tests, 38/38 smoke; verified end to end through Claude Code.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 13:25:35 +02:00
ac08e17b48 Expose submissions: list_submissions, and get_task shows your own
Adds what the API actually permits, which is less than the request asked
for and worth being precise about.

GET /api/v3/submissions/status/task/{taskId} is the only submission
route — no list, no fetch-by-id — so a task id is the only way in. The
probe in the report missed it by trying /api/v3/submissions (404). Its
payload is {id, submitters, isSubmitted, isGraded, grade,
submittingCourseGroupName} and nothing more: no submitted text, no grade
comment, no graded-at. Those lived on /api/v1, which this instance does
not serve at all (404 across the board, confirmed — not the proxy). So
"what feedback did I get" is answerable only when the feedback is a file.

Submitted files are reachable, which covers the main workflow:
get_task now shows the submission id, graded state, grade, group, and
the handed-in files with ids ready for download_file. list_submissions
surveys tasks for "what have I handed in" and "what is still ungraded".
Both state the text/feedback gap rather than implying none was given.

Two things found while building it:

files-storage ignores the parentType path segment when listing —
.../gradings/{id} returns the same records, saying parentType
"submissions". Filtering on each record's own parentType, or a student's
own upload gets reported back as teacher feedback.

get_task could not find this task at all: the task lists only cover the
dashboard, and group-project tasks are absent from both, so it claimed
the id was wrong for a task the account can plainly see. It now falls
back to scanning course pages.

Also bounds live search by measured cost: resolving attachments needs a
request per board element, which is 2s for one course but 325s for all
of them — beyond any client timeout. An unscoped fresh search now reads
text only and says so.

38/38 smoke checks; verified end to end through Claude Code against a
real graded group submission.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-12 23:19:11 +02:00
634b004985 Fix get_board on boards with more than 20 cards
GET /api/v3/cards?ids= accepts at most 20 ids. Above that the request
fails with 400 "each value in ids must be a mongodb id" — which blames
the ids when the real problem is how many there are. Express/NestJS
parse the query string with qs, whose default arrayLimit is 20; past it
the repeated params stop being an array and become an object keyed "0",
"1", …, and @IsMongoId({ each: true }) then rejects every value.

I had chunked at 40, having read the controller and its DTO and found no
documented ceiling. The limit is not there — it is in the query parser
underneath them, which I did not think to check. Verified live: 20 ids
return 200, 21 return 400 with identical ids.

The worse half of this was mine alone. The crawler caught assembleBoard
failures and dropped them, so every board over 20 cards vanished from
the index while the crawl reported "failures: none". Board errors now go
into Snapshot.failures and are surfaced by refresh_index.

Impact of both fixes on a full re-crawl: 205 files -> 255, and the
reported board (27 cards, 18 files) reads fully. The two failures that
remain are genuine 403s — boards this account cannot see — and are now
visible rather than silent.

Thanks to the bug report, which had the root cause exactly right.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-12 22:37:47 +02:00
a62321eb4a docs: register MCP servers at user scope, not the default
Claude Code's default MCP scope is `local`, which is per-project: the
server is stored under that directory in ~/.claude.json and does not
appear in `claude mcp list` from anywhere else. docs/LOCAL.md showed the
default and so produced exactly that confusion.

User scope is right here — the point is to ask about coursework from any
directory. Project scope would write a .mcp.json into the repo, which is
wrong when the HTTP config carries a bearer token.

Also adds the symptom and the fix, since "it is not in the list" has one
overwhelmingly likely cause.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-12 22:10:41 +02:00
e4d087682d Retry transient upstream failures; fix schulcloud --help
A full crawl made the instance answer 4 of 26 course pages with an nginx
503 "temporarily unavailable" front-page — it pushes back when several
hundred requests arrive quickly. Nothing retried, so the index was
quietly incomplete: 22 courses and 153 files rather than 26 and 205,
with the failures recorded per course rather than surfaced as a problem.

The client now retries 429/500/502/503/504 and transient network errors
with exponential backoff plus jitter (so parallel crawl workers do not
retry in lockstep), honouring Retry-After when sent. Every call here is
an idempotent GET, so retrying is safe. 401 and 404 are deliberately not
retried: an expired token will not recover, and neither will a bad id.

Re-crawled after the fix: 26 courses, 205 files, zero failures.

Also: `schulcloud --help` printed 'Unknown command "--help"' because a
leading flag was parsed as the command name.

63 tests.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-12 22:07:02 +02:00
e6f2258df9 Local dev setup; fix mirror volume ownership and a test footgun
Adds docker-compose.override.yml (local-only: publishes the server on
127.0.0.1:8080 and Postgres on 127.0.0.1:55432, crawls on demand) and
docs/LOCAL.md covering the stack, Claude Code registration over both
transports, the CLI, and the test suites.

Two bugs that only running the real container could find:

The mirror volume was root-owned while the container runs as node, so
every file write failed with EACCES. Docker initialises a named volume
from the image directory including its ownership, so the fix is to
create /data/mirror owned by node in the image. This was easy to miss
because the indexer records a per-file failure rather than crashing —
the crawl "succeeded" with 4 skipped. Earlier direct-node testing missed
it entirely by writing to a scratch dir owned by the developer.

The store tests TRUNCATE, and pointing TEST_DATABASE_URL at the dev
database put their fixtures into real data. They now refuse any database
whose name does not contain "test".

Verified against the rebuilt container: 4 files mirrored, image-only PDF
detection firing in the real pipeline, both transports showing
✔ Connected in `claude mcp list`, and a whoami tool call driven end to
end through `claude -p`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-12 21:57:35 +02:00
359c46afad Add the schulcloud CLI, and document the split
The CLI talks only to the Pi's /api surface and holds no Schulcloud
credential — only the same bearer token the Claude connector uses. That
is not layering for its own sake: a Schulcloud session dies after two
hours idle and a CLI process lives for seconds, so a CLI with its own
token would be dead most times you reached for it. Routing through the
Pi means one session, one keepalive, one monthly cookie paste.

sync is a one-way mirror, which follows from the data rather than from
scope-cutting: file records are immutable upstream, so there is no
versioning, no conflict resolution and no merge. State is keyed by file
record id with the path as derived output, so an upstream rename moves
the local file instead of duplicating it — verified against the live
server. Verification is size-only because the download endpoint exposes
no ETag and Schulcloud publishes no hash; size still catches the failure
that happens, a truncated download. Downloads land on a .part neighbour
and are renamed, so an interrupted run leaves no half-file that a later
run mistakes for complete. Deletions are reported but not propagated —
a teacher removing a worksheet is no reason to destroy the student's
copy — with --prune to opt in.

what_changed now clamps to the oldest stored generation instead of
refusing, and says it did: "what's new this week" is a reasonable
question to ask a two-day-old index.

Two build bugs caught by the checks rather than by luck: the smoke
harness constructed the app without services, so the index-backed tools
were never exercised; and the Docker build could not see
scripts/copy-assets.mjs, so the image would have shipped without
migrations and silently degraded to live-only.

67 unit tests (9 needing Postgres), smoke green both ways — 34 checks
with an index, 32 without, because graceful degradation is a supported
mode and not a fallback nobody runs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-12 21:26:33 +02:00
c79f1b120d Index-backed search, /api surfaces, image-only PDF detection
search now queries the Postgres index and states its freshness in every
result, with fresh=true bypassing it for a live crawl — the agent can
always get current data rather than being quietly misled by a stale
index. Adds refresh_index (per-course by default; a full crawl is ~270
requests), what_changed (generation diff — the API has no changed-since
filter of any kind), and index_status.

/api gives the CLI its backend behind the same bearer token as /mcp:
GET /manifest (cursor + per-file status), GET /files/:id (served from
the mirror with Range support, falling back to a live proxy for files
too large to mirror), GET /status, POST /refresh. Bytes go over plain
HTTP rather than MCP because base64 in JSON-RPC costs a third more and
buffers whole files. An unresolvable manifest cursor returns 409 rather
than silently meaning "everything is new", so a client cannot be tricked
into a full re-download.

Verified end to end against the live instance and a real Postgres:
crawl -> index -> German FTS -> manifest -> ranged download, with 401
on missing token, 400 on a malformed id, and 429 on a too-soon refresh.

Two findings worth recording. The build silently omitted the .sql
migrations from dist, which the store's graceful degradation turned into
"running without the index" rather than a crash — now copied by a build
step. And 3 of 4 sampled course PDFs have no embedded fonts at all: they
are scans, so extraction legitimately yields nothing. That is now
detected and reported as image-only with OCR named as the missing piece,
instead of an indistinguishable "0 characters". It revises the roadmap's
"OCR not needed" note, which held for reading images but not for
indexing them.

49 tests pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-12 21:16:38 +02:00
a18b526267 Add Postgres store, path safety, and the crawl indexer
Store: crawl generations as the sync cursor. Diffs compare generations
on entity identity plus a content digest, never on upstream timestamps —
GET /course-rooms/{id}/board returns request time as updatedAt for most
elements, so a timestamp cursor would report every board as changed on
every crawl. Identity diffing also yields deletions, which no timestamp
scheme can. A per-course crawl carries the other courses' rows forward
so every completed generation is a complete picture and any two diff
directly; without that a partial crawl reads as a mass deletion.

FTS uses the german dictionary with weighted title/body, plus a pg_trgm
arm because stemming will not match "Datenschutz" inside
"Datenschutzgrundverordnung" and German compounds make that the common
case. file_texts is keyed by file record id and deliberately outlives
generations: records are immutable upstream, so text extracted once is
valid forever and a re-crawl of unchanged content costs nothing.

Store.open returns undefined instead of throwing when Postgres is
unreachable — the index is an accelerator, and a Pi that loses its
database should get slower, not broken.

core/paths.ts is the security boundary for the mirror. Course titles,
card titles and filenames are all user-supplied upstream, so this is
where a hostile name stops being text and becomes a path. Two bugs found
by its own tests: "///" produced "---" instead of falling back, and dot
runs survived mid-component. Now no ".." can survive anywhere, which
makes the invariant checkable rather than a claim about ordering.

Indexer coalesces concurrent refreshes onto one run and enforces a
minimum interval, since a full crawl is ~270 requests from an account
that looks like a student.

9 store tests against a real Postgres (mocks would test nothing here)
and 13 path tests; 47 total.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-12 21:10:53 +02:00
81dd633863 Extract core/, lift the crawler out of the search tool
Moves the reusable half into src/core/ (client, types, board, extract,
text, keepalive) and the MCP half into src/mcp/. The layering was
already clean — nothing in core imported app code or read process.env —
so this is a move, not a redesign, and the smoke suite stayed the oracle
throughout.

The substantive part is core/crawl.ts. The course->board->card->element
->file traversal previously existed only inside tools/search.ts, and the
indexer, what's-new diff and file mirror all need it. It now returns a
typed Snapshot with breadcrumbs, sorted so two crawls of unchanged
content compare equal. Metadata only: downloading and extracting bytes
is an order of magnitude more expensive and only the indexer wants it.

core/match.ts holds the keyword matching, which makes it testable
without a network, and core/text.ts gains the fold/tokenize/snippet
helpers (accent folding is not optional for German).

search now finds strictly more than before — 5 hits vs 3 for
"Datenschutz" — because the snapshot surfaces file-name matches the old
streaming walk skipped. 34 unit tests and 30/30 smoke checks pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-12 21:04:52 +02:00
c9bcd3de31 Record extension options and decisions in docs/ROADMAP.md
Nothing implemented. Captures the setup decisions (bearer token,
stateless, inline extraction) and, for any future persistence, that it
goes in the Pi's existing PostgreSQL under its own database and user.

Worth keeping because the build surfaced facts that are expensive to
rediscover: search covers file names but never file contents, so the 93
PDFs here are opaque to it; one search costs ~270 upstream requests;
extracted file text is immutable per fileRecord and so cacheable
forever; German needs the german FTS dictionary plus pg_trgm, since
compounds defeat stemming; and "what's new since X" is impossible today
because the API has no changed-since filter anywhere.

Also records what is ruled out and why — collaborative text editor
contents are not retrievable, OCR is unnecessary since images go to
Claude directly, and write tools would forfeit the read-only property
that makes the public endpoint acceptable.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-12 17:12:52 +02:00
bf2e542182 Add keepalive-status, a log-only session health check
Summarises the running container's keepalive: extension count, latest
budget, transient failures, rejections, restarts.

It deliberately makes no API call. Any authenticated request slides the
session TTL, so a checker that talked to Schulcloud would be sustaining
the session itself and could not report on whether the keepalive is
doing it — the same confound that made the first endurance test
ambiguous. Reading container logs observes without participating.

Warns when the reported budget drops below 7000s, which is the early
signal that extensions have stopped taking effect.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-12 16:39:50 +02:00
9ce869f3fb Root cause: an open Schulportal tab revokes the shared token
Neither of my two hypotheses was right, and the upstream source was
correct all along. The jwt cookie copied from the browser IS the
browser's session token — same jti — so this server and the tab share
one session, and the tab ends it:

  1. nuxt-client sets a purely client-side timer, sessionTimeoutTimestamp
     = now + JWT_TIMEOUT_SECONDS, reset only on route change
     (watch(router.currentRoute, startTimer)) — never by API activity and
     never read back from the server's TTL.
  2. AutoLogoutWarning.vue warns at JWT_SHOW_TIMEOUT_WARNING_SECONDS.
  3. At zero, autoLogout() -> location.replace('/logout?auto-logout=true').
  4. schulcloud-client controllers/login.js:439 -> POST /api/v3/logout
     -> removeJwtFromWhitelist(jwt) -> the shared key is deleted.

That explains the endurance failure exactly: the GET pings at t+0/30/60/90
were sliding the Valkey TTL correctly, and then the tab deleted the key.
It also explains the ~1h warning dialog appearing in a tab the user
considers in use — the timer only resets on navigation.

So the sliding TTL is real and a keepalive does hold a session to the
30-day ceiling. The operational fix is not to ping harder but to close
the Schulportal window after copying the cookie; a private window is the
tidy way. This is now the loudest caveat in the token-copying steps,
because it is the single easiest way to break the setup.

Keeping refresh-session rather than reverting to GET, now for a reason
that stands on its own: it states the intent contractually instead of
relying on extend-on-check as a side effect of an unrelated read (that
whitelist has been refactored twice in 2026, and a GET keepalive would
fail silently if it went away), and its budget readout makes session
health visible in the log.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-12 16:32:42 +02:00
60ca4d3eba Keepalive via refresh-session; GET pings measured insufficient
The endurance test refuted the sliding-window model I committed earlier.
A keepalive doing only GET /api/v3/me succeeded at t+0/30/60/90 and was
still rejected by t+120 — consistent with the session ending ~2h after
LOGIN (t+107), and inconsistent with 2h after the last request, which
would have been t+210.

This is a live-vs-source divergence, not a misreading: both the current
JwtWhitelistAdapter and the legacy Feathers ensureTokenIsWhitelisted
re-set the Valkey TTL on every authenticated request, so the source
reads as a sliding window. The instance does not behave that way.

So the keepalive now calls POST /authentication/refresh-session, the
endpoint behind the UI's "Sitzung verlängern" button, which a separate
100s test showed does hold the reported budget at 7200s. It is the only
non-GET request in the server: no body, touches only our own session,
cannot read or modify user data, and is not exposed as a tool, so no
model-driven call can ever be a POST. It logs the returned budget, which
makes a failing extension visible before the session is lost.

Whether this is sufficient is NOT established. Two mechanisms still fit:
an idle TTL that reads fail to refresh (keepalive works), or an absolute
cap/revocation anchored at login — e.g. the IDP's back-channel logout,
which clears every token for the account rather than one. Added
scripts/session-diagnose.mjs to settle it: it logs the budget every 10
min, so a decaying series indicates the former and an abrupt 401 at
7200s the latter. Docs state the open question rather than asserting a
mechanism.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-12 15:46:54 +02:00
d657ece436 Fix session lifetime: 2h sliding idle timeout, not 30 days
The JWT's exp claim says 30 days, and I took that as the session
lifetime. It is only an outer ceiling. The server also keeps a per-token
whitelist entry in Valkey (jwt:{accountId}:{jti}) whose TTL is
JWT_TIMEOUT_SECONDS — 7200s on this instance — and JwtStrategy.validate
re-sets it on every authenticated request. Two hours idle and the token
is rejected with 29 days still on exp.

Proven, not inferred: the token from yesterday returned 401 at 13.8h old.
The live instance publishes the values unauthenticated at
GET /api/v3/config/public — JWT_TIMEOUT_SECONDS 7200,
JWT_SHOW_TIMEOUT_WARNING_SECONDS 3600, the latter being exactly the
one-hour UI prompt that prompted this investigation.

refresh-session turns out not to be special: it extends through the same
guard as any other route, and uniquely only in returning the remaining
TTL. So the keepalive uses GET /api/v3/me instead, and the server stays
GET-only; the one POST in the repo is in scripts/probe.mjs, where it
reports the idle budget.

JWT_EXTENDED_TIMEOUT_SECONDS (~1 month) exists in the config schema but
is vestigial: privateDevice has no references in the current NestJS
source, and generateJwtAndAddToWhitelist never overrides the TTL.

Also fixes a real breakage this surfaced: TypeScript parameter
properties are rejected by Node's type stripping, so `npm run dev` and
`npm test` both failed on any file reaching them. Rewritten as explicit
fields, and noted in CLAUDE.md.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-12 13:07:22 +02:00
35125b7683 Initial schulcloud-mcp server
Read-only MCP server exposing a Schulcloud account to Claude: courses,
column boards, lessons, tasks, and file downloads with text extraction.

The API surface was verified against the live instance rather than
inferred from upstream source, which changed several design decisions:

- The `jwt` cookie works verbatim as `Authorization: Bearer` and lasts 30
  days, so there is no cookie jar and no refresh-session timer.
- Course contents live at /api/v3/course-rooms/{courseId}/board; there is
  no GET /api/v3/courses/{id}.
- Files are a separate service (/api/v3/file/*) with its own OpenAPI doc.
- Board file elements carry no file id; attachments are resolved by
  listing files-storage with parentType=boardnodes and the element id.

Read-only by construction: every client method is a GET, including the
api_get escape hatch. The endpoint is internet-facing by necessity, so a
leaked token being unable to act as the user is the key safety property.

Deploys as a container behind the Pi's existing Caddy, guarded by a
constant-time bearer check. Stateless — no database.

Verified: 28 unit tests, plus a 30-check end-to-end run driving a real
MCP client over Streamable HTTP against the live account.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 23:52:12 +02:00