Read the user's own lesson notes, and the class register behind them

Schulcloud says what was uploaded and WebUntis says what was scheduled.
Neither says what was *taught* — which point the teacher laboured, which
example landed, what "will definitely come up". That lives in two places
this server could not reach: the notes the user takes in the lesson, and
WebUntis' class register.

Notes are a directory of Markdown files (NOTES_DIR), not a table. They
have to be writable from a phone in a classroom, readable when Postgres
is down, and outlive this project, and files are the only shape that is
all three — so the files are the truth and the index is a view of them,
the same split as file_texts and the mirror. list_notes and get_note read
disk, so they answer before the first crawl; search, what_changed and all
three German prompts read them alongside the Schulcloud material.

add_note writes one, and is the only thing in this server that writes
anything. That is not a hole in the read-only invariant but a different
store: it is bounded to NOTES_DIR by the same safeComponent/resolveWithin
pair that stops a hostile Schulcloud filename escaping the mirror, so a
note titled ../../.ssh/authorized_keys becomes a filename. Schulcloud and
WebUntis stay GET-only and allowlisted respectively. NOTES_READONLY
refuses writes outright.

Appending targets the *lesson*, not the title: "halt das auch noch fest"
mid-lesson carries a new title, and deriving the path from it would start
a second note every time, which is the one thing append exists to prevent.

Notes.app has no export — its bodies are compressed protobuf and the
iCloud copy is encrypted — so scripting the app is not the clumsy route
to the notes but the only one. scripts/export-apple-notes.js reads them
through AppleScript into one JSON object per line, and `schulcloud note
import` converts the HTML to Markdown, takes the Notes folder as the
subject and the *creation* date as the lesson's date. Attachments cannot
come across; a note that was a photo of the board imports as a line
saying so, because importing it empty would hide the loss.

The class register needed one API property to become cheap:
getLessonTopic2017 answers per *series*, not per period, so a term is
reconstructed by asking about the latest period of each lesson series and
merging back by id — a few dozen calls for a school year rather than one
per lesson. untis_lesson_topics now takes a subject as well as a period
id, and UNTIS_HISTORY_DAYS of register goes into the index under a kind
of its own, so "what did we actually do before the test" is searchable.

Sharing the snapshot rather than duplicating it caught one thing on the
way: the search tool's live path had to learn notes too, or fresh=true
would have quietly disagreed with the index.

305 tests; 88/89 smoke against the local instance, the one failure being
the H5P service that instance does not run. The live smoke could not be
retaken: that session has lapsed and needs a fresh jwt cookie.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
MechaCat02
2026-09-18 21:45:39 +02:00
parent ad8ba28313
commit af4464decb
34 changed files with 3078 additions and 61 deletions

190
src/cli/apple-notes.ts Normal file
View File

@@ -0,0 +1,190 @@
import { decodeEntities } from '../core/text.ts';
/**
* Turning an Apple Notes export into notes this server can read.
*
* Notes.app stores a note's body as HTML and exposes it through AppleScript,
* which is the only interface it has — there is no file on disk to copy, no
* export format worth the name, and iCloud's copy is encrypted. So
* `scripts/export-apple-notes.js` reads the notes through that interface and
* writes one JSON object per line; this converts them.
*
* Kept out of `core/` because nothing on the server needs it: a migration runs
* once, from the Mac that has the notes, and the server only ever sees the
* Markdown that comes out.
*/
/** One note as `scripts/export-apple-notes.js` writes it. */
export interface AppleNote {
id: string;
name: string;
/** The note's HTML body. */
body: string;
/** The Notes folder it sits in — "Notizen", "Deutsch", "Schule/LF07". */
folder?: string;
/** ISO timestamps from Notes.app. */
created?: string;
modified?: string;
/** Set when Notes refused the body, e.g. a locked note. */
error?: string;
}
export interface ConvertedNote {
title: string;
text: string;
date?: string;
subject?: string;
source: string;
}
/**
* An exported note as a note here.
*
* The creation date becomes the note's date because that is the day of the
* lesson it was taken in — the modification date is whenever it was last
* tidied, which is not a school day at all.
*/
export function convertAppleNote(note: AppleNote, options: { subject?: string } = {}): ConvertedNote {
const body = htmlToMarkdown(note.body);
const title = (note.name || firstLine(body) || 'Notiz').trim();
// Notes repeats the title as the first line of the body; keeping both would
// give every migrated note a duplicated heading.
const text = stripLeadingTitle(body, title);
return {
title,
text,
...(dayOf(note.created) ? { date: dayOf(note.created)! } : {}),
...(subjectFor(note, options.subject) ? { subject: subjectFor(note, options.subject)! } : {}),
source: 'apple-notes',
};
}
/**
* The subject a note belongs to: what the caller said, else its Notes folder.
*
* The folder is the only structure Notes has, and someone keeping lesson notes
* has almost certainly used it for exactly this. A note loose in the default
* folder gets no subject rather than a wrong one.
*/
function subjectFor(note: AppleNote, override: string | undefined): string | undefined {
if (override) return override;
const folder = note.folder?.trim();
if (!folder) return undefined;
// Notes' own default folders say nothing about a subject.
if (/^(notes|notizen|alle .*|all .*|recently deleted|zuletzt gelöscht)$/i.test(folder)) return undefined;
// A nested folder arrives as "Schule/Deutsch"; the leaf is the subject.
return folder.split('/').pop()!.trim() || undefined;
}
/**
* Apple Notes HTML as Markdown.
*
* Deliberately small. Notes emits a narrow set of tags — divs, breaks, lists,
* headings, bold/italic/underline, links and tables — and the goal is readable
* text that keeps its structure, not a faithful rendering. Anything unknown
* loses its tag and keeps its words, which is the right failure for a note.
*/
export function htmlToMarkdown(html: string | undefined | null): string {
if (!html) return '';
let value = html;
// Drop what carries no text at all before anything else looks at it.
value = value.replace(/<(script|style|head)[^>]*>[\s\S]*?<\/\1>/gi, '');
// Attachments (images, scans, drawings) come through as <object>: they have
// no text, and silently dropping them would hide that the note had one.
value = value.replace(/<object\b[^>]*>[\s\S]*?<\/object>/gi, '\n[Anhang aus Apple Notes — nicht übernommen]\n');
value = value.replace(/<img\b[^>]*>/gi, '\n[Bild aus Apple Notes — nicht übernommen]\n');
value = value.replace(/<br\s*\/?>/gi, '\n');
// `li` is deliberately absent: the next `<li>` already opens a line, and
// closing one here too would put a blank line between every bullet, which
// Markdown renders as a loose list.
value = value.replace(/<\/(p|div|tr|h[1-6]|blockquote)>/gi, '\n');
value = value.replace(/<h([1-6])[^>]*>/gi, (_all, level: string) => `\n${'#'.repeat(Number(level))} `);
// A checklist is a list in Notes and a task list in Markdown; the checked
// state lives on the li, so it has to be read before the tag is stripped.
value = value.replace(/<li\b[^>]*\bchecked\b[^>]*>/gi, '\n- [x] ');
value = value.replace(/<li[^>]*>/gi, '\n- ');
// A blank line after a list, or whatever follows is absorbed into the last
// bullet as a lazy continuation.
value = value.replace(/<\/(ul|ol|table)>/gi, '\n\n');
value = value.replace(/<(b|strong)>([\s\S]*?)<\/\1>/gi, (_all, _tag, inner: string) => emphasise(inner, '**'));
value = value.replace(/<(i|em)>([\s\S]*?)<\/\1>/gi, (_all, _tag, inner: string) => emphasise(inner, '_'));
value = value.replace(/<a\b[^>]*href=["']([^"']+)["'][^>]*>([\s\S]*?)<\/a>/gi, (_all, href: string, inner: string) => {
const label = stripTags(inner).trim();
return label ? `[${label}](${href})` : href;
});
// Table cells become separators rather than vanishing, or a row of figures
// runs into one number.
value = value.replace(/<\/(td|th)>/gi, ' | ');
value = stripTags(value);
value = decodeEntities(value);
return value
.split('\n')
.map((line) => line.replace(/[ \t ]+/g, ' ').replace(/ \| $/, '').trimEnd())
.join('\n')
.replace(/\n{3,}/g, '\n\n')
.trim();
}
/** Emphasis only around text that has some: `** **` renders as literal stars. */
function emphasise(inner: string, marker: string): string {
const text = inner.replace(/<br\s*\/?>/gi, '\n');
const body = stripTags(text).trim();
if (!body) return '';
return `${marker}${body}${marker}`;
}
function stripTags(value: string): string {
return value.replace(/<[^>]+>/g, '');
}
function firstLine(body: string): string | undefined {
return body
.split('\n')
.map((line) => line.replace(/^#+\s*/, '').trim())
.find((line) => line.length > 0);
}
/**
* Removes the title if the body repeats it.
*
* Notes shows a note's first line as its name, so `name` and the first line of
* `body` are usually the same string.
*/
function stripLeadingTitle(body: string, title: string): string {
const lines = body.split('\n');
const firstIndex = lines.findIndex((line) => line.trim().length > 0);
if (firstIndex === -1) return '';
const first = lines[firstIndex]!.replace(/^#+\s*/, '').replace(/^\*\*(.*)\*\*$/, '$1').trim();
if (first !== title.trim()) return body.trim();
return lines.slice(firstIndex + 1).join('\n').trim();
}
/** An ISO timestamp as a school day, or nothing when Notes gave none. */
function dayOf(value: string | undefined): string | undefined {
if (!value) return undefined;
const at = new Date(value);
if (Number.isNaN(at.getTime())) return undefined;
// The export writes local time, which is the timezone the note was taken in.
return value.slice(0, 10).match(/^\d{4}-\d{2}-\d{2}$/) ? value.slice(0, 10) : at.toISOString().slice(0, 10);
}
/** Parses the export file: one JSON object per line, blank lines ignored. */
export function parseExport(contents: string): AppleNote[] {
const notes: AppleNote[] = [];
for (const [index, line] of contents.split('\n').entries()) {
const trimmed = line.trim();
if (!trimmed) continue;
try {
notes.push(JSON.parse(trimmed) as AppleNote);
} catch {
// One unparsable line must not cost the export; say which.
throw new Error(`Line ${index + 1} of the export is not JSON. Re-run scripts/export-apple-notes.js.`);
}
}
return notes;
}