Read the user's own lesson notes, and the class register behind them
Schulcloud says what was uploaded and WebUntis says what was scheduled. Neither says what was *taught* — which point the teacher laboured, which example landed, what "will definitely come up". That lives in two places this server could not reach: the notes the user takes in the lesson, and WebUntis' class register. Notes are a directory of Markdown files (NOTES_DIR), not a table. They have to be writable from a phone in a classroom, readable when Postgres is down, and outlive this project, and files are the only shape that is all three — so the files are the truth and the index is a view of them, the same split as file_texts and the mirror. list_notes and get_note read disk, so they answer before the first crawl; search, what_changed and all three German prompts read them alongside the Schulcloud material. add_note writes one, and is the only thing in this server that writes anything. That is not a hole in the read-only invariant but a different store: it is bounded to NOTES_DIR by the same safeComponent/resolveWithin pair that stops a hostile Schulcloud filename escaping the mirror, so a note titled ../../.ssh/authorized_keys becomes a filename. Schulcloud and WebUntis stay GET-only and allowlisted respectively. NOTES_READONLY refuses writes outright. Appending targets the *lesson*, not the title: "halt das auch noch fest" mid-lesson carries a new title, and deriving the path from it would start a second note every time, which is the one thing append exists to prevent. Notes.app has no export — its bodies are compressed protobuf and the iCloud copy is encrypted — so scripting the app is not the clumsy route to the notes but the only one. scripts/export-apple-notes.js reads them through AppleScript into one JSON object per line, and `schulcloud note import` converts the HTML to Markdown, takes the Notes folder as the subject and the *creation* date as the lesson's date. Attachments cannot come across; a note that was a photo of the board imports as a line saying so, because importing it empty would hide the loss. The class register needed one API property to become cheap: getLessonTopic2017 answers per *series*, not per period, so a term is reconstructed by asking about the latest period of each lesson series and merging back by id — a few dozen calls for a school year rather than one per lesson. untis_lesson_topics now takes a subject as well as a period id, and UNTIS_HISTORY_DAYS of register goes into the index under a kind of its own, so "what did we actually do before the test" is searchable. Sharing the snapshot rather than duplicating it caught one thing on the way: the search tool's live path had to learn notes too, or fresh=true would have quietly disagreed with the index. 305 tests; 88/89 smoke against the local instance, the one failure being the H5P service that instance does not run. The live smoke could not be retaken: that session has lapsed and needs a fresh jwt cookie. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
MechaCat02
140
scripts/export-apple-notes.js
Executable file
140
scripts/export-apple-notes.js
Executable file
@@ -0,0 +1,140 @@
|
||||
#!/usr/bin/env osascript -l JavaScript
|
||||
/**
|
||||
* Exports Apple Notes to one JSON object per line, on stdout.
|
||||
*
|
||||
* Run this **on the Mac that has the notes**:
|
||||
*
|
||||
* osascript -l JavaScript scripts/export-apple-notes.js > notes.ndjson
|
||||
* osascript -l JavaScript scripts/export-apple-notes.js --folder Deutsch > deutsch.ndjson
|
||||
*
|
||||
* then hand the file to `schulcloud note import notes.ndjson`.
|
||||
*
|
||||
* Why this exists at all: Notes.app has no export. Its database is a Core Data
|
||||
* store whose bodies are compressed protobuf, and the iCloud copy is encrypted,
|
||||
* so scripting the app is not the clumsy route to the notes — it is the only
|
||||
* one. The first run raises a macOS permission dialog ("Terminal wants access
|
||||
* to Notes"); without it every note comes back empty.
|
||||
*
|
||||
* JXA and not AppleScript because it can serialise JSON, and because reading
|
||||
* properties one note at a time is what keeps a locked note from aborting the
|
||||
* run rather than a language preference.
|
||||
*/
|
||||
|
||||
ObjC.import('stdlib');
|
||||
|
||||
function run(argv) {
|
||||
const options = parseArguments(argv);
|
||||
const notes = Application('Notes');
|
||||
notes.includeStandardAdditions = true;
|
||||
|
||||
let items;
|
||||
try {
|
||||
items = notes.notes();
|
||||
} catch (error) {
|
||||
return fail(
|
||||
'Could not read Notes. Grant the terminal access under System Settings → Privacy & Security → ' +
|
||||
'Automation, then run this again.\n' + error,
|
||||
);
|
||||
}
|
||||
|
||||
let written = 0;
|
||||
let skipped = 0;
|
||||
for (let i = 0; i < items.length; i++) {
|
||||
const note = items[i];
|
||||
const record = readNote(note);
|
||||
if (!record) {
|
||||
skipped++;
|
||||
continue;
|
||||
}
|
||||
if (options.folder && (record.folder || '').toLowerCase().indexOf(options.folder.toLowerCase()) === -1) continue;
|
||||
// One object per line, so a huge export streams and a bad note costs one line.
|
||||
console.log(JSON.stringify(record));
|
||||
written++;
|
||||
}
|
||||
|
||||
// stderr, so it never lands in the file being redirected.
|
||||
log('Exported ' + written + ' note(s)' + (skipped > 0 ? ', skipped ' + skipped + ' unreadable' : '') + '.');
|
||||
return '';
|
||||
}
|
||||
|
||||
function readNote(note) {
|
||||
try {
|
||||
// Read the body first: it is the property a locked note refuses, and
|
||||
// there is no point building a record we cannot fill.
|
||||
const body = note.body();
|
||||
return {
|
||||
id: safe(function () { return note.id(); }, ''),
|
||||
name: safe(function () { return note.name(); }, ''),
|
||||
body: body || '',
|
||||
folder: safe(function () { return folderPath(note.container()); }, ''),
|
||||
created: safe(function () { return iso(note.creationDate()); }, ''),
|
||||
modified: safe(function () { return iso(note.modificationDate()); }, ''),
|
||||
};
|
||||
} catch (error) {
|
||||
// A locked note, or one iCloud has not downloaded. Reported, not dropped
|
||||
// silently: a missing lesson is worse than a line in the file.
|
||||
return {
|
||||
id: safe(function () { return note.id(); }, ''),
|
||||
name: safe(function () { return note.name(); }, '(unreadable)'),
|
||||
body: '',
|
||||
error: String(error),
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/** "Schule/Deutsch" — the import takes the last segment as the subject. */
|
||||
function folderPath(container) {
|
||||
const parts = [];
|
||||
let current = container;
|
||||
for (let depth = 0; current && depth < 8; depth++) {
|
||||
const name = safe(function () { return current.name(); }, '');
|
||||
if (!name) break;
|
||||
parts.unshift(name);
|
||||
current = safe(function () { return current.container(); }, null);
|
||||
}
|
||||
return parts.join('/');
|
||||
}
|
||||
|
||||
/**
|
||||
* A Date as local ISO, not UTC.
|
||||
*
|
||||
* `toISOString` would shift a note taken at 08:30 in Erfurt back to the
|
||||
* previous day for anything written before 01:00 or 02:00, and the date is
|
||||
* the whole point of the record.
|
||||
*/
|
||||
function iso(date) {
|
||||
if (!date) return '';
|
||||
const pad = function (value) { return String(value).padStart(2, '0'); };
|
||||
return (
|
||||
date.getFullYear() + '-' + pad(date.getMonth() + 1) + '-' + pad(date.getDate()) +
|
||||
'T' + pad(date.getHours()) + ':' + pad(date.getMinutes()) + ':' + pad(date.getSeconds())
|
||||
);
|
||||
}
|
||||
|
||||
function safe(read, fallback) {
|
||||
try {
|
||||
const value = read();
|
||||
return value === undefined || value === null ? fallback : value;
|
||||
} catch (error) {
|
||||
return fallback;
|
||||
}
|
||||
}
|
||||
|
||||
function parseArguments(argv) {
|
||||
const options = { folder: '' };
|
||||
for (let i = 0; i < argv.length; i++) {
|
||||
if (argv[i] === '--folder' && argv[i + 1]) options.folder = argv[++i];
|
||||
}
|
||||
return options;
|
||||
}
|
||||
|
||||
function log(message) {
|
||||
$.NSFileHandle.fileHandleWithStandardError.writeData(
|
||||
$.NSString.alloc.initWithUTF8String(message + '\n').dataUsingEncoding($.NSUTF8StringEncoding),
|
||||
);
|
||||
}
|
||||
|
||||
function fail(message) {
|
||||
log(message);
|
||||
$.exit(1);
|
||||
}
|
||||
@@ -25,6 +25,11 @@ process.env.MCP_CONNECTOR_TOKEN = CONNECTOR_TOKEN;
|
||||
// A state directory of its own, so the run can neither read nor leave a saved token.
|
||||
const STATE_DIR = await mkdtemp(join(tmpdir(), 'schulcloud-smoke-state-'));
|
||||
process.env.STATE_DIR = STATE_DIR;
|
||||
// And a notes directory of its own. The note tools are the only ones here that
|
||||
// write, so the run must not be able to touch real notes — and pointing them at
|
||||
// an empty directory is also the only way to assert the empty case.
|
||||
const NOTES_DIR = await mkdtemp(join(tmpdir(), 'schulcloud-smoke-notes-'));
|
||||
process.env.NOTES_DIR = NOTES_DIR;
|
||||
// The app is bound by this script on an ephemeral port, so config.port is unused.
|
||||
|
||||
const config = loadConfig();
|
||||
@@ -480,6 +485,73 @@ if (taskId) {
|
||||
check('list_submissions unscoped', !all.isError, all.text.split('\n')[0]);
|
||||
}
|
||||
|
||||
console.log('\n== own notes ==');
|
||||
// The user's own lesson notes: the one store here that is neither Schulcloud's
|
||||
// nor WebUntis', and the one thing this server can write. Every check runs
|
||||
// against the throwaway NOTES_DIR above.
|
||||
{
|
||||
const noteTools = names.filter((name) => ['list_notes', 'get_note', 'add_note'].includes(name));
|
||||
check('the note tools are offered when NOTES_DIR is set', noteTools.length === 3, noteTools.join(', ') || 'none');
|
||||
|
||||
const empty = await call('list_notes');
|
||||
check(
|
||||
'an empty notes directory is explained, not reported as a failure',
|
||||
!empty.isError && /no notes yet/i.test(empty.text),
|
||||
empty.text.split('\n')[0],
|
||||
);
|
||||
|
||||
const added = await call('add_note', {
|
||||
title: 'Erörterung',
|
||||
text: 'Dreischritt: These, Argument, Fazit. Gegenargument nicht vergessen.',
|
||||
subject: 'Deutsch',
|
||||
date: '2026-09-15',
|
||||
tags: ['klausur'],
|
||||
});
|
||||
check('add_note saves a note', !added.isError && /Deutsch\/2026-09-15 Erörterung\.md/.test(added.text), added.text.split('\n')[0]);
|
||||
|
||||
const listed = await call('list_notes', { subject: 'deut' });
|
||||
check('list_notes finds it by a fragment of the subject', !listed.isError && /Erörterung/.test(listed.text), listed.text.split('\n')[0]);
|
||||
|
||||
const one = await call('get_note', { path: 'Deutsch/2026-09-15 Erörterung.md' });
|
||||
check('get_note returns the note in full', !one.isError && /Gegenargument/.test(one.text), one.text.split('\n')[0]);
|
||||
|
||||
const appended = await call('add_note', {
|
||||
title: 'Nachtrag',
|
||||
text: 'Beispiel: Handyverbot an Schulen.',
|
||||
subject: 'Deutsch',
|
||||
date: '2026-09-15',
|
||||
append: true,
|
||||
});
|
||||
const afterAppend = await call('get_note', { path: 'Deutsch/2026-09-15 Erörterung.md' });
|
||||
check(
|
||||
'append adds to the same note rather than starting a second one',
|
||||
!appended.isError && /Handyverbot/.test(afterAppend.text) && /Gegenargument/.test(afterAppend.text),
|
||||
appended.text.split('\n')[0],
|
||||
);
|
||||
|
||||
const missing = await call('get_note', { path: 'Deutsch/gibt-es-nicht.md' });
|
||||
check('a missing note is a tool error naming the path', missing.isError && /no note at/i.test(missing.text), missing.text.split('\n')[0]);
|
||||
|
||||
// The title reaches the filesystem, so it is untrusted input at exactly the
|
||||
// boundary core/paths.ts exists to guard.
|
||||
const hostile = await call('add_note', { title: '../../../etc/passwd', text: 'x', subject: '../..', date: '2026-09-15' });
|
||||
// The title is echoed back verbatim — it is the user's own — so the check is
|
||||
// on the path the note actually landed at, in backticks.
|
||||
const hostilePath = hostile.text.match(/`([^`]+)`/)?.[1] ?? '';
|
||||
check(
|
||||
'a note cannot be written outside the notes directory',
|
||||
!hostile.isError && hostilePath.length > 0 && !hostilePath.split('/').includes('..'),
|
||||
hostilePath,
|
||||
);
|
||||
|
||||
const fresh = await call('search', { query: 'Gegenargument', fresh: true, courseId: courseIds[0] });
|
||||
check(
|
||||
'a live search reads the notes too, so it agrees with the index',
|
||||
!fresh.isError && /Gegenargument/.test(fresh.text),
|
||||
fresh.text.split('\n')[0],
|
||||
);
|
||||
}
|
||||
|
||||
console.log('\n== index tools ==');
|
||||
// These degrade gracefully without DATABASE_URL, so assert on either outcome
|
||||
// rather than requiring a database for the smoke run to be meaningful.
|
||||
@@ -509,6 +581,13 @@ if (hasIndex) {
|
||||
!indexed.isError && /refreshed/i.test(indexed.text),
|
||||
indexed.text.split('\n')[0],
|
||||
);
|
||||
|
||||
// Notes are only picked up by a *full* crawl, and a full crawl walks every
|
||||
// course and every file-manager folder — minutes, not seconds. Indexing them
|
||||
// is covered by test/store.test.ts against a real Postgres instead; what the
|
||||
// smoke checks here is that the kind filter exists and answers.
|
||||
const byKind = await call('search', { query: 'Gegenargument', kinds: ['note'] });
|
||||
check('search accepts the note kind', !byKind.isError, byKind.text.split('\n')[0]);
|
||||
}
|
||||
|
||||
console.log('\n== WebUntis ==');
|
||||
@@ -580,6 +659,26 @@ if (hasUntis) {
|
||||
check('untis_lesson_topics reads what previous lessons covered', true, 'skipped: no lesson in the window');
|
||||
}
|
||||
|
||||
// The subject form is the one that reconstructs a term without a period id.
|
||||
const subject = month.text.match(/\*\*([A-Za-zÄÖÜäöü0-9]{2,10})\*\*/)?.[1];
|
||||
if (subject) {
|
||||
const bySubject = await call('untis_lesson_topics', { subject, from: '2026-06-01', to: end, limit: 5 });
|
||||
check(
|
||||
`untis_lesson_topics reads a whole term by subject ("${subject}")`,
|
||||
!bySubject.isError && (/Unterricht „/.test(bySubject.text) || /No lessons of|nothing was recorded/.test(bySubject.text)),
|
||||
bySubject.text.split('\n')[0],
|
||||
);
|
||||
} else {
|
||||
check('untis_lesson_topics reads a whole term by subject', true, 'skipped: no subject in the window');
|
||||
}
|
||||
|
||||
const neither = await call('untis_lesson_topics', {});
|
||||
check(
|
||||
'untis_lesson_topics asks for a subject or a period, not neither',
|
||||
neither.isError && /subject/.test(neither.text),
|
||||
neither.text.split('\n')[0],
|
||||
);
|
||||
|
||||
const unreal = await call('untis_timetable', { from: '2026-02-30' });
|
||||
check(
|
||||
'a date that does not exist is refused rather than rolled over',
|
||||
@@ -713,6 +812,7 @@ await client.close();
|
||||
httpServer.close();
|
||||
await closeServices(services);
|
||||
await rm(STATE_DIR, { recursive: true, force: true });
|
||||
await rm(NOTES_DIR, { recursive: true, force: true });
|
||||
|
||||
console.log(`\n${results.length - failures}/${results.length} checks passed`);
|
||||
process.exit(failures === 0 ? 0 : 1);
|
||||
|
||||
Reference in New Issue
Block a user