Browse the file manager ("Dateien") as a filesystem

Many teachers never use topics or boards; their material sits in the course's file area, and the tools answered "0 files" for courses holding dozens of worksheets — 21 of 26 courses on the live account. Persönliche, Kurs-, Team- and Geteilte Dateien live in the legacy file store, not in files-storage, and its service is not in the public ingress. The only way in is the legacy client: HTML listings, and GET /files/signedurl for a pre-signed download. core/legacy-files.ts turns that into one path tree — /my, /courses/<course>, /teams/<team>, /shared — resolving names that contain "/", ids anywhere in a path, and wrong or ambiguous names with a message saying what is there. A listing that does not parse throws; it never reads as an empty folder. Some of the legacy client's GET routes write (GET /files/share/ mints a share token), so getFileManagerPage allows only the listing routes, by pattern. Signed URLs are fetched with no credentials and must be https. - MCP: fs_list, fs_tree, fs_find and fs_read; get_course lists course files. - CLI: schulcloud fs ls, tree, find and get, recursive and resumable. - API: /api/fs/list, tree, find and file. - Index: the crawl walks the file manager (INDEX_FILE_MANAGER, on by default), so search covers the text inside those files and sync mirrors them under <course>/Kurs-Dateien. The local instance gains a fixture for all four areas. It needed a loopback, so signed URLs open from the host, and a pre-created bucket, since MinIO does not implement PutBucketCors. 135 tests. Smoke 55/55 live; 57/57 and 55/55 on the local instance. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 20:19:16 +02:00
parent 10c6544579
commit bed3923902
32 changed files with 2696 additions and 106 deletions
--- a/local-instance/README.md
+++ b/local-instance/README.md
@@ -89,6 +89,7 @@ Every one of these is marked in the env files at the line it affects.

 ```
 docker-compose.yml    the stack; profiles: (default) | tools | av | preview
+                      (minio-loopback: see "Simulating a teacher")
 env/                  one file per service, all values local-only
 proxy/nginx.conf      GENERATED — the single origin, see scripts/gen-proxy-conf.py
 scripts/seed.sh       loads the demo school via the management app
@@ -183,14 +184,40 @@ node scripts/simulate-teacher.mjs create   # course, room, topic, task, board,
                                           # Etherpad pad, folder, files
 node scripts/simulate-teacher.mjs update   # rename and rewrite all of it
 node scripts/simulate-teacher.mjs delete   # remove it again
+node scripts/simulate-teacher.mjs files    # only the file-manager part, onto an existing fixture
 ```

+It also fills the **file manager** ("Dateien") — the legacy file system behind
+Persönliche, Kurs-, Team- and Geteilte Dateien, a separate store from the board
+files above — with a course folder tree ("Arbeitsblätter/Woche 1"), a team
+folder, a folder in the demo student's own files, and a teacher's file shared
+read-only with the student. The student is added to the adopted team so the
+team files are visible from the account under test. Every file carries a unique
+search term, so search over the index can be checked per area.
+
+Legacy file uploads only work because of two changes to the stack:
+
+- **`minio-loopback`.** The legacy service signs upload and download URLs for
+  its storage provider's single endpoint, and uses that endpoint for its own S3
+  calls. `minio:9000` works inside the compose network and nowhere else, so the
+  browser and the MCP server on the host were handed URLs they could not open.
+  `seed.sh` now registers `http://localhost:9900`, and this socat sidecar,
+  sharing the api container's network namespace, forwards that address to
+  MinIO — the same url then works from the api, the browser and the host.
+  After recreating `api`, recreate `minio-loopback` too: it lives in api's
+  network namespace.
+- **The school bucket is created up front.** The legacy service makes
+  `bucket-<schoolId>` on first upload and then calls `PutBucketCors`, which
+  MinIO does not implement, so the first upload failed with *"A header you
+  provided implies functionality that is not implemented"*. `minio-init.sh`
+  creates the demo school's bucket, and an existing bucket skips both calls.
+
 Ids are kept in `.simulate-teacher.json` between phases, so the MCP server can
 be pointed at the instance in between:

 ```bash
 eval "$(./scripts/mcp-env.sh)"      # as the demo student
-cd .. && npm run smoke              # 39 checks against the local instance
+cd .. && npm run smoke              # 57 checks against the local instance
 ```

 `mcp-env.sh` points the index at its own database, `schulcloud_local`, and the
--- a/local-instance/docker-compose.yml
+++ b/local-instance/docker-compose.yml
@@ -154,6 +154,24 @@ services:
      minio: {condition: service_healthy}
    restart: unless-stopped

+  minio-loopback:
+    # The legacy file service (the file manager: Persönliche/Kurs-/Team-Dateien)
+    # signs upload and download URLs for its storage provider's one endpoint, and
+    # the same endpoint serves its own S3 calls. "minio:9000" works inside the
+    # compose network and nowhere else, so a browser or the MCP server on this
+    # machine was handed URLs it could not open: uploads through the UI failed
+    # and downloads could not be tested. seed.sh registers the provider as
+    # http://localhost:9900 instead, and this forwards that address to MinIO from
+    # inside the api container's own network namespace — so the one URL now
+    # works from the api, from the browser and from the host alike.
+    image: alpine/socat:1.8.0.1
+    network_mode: "service:api"
+    command: ["TCP-LISTEN:9900,fork,reuseaddr,bind=127.0.0.1", "TCP:minio:9000"]
+    depends_on:
+      api: {condition: service_started}
+      minio: {condition: service_healthy}
+    restart: unless-stopped
+
  file-preview:
    # Generates thumbnails via ImageMagick, driven off RabbitMQ. Optional: with
    # it absent, files still upload and download, they just have no preview.
--- a/local-instance/scripts/minio-init.sh
+++ b/local-instance/scripts/minio-init.sh
@@ -11,9 +11,16 @@ for bucket in \
 	h5p-content-bucket `   # h5p-editor content` \
 	h5p-library-bucket `   # h5p content types` \
 	ydocs `                # tldraw whiteboard documents` \
-	fwu-content            # FWU media, unused but cheap to create
+	fwu-content `          # FWU media, unused but cheap to create` \
+	bucket-5f2987e020834114b8efd6f6 # legacy file manager, demo school (see below)
 do
 	mc mb --ignore-existing "local/$bucket"
 done

+# The legacy file manager keeps one bucket per school, "bucket-<schoolId>", and
+# creates it on first upload — then calls PutBucketCors, which MinIO does not
+# implement, so the first upload fails with "A header you provided implies
+# functionality that is not implemented". A bucket that already exists skips
+# both calls. 5f2987e020834114b8efd6f6 is the demo school's fixed seed id.
+
 mc ls local
--- a/local-instance/scripts/seed.sh
+++ b/local-instance/scripts/seed.sh
@@ -39,7 +39,10 @@ SECRET=$(curl -fsS -X POST "$MGMT/encrypt-plain-text" \
 	const id = ObjectId('62949a4003839b6162aa566b');
 	db.storageproviders.replaceOne({ _id: id }, {
 		_id: id, isShared: true, region: 'eu-central-1', type: 'S3',
-		endpointUrl: 'http://minio:9000',
+		// Not minio:9000: this one endpoint also goes into every signed URL, and
+		// those are opened by the browser and the MCP server on the host. The
+		// minio-loopback service makes localhost:9900 reach MinIO from the api too.
+		endpointUrl: 'http://localhost:9900',
 		accessKeyId: 'miniouser',
 		secretAccessKey: '$SECRET',
 		maxBuckets: 150, freeBuckets: 138,
--- a/local-instance/scripts/simulate-teacher.mjs
+++ b/local-instance/scripts/simulate-teacher.mjs
@@ -11,6 +11,7 @@
 *   node scripts/simulate-teacher.mjs create   # build the fixture, print ids
 *   node scripts/simulate-teacher.mjs update   # rename/edit everything it made
 *   node scripts/simulate-teacher.mjs delete   # remove it again
+ *   node scripts/simulate-teacher.mjs files    # (re)build only the file-manager part
 *
 * State lives in .simulate-teacher.json so the phases can be run one at a time
 * with MCP checks in between.
@@ -295,8 +296,9 @@ async function create() {
 	log(`second board left unpublished on purpose: ${draft.id}`);

 	saveState(s);
+	await fileManager();
 	console.log(`\nstate written to ${STATE}`);
-	summary(s);
+	summary(loadState());
 }

 /** files-storage attaches bytes to a board node (element) id, not to the card. */
@@ -307,6 +309,146 @@ async function upload(parentId, name, type, body) {
 	return record.id;
 }

+// ---------------------------------------------------------- file manager ---
+//
+// The file manager ("Dateien": Persönliche, Kurs-, Team- and Geteilte Dateien)
+// is the legacy file system, a different store from files-storage above, with a
+// real folder tree. Many teachers use nothing else, so the MCP server's fs_*
+// tools need content there. Its services are only reachable on the server's own
+// port, like the other /api/v1 writes in this script.
+
+const STUDENT_PASSWORD = process.env.SIM_STUDENT_PASSWORD ?? 'schulcloud';
+const TEAM_MEMBER_ROLE = '5bb5c190fb457b1c3c0c7e0f'; // "teammember" in the seed
+
+/** Runs `fn` signed in as another account, then restores the teacher. */
+async function asUser(email, password, fn) {
+	const saved = { jwt, me };
+	const res = await fetch(`${API}/api/v3/authentication/local`, {
+		method: 'POST',
+		headers: { 'Content-Type': 'application/json' },
+		body: JSON.stringify({ username: email, password }),
+	});
+	if (!res.ok) throw new Error(`login as ${email} failed: ${res.status}`);
+	jwt = (await res.json()).accessToken;
+	me = await v3('GET', '/me');
+	try {
+		return await fn();
+	} finally {
+		({ jwt, me } = saved);
+	}
+}
+
+async function legacyDir(name, owner, parent) {
+	return (await v1('POST', '/fileStorage/directories', { name, owner, parent }))._id;
+}
+
+/**
+ * The browser's own upload sequence: a signed PUT url, the bytes, then the
+ * record. The url comes from the server, so it is held to the same localhost
+ * rule as everything else this script writes to.
+ */
+async function legacyUpload({ owner, parent, name, type, body }) {
+	const bytes = Buffer.from(body);
+	const signed = await v1('POST', '/fileStorage/signedUrl', { parent, filename: name, fileType: type });
+	const { hostname } = new URL(signed.url);
+	if (hostname !== '127.0.0.1' && hostname !== 'localhost') {
+		throw new Error(`refusing to upload to ${hostname}: not a localhost address (see minio-loopback in docker-compose.yml)`);
+	}
+	const put = await fetch(signed.url, { method: 'PUT', headers: signed.header, body: bytes });
+	if (!put.ok) throw new Error(`PUT ${name} to storage -> ${put.status} ${(await put.text()).slice(0, 200)}`);
+	const record = await v1('POST', '/fileStorage', {
+		name,
+		owner,
+		parent,
+		type,
+		size: bytes.length,
+		storageFileName: signed.header['x-amz-meta-flat-name'],
+	});
+	return record._id;
+}
+
+async function fileManager() {
+	const s = loadState();
+	if (!s.courseId) throw new Error('run `create` first: the file-manager fixture lives in its course');
+	const keep = (key, value) => {
+		s[key] = value;
+		saveState(s);
+	};
+	const student = await studentId();
+
+	step('file manager: Kurs-Dateien');
+	keep('fmCourseRootFileId', await legacyUpload({
+		owner: s.courseId, name: 'Kursplan.txt', type: 'text/plain',
+		body: 'Kursplan Biologie\n\nThemen: Zelle, Gewebe, Organe. Stichwort: Photosynthese-Lichtreaktion.\n',
+	}));
+	keep('fmCourseDirId', await legacyDir('Arbeitsblätter', s.courseId));
+	keep('fmCourseFileId', await legacyUpload({
+		owner: s.courseId, parent: s.fmCourseDirId, name: 'Blatt 1 - Zellorganellen.txt', type: 'text/plain',
+		body: 'Blatt 1: Zellorganellen\n\nBeschrifte die Mitochondrienmembran und das endoplasmatische Retikulum.\n',
+	}));
+	keep('fmCourseSubDirId', await legacyDir('Woche 1', s.courseId, s.fmCourseDirId));
+	keep('fmCourseDeepFileId', await legacyUpload({
+		owner: s.courseId, parent: s.fmCourseSubDirId, name: 'Blatt 2 - Gewebe.txt', type: 'text/plain',
+		body: 'Blatt 2: Gewebe\n\nVergleiche Epithelgewebe und Bindegewebe.\n',
+	}));
+	log(`course root file, folder "Arbeitsblätter" with a file, and "Woche 1" nested inside it`);
+
+	if (s.teamId) {
+		step('file manager: Team-Dateien');
+		// Teams cannot be created (see the README), and the adopted one does not
+		// include the demo student, whose view is the one under test.
+		const team = await v1('GET', `/teams/${s.teamId}`);
+		if (!team.userIds.some((entry) => String(entry.userId?._id ?? entry.userId) === student)) {
+			const userIds = team.userIds.map((entry) => ({
+				userId: String(entry.userId?._id ?? entry.userId),
+				role: String(entry.role?._id ?? entry.role),
+				schoolId: String(entry.schoolId?._id ?? entry.schoolId),
+			}));
+			userIds.push({ userId: student, role: TEAM_MEMBER_ROLE, schoolId: me.school.id });
+			await v1('PATCH', `/teams/${s.teamId}`, { userIds });
+			keep('fmStudentAddedToTeam', true);
+			log('demo student added to the team');
+		}
+		keep('fmTeamDirId', await legacyDir('Projekt', s.teamId));
+		keep('fmTeamFileId', await legacyUpload({
+			owner: s.teamId, parent: s.fmTeamDirId, name: 'Projektplan.txt', type: 'text/plain',
+			body: 'Projektplan\n\nMeilenstein Chlorophyll bis Freitag.\n',
+		}));
+		log('team folder "Projekt" with a file');
+	}
+
+	step('file manager: Persönliche Dateien (the student\'s own)');
+	// No `owner` for personal files, exactly as the upload page sends none: the
+	// server decides the owner model as "a course, or else a team", so passing a
+	// user id records the folder as a team's, and every later permission check on
+	// it then dereferences a team that does not exist.
+	await asUser(STUDENT_EMAIL, STUDENT_PASSWORD, async () => {
+		keep('fmStudentDirId', await legacyDir('Notizen'));
+		keep('fmStudentFileId', await legacyUpload({
+			parent: s.fmStudentDirId, name: 'Lernzettel.txt', type: 'text/plain',
+			body: 'Lernzettel\n\nRibosomenfabrik: Proteinbiosynthese am rauen ER.\n',
+		}));
+	});
+	log('student folder "Notizen" with a file');
+
+	step('file manager: Geteilte Dateien');
+	keep('fmSharedFileId', await legacyUpload({
+		name: 'Geteilt vom Lehrer.txt', type: 'text/plain',
+		body: 'Zusatzmaterial\n\nDie Zellkernhuelle trennt Kernplasma und Zytoplasma.\n',
+	}));
+	// What accepting a share link does in the legacy client: a read-only user
+	// permission. Not the permission service, which writes `refOwnerModel` where
+	// the "shared with me" query reads `refPermModel`, so its shares never show.
+	const shared = await v1('GET', `/files/${s.fmSharedFileId}`);
+	await v1('PATCH', `/files/${s.fmSharedFileId}`, {
+		permissions: [
+			...shared.permissions,
+			{ refId: student, refPermModel: 'user', read: true, write: false, delete: false, create: false },
+		],
+	});
+	log('teacher file shared read-only with the student');
+}
+
 // ---------------------------------------------------------------- update ---

 async function update() {
@@ -348,6 +490,17 @@ async function update() {
 	await files('PATCH', `/rename/${s.fileId}`, { fileName: 'nervensystem-notiz-v2.txt' });
 	log('file renamed');

+	if (s.fmCourseFileId) {
+		step('file manager edits');
+		await v1('POST', '/fileStorage/rename', { id: s.fmCourseFileId, newName: 'Blatt 1 - Zellorganellen (korrigiert).txt' });
+		log('course file renamed');
+		s.fmCourseAddedFileId = await legacyUpload({
+			owner: s.courseId, parent: s.fmCourseSubDirId, name: 'Blatt 3 - Organe.txt', type: 'text/plain',
+			body: 'Blatt 3: Organe\n\nNeuer Suchbegriff: Nephronschleife.\n',
+		});
+		log('course file added in "Woche 1" (new search term: Nephronschleife)');
+	}
+
 	saveState({ ...s, updated: true });
 }

@@ -376,6 +529,46 @@ async function remove() {
 		['second room', () => v3('DELETE', `/rooms/${s.roomWithoutStudentId}`)],
 		['course', () => v1('DELETE', `/courses/${s.courseId}`)],
 	];
+	// File-manager content goes first, while its course and team still exist.
+	const fileManagerTries = [
+		['file-manager course files', async () => {
+			for (const id of [s.fmCourseAddedFileId, s.fmCourseDeepFileId, s.fmCourseFileId, s.fmCourseRootFileId]) {
+				if (id) await v1('DELETE', `/fileStorage?_id=${id}`);
+			}
+		}],
+		['file-manager course folders', async () => {
+			for (const id of [s.fmCourseSubDirId, s.fmCourseDirId]) if (id) await v1('DELETE', `/fileStorage/directories?_id=${id}`);
+		}],
+		['file-manager team content', async () => {
+			if (s.fmTeamFileId) await v1('DELETE', `/fileStorage?_id=${s.fmTeamFileId}`);
+			if (s.fmTeamDirId) await v1('DELETE', `/fileStorage/directories?_id=${s.fmTeamDirId}`);
+		}],
+		['shared file', async () => {
+			if (s.fmSharedFileId) await v1('DELETE', `/fileStorage?_id=${s.fmSharedFileId}`);
+		}],
+		['student personal files', async () => {
+			if (!s.fmStudentFileId && !s.fmStudentDirId) return;
+			await asUser(STUDENT_EMAIL, STUDENT_PASSWORD, async () => {
+				if (s.fmStudentFileId) await v1('DELETE', `/fileStorage?_id=${s.fmStudentFileId}`);
+				if (s.fmStudentDirId) await v1('DELETE', `/fileStorage/directories?_id=${s.fmStudentDirId}`);
+			});
+		}],
+		['student team membership', async () => {
+			if (!s.fmStudentAddedToTeam || !s.teamId) return;
+			const student = await studentId();
+			const team = await v1('GET', `/teams/${s.teamId}`);
+			const userIds = team.userIds
+				.map((entry) => ({
+					userId: String(entry.userId?._id ?? entry.userId),
+					role: String(entry.role?._id ?? entry.role),
+					schoolId: String(entry.schoolId?._id ?? entry.schoolId),
+				}))
+				.filter((entry) => entry.userId !== student);
+			await v1('PATCH', `/teams/${s.teamId}`, { userIds });
+		}],
+	];
+	tries.unshift(...fileManagerTries);
+
 	for (const [what, fn] of tries) {
 		try {
 			await fn();
@@ -398,8 +591,11 @@ await login();
 if (phase === 'create') await create();
 else if (phase === 'update') await update();
 else if (phase === 'delete') await remove();
-else if (phase === 'show') summary(loadState());
+else if (phase === 'files') {
+	await fileManager();
+	summary(loadState());
+} else if (phase === 'show') summary(loadState());
 else {
-	console.error(`unknown phase ${phase}; expected create | update | delete | show`);
+	console.error(`unknown phase ${phase}; expected create | update | delete | files | show`);
 	process.exit(2);
 }