Browse the file manager ("Dateien") as a filesystem
Many teachers never use topics or boards; their material sits in the course's file area, and the tools answered "0 files" for courses holding dozens of worksheets — 21 of 26 courses on the live account. Persönliche, Kurs-, Team- and Geteilte Dateien live in the legacy file store, not in files-storage, and its service is not in the public ingress. The only way in is the legacy client: HTML listings, and GET /files/signedurl for a pre-signed download. core/legacy-files.ts turns that into one path tree — /my, /courses/<course>, /teams/<team>, /shared — resolving names that contain "/", ids anywhere in a path, and wrong or ambiguous names with a message saying what is there. A listing that does not parse throws; it never reads as an empty folder. Some of the legacy client's GET routes write (GET /files/share/ mints a share token), so getFileManagerPage allows only the listing routes, by pattern. Signed URLs are fetched with no credentials and must be https. - MCP: fs_list, fs_tree, fs_find and fs_read; get_course lists course files. - CLI: schulcloud fs ls, tree, find and get, recursive and resumable. - API: /api/fs/list, tree, find and file. - Index: the crawl walks the file manager (INDEX_FILE_MANAGER, on by default), so search covers the text inside those files and sync mirrors them under <course>/Kurs-Dateien. The local instance gains a fixture for all four areas. It needed a loopback, so signed URLs open from the host, and a pre-created bucket, since MinIO does not implement PutBucketCors. 135 tests. Smoke 55/55 live; 57/57 and 55/55 on the local instance. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
63
docs/API.md
63
docs/API.md
@@ -225,6 +225,69 @@ hops and without Etherpad's API key; `core/etherpad.ts` does this. The url is
|
||||
built from the server's `ETHERPAD__PAD_URI`, so it must be checked against the
|
||||
instance host before the session cookie is sent to it.
|
||||
|
||||
## The file manager ("Dateien") is a third store
|
||||
|
||||
Persönliche Dateien, Kurs-Dateien, Team-Dateien and Geteilte Dateien are the
|
||||
**legacy file system**: a `files` collection with real folders (`isDirectory`,
|
||||
`parent`, `owner`, `refOwnerModel`), served by the legacy Feathers
|
||||
`fileStorage` service. It shares nothing with files-storage. Asking
|
||||
`/api/v3/file/list/school/{school}/courses/{courseId}` answers **0** for a
|
||||
course whose file manager holds dozens of worksheets — measured: 21 of 26
|
||||
courses on the live account keep files here, some nothing else.
|
||||
|
||||
**Its service is not in the public ingress**, so the only way in is the legacy
|
||||
client. Verified live:
|
||||
|
||||
| Route (legacy client, `jwt` cookie) | Returns | Notes |
|
||||
|---|---|---|
|
||||
| `GET /files/my/` , `/files/my/{folder}` | HTML listing | personal root / one folder |
|
||||
| `GET /files/courses/` | HTML listing | the courses, *as folders* (ids = course ids) |
|
||||
| `GET /files/courses/{course}` , `/files/courses/{course}/{folder}` | HTML listing | **one** folder segment at any depth |
|
||||
| `GET /files/teams/…` | HTML listing | same shape as courses |
|
||||
| `GET /files/shared/` | HTML listing | flat; shared *folders* cannot be opened (no route; the UI's link 404s) |
|
||||
| `GET /files/signedurl?file={id}&name={name}` | `{"url": …}` | pre-signed S3 url, **another host** (live: `s3.hidrive.strato.com`) |
|
||||
| `GET /files/permittedDirectories/` | JSON tree | **lists every course with no folders in any** — see below |
|
||||
| `GET /files/search/?q=` | HTML | **504** on live: an unindexed regex over every file record |
|
||||
|
||||
Listings are parsed on the attributes the page's own scripts use:
|
||||
`data-folder-id` with the name inside `.card-title-directory` (emitted
|
||||
**unescaped**, `{{{stripOnlyScript name}}}`), and `data-file-id`,
|
||||
`data-file-name`, `data-file-size`, `data-file-viewer-type` on each
|
||||
`.card.file`. A blocked file carries `btn-file-danger` and no viewer type. No
|
||||
dates are rendered.
|
||||
|
||||
**`permittedDirectories` is broken for courses.** The directory service's
|
||||
query matches course folders on `refOwnerModel: 'courses'`; the records say
|
||||
`'course'`. Live result: 26 courses, 0 folders; personal folders come through.
|
||||
Listings are the only complete view — which is also what the UI shows.
|
||||
|
||||
**Some GET routes write.** `GET /files/share/?file=` mints a share token when
|
||||
the file has none (`PATCH /fileStorage/shared/{id}`), and
|
||||
`GET /files/file?…&share=…` grants the caller a permission on the file.
|
||||
`GET /files/fileModel/{id}/proxy` forwards to the latter. A GET-only client is
|
||||
therefore *not* read-only against this surface by itself: `core/client.ts`
|
||||
allows only the listing routes and `/files/signedurl`, by pattern.
|
||||
|
||||
**The signed-url service returns its error instead of throwing it**
|
||||
(`.catch((err) => new Forbidden(err))`), so a refused file is a 200 whose body
|
||||
has no `url`.
|
||||
|
||||
**Course names contain `/`** in real data ("LF07 - FIA24A/B - Sb/Ha",
|
||||
"FIA24/FIP24 IT LF12"), and a real file is literally called `..docx`. Paths
|
||||
built from names cannot be split naively; `core/legacy-files.ts` resolves by
|
||||
trying joined segments, and accepts ids as segments.
|
||||
|
||||
Two more, seen while building the local fixture:
|
||||
|
||||
- **`getRefOwnerModel(owner)` answers "a course, or else `teams`".** Any owner
|
||||
id that is not a course — a *user* included — is recorded as a team's. The
|
||||
upload page never sends an owner for personal files (`data-owner=""`), so the
|
||||
server defaults to the creator and records `user`; send the user id and every
|
||||
later permission check dereferences a team that does not exist.
|
||||
- **The file permission service writes `refOwnerModel`** where the "shared with
|
||||
me" query reads `refPermModel`, so a share made through it never appears
|
||||
under Geteilte Dateien. The share-link flow patches `/files/{id}` directly.
|
||||
|
||||
## Re-verifying after an upstream release
|
||||
|
||||
`npm run probe` re-checks every assumption above against the live instance and
|
||||
|
||||
33
docs/CLI.md
33
docs/CLI.md
@@ -50,6 +50,39 @@ alongside courses, with their files under the room's name rather than a course's
|
||||
what changed: that is a handful of requests, where a full re-crawl reads every
|
||||
course. The server refuses a repeat within a minute unless you pass `--force`.
|
||||
|
||||
### The file manager (`fs`)
|
||||
|
||||
The Schulcloud file manager ("Dateien") — Persönliche, Kurs-, Team- and
|
||||
Geteilte Dateien — browsed like a filesystem, live:
|
||||
|
||||
```
|
||||
schulcloud fs ls [path] [--long]
|
||||
schulcloud fs tree [path] [--depth <n>] [--max-folders <n>]
|
||||
schulcloud fs find <name> [--path <path>] [--type file|folder] [--long]
|
||||
schulcloud fs get <path> [--out <path>] [--force] [--jobs <n>]
|
||||
```
|
||||
|
||||
```console
|
||||
$ schulcloud fs ls /courses
|
||||
$ schulcloud fs tree "/courses/FIA24B - SK (Rh)"
|
||||
$ schulcloud fs find "*Erben*" --path /courses
|
||||
$ schulcloud fs get "/courses/FIA24B - SK (Rh)/02_Erbrecht" --out ~/Erbrecht
|
||||
```
|
||||
|
||||
The tree is `/my`, `/courses/<course>`, `/teams/<team>` and `/shared`; the
|
||||
German names ("/Kurs-Dateien") work too. Names may contain `/` — course names
|
||||
often do — and still resolve; any segment may also be an id from `--long`.
|
||||
|
||||
`fs find` matches any part of a name, or, given `*` or `?`, the whole name as
|
||||
`find -name` does. `fs get` on a folder downloads everything below it, keeps
|
||||
the structure, and skips files already present at the same size — so re-running
|
||||
it resumes. Each folder is one page load on the server, so large trees take a
|
||||
while.
|
||||
|
||||
`sync` mirrors these files too, under `<course>/Kurs-Dateien/…`,
|
||||
`Persönliche Dateien/…`, `Team-Dateien/<team>/…` and `Geteilte Dateien/`, once
|
||||
the server's index includes them (`INDEX_FILE_MANAGER`, on by default).
|
||||
|
||||
## How sync works
|
||||
|
||||
It is a **one-way mirror, not a two-way sync**, and that follows from the data
|
||||
|
||||
@@ -103,9 +103,9 @@ node dist/bin/cli.js sync
|
||||
## Run the test suites
|
||||
|
||||
```bash
|
||||
npm test # 58 offline tests
|
||||
npm test # 135 offline tests
|
||||
npm run smoke # end-to-end against the live instance, live-only mode
|
||||
DATABASE_URL=… npm run smoke # end-to-end with the index (34 checks)
|
||||
DATABASE_URL=… npm run smoke # end-to-end with the index (57 checks)
|
||||
```
|
||||
|
||||
Store tests need a database and skip without one:
|
||||
|
||||
Reference in New Issue
Block a user