Add a local Schulcloud instance modelled on the live target
A Docker Compose stack that runs the deployed images (quay.io/schulcloudverbund/*, thr theme, tag 33.40 — the versions schulcloud-thueringen.de reports) rather than a rebuild of main, so what we develop against is the deployed artefact. It exists to produce the states we can otherwise only observe read-only: log in as the teacher, grade, then read it back the way the MCP server does. Faithful where it matters and honest where it isn't: - Feature flags in env/api.env are a replay of GET /api/v3/config/public from the live instance, not a hand-picked set; instance identity mirrors the thr group_vars from dof_app_deploy. - The proxy is generated from the deployment's own ingress table (scripts/gen-proxy-conf.py) so the legacy-client / SPA / API path split matches production; getting it wrong tests a different application. - Valkey runs in `single` mode so the JWT whitelist expires sessions the way production does, rather than the in-memory shortcut that hides it. - No external OAuth / Schulportal login (excluded by request and not reproducible locally), no BigBlueButton; each divergence is marked at the line it affects. Everything binds to 127.0.0.1 and uses the upstream dev credentials, which are public. Profiles keep the heavy pieces opt-in: `tools` adds Etherpad/H5P/tldraw/ Collabora, `av` adds ClamAV, `preview` adds thumbnailing. seed.sh loads the upstream demo school (the same call the deployment's init job makes) and registers MinIO as the legacy storage provider, which has no seed data on purpose. The demo data already contains the grading states that are hard to obtain from the real account — a feedback-only grade and a 100% one — which is what surfaced the past-due submitted-text scrape gap. One config finding baked in: file-storage and h5p validate a token's issuer/audience against JWT_DOMAIN (default "localhost"), while the API stamps SC_DOMAIN; without keeping them equal, the homework page's file lookups 401. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
MechaCat02
190
local-instance/scripts/gen-proxy-conf.py
Normal file
190
local-instance/scripts/gen-proxy-conf.py
Normal file
@@ -0,0 +1,190 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Regenerate proxy/nginx.conf from the real deployment's ingress table.
|
||||
|
||||
The live instance is a Kubernetes ingress that splits one origin across the
|
||||
legacy client, the new SPA and several APIs. Which path goes where is not
|
||||
documented in prose — it is the table in
|
||||
dof_app_deploy/ansible/group_vars/all/x_ingress.yml plus a per-path ingress in
|
||||
each service repo. Transcribing 46 rules by hand invites exactly the drift that
|
||||
would make this instance lie about the real one, so we generate them.
|
||||
|
||||
Usage: python3 scripts/gen-proxy-conf.py > proxy/nginx.conf
|
||||
Needs the upstream clones in ../vendor (see README.md).
|
||||
"""
|
||||
import pathlib
|
||||
import sys
|
||||
|
||||
try:
|
||||
import yaml
|
||||
except ImportError:
|
||||
sys.exit('needs PyYAML: pip install pyyaml')
|
||||
|
||||
VENDOR = pathlib.Path(__file__).resolve().parents[2] / 'vendor'
|
||||
INGRESS = VENDOR / 'dof_app_deploy/ansible/group_vars/all/x_ingress.yml'
|
||||
|
||||
# Kubernetes service name -> compose upstream. None means the real deployment
|
||||
# deliberately 404s that path.
|
||||
UPSTREAM = {
|
||||
'client-svc': 'client:3100',
|
||||
'nuxtclient-svc': 'nuxt:4000',
|
||||
'api-svc': 'api:3030',
|
||||
'default-backend-404-svc': None,
|
||||
'version-aggregator-svc': None, # replaced by our own /version below
|
||||
None: None,
|
||||
}
|
||||
|
||||
# Routes that live in the individual service repos' own ingress templates
|
||||
# rather than the shared table, plus the two websocket endpoints.
|
||||
# (path, upstream, websocket, rewrite-or-None)
|
||||
EXTRA = [
|
||||
('/api/v3/file/', 'file-storage:4444', False, None),
|
||||
('/api/v3/wopi/', 'file-storage:4444', False, None),
|
||||
('/api/v3/h5p-editor/h5pstatics/', 'h5p-staticfiles:8080', False,
|
||||
'^/api/v3/h5p-editor/h5pstatics/(.*)$ /h5pstatics/$1'),
|
||||
('/api/v3/h5p-editor/', 'h5p-editor:4448', False, None),
|
||||
('/api/v3/', 'api:3030', False, None),
|
||||
('/admin/api/v1', 'admin-api:4030', False, None),
|
||||
('/board-collaboration', 'board-collaboration:4450', True, None),
|
||||
('/tldraw-server', 'tldraw-server:3345', True, None),
|
||||
('/api/tldraw', 'tldraw-server:3345', False, None),
|
||||
]
|
||||
|
||||
PREAMBLE = '''# GENERATED by scripts/gen-proxy-conf.py — do not edit by hand.
|
||||
#
|
||||
# One origin in front of the whole stack, the way the real instance is fronted
|
||||
# by its Kubernetes ingress. The path split between the legacy client and the
|
||||
# new SPA is not cosmetic: get it wrong and you are testing a different
|
||||
# application from the one the students use.
|
||||
|
||||
map $http_upgrade $connection_upgrade {
|
||||
default upgrade;
|
||||
'' close;
|
||||
}
|
||||
|
||||
server {
|
||||
listen 4400;
|
||||
server_name localhost;
|
||||
|
||||
# Docker's embedded DNS. Every proxy_pass below goes through a variable so
|
||||
# that names resolve per request rather than at startup — otherwise this
|
||||
# container refuses to boot whenever an optional profile (tools, av) is
|
||||
# down, which is the normal case.
|
||||
resolver 127.0.0.11 valid=10s ipv6=off;
|
||||
|
||||
# Course files and H5P uploads are large; the ingress allows the same.
|
||||
client_max_body_size 2600m;
|
||||
proxy_read_timeout 300s;
|
||||
proxy_send_timeout 300s;
|
||||
|
||||
# version-aggregator-svc upstream; /serverversion and /nuxtversion are the
|
||||
# real per-app endpoints and are routed below.
|
||||
location = /version {
|
||||
default_type application/json;
|
||||
return 200 '{"local-instance":true,"see":["/serverversion","/nuxtversion"]}';
|
||||
}
|
||||
'''
|
||||
|
||||
COMMON = ''' proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
'''
|
||||
|
||||
WS = ''' proxy_http_version 1.1;
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection $connection_upgrade;
|
||||
'''
|
||||
|
||||
GONE = '''
|
||||
location {path} {{
|
||||
return 404;
|
||||
}}
|
||||
'''
|
||||
|
||||
# Etherpad is mounted under a prefix it knows nothing about, so the deployment
|
||||
# runs a dedicated nginx in front of it that rewrites the prefix away and
|
||||
# proxies socket.io separately. Copied from
|
||||
# dof_app_deploy/ansible/roles/dof_etherpad/templates/nginx-configmap-files.yml.j2
|
||||
# — a plain proxy_pass gets you a pad that loads and then never syncs.
|
||||
ETHERPAD = '''
|
||||
location ^~ /etherpad/admin { return 404; }
|
||||
location ^~ /etherpad/stats { return 404; }
|
||||
|
||||
location /etherpad/socket.io {
|
||||
set $up_etherpad etherpad:9001;
|
||||
rewrite /etherpad/socket.io/(.*) /socket.io/$1 break;
|
||||
proxy_pass http://$up_etherpad;
|
||||
proxy_redirect / /etherpad/;
|
||||
proxy_set_header Host $http_host;
|
||||
proxy_buffering off;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $remote_addr;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection $connection_upgrade;
|
||||
}
|
||||
|
||||
location /etherpad {
|
||||
set $up_etherpad etherpad:9001;
|
||||
rewrite ^/etherpad$ /etherpad/ permanent;
|
||||
rewrite /etherpad/(.*) /$1 break;
|
||||
proxy_pass http://$up_etherpad;
|
||||
proxy_pass_header Server;
|
||||
proxy_redirect / /etherpad/;
|
||||
proxy_set_header Host $host;
|
||||
proxy_buffering off;
|
||||
}
|
||||
'''
|
||||
|
||||
|
||||
def block(path: str, upstream: str, ws: bool, rewrite: str | None, var: str) -> str:
|
||||
out = [f'\n\tlocation {path} {{\n', f'\t\tset ${var} {upstream};\n']
|
||||
if rewrite:
|
||||
out.append(f'\t\trewrite {rewrite} break;\n')
|
||||
out.append(f'\t\tproxy_pass http://${var};\n')
|
||||
out.append(COMMON)
|
||||
if ws:
|
||||
out.append(WS)
|
||||
out.append('\t}\n')
|
||||
return ''.join(out)
|
||||
|
||||
|
||||
def varname(path: str) -> str:
|
||||
safe = ''.join(c if c.isalnum() else '_' for c in path.strip('/')) or 'root'
|
||||
return f'up_{safe}'
|
||||
|
||||
|
||||
def main() -> None:
|
||||
table = yaml.safe_load(INGRESS.read_text())['default_ingress']
|
||||
|
||||
seen: set[str] = set()
|
||||
out = [PREAMBLE]
|
||||
|
||||
out.append('\n\t# --- service-owned ingresses and websockets ---\n')
|
||||
for path, upstream, ws, rewrite in EXTRA:
|
||||
seen.add(path)
|
||||
out.append(block(path, upstream, ws, rewrite, varname(path)))
|
||||
|
||||
out.append('\n\t# --- from dof_app_deploy x_ingress.yml ---\n')
|
||||
for name, entry in table.items():
|
||||
path = entry.get('path')
|
||||
if path is None or path in seen:
|
||||
continue
|
||||
seen.add(path)
|
||||
if path == '/etherpad':
|
||||
out.append(ETHERPAD)
|
||||
continue
|
||||
upstream = UPSTREAM.get(entry.get('serviceName'), 'MISSING')
|
||||
if upstream == 'MISSING':
|
||||
sys.exit(f'unknown serviceName for {name}: {entry.get("serviceName")}')
|
||||
out.append(f'\n\t# {name}')
|
||||
out.append(GONE.format(path=path) if upstream is None
|
||||
else block(path, upstream, False, None, varname(path)))
|
||||
|
||||
out.append('}\n')
|
||||
sys.stdout.write(''.join(out))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
19
local-instance/scripts/minio-init.sh
Executable file
19
local-instance/scripts/minio-init.sh
Executable file
@@ -0,0 +1,19 @@
|
||||
#!/bin/sh
|
||||
# Creates the buckets each service expects. MinIO does not create them on
|
||||
# demand: file-storage reports a generic 500 on upload if its bucket is
|
||||
# missing, and the h5p library job fails halfway through.
|
||||
set -eu
|
||||
|
||||
mc alias set local http://minio:9000 miniouser miniouser
|
||||
|
||||
for bucket in \
|
||||
schulcloud ` # files-storage (the /api/v3/file API)` \
|
||||
h5p-content-bucket ` # h5p-editor content` \
|
||||
h5p-library-bucket ` # h5p content types` \
|
||||
ydocs ` # tldraw whiteboard documents` \
|
||||
fwu-content # FWU media, unused but cheap to create
|
||||
do
|
||||
mc mb --ignore-existing "local/$bucket"
|
||||
done
|
||||
|
||||
mc ls local
|
||||
69
local-instance/scripts/seed.sh
Executable file
69
local-instance/scripts/seed.sh
Executable file
@@ -0,0 +1,69 @@
|
||||
#!/usr/bin/env bash
|
||||
# Seed the local instance with the upstream demo school.
|
||||
#
|
||||
# This is what the real deployment's init job does (dof_app_deploy
|
||||
# .../schulcloud-server-init/templates/configmap_file_init.yml.j2): it asks the
|
||||
# management app to load backup/setup/*.json, which ships inside the server
|
||||
# image. Safe to re-run — collections are replaced, not appended to.
|
||||
set -euo pipefail
|
||||
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
MGMT=http://127.0.0.1:3333/api/management/database
|
||||
COMPOSE=(docker compose)
|
||||
|
||||
echo "==> waiting for the management app"
|
||||
for _ in $(seq 1 60); do
|
||||
if curl -fsS -o /dev/null "$MGMT/../../docs" 2>/dev/null || curl -fsS -o /dev/null -X POST "$MGMT/sync-indexes" 2>/dev/null; then
|
||||
break
|
||||
fi
|
||||
sleep 5
|
||||
done
|
||||
|
||||
echo "==> seeding collections (this takes a minute or two)"
|
||||
curl -fsS --retry 30 --retry-all-errors --retry-delay 10 \
|
||||
-X POST "$MGMT/seed?with-indexes=true" >/dev/null
|
||||
echo " done"
|
||||
|
||||
# The legacy file service (course/topic attachments, as opposed to the newer
|
||||
# /api/v3/file API) reads its S3 credentials from a storageproviders document
|
||||
# rather than from the environment, and there is deliberately no seed data for
|
||||
# it. Without this, legacy uploads fail with a provider-not-found error.
|
||||
echo "==> registering MinIO as the legacy storage provider"
|
||||
S3_KEY=$(grep -E '^S3_KEY=' env/shared.env | cut -d= -f2-)
|
||||
SECRET=$(curl -fsS -X POST "$MGMT/encrypt-plain-text" \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d "$(printf '{"plainText":"miniouser","key":"%s"}' "$S3_KEY")")
|
||||
|
||||
"${COMPOSE[@]}" exec -T mongo mongosh schulcloud --quiet --eval "
|
||||
const id = ObjectId('62949a4003839b6162aa566b');
|
||||
db.storageproviders.replaceOne({ _id: id }, {
|
||||
_id: id, isShared: true, region: 'eu-central-1', type: 'S3',
|
||||
endpointUrl: 'http://minio:9000',
|
||||
accessKeyId: 'miniouser',
|
||||
secretAccessKey: '$SECRET',
|
||||
maxBuckets: 150, freeBuckets: 138,
|
||||
createdAt: new Date(), updatedAt: new Date(), __v: 0,
|
||||
}, { upsert: true });
|
||||
const r = db.schools.updateMany({}, { \$set: { storageProvider: id } });
|
||||
print(' schools linked to the provider: ' + r.modifiedCount);
|
||||
"
|
||||
|
||||
cat <<'ACCOUNTS'
|
||||
|
||||
==> ready — http://localhost:4400
|
||||
|
||||
Seeded accounts (the demo password differs by account — upstream quirk):
|
||||
|
||||
demo-schueler@schul-cloud.org student Fritz Schmidt schulcloud
|
||||
^ has graded submissions, incl. a feedback-only and a 100% one
|
||||
demo-lehrer@schul-cloud.org teacher Erika Meier schulcloud
|
||||
klara.fall@schul-cloud.org teacher Klara Fall Schulcloud1!
|
||||
^ owns Fritz's graded Biologie submissions
|
||||
lehrer@schul-cloud.org teacher Cord Carl Schulcloud1!
|
||||
admin@schul-cloud.org admin Thorsten Test Schulcloud1!
|
||||
*.qa@schul-cloud.org various Schulcloud1qa!
|
||||
|
||||
Sign in as the teacher to grade, as the student to see what grading looks
|
||||
like from the side our MCP server reads.
|
||||
ACCOUNTS
|
||||
Reference in New Issue
Block a user