diff --git a/CLAUDE.md b/CLAUDE.md index 9ae2d93..183c0c4 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -26,13 +26,17 @@ npm run probe # verify token + API assumptions against the LIVE instance npm run smoke # full end-to-end: real server + real MCP client + real data npm run keepalive-status # is the deployed container holding its session? npm run session-diagnose # ~2.5h: measure what actually ends the session +npm run publish-image # amd64 + arm64 image to registry.mc02.dev; clean tree only ``` `docker-compose.override.yml` is local-only and publishes the server on `127.0.0.1:8080` (or `MCP_HOST_PORT`) and Postgres on `127.0.0.1:55432`; see `docs/LOCAL.md`. Compose merges it whenever `COMPOSE_FILE` is unset, so the Pi's `.env` sets `COMPOSE_FILE=docker-compose.yml:deploy/docker-compose.pi.yml` -instead; `docs/PI.md` is the setup guide. When `.env` points at the live account, test against the local +instead; `docs/PI.md` is the setup guide. **The Pi never builds**: that file runs +`registry.mc02.dev/schulcloud-mcp` and `!reset`s the build section, and +`npm run publish-image` is how an image gets there — so a change reaches the +Pi only once it is committed and published. When `.env` points at the live account, test against the local instance only through `local-instance/scripts/mcp-env.sh`: it pins its own database (`schulcloud_local`) and mirror, so fixtures cannot reach the live index. diff --git a/README.md b/README.md index 46f24aa..d4b0933 100644 --- a/README.md +++ b/README.md @@ -162,7 +162,7 @@ src/ cli/ CLI config, API client, sync engine bin/ http, stdio and cli entry points docs/ API findings, auth, deployment, CLI, roadmap -deploy/ Caddyfile snippet +deploy/ Caddyfile snippet, the Pi's compose file scripts/ probe, smoke, session diagnostics vendor/ upstream clones, git-ignored, for reference only ``` @@ -180,6 +180,7 @@ npm run probe # check assumptions against the live instance npm run smoke # full end-to-end: real server, real client, real data npm run session-diagnose # instrument what actually ends the session (~2.5h) npm run keepalive-status # is the deployed container holding its session? +npm run publish-image # build amd64 + arm64 and push to registry.mc02.dev — the Pi only pulls npm run typecheck ``` diff --git a/deploy/docker-compose.pi.yml b/deploy/docker-compose.pi.yml index d80c2aa..57a18cd 100644 --- a/deploy/docker-compose.pi.yml +++ b/deploy/docker-compose.pi.yml @@ -10,6 +10,12 @@ services: schulcloud-mcp: + # The Pi runs the image published to the registry and never builds one: + # scripts/publish-image.sh builds it for arm64 and amd64 elsewhere. `!reset` + # drops the build section docker-compose.yml declares, so no `docker compose` + # command here can fall back to building (Compose 2.24 or later). + image: registry.mc02.dev/schulcloud-mcp:${SCHULCLOUD_MCP_TAG:-latest} + build: !reset null environment: # The bundled Postgres, by name on the private backend network. Overrides # any DATABASE_URL in .env. diff --git a/docs/DEPLOYMENT.md b/docs/DEPLOYMENT.md index 813aa87..3cf1fab 100644 --- a/docs/DEPLOYMENT.md +++ b/docs/DEPLOYMENT.md @@ -38,12 +38,14 @@ this server's bearer check. ## First deploy -[PI.md](PI.md) steps 2–5: clone to `/opt/schulcloud-mcp`, write `.env` (the Pi's +[PI.md](PI.md) steps 1–5: log in to `registry.mc02.dev`, clone to +`/opt/schulcloud-mcp` for the compose files, write `.env` (the Pi's `COMPOSE_FILE`, `CADDY_NETWORK` and `POSTGRES_PASSWORD`, generated secrets, the first Schulcloud token), then: ```bash -docker compose up -d --build +docker compose pull +docker compose up -d docker compose logs -f schulcloud-mcp ``` @@ -216,15 +218,44 @@ use stdio: `MCP_AUTH_TOKEN` is irrelevant in stdio mode — there is no network listener. +## Publishing an image + +The Pi never builds: `deploy/docker-compose.pi.yml` runs +`registry.mc02.dev/schulcloud-mcp` and removes the build section it would +otherwise inherit. Images are published from a development machine: + +```bash +npm run publish-image +docker buildx imagetools inspect registry.mc02.dev/schulcloud-mcp:latest # lists linux/amd64 and linux/arm64 +``` + +`scripts/publish-image.sh` refuses a working tree with uncommitted changes, so +a tag always names exactly the code of one commit. It builds for `linux/amd64` +and `linux/arm64` and pushes two tags: `latest`, and the short commit id. It +also labels the image with the full commit and the repository. + +Two things the build machine needs: + +- **arm64 emulation on an x86 machine.** The script checks for it and prints + the command that registers QEMU — + `docker run --privileged --rm tonistiigi/binfmt --install arm64` — which lasts + until the next reboot. +- **A multi-platform push.** Docker's containerd image store supports it + directly. Without that store, create a builder first with + `docker buildx create --use`. + ## Updating ```bash -cd /opt/schulcloud-mcp && git pull -docker compose up -d --build -docker compose exec schulcloud-mcp node -e "1" # sanity -npm run probe # re-verify the API assumptions +npm run publish-image # on a development machine +cd /opt/schulcloud-mcp && git pull # on the Pi: compose files and docs +docker compose pull && docker compose up -d +npm run probe # on a development machine: re-verify the API assumptions ``` +A `.env` that pins `SCHULCLOUD_MCP_TAG` needs the new commit id before the pull. +Setting the previous one rolls back. + ## Operational notes - **Restart policy** is `unless-stopped`; the container comes back after a diff --git a/docs/PI.md b/docs/PI.md index d91e6bb..a11d0f3 100644 --- a/docs/PI.md +++ b/docs/PI.md @@ -4,6 +4,10 @@ A step-by-step guide from a Pi with Docker to a working claude.ai connector. [DEPLOYMENT.md](DEPLOYMENT.md) explains *why* each piece is there; this page is the order to do it in. +The Pi never builds the server. It pulls the image published to +`registry.mc02.dev`, built for arm64 and amd64 by `npm run publish-image` on a +development machine ([DEPLOYMENT.md](DEPLOYMENT.md#publishing-an-image)). + ``` claude.ai ─┐ ┌──────────── Pi (home, always on) ─────────────┐ Claude Code├─HTTPS─▶ VPS ─TCP─▶ │ Caddy ─▶ schulcloud-mcp ─▶ schulcloud-thueringen.de @@ -18,7 +22,8 @@ CLI ───────┘ (public IP, │ (TLS) └─ postgres (p OS is plenty; put Docker's data on an SSD rather than the SD card if you can, since Postgres and the file mirror (about 1.3 GB on the current account) write to it. -- **Docker Engine with the Compose plugin** on the Pi. +- **Docker Engine with the Compose plugin**, Compose 2.24 or later, on the Pi. +- **A login for `registry.mc02.dev`**, where the image is published. - **Caddy running in a container on the Pi**, serving ports 80 and 443. This server joins its Docker network; it opens no port of its own. - **The VPS**, with a public IPv4 address, forwarding TCP ports 80 and 443 to the @@ -34,13 +39,16 @@ CLI ───────┘ (public IP, │ (TLS) └─ postgres (p sudo apt update && sudo apt full-upgrade -y curl -fsSL https://get.docker.com | sh sudo usermod -aG docker "$USER" # then log out and back in -docker compose version +docker compose version # v2.24 or later timedatectl # "System clock synchronized: yes" +docker login registry.mc02.dev ``` -The clock matters: token expiry and TLS certificates are both judged by it. +The clock matters: token expiry and TLS certificates are both judged by it. The +login is needed to pull, not only to push; Docker keeps it in +`~/.docker/config.json`. -## 2. Get the code +## 2. Get the compose files ```bash sudo mkdir -p /opt/schulcloud-mcp && sudo chown "$USER": /opt/schulcloud-mcp @@ -48,7 +56,9 @@ git clone https://git.mc02.dev/fabi/Schulcloud-MCP.git /opt/schulcloud-mcp cd /opt/schulcloud-mcp ``` -Every command below runs in `/opt/schulcloud-mcp`. +Every command below runs in `/opt/schulcloud-mcp`. The checkout provides the +compose files, `.env.example` and the Caddy snippet; its source code is not built +here. ## 3. Configure @@ -74,6 +84,7 @@ POSTGRES_PASSWORD=$(openssl rand -hex 24) MCP_AUTH_TOKEN=$(openssl rand -hex 32) MCP_PATH_SECRET=$(openssl rand -hex 32) INDEX_PERSONAL_FILES=true +# SCHULCLOUD_MCP_TAG=latest EOF ``` @@ -81,12 +92,13 @@ What those lines do: | Setting | | |---|---| -| `COMPOSE_FILE` | Makes every `docker compose` command here use [`deploy/docker-compose.pi.yml`](../deploy/docker-compose.pi.yml), which joins Caddy's network and wires up the database. It also keeps `docker-compose.override.yml` out — that one is for local development and would publish ports and switch the crawl timer off. | +| `COMPOSE_FILE` | Makes every `docker compose` command here use [`deploy/docker-compose.pi.yml`](../deploy/docker-compose.pi.yml), which runs the registry image — removing the build section, so nothing can be built here — joins Caddy's network and wires up the database. It also keeps `docker-compose.override.yml` out — that one is for local development and would publish ports and switch the crawl timer off. | | `CADDY_NETWORK` | The network Caddy reaches this server on, by the name `schulcloud-mcp`. | | `POSTGRES_PASSWORD` | The bundled Postgres, which sits on a private network with this server only. Hex, so it needs no escaping inside the connection URL. | | `MCP_AUTH_TOKEN` | What Claude Code, the CLI and the `/token` page present. | | `MCP_PATH_SECRET` | The secret path claude.ai uses until OAuth exists. | | `INDEX_PERSONAL_FILES` | Also indexes your own files and handed-in work, including teachers' feedback. Optional. | +| `SCHULCLOUD_MCP_TAG` | Which published image to run. Unset means `latest`; a commit id such as `bac9130` pins it, so updates happen only when you change it. Optional. | **Copy `MCP_AUTH_TOKEN` and `MCP_PATH_SECRET` into your password manager now** — you need both again in step 9, and neither is ever printed by the server: @@ -110,11 +122,15 @@ restart (step 11). ## 5. Start the server ```bash -docker compose up -d --build # the first build takes a few minutes on a Pi +docker compose pull # the published image, and Postgres +docker compose up -d docker compose ps docker compose logs -f schulcloud-mcp ``` +Nothing is built on the Pi: `pull` fetches the arm64 variant of the image, and +with the Pi file in place no `docker compose` command can fall back to a build. + Expect, within a few seconds: ``` @@ -339,15 +355,24 @@ hours, say. ## Updating +A new version is published first, from a development machine: +`npm run publish-image` ([DEPLOYMENT.md](DEPLOYMENT.md#publishing-an-image)). +Then, on the Pi: + ```bash cd /opt/schulcloud-mcp -git pull -docker compose up -d --build +git pull # compose files and docs +docker compose pull # the newly published image +docker compose up -d docker compose logs --tail 20 schulcloud-mcp ``` -`COMPOSE_FILE` in `.env` keeps applying the Pi settings. A rebuild keeps the -database, the mirror and a replaced token, all of which live in volumes. +`COMPOSE_FILE` in `.env` keeps applying the Pi settings. Recreating the container +keeps the database, the mirror and a replaced token, all of which live in volumes. + +With `SCHULCLOUD_MCP_TAG` pinned, set it to the new commit id before pulling. +Going back works the same way: set the previous commit id, then `pull` and +`up -d`. ## Backups @@ -370,12 +395,18 @@ database, the mirror and a replaced token, all of which live in volumes. | Caddy never obtains a certificate | Port 80 not forwarded, or DNS not yet pointing at the VPS | Step 7 | | `network … declared as external, but could not be found` | Wrong `CADDY_NETWORK` | Step 3 | | `required variable … is missing a value` | A line missing from `.env` | Step 3 | +| `unauthorized` or `pull access denied` from `docker compose pull` | Not logged in to the registry on the Pi | `docker login registry.mc02.dev` | +| `no matching manifest for linux/arm64` | The image was published for amd64 only | Publish again with `npm run publish-image`, which builds both | +| Compose rejects `!reset` in `deploy/docker-compose.pi.yml` | Compose older than 2.24 | Update Docker (step 1) | | `EACCES` for `/data/state` or `/data/mirror` in the logs | A volume created by an old image, owned by root | `docker compose down`, `docker volume rm schulcloud-mcp_state` (or `_mirror`), `docker compose up -d` | ## Security checklist - `.env` is mode 600 and in no unencrypted backup. - `docker compose ps` shows no published ports for `schulcloud-mcp` or `schulcloud-mcp-db`. +- The registry login in `~/.docker/config.json` is only base64-encoded. Keep the + Pi user's home private, and use an account that may only pull if your + registry can issue one. - The VPS forwards raw TCP and opens only 80, 443, 51820/udp and SSH. - The secret path stays out of Caddy's access log — this prints a count, never the secret: diff --git a/package.json b/package.json index 53bd0cf..f06cb05 100644 --- a/package.json +++ b/package.json @@ -22,6 +22,7 @@ "smoke": "node --env-file=.env scripts/smoke.mjs", "session-diagnose": "node --env-file=.env scripts/session-diagnose.mjs", "keepalive-status": "bash scripts/keepalive-status.sh", + "publish-image": "bash scripts/publish-image.sh", "cli": "node dist/bin/cli.js" }, "dependencies": { diff --git a/scripts/publish-image.sh b/scripts/publish-image.sh new file mode 100755 index 0000000..7b8c408 --- /dev/null +++ b/scripts/publish-image.sh @@ -0,0 +1,48 @@ +#!/usr/bin/env bash +# Builds the server image for the Pi (arm64) and x86 hosts (amd64) and pushes it +# to the registry, tagged `latest` and with the commit it was built from. +# +# The Pi never builds: deploy/docker-compose.pi.yml pulls what this pushes. +# Only a clean working tree is built, so a commit tag names exactly the code in +# that commit — a tag built from uncommitted edits would name nothing. +# +# npm run publish-image # registry.mc02.dev/schulcloud-mcp +# IMAGE=registry.example/other npm run publish-image +set -euo pipefail + +IMAGE=${IMAGE:-registry.mc02.dev/schulcloud-mcp} +PLATFORMS=${PLATFORMS:-linux/amd64,linux/arm64} + +cd "$(dirname "$0")/.." + +if [ -n "$(git status --porcelain)" ]; then + echo "The working tree has uncommitted changes. Commit them first: the image's tag must name exactly one commit." >&2 + exit 1 +fi + +# An x86 machine builds arm64 under QEMU emulation, which has to be registered +# once per boot. Registering needs a privileged container, so say how rather +# than doing it unasked. +if [[ "$PLATFORMS" == *linux/arm64* ]] && ! docker buildx inspect | grep -q 'linux/arm64'; then + echo "This builder cannot build linux/arm64. Register emulation (lasts until the next reboot) with:" >&2 + echo " docker run --privileged --rm tonistiigi/binfmt --install arm64" >&2 + exit 1 +fi + +revision=$(git rev-parse HEAD) +tag=$(git rev-parse --short HEAD) + +# One multi-platform push needs either Docker's containerd image store or a +# docker-container builder (`docker buildx create --use`); buildx says which +# is missing if neither is there. +docker buildx build \ + --platform "$PLATFORMS" \ + --tag "$IMAGE:latest" \ + --tag "$IMAGE:$tag" \ + --label org.opencontainers.image.title=schulcloud-mcp \ + --label org.opencontainers.image.source=https://git.mc02.dev/fabi/Schulcloud-MCP \ + --label "org.opencontainers.image.revision=$revision" \ + --label "org.opencontainers.image.created=$(date -u +%Y-%m-%dT%H:%M:%SZ)" \ + --push . + +echo "Pushed $IMAGE:$tag and $IMAGE:latest for $PLATFORMS."