From 05ea29fbd0429f3d932367e05c35950e09f7a1d9 Mon Sep 17 00:00:00 2001 From: MechaCat02 Date: Sat, 20 Jun 2026 13:54:23 +0200 Subject: [PATCH] docs: add comprehensive developer guide (mdBook) Add a browsable mdBook developer guide under docs/dev-guide/ covering the full PiCloud v1.1.9 surface for developers building on the platform: - Guide: introduction, 10-minute quickstart, core concepts, writing scripts - Tutorials: 5 end-to-end example apps (URL shortener, webhook receiver, TODO API with auth, scheduled report, file-upload service) + feature matrix - SDK reference: kv/docs/files, pubsub/queue, http, email, users, secrets, invoke/retry/dead_letters, stdlib, ctx & events - HTTP API reference: every /api/v1 endpoint, auth, and capability - CLI reference: the `pic` client + the server admin recovery command - Config: every PICLOUD_* env var, capabilities & roles - Deployment: Docker Compose, bare binary, Caddy/TLS, production checklist - Operations: security, best practices, troubleshooting Every example was built and run against a live instance; SDK signatures, endpoint paths, and env vars were verified against source. Build the site with `mdbook serve docs/dev-guide`. A pointer is added to README. Co-Authored-By: Claude Opus 4.8 --- README.md | 14 ++ docs/dev-guide/.gitignore | 2 + docs/dev-guide/book.toml | 23 ++ docs/dev-guide/src/SUMMARY.md | 66 ++++++ docs/dev-guide/src/deploy/bare-binary.md | 82 +++++++ docs/dev-guide/src/deploy/caddy-tls.md | 71 ++++++ docs/dev-guide/src/deploy/docker-compose.md | 88 ++++++++ .../src/deploy/production-checklist.md | 77 +++++++ docs/dev-guide/src/examples/file-upload.md | 126 +++++++++++ docs/dev-guide/src/examples/index.md | 59 +++++ .../src/examples/scheduled-report.md | 124 +++++++++++ docs/dev-guide/src/examples/todo-api.md | 158 ++++++++++++++ docs/dev-guide/src/examples/url-shortener.md | 118 ++++++++++ .../src/examples/webhook-receiver.md | 148 +++++++++++++ docs/dev-guide/src/guide/concepts.md | 177 +++++++++++++++ docs/dev-guide/src/guide/quickstart.md | 170 +++++++++++++++ docs/dev-guide/src/guide/writing-scripts.md | 206 ++++++++++++++++++ docs/dev-guide/src/introduction.md | 124 +++++++++++ .../src/operations/best-practices.md | 99 +++++++++ docs/dev-guide/src/operations/security.md | 105 +++++++++ .../src/operations/troubleshooting.md | 106 +++++++++ docs/dev-guide/src/reference/cli/pic.md | 97 +++++++++ .../src/reference/cli/server-admin.md | 42 ++++ .../src/reference/config/capabilities.md | 73 +++++++ .../src/reference/config/env-vars.md | 151 +++++++++++++ .../src/reference/rest-api/access.md | 84 +++++++ .../src/reference/rest-api/app-users.md | 43 ++++ docs/dev-guide/src/reference/rest-api/apps.md | 62 ++++++ .../src/reference/rest-api/data-admin.md | 61 ++++++ .../src/reference/rest-api/overview.md | 106 +++++++++ .../src/reference/rest-api/queues.md | 46 ++++ .../src/reference/rest-api/scripts.md | 101 +++++++++ .../src/reference/rest-api/topics.md | 57 +++++ .../src/reference/rest-api/triggers.md | 75 +++++++ .../src/reference/sdk/composition.md | 91 ++++++++ .../src/reference/sdk/ctx-and-events.md | 106 +++++++++ docs/dev-guide/src/reference/sdk/email.md | 50 +++++ docs/dev-guide/src/reference/sdk/http.md | 77 +++++++ docs/dev-guide/src/reference/sdk/messaging.md | 73 +++++++ docs/dev-guide/src/reference/sdk/overview.md | 96 ++++++++ docs/dev-guide/src/reference/sdk/secrets.md | 51 +++++ docs/dev-guide/src/reference/sdk/stdlib.md | 100 +++++++++ docs/dev-guide/src/reference/sdk/storage.md | 117 ++++++++++ docs/dev-guide/src/reference/sdk/users.md | 87 ++++++++ 44 files changed, 3989 insertions(+) create mode 100644 docs/dev-guide/.gitignore create mode 100644 docs/dev-guide/book.toml create mode 100644 docs/dev-guide/src/SUMMARY.md create mode 100644 docs/dev-guide/src/deploy/bare-binary.md create mode 100644 docs/dev-guide/src/deploy/caddy-tls.md create mode 100644 docs/dev-guide/src/deploy/docker-compose.md create mode 100644 docs/dev-guide/src/deploy/production-checklist.md create mode 100644 docs/dev-guide/src/examples/file-upload.md create mode 100644 docs/dev-guide/src/examples/index.md create mode 100644 docs/dev-guide/src/examples/scheduled-report.md create mode 100644 docs/dev-guide/src/examples/todo-api.md create mode 100644 docs/dev-guide/src/examples/url-shortener.md create mode 100644 docs/dev-guide/src/examples/webhook-receiver.md create mode 100644 docs/dev-guide/src/guide/concepts.md create mode 100644 docs/dev-guide/src/guide/quickstart.md create mode 100644 docs/dev-guide/src/guide/writing-scripts.md create mode 100644 docs/dev-guide/src/introduction.md create mode 100644 docs/dev-guide/src/operations/best-practices.md create mode 100644 docs/dev-guide/src/operations/security.md create mode 100644 docs/dev-guide/src/operations/troubleshooting.md create mode 100644 docs/dev-guide/src/reference/cli/pic.md create mode 100644 docs/dev-guide/src/reference/cli/server-admin.md create mode 100644 docs/dev-guide/src/reference/config/capabilities.md create mode 100644 docs/dev-guide/src/reference/config/env-vars.md create mode 100644 docs/dev-guide/src/reference/rest-api/access.md create mode 100644 docs/dev-guide/src/reference/rest-api/app-users.md create mode 100644 docs/dev-guide/src/reference/rest-api/apps.md create mode 100644 docs/dev-guide/src/reference/rest-api/data-admin.md create mode 100644 docs/dev-guide/src/reference/rest-api/overview.md create mode 100644 docs/dev-guide/src/reference/rest-api/queues.md create mode 100644 docs/dev-guide/src/reference/rest-api/scripts.md create mode 100644 docs/dev-guide/src/reference/rest-api/topics.md create mode 100644 docs/dev-guide/src/reference/rest-api/triggers.md create mode 100644 docs/dev-guide/src/reference/sdk/composition.md create mode 100644 docs/dev-guide/src/reference/sdk/ctx-and-events.md create mode 100644 docs/dev-guide/src/reference/sdk/email.md create mode 100644 docs/dev-guide/src/reference/sdk/http.md create mode 100644 docs/dev-guide/src/reference/sdk/messaging.md create mode 100644 docs/dev-guide/src/reference/sdk/overview.md create mode 100644 docs/dev-guide/src/reference/sdk/secrets.md create mode 100644 docs/dev-guide/src/reference/sdk/stdlib.md create mode 100644 docs/dev-guide/src/reference/sdk/storage.md create mode 100644 docs/dev-guide/src/reference/sdk/users.md diff --git a/README.md b/README.md index 04efb8a..f6c19f6 100644 --- a/README.md +++ b/README.md @@ -28,6 +28,20 @@ cargo check --workspace cargo run -p picloud ``` +## Documentation + +The **Developer Guide** in [`docs/dev-guide/`](docs/dev-guide/) is the place to start using PiCloud — +quickstart, core concepts, full SDK / HTTP-API / CLI reference, five end-to-end example apps, and +deployment + security guides. Build and read it locally with [mdBook](https://rust-lang.github.io/mdBook/): + +```sh +cargo install mdbook # once +mdbook serve docs/dev-guide # then open http://localhost:3000 +``` + +Architecture and contributor notes live alongside it in [`docs/`](docs/) (`sdk-shape.md`, +`stdlib-reference.md`, `versioning.md`, …) and in [`serverless_cloud_blueprint.md`](serverless_cloud_blueprint.md). + ## Repository Layout ``` diff --git a/docs/dev-guide/.gitignore b/docs/dev-guide/.gitignore new file mode 100644 index 0000000..35a1c91 --- /dev/null +++ b/docs/dev-guide/.gitignore @@ -0,0 +1,2 @@ +# mdBook build output — generated by `mdbook build`, not source. +book/ diff --git a/docs/dev-guide/book.toml b/docs/dev-guide/book.toml new file mode 100644 index 0000000..e6b36cb --- /dev/null +++ b/docs/dev-guide/book.toml @@ -0,0 +1,23 @@ +[book] +title = "PiCloud Developer Guide" +description = "How to build and run serverless apps on PiCloud — write Rhai scripts, get HTTP endpoints." +authors = ["PiCloud"] +language = "en" +src = "src" + +[output.html] +default-theme = "navy" +preferred-dark-theme = "navy" +smart-punctuation = true +git-repository-url = "" +edit-url-template = "" + +[output.html.fold] +enable = true +level = 1 + +[output.html.search] +enable = true +limit-results = 30 +use-boolean-and = true +boost-title = 2 diff --git a/docs/dev-guide/src/SUMMARY.md b/docs/dev-guide/src/SUMMARY.md new file mode 100644 index 0000000..9b7a2ab --- /dev/null +++ b/docs/dev-guide/src/SUMMARY.md @@ -0,0 +1,66 @@ +# Summary + +[Introduction](introduction.md) + +# Guide + +- [Quickstart](guide/quickstart.md) +- [Core concepts](guide/concepts.md) +- [Writing scripts](guide/writing-scripts.md) + +# Tutorials + +- [Overview & feature matrix](examples/index.md) +- [URL shortener](examples/url-shortener.md) +- [Webhook receiver](examples/webhook-receiver.md) +- [TODO API with auth](examples/todo-api.md) +- [Scheduled report](examples/scheduled-report.md) +- [File-upload service](examples/file-upload.md) + +# SDK reference + +- [Overview](reference/sdk/overview.md) +- [The execution context & events](reference/sdk/ctx-and-events.md) +- [Storage: kv, docs, files](reference/sdk/storage.md) +- [Messaging: pubsub, queue](reference/sdk/messaging.md) +- [Outbound HTTP](reference/sdk/http.md) +- [Email](reference/sdk/email.md) +- [Users & auth](reference/sdk/users.md) +- [Secrets](reference/sdk/secrets.md) +- [Composition: invoke, retry, dead_letters](reference/sdk/composition.md) +- [Standard library](reference/sdk/stdlib.md) + +# HTTP API reference + +- [Overview & authentication](reference/rest-api/overview.md) +- [Apps & domains](reference/rest-api/apps.md) +- [Scripts, routes & logs](reference/rest-api/scripts.md) +- [Triggers](reference/rest-api/triggers.md) +- [Topics & realtime](reference/rest-api/topics.md) +- [Secrets, KV & files](reference/rest-api/data-admin.md) +- [Queues & dead-letters](reference/rest-api/queues.md) +- [App users](reference/rest-api/app-users.md) +- [Members, admins & API keys](reference/rest-api/access.md) + +# CLI reference + +- [The `pic` CLI](reference/cli/pic.md) +- [Server admin commands](reference/cli/server-admin.md) + +# Configuration + +- [Environment variables](reference/config/env-vars.md) +- [Capabilities & roles](reference/config/capabilities.md) + +# Deployment + +- [Docker Compose](deploy/docker-compose.md) +- [Running the bare binary](deploy/bare-binary.md) +- [Caddy & TLS](deploy/caddy-tls.md) +- [Production checklist](deploy/production-checklist.md) + +# Operations + +- [Security](operations/security.md) +- [Best practices](operations/best-practices.md) +- [Troubleshooting](operations/troubleshooting.md) diff --git a/docs/dev-guide/src/deploy/bare-binary.md b/docs/dev-guide/src/deploy/bare-binary.md new file mode 100644 index 0000000..738a1d5 --- /dev/null +++ b/docs/dev-guide/src/deploy/bare-binary.md @@ -0,0 +1,82 @@ +# Running the bare binary + +You can run `picloud` directly against a Postgres you provide — handy for development against the +source, debugging, or a setup where you manage the database and proxy yourself. + +## Prerequisites + +- **Rust 1.92+** (pinned in `rust-toolchain.toml`). +- **PostgreSQL 15+** reachable via `DATABASE_URL`. The easiest Postgres is the compose one: + `docker compose up -d postgres` (publishes `127.0.0.1:15432`). + +## Build and run + +```sh +cargo build -p picloud # or --release + +export DATABASE_URL="postgres://picloud:picloud@localhost:15432/picloud" +export PICLOUD_BIND="0.0.0.0:18080" # 8080 is a common conflict — pick a free port +export PICLOUD_PUBLIC_BASE_URL="http://localhost:18080" +# Dev master key (local only): +export PICLOUD_DEV_MODE=true +export PICLOUD_DEV_INSECURE_KEY="i-understand-this-is-insecure" +# Bootstrap admin (only needed on a fresh DB): +export PICLOUD_ADMIN_USERNAME="admin" +export PICLOUD_ADMIN_PASSWORD="change-me" + +cargo run -p picloud +``` + +It applies migrations, seeds the bootstrap admin + `hello` example on a fresh DB, and listens on +`PICLOUD_BIND`. Verify: + +```sh +curl localhost:18080/healthz # ok +curl localhost:18080/version +``` + +**Minimal requirements:** just `DATABASE_URL` and a master key — either `PICLOUD_SECRET_KEY` (base64 of +32 bytes) *or* the dev-mode pair above. `PICLOUD_DEV_MODE=true` **alone** aborts at startup; you must +also set `PICLOUD_DEV_INSECURE_KEY`. The bootstrap admin vars are only consulted when `admin_users` is +empty. Full list: [Environment variables](../reference/config/env-vars.md). + +## A real master key + +For anything beyond throwaway local use, generate a proper key instead of dev mode: + +```sh +export PICLOUD_SECRET_KEY="$(head -c 32 /dev/urandom | base64)" +# (then DON'T set PICLOUD_DEV_MODE / PICLOUD_DEV_INSECURE_KEY) +``` + +Keep this stable — see the [rotation caveat](../operations/security.md#secrets-and-the-master-key). + +## Running the dashboard in dev + +The SvelteKit dashboard has its own Vite dev server with hot reload: + +```sh +cd dashboard +npm install +npm run dev # serves on http://localhost:5173 +``` + +> **Port footgun.** Vite proxies `/api` and `/healthz` to **`http://127.0.0.1:18080`** by default — not +> 8080. So either run `picloud` on `18080` (as above), or point Vite elsewhere: +> `PICLOUD_API=http://localhost:9000 npm run dev`. The dashboard dev port is `5173` +> (`PICLOUD_DASHBOARD_PORT` to change). + +For a production-style static build, `npm run build` emits a static SPA (served by Caddy in the +compose stack). You usually don't need the Vite dev server unless you're hacking on the dashboard +itself — the [compose stack](docker-compose.md) already serves a built dashboard at `/admin`. + +## The `pic` CLI + +Build it alongside: `cargo build -p picloud-cli` → `target/debug/pic`. Point it at your instance: + +```sh +printf "$PICLOUD_ADMIN_PASSWORD" | pic login --url http://localhost:18080 --username admin --password-stdin +pic whoami +``` + +See the [CLI reference](../reference/cli/pic.md). diff --git a/docs/dev-guide/src/deploy/caddy-tls.md b/docs/dev-guide/src/deploy/caddy-tls.md new file mode 100644 index 0000000..fba85f8 --- /dev/null +++ b/docs/dev-guide/src/deploy/caddy-tls.md @@ -0,0 +1,71 @@ +# Caddy & TLS + +Caddy is the single entry point in front of PiCloud. The same topology serves dev and production — only +the upstream targets and TLS settings change. + +## Routing topology + +Caddy routes by path prefix (from `caddy/Caddyfile`): + +| Path | Goes to | +|---|---| +| `/healthz`, `/version` | picloud (liveness + version) | +| `/api/v1/admin/*` | picloud (control plane) | +| `/api/v1/execute/*` | picloud (execute-by-id) | +| `/api/*` (anything else) | **404** — reserved for future API versions | +| `/admin/*` | the dashboard SPA | +| **everything else** | picloud's **user-route matcher** — your scripts' routes | + +That final catch-all is what makes arbitrary user paths like `/hello` or `/r/abc123` work: Caddy +forwards them to picloud, which matches them against the [route table](../guide/concepts.md#routes-and-domains) +(or returns a JSON `404`). This is why your routes can't use the reserved prefixes `/api/`, `/admin/`, +`/healthz`, `/version` — Caddy would never forward them to the matcher. + +## Security headers & body limit + +The Caddyfile adds defense-in-depth headers and a hard body cap: + +- `X-Content-Type-Options: nosniff` and `Referrer-Policy: no-referrer` on **every** response. +- A strict CSP, `X-Frame-Options: DENY`, `Cache-Control: no-store`, and a locked-down + `Permissions-Policy` on the **dashboard** and **admin API**. +- A **12 MB request-body ceiling** at the proxy (just above the orchestrator's 10 MiB user-route read). +- **User-route responses get no CSP on purpose** — your scripts own their own response headers. The + `?` (set-if-missing) operator means a script's own header wins over the default. + +## Production: HTTPS with Let's Encrypt + +Layer the production overlay, which swaps in `caddy/Caddyfile.prod` and exposes 80/443: + +```sh +export PICLOUD_DOMAIN="picloud.example.com" +export PICLOUD_ADMIN_EMAIL="you@example.com" +export POSTGRES_PASSWORD="$(head -c 24 /dev/urandom | base64)" +export PICLOUD_SECRET_KEY="$(head -c 32 /dev/urandom | base64)" +export PICLOUD_ADMIN_USERNAME="admin" +export PICLOUD_ADMIN_PASSWORD="…" +export PICLOUD_PUBLIC_BASE_URL="https://picloud.example.com" + +docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d --build +``` + +Caddy obtains and renews a certificate for `PICLOUD_DOMAIN` automatically (HTTP/TLS-ALPN challenge — the +host must be reachable on 80/443 from the internet). The prod overlay also: + +- **removes the published Postgres port** (DB reachable only inside the compose network); +- adds **HSTS** to every response (on top of the dev headers); +- sets `restart: unless-stopped` on all services. + +The prod Caddyfile reads `PICLOUD_DOMAIN` and `PICLOUD_ADMIN_EMAIL` from the environment, so set them +before `up`. + +## Custom domains for your apps + +PiCloud's app [domains](../reference/rest-api/apps.md#domains) are matched by picloud *after* Caddy +forwards the request — they're independent of Caddy's own host config. For multi-tenant setups +(`{tenant}.example.com`), point a wildcard DNS record + Caddy cert at the box, then let each app claim +its host pattern. The most-specific app claim wins; unclaimed hosts get a `404 no app claims host`. + +## Adding a future API version + +When `/api/v2/...` ships, add a `handle /api/v2/admin/* { … }` block before the catch-all `/api/*` +404, mirroring the v1 block. The v1 routes stay live through the deprecation window. diff --git a/docs/dev-guide/src/deploy/docker-compose.md b/docs/dev-guide/src/deploy/docker-compose.md new file mode 100644 index 0000000..70be9ac --- /dev/null +++ b/docs/dev-guide/src/deploy/docker-compose.md @@ -0,0 +1,88 @@ +# Deploy with Docker Compose + +The default stack runs the whole system — Postgres, the `picloud` binary, the dashboard, and a Caddy +reverse proxy — behind one port. It's the recommended way to run PiCloud for local dev and single-node +production. + +## The stack + +`docker-compose.yml` defines four services: + +| Service | Role | Exposed | +|---|---|---| +| `postgres` | the database | host `127.0.0.1:15432` (dev convenience; removed in prod) | +| `picloud` | the all-in-one binary | internal `:8080` only | +| `dashboard` | the SvelteKit SPA | internal `:80` only | +| `caddy` | reverse proxy / entry point | host `:${PICLOUD_HOST_PORT:-8000}` → `:80` | + +Only Caddy is published. It routes by path: `/healthz`, `/version`, `/api/v1/admin/*`, +`/api/v1/execute/*` → picloud; `/admin/*` → dashboard; everything else → picloud's user-route matcher. +See [Caddy & TLS](caddy-tls.md). + +## First run + +```sh +cp .env.example .env +# edit .env — at minimum set PICLOUD_ADMIN_USERNAME and PICLOUD_ADMIN_PASSWORD +docker compose up -d --build +``` + +On first boot picloud runs migrations, seeds the bootstrap admin from `PICLOUD_ADMIN_USERNAME` / +`PICLOUD_ADMIN_PASSWORD` (as instance `owner`), and seeds a `hello` example into the `default` app. +Then: + +```sh +curl localhost:8000/healthz # ok +curl localhost:8000/version # {"product":"1.1.9","sdk":"1.10",...} +curl localhost:8000/hello # {"message":"Hello, world!"} +open http://localhost:8000/admin +``` + +> **The compose file makes `PICLOUD_ADMIN_USERNAME`/`PICLOUD_ADMIN_PASSWORD` mandatory** (it uses +> `${VAR:?…}`), so `docker compose up` errors out if they're unset — even though the binary itself only +> needs them on a fresh DB. This is intentional: it stops you from shipping with no admin. + +## The dev `.env` + +The shipped `.env.example` runs in **dev mode**: it sets `PICLOUD_DEV_MODE=true`, and +`docker-compose.override.yml` (gitignored, applied automatically) supplies +`PICLOUD_DEV_INSECURE_KEY=i-understand-this-is-insecure` so the stack boots with a deterministic, +world-known master key. **This is for local dev only** — at-rest encryption is meaningless with a public +key. For anything real, set a true `PICLOUD_SECRET_KEY` and don't use dev mode (see +[Production checklist](production-checklist.md)). + +Key `.env` settings: + +| Variable | Purpose | +|---|---| +| `PICLOUD_HOST_PORT` | host port Caddy listens on (default 8000) | +| `POSTGRES_PASSWORD` | DB password — **change for prod** | +| `PICLOUD_POSTGRES_HOST_PORT` | host port for Postgres (dev only; default 15432) | +| `PICLOUD_PUBLIC_BASE_URL` | the URL users actually reach (rendered in the dashboard, `/version`) | +| `PICLOUD_ADMIN_USERNAME` / `PICLOUD_ADMIN_PASSWORD` | bootstrap admin | +| `RUST_LOG` | log verbosity | + +The full variable list is in [Environment variables](../reference/config/env-vars.md). + +## Everyday operations + +```sh +docker compose ps # service health +docker compose logs -f picloud # tail the server +docker compose exec postgres psql -U picloud picloud # poke the DB (dev) +docker compose down # stop (keeps data) +docker compose down -v # stop and WIPE Postgres + Caddy volumes +docker compose up -d --build # rebuild after a code change +``` + +Data lives in the `postgres_data` volume (and, for `files`, inside the picloud container's +`PICLOUD_FILES_ROOT` — mount a volume there if you use `files` in production). Recover a locked-out +admin with [`picloud admin reset-password`](../reference/cli/server-admin.md): + +```sh +docker compose exec picloud picloud admin reset-password admin +``` + +For production (real domain, HTTPS, no published DB port), layer the prod overlay — see +[Caddy & TLS](caddy-tls.md) and the [Production checklist](production-checklist.md). To run the binary +without Docker, see [Running the bare binary](bare-binary.md). diff --git a/docs/dev-guide/src/deploy/production-checklist.md b/docs/dev-guide/src/deploy/production-checklist.md new file mode 100644 index 0000000..360c126 --- /dev/null +++ b/docs/dev-guide/src/deploy/production-checklist.md @@ -0,0 +1,77 @@ +# Production checklist + +Before exposing PiCloud to real traffic, walk this list. Items marked **critical** can compromise the +whole instance if skipped. + +## Secrets & keys + +- [ ] **Critical: set a real `PICLOUD_SECRET_KEY`** (base64 of 32 random bytes) and **do not** use + `PICLOUD_DEV_MODE` / `PICLOUD_DEV_INSECURE_KEY`. The dev key is world-known — every `secrets` + value and realtime key would be "encrypted" with a public value. + ```sh + export PICLOUD_SECRET_KEY="$(head -c 32 /dev/urandom | base64)" + ``` +- [ ] **Store the key in a secret manager**, not in a committed `.env`. If you lose it, every encrypted + secret becomes undecryptable. **Rotating it orphans existing ciphertext** — there's no auto + re-encryption; rotate deliberately and re-`set` your secrets. See + [Security → master key](../operations/security.md#secrets-and-the-master-key). +- [ ] **Critical: change `POSTGRES_PASSWORD`** from the dev default. +- [ ] Prefer `PICLOUD_ADMIN_PASSWORD_HASH` (a pre-computed Argon2id PHC string) over a raw + `PICLOUD_ADMIN_PASSWORD`, so the plaintext never lands in env/compose. + +## Network & TLS + +- [ ] Deploy with the **prod overlay** for automatic HTTPS: + `docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d`. See + [Caddy & TLS](caddy-tls.md). +- [ ] Set `PICLOUD_DOMAIN`, `PICLOUD_ADMIN_EMAIL`, and `PICLOUD_PUBLIC_BASE_URL` (to your `https://` + origin). +- [ ] Confirm **Postgres is not publicly exposed** (the prod overlay removes the host port mapping). +- [ ] **Critical: do not set `PICLOUD_HTTP_ALLOW_PRIVATE`** — leaving the SSRF guard on stops scripts + probing your internal network via `http::*`. + +## Data & durability + +- [ ] Back up the **Postgres volume** *and* the **`files` blob storage** (`PICLOUD_FILES_ROOT`) + together — file metadata and bytes live in different places. +- [ ] Mount a persistent volume for `PICLOUD_FILES_ROOT` if you use the `files` SDK (otherwise blobs + live inside the container and vanish on `down`). +- [ ] Set retention to taste: `PICLOUD_DEAD_LETTER_RETENTION_DAYS`, + `PICLOUD_ABANDONED_EXECUTIONS_RETENTION_DAYS`. + +## Capacity & limits + +- [ ] Tune `PICLOUD_MAX_CONCURRENT_EXECUTIONS` and `PICLOUD_DB_MAX_CONNECTIONS` together to your + hardware. Past the concurrency cap, data-plane requests get `503 Retry-After: 1` — make sure + clients handle it. +- [ ] Review the [sandbox ceilings](../reference/config/env-vars.md#sandbox-ceilings) + (`PICLOUD_SANDBOX_MAX_*`) — the defaults are conservative; raise only deliberately. +- [ ] Review the [size caps](../reference/config/env-vars.md#data-plane-size-caps) for `kv`, `docs`, + `queue`, `pubsub`, `files`. + +## Email + +- [ ] Configure `PICLOUD_SMTP_*` if any script (or the `users` flows) sends mail — otherwise + `email::send` throws `NotConfigured` (the dev sink does **not** exist outside dev mode). + +## Application hygiene + +- [ ] Remember the [data plane is public](../operations/security.md#the-data-plane-is-public): every + route runs with full app authority. Audit each public route — does it expose data it shouldn't? +- [ ] Put auth (`users::verify`, a shared secret, etc.) on every route that needs it; the platform + won't. +- [ ] Scope **API keys** narrowly (least-privilege scopes, bind to one app, set `expires_at`). +- [ ] Use **per-app members** with the lowest role that works (`viewer`/`editor`/`app_admin`) rather + than handing out instance `admin`. +- [ ] Prefer **`async` dispatch** for slow or fire-and-forget work so the request path stays fast. + +## Observability + +- [ ] Set `RUST_LOG` sensibly (`info,picloud=info` in prod). +- [ ] Alert on the unresolved **dead-letter** count (`pic dead-letters count --app …` / + `GET …/dead_letters/count`). +- [ ] Remember that `log::` output is captured for **async/triggered** runs, not synchronous HTTP runs + ([why](../guide/writing-scripts.md#logging)). + +When all boxes are checked, you're running on a real key, behind HTTPS, with a private database, +backed up, and with auth on the routes that need it. diff --git a/docs/dev-guide/src/examples/file-upload.md b/docs/dev-guide/src/examples/file-upload.md new file mode 100644 index 0000000..4e28192 --- /dev/null +++ b/docs/dev-guide/src/examples/file-upload.md @@ -0,0 +1,126 @@ +# Tutorial: File-upload service + +Accept file uploads, store them as blobs, announce each upload on a [pubsub](../reference/sdk/messaging.md#pubsub) +topic, and stream those announcements to a browser over [SSE](../reference/rest-api/topics.md). This +exercises [`files`](../reference/sdk/storage.md#files), [`base64`](../reference/sdk/stdlib.md#base64), +`pubsub::publish_durable`, and a realtime topic — and surfaces two real platform constraints about +binary I/O. + +> **Two things to know up front (both verified below):** +> 1. **There are no raw request bytes** — `ctx.request.body` is JSON. So uploads arrive as +> **base64 inside a JSON field**, which the script decodes. +> 2. **Script responses are always JSON** — a script *cannot* stream raw bytes back through its +> response (a blob body serializes to a hex JSON string). So downloads either return **base64 in +> JSON** (shown here) or use the **admin files endpoint**, which serves true raw bytes. + +## 1. App, host, topic + +```sh +pic apps create vault --name "File Vault" +pic apps domains add vault vault.localhost +# register a public, externally-subscribable topic (note: --app is a flag, name is positional) +pic topics create --app vault file.uploaded --external --auth-mode public +``` + +## 2. Upload + +`upload.rhai` — decode the base64 body, store the blob, publish: + +```rhai +let b = ctx.request.body; +if type_of(b) != "map" || !b.contains("b64") { + return #{ statusCode: 400, body: #{ error: "expected JSON { name, content_type, b64 }" } }; +} +let bytes = base64::decode(b.b64); // Blob +let id = files::collection("uploads").create(#{ + name: b.name, content_type: b.content_type, data: bytes +}); +pubsub::publish_durable("file.uploaded", #{ id: id, name: b.name, size: bytes.len() }); +return #{ statusCode: 201, body: #{ id: id, size: bytes.len() } }; +``` + +## 3. Download (base64 in JSON) + +`download.rhai`: + +```rhai +let id = ctx.request.params.id; +let meta = files::collection("uploads").head(id); // metadata, () if missing +if meta == () { return #{ statusCode: 404, body: #{ error: "not found" } }; } +let bytes = files::collection("uploads").get(id); // the Blob +return #{ statusCode: 200, body: #{ + name: meta.name, content_type: meta.content_type, b64: base64::encode(bytes) +} }; +``` + +## 4. Deploy and route + +```sh +pic scripts deploy upload.rhai --app vault --name upload +pic scripts deploy download.rhai --app vault --name download +UP=$(pic --output json scripts ls --app vault | jq -r '.[]|select(.name=="upload").id') +DL=$(pic --output json scripts ls --app vault | jq -r '.[]|select(.name=="download").id') +pic routes create --script $UP --path /files --method POST +pic routes create --script $DL --path '/files/:id/data' --path-kind param --method GET +``` + +## 5. Upload and download + +```sh +H='Host: vault.localhost' +B64=$(printf 'hello pics' | base64) # aGVsbG8gcGljcw== + +curl -s -X POST http://localhost:18080/files -H "$H" -H 'Content-Type: application/json' \ + -d "{\"name\":\"greeting.txt\",\"content_type\":\"text/plain\",\"b64\":\"$B64\"}" +# {"id":"98698d33-…","size":10} + +curl -s http://localhost:18080/files//data -H "$H" +# {"b64":"aGVsbG8gcGljcw==","content_type":"text/plain","name":"greeting.txt"} +# → base64-decode "b64" on the client to recover the bytes +``` + +Need true raw bytes (e.g. to `` an upload)? Use the authenticated admin endpoint, which sets +`Content-Type`/`Content-Disposition` and streams the bytes: + +```sh +curl http://localhost:18080/api/v1/admin/apps/$(pic --output json apps show vault | jq -r .id)/files/uploads/ \ + -H "Authorization: Bearer $TOKEN" -o greeting.txt +``` + +…or the CLI: `pic files get --app vault --collection uploads --id --out greeting.txt`. + +## 6. Watch uploads live (SSE) + +Subscribe to the topic; every upload pushes an event: + +```sh +curl -N http://localhost:18080/realtime/topics/file.uploaded -H 'Host: vault.localhost' +``` + +Upload another file from a second terminal and the subscriber prints: + +```text +data: {"message":{"id":"1313221c-…","name":"c.txt","size":10}, + "published_at":"2026-…Z","topic":"file.uploaded"} +``` + +The published payload is under `message`. In a browser that's +`new EventSource('https://your-host/realtime/topics/file.uploaded')`. For non-public topics, mint a +subscriber token (`pubsub::subscriber_token`) or use an app-user session — see +[Topics & realtime](../reference/rest-api/topics.md). + +## Notes, constraints & next steps + +- **Size caps.** Per-file blobs cap at `PICLOUD_FILES_MAX_FILE_SIZE_BYTES` (100 MiB), but the upload + also passes through the request-body limit (~10 MiB) **and** base64 inflates by ~33%. For large + files you'll want chunking or a different ingest path; this base64-in-JSON pattern suits modest + files (avatars, attachments, docs). +- **Blobs in / metadata out.** `files::...head(id)` returns metadata only (name, content_type, size, + checksum) — cheap; `...get(id)` returns the bytes. +- **Storage location.** Bytes live on disk under `PICLOUD_FILES_ROOT`; metadata in Postgres. Back both + up together. +- **`files` triggers.** A blob create/update/delete can fire a [`files` trigger](../reference/rest-api/triggers.md) + (`ctx.event.files`, metadata only) — e.g. to generate a thumbnail or scan an upload. + +Next: validate `content_type` and reject disallowed types; generate thumbnails in a `files`-trigger +consumer; or gate uploads behind the [auth pattern from the TODO tutorial](todo-api.md). diff --git a/docs/dev-guide/src/examples/index.md b/docs/dev-guide/src/examples/index.md new file mode 100644 index 0000000..b034a3c --- /dev/null +++ b/docs/dev-guide/src/examples/index.md @@ -0,0 +1,59 @@ +# Tutorials — overview & feature matrix + +Five complete, copy-pasteable example apps. Each was built and run against a real instance while +writing this guide — the commands and outputs are what actually happened. Work through them in any +order; together they touch most of the platform. + +Every tutorial assumes the [Quickstart](../guide/quickstart.md) stack is running and you're logged in +with `pic`. Each creates its own [app](../guide/concepts.md#apps) and claims a `*.localhost` host, so +you address it with a `Host:` header (`curl -H 'Host: links.localhost' …`). + +> **A note on ports.** These tutorials use `http://localhost:18080` (a bare binary — see +> [Running the bare binary](../deploy/bare-binary.md)). If you followed the Quickstart's **Docker +> Compose** stack, substitute **`8000`** for `18080` everywhere (including the `short_url` the URL +> shortener builds). Same behavior, different port. + +## Which tutorial shows which feature + +| | [URL shortener](url-shortener.md) | [Webhook receiver](webhook-receiver.md) | [TODO API](todo-api.md) | [Scheduled report](scheduled-report.md) | [File upload](file-upload.md) | +|---|:--:|:--:|:--:|:--:|:--:| +| `kv` | ● | ● | | ● | | +| `docs` | | | ● | | | +| `files` | | | | | ● | +| `secrets` | | ● | | | | +| `http` (outbound) | | ● | | | | +| `email` | | | | ● | | +| `users` (auth) | | | ● | | | +| `pubsub` + SSE | | | | | ● | +| `queue` | | ● | | | | +| `invoke`/`retry` | | ● | | | | +| dead-letters | | ● | | | | +| stdlib (`random`/`base64`/`time`) | ● | | | ● | ● | +| route params (`:id`) | ● | | ● | | ● | +| async dispatch (`202`) | | ● | | | | +| cron trigger | | | | ● | | +| queue trigger | | ● | | | | +| modules (`import`) | | | | ● | | + +## The five apps + +1. **[URL shortener](url-shortener.md)** — `POST /shorten` + `GET /r/:code`. The gentlest start: KV, + a param route, a random code, a `302` redirect. *(~10 min.)* +2. **[Webhook receiver](webhook-receiver.md)** — authenticate with a shared secret, accept with `202`, + process on a durable queue with retries, forward over HTTP, handle failures as dead-letters. The + most feature-dense tutorial. *(~25 min.)* +3. **[TODO API with auth](todo-api.md)** — real multi-user signup/login built from the `users` SDK, + per-user data in `docs`. Shows that PiCloud has no built-in auth endpoints — you compose them. + *(~20 min.)* +4. **[Scheduled report](scheduled-report.md)** — a cron trigger that aggregates data, formats it via an + imported module, and emails a summary (verified against the dev mail sink). *(~15 min.)* +5. **[File-upload service](file-upload.md)** — store blobs, announce uploads on a pubsub topic, stream + them to a browser via SSE. Clears up how binary I/O works (base64-in/JSON-out). *(~20 min.)* + +## Patterns they share + +- **One app per project**, each claiming its own host — that's the isolation boundary. +- **Deploy from a file** with `pic scripts deploy file.rhai --app `, then bind routes. +- **Inspect with the admin side** — `pic kv get`, `pic logs`, `pic dead-letters ls`, the dashboard. +- **Read the "Notes, constraints & next steps"** box at the end of each — that's where the real-world + caveats and security gotchas live. diff --git a/docs/dev-guide/src/examples/scheduled-report.md b/docs/dev-guide/src/examples/scheduled-report.md new file mode 100644 index 0000000..6ea91fb --- /dev/null +++ b/docs/dev-guide/src/examples/scheduled-report.md @@ -0,0 +1,124 @@ +# Tutorial: Scheduled report + +Run a job on a schedule that aggregates data and emails a summary. This exercises a +[`cron` trigger](../reference/rest-api/triggers.md), a reusable [`module`](../guide/writing-scripts.md#modules-and-imports) +script imported with `import`, [`kv`](../reference/sdk/storage.md#kv) reads, and +[`email::send_html`](../reference/sdk/email.md) — verified here against the **dev email sink** so you +need no real SMTP server. + +## 1. App + +```sh +pic apps create reports --name "Scheduled Reports" +``` + +(No domain needed — a cron-triggered script isn't reached over HTTP.) + +## 2. A shared formatting module + +`report_fmt.rhai` (`kind: module` — only `fn`/`const`, no top-level statements): + +```rhai +fn summary(n) { + `Signups so far: ${n}` +} +``` + +## 3. The report worker + +`reporter.rhai` reads a metric, formats it via the module, and emails it. It handles both a real cron +firing (`ctx.event.cron`) and a manual test run (no event): + +```rhai +import "report_fmt" as fmt; + +let hits = kv::collection("metrics").get("signups"); +if hits == () { hits = 0; } + +let when = if "event" in ctx && ctx.event.source == "cron" { + ctx.event.cron.scheduled_at // RFC 3339 string of the scheduled tick +} else { + time::now() +}; + +let line = fmt::summary(hits); +email::send_html(#{ + to: "ops@example.com", + from: "reports@reports.app", + subject: `Report @ ${when}`, + text: line, // plain-text fallback + html: `

${line}

` +}); +log::info("report sent", #{ when: when, line: line }); +return #{ statusCode: 200, body: #{ sent: true, line: line } }; +``` + +## 4. Deploy + +```sh +pic scripts deploy report_fmt.rhai --app reports --kind module --name report_fmt +pic scripts deploy reporter.rhai --app reports --name reporter +RP=$(pic --output json scripts ls --app reports | jq -r '.[]|select(.name=="reporter").id') +``` + +## 5. Test it once, by hand + +You can run any script directly by id (no route needed) with `execute`. There's no cron event this way, +so the worker falls back to `time::now()`: + +```sh +curl -s -X POST http://localhost:18080/api/v1/execute/$RP -H 'Content-Type: application/json' -d '{}' +# {"line":"Signups so far: 0","sent":true} +``` + +Because the dev stack has no SMTP relay, the mail went to the **in-memory dev sink**. Read it back +(instance owner/admin only; this route exists only in dev mode): + +```sh +curl -s http://localhost:18080/api/v1/admin/dev/emails -H "Authorization: Bearer $TOKEN" +``` +```json +[{"captured_at":"2026-…Z","from":"reports@reports.app","to":["ops@example.com"], + "raw":"From: reports@reports.app\r\nSubject: Report @ 2026-…Z\r\n…multipart/alternative…"}] +``` + +## 6. Put it on a schedule + +Cron expressions are **6 fields (with seconds)**. This fires every 10 seconds — handy for a demo: + +```sh +pic triggers create-cron --app reports --script $RP --schedule '*/10 * * * * *' +``` + +Within ~10 seconds it fires; confirm via the execution log (source `cron`), where — because cron runs +are asynchronous — the `log::` output is captured: + +```sh +pic logs $RP --source cron --limit 3 +# … status=success source=cron … (the "report sent" log entry is recorded) +``` + +Then replace it with a real schedule and timezone, e.g. 8 AM daily in Berlin, and remove the demo one: + +```sh +pic triggers create-cron --app reports --script $RP --schedule '0 0 8 * * *' --timezone Europe/Berlin +pic triggers ls --app reports +pic triggers rm --app reports +``` + +## Notes, constraints & next steps + +- **Module fns don't see `ctx`.** An imported module gets no access to the caller's scope — pass what + it needs as arguments (here, `fmt::summary(hits)`). Module fns *can* call SDK namespaces. +- **Cron resolution is seconds, 6 fields.** `*/10 * * * * *` = every 10s; `0 0 8 * * *` = 08:00:00 + daily. Set `--timezone` (IANA name) or it runs in UTC. +- **Real SMTP.** Configure `PICLOUD_SMTP_*` ([env vars](../reference/config/env-vars.md)) for actual + delivery; without it (outside dev mode) `email::send_html` throws `NotConfigured`. +- **Where does the metric come from?** Here it's a `kv` key you'd increment elsewhere in your app (the + [quickstart counter](../guide/quickstart.md#step-5-write-your-first-script) pattern). A report often + aggregates [`docs`](../reference/sdk/storage.md#docs) instead — `docs.find(...)` then fold. +- **Don't over-fire.** Each tick is a full execution counted against your concurrency budget; a + 10-second cron in production is rarely what you want. + +Next: have the report run per user (loop `users::list`, email each), or push results to a dashboard +over [pubsub + SSE](file-upload.md) instead of email. diff --git a/docs/dev-guide/src/examples/todo-api.md b/docs/dev-guide/src/examples/todo-api.md new file mode 100644 index 0000000..dfb226d --- /dev/null +++ b/docs/dev-guide/src/examples/todo-api.md @@ -0,0 +1,158 @@ +# Tutorial: TODO API with auth + +Build a multi-user TODO API where each user signs up, logs in, and sees only their own items. This is +the tutorial that shows how **app-user authentication** actually works in PiCloud: there is no built-in +`/signup` or `/login` — you compose them from the [`users`](../reference/sdk/users.md) SDK and bind +them to your own routes. Data lives in [`docs`](../reference/sdk/storage.md#docs). + +> **The key idea:** PiCloud gives you `users::create`, `users::login`, `users::verify` — the +> *primitives*. The endpoints (`/auth/signup`, the bearer-token check on protected routes) are yours to +> write. This is more work than a turnkey auth box, but it means auth is *your* code with no hidden +> behavior. See [Concepts → authentication](../guide/concepts.md#authentication-three-kinds-of-identity). + +## 1. App and host + +```sh +pic apps create todo --name "TODO API" +pic apps domains add todo todo.localhost +``` + +## 2. Signup + +`signup.rhai` — create the user, then log them straight in: + +```rhai +let b = ctx.request.body; +if type_of(b) != "map" || !b.contains("email") || !b.contains("password") { + return #{ statusCode: 400, body: #{ error: "email and password required" } }; +} +if !users::email_available(b.email) { // anonymous-safe pre-check + return #{ statusCode: 409, body: #{ error: "email already registered" } }; +} +let display = if b.contains("display_name") { b.display_name } else { () }; +users::create(#{ email: b.email, password: b.password, display_name: display }); +let token = users::login(b.email, b.password); // mint a session token +return #{ statusCode: 201, body: #{ token: token } }; +``` + +## 3. Login + +`login.rhai`: + +```rhai +let b = ctx.request.body; +let token = users::login(b.email, b.password); // () on bad credentials +if token == () { return #{ statusCode: 401, body: #{ error: "invalid credentials" } }; } +return #{ statusCode: 200, body: #{ token: token } }; +``` + +## 4. The TODO endpoints (one script, branches on method) + +A single `todos.rhai` handles `POST /todos`, `GET /todos`, and `DELETE /todos/:id`. Each begins by +turning the `Authorization: Bearer …` header into a user with `users::verify`: + +```rhai +// --- authenticate --- +let hdr = if "authorization" in ctx.request.headers { ctx.request.headers["authorization"] } else { "" }; +let token = if hdr.starts_with("Bearer ") { hdr.sub_string(7) } else { hdr }; +let me = users::verify(token); // () if invalid/expired +if me == () { return #{ statusCode: 401, body: #{ error: "unauthorized" } }; } + +// --- handle the request --- +let todos = docs::collection("todos"); +let method = ctx.request.method; + +if method == "POST" { + let id = todos.create(#{ owner: me.id, text: ctx.request.body.text, done: false }); + return #{ statusCode: 201, body: #{ id: id } }; +} +if method == "GET" { + let mine = todos.find(#{ owner: me.id }); // only this user's docs + return #{ statusCode: 200, body: #{ todos: mine } }; +} +if method == "DELETE" { + let id = ctx.request.params.id; + let row = todos.get(id); + // 404 if missing OR not owned by the caller — never reveal another user's item + if row == () || row.data.owner != me.id { return #{ statusCode: 404, body: #{ error: "not found" } }; } + todos.delete(id); + return #{ statusCode: 204, body: () }; +} +return #{ statusCode: 405, body: #{ error: "method not allowed" } }; +``` + +## 5. Deploy and bind + +```sh +pic scripts deploy signup.rhai --app todo --name signup +pic scripts deploy login.rhai --app todo --name login +pic scripts deploy todos.rhai --app todo --name todos + +SU=$(pic --output json scripts ls --app todo | jq -r '.[]|select(.name=="signup").id') +LI=$(pic --output json scripts ls --app todo | jq -r '.[]|select(.name=="login").id') +TD=$(pic --output json scripts ls --app todo | jq -r '.[]|select(.name=="todos").id') + +pic routes create --script $SU --path /auth/signup --method POST +pic routes create --script $LI --path /auth/login --method POST +pic routes create --script $TD --path /todos --method POST +pic routes create --script $TD --path /todos --method GET +pic routes create --script $TD --path '/todos/:id' --path-kind param --method DELETE +``` + +(Binding several routes to one script is normal — the script branches on `ctx.request.method`.) + +## 6. Use it + +```sh +H='Host: todo.localhost' +# sign up → get a token +curl -s -X POST http://localhost:18080/auth/signup -H "$H" -H 'Content-Type: application/json' \ + -d '{"email":"ada@example.com","password":"lovelace99","display_name":"Ada"}' +# {"token":"JCMgEH6i_DLtNggsEE9KspGNaUHIYvBcV0p4jG9fKZ8"} + +TOK=… # paste the token + +# unauthenticated request is rejected +curl -s -o /dev/null -w '%{http_code}\n' http://localhost:18080/todos -H "$H" # 401 + +# create a couple of todos +curl -s -X POST http://localhost:18080/todos -H "$H" -H "Authorization: Bearer $TOK" \ + -H 'Content-Type: application/json' -d '{"text":"write docs"}' +# {"id":"391bc9df-…"} + +# list them — each is a docs envelope (your fields under `data`) +curl -s http://localhost:18080/todos -H "$H" -H "Authorization: Bearer $TOK" +``` +```json +{"todos":[ + {"id":"0816b607-…","data":{"done":false,"owner":"8c7716bc-…","text":"ship it"}, + "created_at":"2026-…Z","updated_at":"2026-…Z"}, + {"id":"391bc9df-…","data":{"done":false,"owner":"8c7716bc-…","text":"write docs"}, + "created_at":"2026-…Z","updated_at":"2026-…Z"} +]} +``` + +Operators can see registered users (but never their passwords) from the admin side: + +```sh +pic users ls --app todo +# id … email ada@example.com … display_name Ada … last_login_at … created_at … +``` + +## Notes, constraints & next steps + +- **`find_by_email` is privileged.** Anonymous (public) scripts can't call it — it's an + anti-enumeration guard. `email_available` *is* anonymous-safe (a signup form needs it) but isn't + throttled, so rate-limit it if abuse matters. [Security →](../operations/security.md#user-enumeration) +- **Ownership checks are yours.** The platform scopes data to the *app*, not to a *user*. "Only my + todos" is enforced by the `owner` field + the `row.data.owner != me.id` check — there's no automatic + per-user row security. Get this right on every protected handler. +- **`docs.update` replaces `data` wholesale.** To toggle `done`, read the doc, set the field, and + `update` with the full map. +- **Sessions slide.** `users::verify` extends the session TTL on each call + (`PICLOUD_APP_USER_SESSION_TTL_HOURS`). Use `users::logout(token)` to end one. +- **Add roles** with `users::add_role(me.id, "pro")` and gate features on `users::has_role(...)`. + +Next: add email verification (`users::send_verification_email` + a `/auth/verify` route) and password +reset — the [users SDK reference](../reference/sdk/users.md#email-tied-flows) lists the flow. The +[scheduled-report tutorial](scheduled-report.md) shows how to email users on a cron. diff --git a/docs/dev-guide/src/examples/url-shortener.md b/docs/dev-guide/src/examples/url-shortener.md new file mode 100644 index 0000000..664224f --- /dev/null +++ b/docs/dev-guide/src/examples/url-shortener.md @@ -0,0 +1,118 @@ +# Tutorial: URL shortener + +Build a classic link shortener: `POST /shorten` stores a long URL under a random code; `GET /r/:code` +redirects to it. You'll use [`kv`](../reference/sdk/storage.md#kv) storage, a +[`:param` route](../guide/concepts.md#routes-and-domains), the [`random`](../reference/sdk/stdlib.md#random) +stdlib, and a `302` [response envelope](../guide/writing-scripts.md#the-response-envelope-in-detail). + +**Prereqs:** the [Quickstart](../guide/quickstart.md) stack running, and you're logged in with `pic`. +We use `http://localhost:18080`; adjust the port to yours. + +## 1. Create the app and claim a host + +Each tutorial gets its own [app](../guide/concepts.md#apps) for isolation. Locally we claim a +`*.localhost` host and pass it with a `Host:` header. + +```sh +pic apps create links --name "Link Shortener" +pic apps domains add links links.localhost +``` + +## 2. The "shorten" script + +`shorten.rhai`: + +```rhai +// POST /shorten body: { "url": "https://..." } -> { code, short_url } +let body = ctx.request.body; +if type_of(body) != "map" || !body.contains("url") { + return #{ statusCode: 400, body: #{ error: "json body with a 'url' field required" } }; +} +let links = kv::collection("links"); +let code = random::string(6); // 6 alphanumeric chars +links.set(code, body.url); +return #{ + statusCode: 201, + body: #{ code: code, short_url: `http://links.localhost:18080/r/${code}` } +}; +``` + +## 3. The "redirect" script + +`redirect.rhai`: + +```rhai +// GET /r/:code -> 302 to the stored URL, or 404 +let code = ctx.request.params.code; +let target = kv::collection("links").get(code); // () when absent +if target == () { + return #{ statusCode: 404, body: #{ error: "unknown code" } }; +} +return #{ statusCode: 302, headers: #{ "Location": target }, body: () }; +``` + +## 4. Deploy and bind routes + +```sh +pic scripts deploy shorten.rhai --app links --name shorten +pic scripts deploy redirect.rhai --app links --name redirect + +SH=$(pic --output json scripts ls --app links | jq -r '.[]|select(.name=="shorten").id') +RD=$(pic --output json scripts ls --app links | jq -r '.[]|select(.name=="redirect").id') + +pic routes create --script $SH --path /shorten --method POST +pic routes create --script $RD --path '/r/:code' --path-kind param --method GET +``` + +## 5. Try it + +```sh +curl -X POST http://localhost:18080/shorten -H 'Host: links.localhost' \ + -H 'Content-Type: application/json' -d '{"url":"https://example.com/some/long/path"}' +``` +```json +{"code":"pE4wVV","short_url":"http://links.localhost:18080/r/pE4wVV"} +``` + +```sh +curl -i http://localhost:18080/r/pE4wVV -H 'Host: links.localhost' +``` +```text +HTTP/1.1 302 Found +location: https://example.com/some/long/path +``` + +```sh +curl -i http://localhost:18080/r/nope -H 'Host: links.localhost' +# HTTP/1.1 404 Not Found +``` + +## 6. Look inside + +Every stored link is just a KV entry — inspect them straight from the admin side: + +```sh +pic kv ls --app links --collection links +pic kv get --app links --collection links pE4wVV +# "https://example.com/some/long/path" +``` + +(`pic kv get` prints the stored value as JSON — a string value comes back quoted.) + +## Notes, constraints & next steps + +- **Collisions.** `random::string(6)` over 62 symbols is only ~36 bits (6 × log₂62) — fine for a + demo, but at scale two codes will eventually collide and the second `set` would silently overwrite + the first. For production, check `links.has(code)` and regenerate on collision, use more characters, + or use `random::uuid()`. +- **Size cap.** A stored value can't exceed `PICLOUD_KV_MAX_VALUE_BYTES` (256 KiB) — irrelevant for + URLs, but good to know ([overview](../reference/sdk/overview.md#size-caps)). +- **Validate input.** We checked the body is a map with a `url`. A real version should also validate + the URL scheme to avoid storing `javascript:` links you later redirect to. +- **Open redirect.** Redirecting to arbitrary user-supplied URLs is an + [open-redirect](../operations/security.md) vector if codes are guessable and the links are + attacker-controlled — fine here, but think about it for auth flows. + +Next: add per-link click counts (another `kv` collection, like the quickstart counter), or move to +[`docs`](../reference/sdk/storage.md#docs) if you want to store metadata (owner, created-at, hits) per +link and query it — which is exactly what the [TODO API tutorial](todo-api.md) does. diff --git a/docs/dev-guide/src/examples/webhook-receiver.md b/docs/dev-guide/src/examples/webhook-receiver.md new file mode 100644 index 0000000..ef64463 --- /dev/null +++ b/docs/dev-guide/src/examples/webhook-receiver.md @@ -0,0 +1,148 @@ +# Tutorial: Webhook receiver + +Receive webhooks reliably: authenticate the caller with a shared secret, accept fast with `202`, do the +slow work on a durable [queue](../reference/sdk/messaging.md#queue), forward downstream over +[`http`](../reference/sdk/http.md) with [`retry`](../reference/sdk/composition.md#retry), and let +failures land in the [dead-letter](../reference/sdk/composition.md#dead-letters) queue for replay. + +This exercises [`secrets`](../reference/sdk/secrets.md), [async dispatch](../guide/concepts.md#dispatch-modes-sync-vs-async), +`queue::enqueue`, a `queue` trigger, `http` + `retry`, and dead-letters. + +> **Why a shared secret, not HMAC?** The 1.1.9 script SDK has **no hashing/HMAC primitive** — `base64` +> and `hex` exist, but not `sha256`/`hmac`. So a script can't verify a provider's HMAC signature +> itself. (The platform *does* HMAC-verify built-in [inbound-email triggers](../reference/rest-api/triggers.md) +> server-side.) For script-level webhooks, use a shared-secret header, which is what we do here. + +## 1. App, host, and the secret + +```sh +pic apps create hooks --name "Webhook Receiver" +pic apps domains add hooks hooks.localhost +printf 'shhh-secret-token' | pic secrets set --app hooks webhook_token +``` + +## 2. The receiver (sync, returns 202) + +`receiver.rhai` — verify the secret, enqueue, accept: + +```rhai +// POST /ingest header: x-webhook-token: body: the event JSON +let expected = secrets::get("webhook_token"); +let got = if "x-webhook-token" in ctx.request.headers { ctx.request.headers["x-webhook-token"] } else { "" }; +if got != expected { + return #{ statusCode: 401, body: #{ error: "bad or missing webhook token" } }; +} +queue::enqueue("deliveries", ctx.request.body, #{ max_attempts: 2 }); +return #{ statusCode: 202, body: #{ accepted: true } }; +``` + +We verify and enqueue synchronously (so a bad token gets a real `401`), then return `202` — the heavy +lifting happens off the request path. + +## 3. The consumer (runs per queued message) + +`consumer.rhai` — note the event shape: the payload is `ctx.event.queue.message`, and the retry count +is `ctx.event.queue.attempt`: + +```rhai +let msg = ctx.event.queue.message; +log::info("processing delivery", #{ id: msg.id, attempt: ctx.event.queue.attempt }); + +if msg.fail == true { + throw "simulated downstream failure"; // forces retries → dead-letter (for the demo) +} + +// Forward downstream, retrying transient failures. +let policy = retry::policy(#{ max_attempts: 3, backoff: "exponential", base_ms: 200 }); +let resp = retry::run(policy, || http::get("https://example.com")); + +kv::collection("delivered").set(msg.id, `${resp.status}`); +``` + +## 4. Deploy, route, and register the queue trigger + +```sh +pic scripts deploy receiver.rhai --app hooks --name receiver +pic scripts deploy consumer.rhai --app hooks --name consumer + +RCV=$(pic --output json scripts ls --app hooks | jq -r '.[]|select(.name=="receiver").id') +CON=$(pic --output json scripts ls --app hooks | jq -r '.[]|select(.name=="consumer").id') + +pic routes create --script $RCV --path /ingest --method POST + +# Register the consumer on the "deliveries" queue. The per-kind wrapper `pic triggers create-queue` +# works too; we use the JSON form here only to set a short retry backoff so the dead-letter demo is +# quick (the wrapper doesn't expose retry knobs): +pic triggers create-from-json --app hooks --kind queue \ + --body "{\"script_id\":\"$CON\",\"queue_name\":\"deliveries\",\"retry_max_attempts\":1,\"retry_base_ms\":300}" +``` + +## 5. Send some webhooks + +```sh +B=http://localhost:18080/ingest +# bad token → 401 +curl -s -o /dev/null -w '%{http_code}\n' -X POST $B -H 'Host: hooks.localhost' \ + -H 'Content-Type: application/json' -d '{"id":"d1"}' # 401 + +# good token, succeeds → 202, processed in the background +curl -s -o /dev/null -w '%{http_code}\n' -X POST $B -H 'Host: hooks.localhost' \ + -H 'x-webhook-token: shhh-secret-token' -H 'Content-Type: application/json' \ + -d '{"id":"d3","event":"order.created","fail":false}' # 202 + +# good token, fails downstream → 202 now, dead-letter after retries +curl -s -o /dev/null -w '%{http_code}\n' -X POST $B -H 'Host: hooks.localhost' \ + -H 'x-webhook-token: shhh-secret-token' -H 'Content-Type: application/json' \ + -d '{"id":"d4","fail":true}' # 202 +``` + +After a moment, the happy delivery is recorded and its background run's `log::` output is captured +(async/triggered runs persist logs — sync HTTP runs don't): + +```sh +pic kv get --app hooks --collection delivered d3 # "200" +pic logs $CON --source queue --limit 5 # shows the "processing delivery" entries +``` + +## 6. Dead-letters {#dead-letters} + +The `fail` delivery used up its delivery attempts and became a dead-letter. We enqueued it with +`max_attempts: 2`, so the dead-letter row shows `attempts 2`: + +```sh +pic dead-letters count --app hooks # 1 +pic dead-letters ls --app hooks --unresolved +# id … attempts 2 … last_error "script runtime error: Runtime error: simulated downstream failure …" +``` + +Re-enqueue it (e.g. after fixing the downstream), or close it out: + +```sh +pic dead-letters replay --app hooks # Replayed dead-letter +pic dead-letters resolve --app hooks --reason ignored # close without replay +``` + +`replay` marks the row `replayed` and puts the original message back on the queue. `resolve` takes one +of a **fixed set** of reasons — `replayed`, `ignored`, `handled_by_script`, `handler_failed` — not +free text. You can also automate this: register a [`dead_letter` trigger](../reference/rest-api/triggers.md) +that runs a script (with `ctx.event.dead_letter`) to alert or call +[`dead_letters::replay`/`resolve`](../reference/sdk/composition.md#dead-letters). + +## Notes, constraints & next steps + +- **Idempotency.** A queued message can run more than once (a retry after a partial success, or a + visibility-timeout expiry). Use `ctx.event.queue.attempt` and a dedup key (`if + kv::collection("done").has(msg.id) { return; }`). See + [Best practices](../operations/best-practices.md#idempotent-consumers). +- **SSRF.** `http::*` blocks private/loopback targets — `http::get("http://localhost…")` throws + `http: blocked by SSRF policy: loopback`. Forwarding to your own internal services from a script + requires a routable address (or, in dev only, `PICLOUD_HTTP_ALLOW_PRIVATE=true`). + [Security →](../operations/security.md#ssrf) +- **Size cap.** A queued message can't exceed `PICLOUD_QUEUE_MAX_PAYLOAD_BYTES` (256 KiB). +- **Two retry layers.** The in-script `retry::run` retries *within one execution*; the trigger's + `retry_max_attempts` retries the *whole execution*. Pick one as your primary strategy to avoid + multiplicative delays. + +Next: swap the shared-secret check for the platform's [inbound-email trigger](../reference/rest-api/triggers.md) +if you're ingesting mail, or fan a single event out to several consumers with +[`pubsub`](../reference/sdk/messaging.md#pubsub). diff --git a/docs/dev-guide/src/guide/concepts.md b/docs/dev-guide/src/guide/concepts.md new file mode 100644 index 0000000..be41070 --- /dev/null +++ b/docs/dev-guide/src/guide/concepts.md @@ -0,0 +1,177 @@ +# Core concepts + +This chapter is the mental model. Read it once and the rest of the guide clicks into place. + +## The request lifecycle + +```text + ┌─────────────────────────────────────────────────────────────┐ + request ──▶│ 1. Caddy proxy 2. resolve Host → app 3. match the route │ + │ (most-specific (method+path │ + │ domain claim wins) in that app) │ + └─────────────────────────────────────────────────────────────┘ + │ found a script + ▼ + ┌─────────────────────────────────────────────────────────────┐ + │ 4. run the script in a sandbox, with `ctx` populated │ + │ 5. script calls the SDK (kv, http, …) — all scoped to the app │ + │ 6. script returns a response envelope → HTTP response │ + └─────────────────────────────────────────────────────────────┘ +``` + +Two phases matter most: **Host → app**, then **route within that app**. A request whose `Host` no app +claims gets `404 {"error":"no app claims host …"}`. A request that hits a claimed host but matches no +route gets `404 {"error":"no route matches …"}`. (The built-in `default` app claims `localhost`, which +is why the quickstart's `curl localhost:8000/...` just works.) + +## Apps + +An **app** is a tenant: an isolated namespace that owns scripts, routes, domain claims, and *all* data +(KV, docs, files, secrets, users, queues, topics). Isolation is the headline guarantee: + +> **A script can only ever touch its own app's data.** Every SDK call derives the app from the +> execution context on the server — never from anything the script passes. There is no API for a +> script to name another app. This is enforced in the platform, not by convention. + +You create apps from the dashboard (Apps → New), the CLI (`pic apps create `), or HTTP +(`POST /api/v1/admin/apps`). An app has a URL-safe **slug** (`^[a-z0-9][a-z0-9-]{0,62}$`) and a +display name. The `default` app exists on every install and is where the seeded example lives. + +Cross-app data sharing does not exist in 1.1.9 — isolation is strict by design. (It is on the roadmap +for a later release.) + +## Scripts + +A **script** is a Rhai program belonging to one app. There are two **kinds**: + +| Kind | Runs on | Can bind routes/triggers? | Top-level statements? | +|---|---|---|---| +| `endpoint` (default) | HTTP requests, events, `invoke()`, `/execute/{id}` | yes | yes | +| `module` | never directly — only `import`ed by other scripts | no | no — only `fn` and `const` | + +A **module** is a library. Another script in the *same app* pulls it in with +`import "modulename" as alias;` and calls `alias::some_fn(...)`. Cross-app imports are impossible (the +import name carries no app). Modules are how you share helpers without copy-paste. See +[Writing scripts](writing-scripts.md#modules-and-imports). + +Each script carries sandbox settings — a wall-clock `timeout_seconds`, a `memory_limit_mb`, and +optional fine-grained [sandbox overrides](writing-scripts.md#sandbox-limits). New scripts default to a +30-second timeout and 256 MB. + +## Routes and domains + +A **route** binds an incoming HTTP request to a script. A route has: + +- a **host** match: `any` (the default, matches whatever host the app claims), `strict` (one exact + host), or `wildcard` (`*.example.com`, optionally capturing the subdomain into a param); +- a **path** match of one of three **path kinds**: + - `exact` — `/webhook` matches only `/webhook`; + - `param` — `/users/:id` matches `/users/42` and exposes `ctx.request.params.id == "42"`; + - `prefix` — `/files/*` matches `/files/a/b/c` and exposes the tail as `ctx.request.rest`; +- an optional **method** (`GET`, `POST`, …); omit it to match any method; +- a **dispatch mode** (below). + +> **Param syntax is deliberately split.** Route paths use `:name` (`/users/:id`). Domain patterns use +> `{name}` (`{tenant}.example.com`). Never mix them. [More →](writing-scripts.md). + +**Domains.** Beyond the `default` app, an app's routes only match once the app **claims** the request's +`Host`. Claim patterns are exact (`api.example.com`), wildcard (`*.example.com`), or parameterized +(`{tenant}.example.com`). The most specific claim wins. Locally you can claim something like +`myapp.localhost` and test with `curl -H 'Host: myapp.localhost' localhost:8000/...` — that's the +pattern the [tutorials](../examples/index.md) use. The port is stripped before matching. + +**Reserved prefixes.** PiCloud refuses to create routes under `/api/`, `/admin/`, `/healthz`, or +`/version` — those belong to the platform. Pick any other path. + +## The response envelope + +An `endpoint` script's return value becomes the HTTP response. The rule +([`engine.rs`](../reference/sdk/ctx-and-events.md)): + +- **Return a map containing a `statusCode` key** → that's the structured envelope: + ```rhai + return #{ + statusCode: 201, + headers: #{ "Content-Type": "application/json", "Location": "/things/7" }, + body: #{ id: 7 } + }; + ``` + `statusCode` is a required integer; `headers` (string→string) and `body` are optional. A `body` that + is a map/array is serialized to JSON. +- **Return anything else** (a map without `statusCode`, a string, a number, an array) → PiCloud wraps + it in a **`200 OK`** with that value as the body. + +So `return #{ ok: true };` and `return #{ statusCode: 200, body: #{ ok: true } };` both produce +`200 {"ok":true}`. Use the full envelope when you need a non-200 status, custom headers, or redirects. + +## Dispatch modes: sync vs async + +Every route (and trigger) has a **dispatch mode**: + +- **`sync`** (default for routes) — the caller waits; the script's envelope is the HTTP response. +- **`async`** — PiCloud immediately returns **`202 Accepted`** with an `execution_id`, and runs the + script in the background via the dispatcher. The caller never sees the script's return value. + +Use `async` for fire-and-forget work (webhooks you don't need to answer in detail, fan-out, slow jobs) +so the client isn't blocked. Triggers default to `async`. [More on choosing →](../operations/best-practices.md#sync-vs-async). + +## Triggers and events + +A **trigger** fires a script on an *event* rather than an HTTP request. PiCloud has eight trigger +kinds: + +| Kind | Fires when… | `ctx.event` carries | +|---|---|---| +| `kv` | a key is set/deleted in matching collections | `op`, `kv.{collection,key,value}` | +| `docs` | a document is created/updated/deleted | `op`, `docs.{collection,id,data,prev_data}` | +| `files` | a blob is created/updated/deleted | `op`, file metadata (never the bytes) | +| `pubsub` | a message is published to a matching topic | `pubsub.{topic,payload}` | +| `queue` | a message is claimed off a named queue | `queue.{…}`, attempt count | +| `cron` | a schedule ticks | `cron.{schedule,timezone,scheduled_at,fired_at}` | +| `email` | mail arrives at the inbound webhook | the parsed message | +| `dead_letter` | another trigger exhausts its retries | the failed event + error | + +The triggered script reads `ctx.event` (absent on plain HTTP invocations, so you can test +`if "event" in ctx`). Triggers are created per-kind — see [Triggers](../reference/rest-api/triggers.md) +and the [`ctx.event` reference](../reference/sdk/ctx-and-events.md#events). + +## Authentication: three kinds of identity + +Keep these straight — they are easy to confuse: + +| Identity | Who | Used for | Managed by | +|---|---|---|---| +| **Admin user** | you / your team | the control plane (dashboard, CLI, admin API) | `pic admins`, bootstrap env vars | +| **API key** | you / a CI job | the control plane, non-interactively | `pic api-keys`, dashboard profile | +| **App user** | *your app's* end-users | whatever *your* scripts decide | the `users` SDK, in your scripts | + +An **admin user** or **API key** is a *principal* on the control plane, with an **instance role** +(`owner` > `admin` > `member`) and, for members, per-app **app roles** (`app_admin` > `editor` > +`viewer`). See [Capabilities & roles](../reference/config/capabilities.md). + +> **There is no built-in end-user signup/login HTTP endpoint.** App-user authentication is the +> [`users` SDK](../reference/sdk/users.md) — you compose `users::create`, `users::login`, +> `users::verify` into *your own* routes. This trips people up; the +> [TODO API tutorial](../examples/todo-api.md) shows the full pattern. + +## The data plane is unauthenticated by default + +Routes you create are **public**. A request to your route runs the script with **full app authority** — +it can read every secret, every KV key, every user in the app. PiCloud does *not* put auth in front of +your routes for you. If a route must be restricted, your script enforces it (check a token, call +`users::verify`, compare an HMAC). This is the single most important security point — +see [Security](../operations/security.md). + +## The five version surfaces + +`GET /version` returns five independent numbers. They move independently on purpose: + +| Surface | Example | What it tracks | Notes | +|---|---|---|---| +| `product` | `1.1.9` | the release build | semver of the whole platform | +| `sdk` | `1.10` | the script-visible API | **`major.minor`** — `1.10` is the *tenth minor*, not `1.1.0`. Read it at runtime as `ctx.sdk_version`. A minor bump only *adds*; existing scripts keep working. | +| `api` | `1` | the HTTP API major | appears in URLs as `/api/v1/...` | +| `schema` | `44` | the DB migration number | monotonic, forward-only | +| `wire` | `1` | inter-node protocol | reserved; cluster mode is a future release | + +The recurring mistake is reading `sdk: "1.10"` as "version 1.1.0". It isn't. It's SDK 1, minor 10. diff --git a/docs/dev-guide/src/guide/quickstart.md b/docs/dev-guide/src/guide/quickstart.md new file mode 100644 index 0000000..843f9d3 --- /dev/null +++ b/docs/dev-guide/src/guide/quickstart.md @@ -0,0 +1,170 @@ +# Quickstart + +This gets you from nothing to a live, custom HTTP endpoint — backed by persistent storage — in about +ten minutes. We'll use the built-in **default app** so there's zero setup friction; later you'll learn +to create your own [apps](concepts.md#apps) with their own [domains](concepts.md#routes-and-domains). + +> Every command and response below was produced by running it. If yours differ, check +> [Troubleshooting](../operations/troubleshooting.md). + +## Step 1 — Boot the stack + +The fastest path is Docker Compose, which brings up Postgres, PiCloud, the dashboard, and a Caddy +reverse proxy behind one port. + +```sh +git clone picloud && cd picloud +cp .env.example .env # then edit: set PICLOUD_ADMIN_USERNAME / PICLOUD_ADMIN_PASSWORD +docker compose up -d --build +``` + +The default `.env` runs in **dev mode** (a deterministic, insecure master key — fine for local, never +for production) and publishes Caddy on host port **8000**. The dashboard is at +`http://localhost:8000/admin`. + +> **Prefer running the binary directly?** See [Running the bare binary](../deploy/bare-binary.md). +> The only difference is the port. This guide uses `http://localhost:8000`; export it once so you can +> paste the rest verbatim: +> +> ```sh +> export PICLOUD=http://localhost:8000 +> ``` + +## Step 2 — Confirm it's alive + +```sh +curl $PICLOUD/healthz +# ok + +curl $PICLOUD/version +``` +```json +{"api":1,"product":"1.1.9","public_base_url":"http://localhost:8000","schema":44,"sdk":"1.10","wire":1} +``` + +That `sdk` field — `"1.10"` — is the SDK version your scripts target. It is a `major.minor` string (the +tenth minor of SDK v1), **not** the product release `1.1.9`. See +[the five version surfaces](concepts.md#the-five-version-surfaces). + +## Step 3 — Log in + +You authenticate once and use a **bearer token** for every control-plane call. Three equivalent ways: + +**Dashboard:** open `http://localhost:8000/admin`, enter your admin username/password. + +**CLI** (recommended for the rest of this guide): +```sh +printf 'YOUR_PASSWORD' | pic login --url $PICLOUD --username admin --password-stdin +# Logged in as admin (owner) at http://localhost:8000 +pic whoami +``` + +**Raw HTTP:** +```sh +TOKEN=$(curl -s -X POST $PICLOUD/api/v1/admin/auth/login \ + -H 'Content-Type: application/json' \ + -d '{"username":"admin","password":"YOUR_PASSWORD"}' | jq -r .token) +``` +The login response is: +```json +{"user":{"id":"…","username":"admin","instance_role":"owner","email":null}, + "token":"V-f-0ey3eEcFKBWH5_6GLCqBkih08bhWtY-5GHCxZ04", + "expires_at":"2026-06-18T19:49:39Z"} +``` +Pass it as `Authorization: Bearer $TOKEN` on admin requests. (The `pic` CLI stores it for you in +`~/.config/picloud/credentials`.) + +## Step 4 — Meet the seeded "hello" script + +A fresh install seeds one example script into the default app, bound to `GET /hello`: + +```sh +curl $PICLOUD/hello +# {"message":"Hello, world!"} + +curl $PICLOUD/hello -H 'Content-Type: application/json' -d '{"name":"Fabi"}' +# {"message":"Hello, Fabi!"} +``` + +Open it in the dashboard (Apps → Default → Scripts → hello) to see its source. It's a good template: +it reads `ctx.request.body` and returns a [response envelope](concepts.md#the-response-envelope). + +## Step 5 — Write your first script + +Create a file `counter.rhai`: + +```rhai +// Count visits, persisted in KV across requests and restarts. +let hits = kv::collection("counters"); +let n = hits.get("home"); // () if the key has never been set +if n == () { n = 0; } +n += 1; +hits.set("home", n); +return #{ statusCode: 200, body: #{ count: n } }; +``` + +Deploy it to the default app: + +```sh +pic scripts deploy counter.rhai --app default --name counter +``` +```json +{"action":"created","id":"3502852c-030b-4e39-82f6-121220095da7","name":"counter","version":"1"} +``` + +Grab that `id` — call it `$SID`. + +> Doing it in the dashboard instead? Apps → Default → Scripts → **New script**, paste the source, +> save. Doing it over raw HTTP? `POST /api/v1/admin/scripts` with +> `{"app_id":"…","name":"counter","source":"…"}` — see [Scripts](../reference/rest-api/scripts.md). + +## Step 6 — Bind a route + +A script does nothing until a [route](concepts.md#routes-and-domains) points at it. + +```sh +pic routes create --script $SID --path /count --method GET +# Created route 9df71836-… (GET * /count) +``` + +Now hit it — the count persists between calls: + +```sh +curl $PICLOUD/count # {"count":1} +curl $PICLOUD/count # {"count":2} +curl $PICLOUD/count # {"count":3} +``` + +You just wrote a stateful endpoint with no database wiring, no migrations, no deploy pipeline. + +## Step 7 — Look behind the curtain + +Read what your script stored, straight from KV: + +```sh +pic kv get --app default --collection counters home +# 3 +``` + +And see every execution, with timing and status: + +```sh +pic logs $SID --limit 3 +``` +```text +created_at source status summary +2026-06-17T19:54:01.265901+00:00 http success - +2026-06-17T19:54:01.161625+00:00 http success - +``` + +The dashboard shows the same under the script's **Executions** tab, including each run's `log::` output. + +## Where to next + +You now understand the core loop: **write a script → bind a route → call it → inspect**. From here: + +- **Understand the model** → [Core concepts](concepts.md): apps, isolation, dispatch, events, versions. +- **Write better scripts** → [Writing scripts](writing-scripts.md): the `ctx` object, the envelope, + error handling, logging, sandbox limits. +- **Build something real** → the [tutorials](../examples/index.md): a URL shortener, a webhook + receiver, an authenticated TODO API, a scheduled report, and a file-upload service. diff --git a/docs/dev-guide/src/guide/writing-scripts.md b/docs/dev-guide/src/guide/writing-scripts.md new file mode 100644 index 0000000..8484f77 --- /dev/null +++ b/docs/dev-guide/src/guide/writing-scripts.md @@ -0,0 +1,206 @@ +# Writing scripts + +Scripts are written in [Rhai](https://rhai.rs/book/) — a small, embeddable language with Rust-flavored +syntax, dynamic typing, maps (`#{ ... }`), arrays, and closures (`|x| ...`). This chapter covers what's +specific to writing scripts *for PiCloud*: the `ctx` object, the response envelope, errors, logging, +modules, and the sandbox. For the SDK calls themselves (`kv`, `http`, …) see the +[SDK reference](../reference/sdk/overview.md). + +## The shape of a script + +An `endpoint` script runs top to bottom; whatever it `return`s (or the value of its last expression) +becomes the [response envelope](concepts.md#the-response-envelope). + +```rhai +// Read input from ctx, do work, return a response. +let body = ctx.request.body; +let name = if type_of(body) == "map" && body.contains("name") { body.name } else { "world" }; +return #{ statusCode: 200, body: #{ message: `Hello, ${name}!` } }; +``` + +There's no `main`, no handler signature, no framework. The whole file *is* the handler. + +## `ctx` — the execution context + +Every run gets a global `ctx` map. The fields: + +| Field | Type | Notes | +|---|---|---| +| `ctx.sdk_version` | string | e.g. `"1.10"` — for feature detection | +| `ctx.execution_id` | string (UUID) | unique per run | +| `ctx.script_id` | string (UUID) | the running script | +| `ctx.script_name` | string | | +| `ctx.request_id` | string (UUID) | correlation id, also in logs | +| `ctx.invocation_type` | string | `"http"`, `"function"` (via `invoke()`), or `"scheduled"` | +| `ctx.request` | map | the request, see below | +| `ctx.event` | map | **present only for triggered runs** — see [events](../reference/sdk/ctx-and-events.md#events) | + +`ctx.request`: + +| Field | Type | Notes | +|---|---|---| +| `ctx.request.path` | string | e.g. `"/users/42"` | +| `ctx.request.method` | string | uppercased, e.g. `"GET"` | +| `ctx.request.headers` | map | **lowercased** keys → values: `ctx.request.headers["authorization"]` | +| `ctx.request.body` | dynamic | **already JSON-parsed** when the body is JSON; a string otherwise; `()` if empty | +| `ctx.request.params` | map | captures from `:name` path segments; empty if none | +| `ctx.request.query` | map | query-string params; empty if none | +| `ctx.request.rest` | string | the tail captured by a `prefix` (`/*`) route; empty otherwise | + +> **`ctx.request.body` is parsed for you.** If the client sends `Content-Type: application/json`, the +> body arrives as a Rhai map/array/scalar — you do **not** call `json::parse` on it. Only parse raw +> strings you get from elsewhere. Header keys are always lowercase. Test for presence before indexing: +> `if "x-api-key" in ctx.request.headers { ... }`. + +> **There are no raw request bytes in `ctx`.** The body is JSON (or a string). To accept a binary +> upload, have the client base64-encode it inside a JSON field and `base64::decode` it in the script — +> see the [file-upload tutorial](../examples/file-upload.md). + +## The response envelope, in detail + +```rhai +// Full control: status, headers, body. +return #{ + statusCode: 302, + headers: #{ "Location": "https://example.com/target" }, + body: () +}; +``` + +- `statusCode` — required integer if the map is to be treated as an envelope. Omit it and the *whole + map* becomes a `200` JSON body instead. +- `headers` — a `string → string` map. Values are stringified. +- `body` — **always serialized as JSON.** A map/array becomes a JSON object/array; a string becomes a + JSON *string* (so `body: "pong"` is sent as `"pong"`, with quotes); `()` becomes `null`. + +Shortcut: returning any non-envelope value yields `200 OK` with that value as the JSON body. `return #{ +ok: true };` ⇒ `200 {"ok":true}`; `return "pong";` ⇒ `200 "pong"`. + +> **Responses are always JSON.** The response body is JSON-encoded regardless of any `Content-Type` +> header you set (the default content type is `application/json`; you can override the *header*, but +> the *bytes* stay JSON). In 1.1.9 a script route therefore **cannot serve raw HTML, plain text, or +> binary** — to return a file's bytes, send base64 in JSON or use the +> [admin files endpoint](../reference/rest-api/data-admin.md#files). See the +> [file-upload tutorial](../examples/file-upload.md). + +## Errors + +A script that **`throw`**s aborts and produces an HTTP **502**: + +```rhai +if ctx.request.body == () { throw "body required"; } +``` +```text +HTTP/1.1 502 Bad Gateway +{"error":"Runtime error: body required (line 1, position 30)"} +``` + +To return a *clean* client error instead of a 502, return an envelope with the status you want: + +```rhai +if ctx.request.body == () { + return #{ statusCode: 400, body: #{ error: "body required" } }; +} +``` + +SDK calls follow a consistent convention so you can decide whether to guard or let it bubble: + +- **They throw** on real failures (DB down, payload over a size cap, authorization denied). +- **They return `()`** for "not found" (`kv::...get` of a missing key, `users::login` with bad + credentials, etc.). Test with `== ()`. +- **They return a `bool`** for predicates (`...has(k)`, `...delete(k)` → was-it-present). + +Wrap risky calls in `try { ... } catch (e) { ... }` when you want to handle failure yourself. + +## Logging + +`log::(message)` or `log::(message, #{ structured: "data" })`, at four levels: + +```rhai +log::info("charge succeeded", #{ order: id, cents: amount }); +log::warn("retrying upstream"); +log::error("gave up", #{ attempts: 3 }); +log::trace("entered branch A"); +``` + +There is **no `log::debug`** (`debug` is a reserved word in Rhai) — use `log::trace`. + +> **Where do these show up?** Log entries are buffered during the run and persisted to the execution +> log **for asynchronous and triggered executions** (async routes, cron/queue/kv/… triggers) — view +> them with `pic logs ` or the dashboard's **Executions** tab. **Synchronous HTTP +> executions do not persist the buffered `log::` output** (the row records timing, status, and code +> only). So `log::` is most useful for background work; for sync endpoints, fold diagnostics into the +> response during development. This is a deliberate hot-path optimization, not a bug. + +## Modules and imports + +Factor shared logic into a **module** script (`kind: "module"`) — it may contain only `fn` and `const` +declarations, no top-level statements: + +```rhai +// module script named "money" +const CURRENCY = "EUR"; +fn format(cents) { + `${CURRENCY} ${cents / 100}.${cents % 100}` +} +``` + +Another script *in the same app* imports it by name: + +```rhai +import "money" as money; +return #{ statusCode: 200, body: #{ price: money::format(4999) } }; +``` + +Deploy a module with `pic scripts deploy money.rhai --app myapp --kind module`. Cross-app imports are +impossible — the import name carries no app, and resolution is scoped to the caller's app. Module names +can't shadow SDK namespaces (`kv`, `http`, `log`, …). The [scheduled-report +tutorial](../examples/scheduled-report.md) uses a module. + +## Sandbox limits + +Every run is bounded. If a script exceeds a limit it's terminated and the request fails (a timeout is +`504`; an operation-budget overrun is `507`). + +| Limit | Default (what scripts get) | What it bounds | +|---|---|---| +| `timeout_seconds` | 30 (max 300) | wall-clock time | +| `memory_limit_mb` | 256 (max 2048) | memory | +| `max_operations` | 1,000,000 | total Rhai operations (a CPU proxy) | +| `max_string_size` | 64 KiB | longest string built | +| `max_array_size` | 10,000 | longest array | +| `max_map_size` | 10,000 | largest map | +| `max_call_levels` | 64 | call-stack depth | +| `max_expr_depth` | 64 | expression nesting | + +You can *lower* (or raise, within ceilings) these per script. Via the CLI: + +```sh +pic scripts deploy heavy.rhai --app myapp --timeout 60 --sandbox max_operations=5000000 +``` + +Per-knob overrides are clamped to admin-configured **ceilings** (defaults: 10M operations, 1 MiB +strings, 100k array/map, 128 call/expr levels; tunable with `PICLOUD_SANDBOX_MAX_*` — see +[env vars](../reference/config/env-vars.md)). An override above the ceiling is rejected at deploy time, +so a typo can't silently create an unrestricted script. + +Separately, the whole instance caps **concurrent** executions (`PICLOUD_MAX_CONCURRENT_EXECUTIONS`, +default 32); past that, new data-plane requests get `503` with `Retry-After: 1` immediately — there's +no queue. Plan for it; see [Best practices](../operations/best-practices.md). + +## Quick reference: idioms + +```rhai +// Default a missing body field +let n = if "count" in ctx.request.body { ctx.request.body.count } else { 0 }; + +// Read a header (always lowercase key) +let auth = if "authorization" in ctx.request.headers { ctx.request.headers["authorization"] } else { "" }; + +// 404 cleanly +let row = kv::collection("things").get(id); +if row == () { return #{ statusCode: 404, body: #{ error: "not found" } }; } + +// Redirect +return #{ statusCode: 302, headers: #{ "Location": target }, body: () }; +``` diff --git a/docs/dev-guide/src/introduction.md b/docs/dev-guide/src/introduction.md new file mode 100644 index 0000000..d13b177 --- /dev/null +++ b/docs/dev-guide/src/introduction.md @@ -0,0 +1,124 @@ +# PiCloud Developer Guide + +**PiCloud is a self-hosted, event-driven serverless platform.** You write small scripts in +[Rhai](https://rhai.rs) (an embedded scripting language with Rust-like syntax), upload them, and bind +them to URLs. PiCloud runs each script in a sandbox when a request arrives — no servers to manage, no +containers to build, no cold-start orchestration to think about. It is built to run comfortably on a +single box (a home server, a VPS, a Raspberry Pi) and scales to a cluster later without a rewrite. + +If you have ever used a "functions" product in a public cloud, the mental model is the same — but the +whole thing is one binary plus a Postgres database that *you* own. + +```text + HTTP request ┌──────────────────────────┐ + │ │ your Rhai script │ + ▼ │ │ + ┌───────────┐ resolve Host → app, ┌──────▶│ let body = ctx.request │ + │ Caddy │──▶ then match the route ─┘ │ kv::collection("x")... │ + │ (proxy) │ (orchestrator) │ return #{ statusCode } │ + └───────────┘ └──────────────────────────┘ + │ │ + │ sandboxed run (executor) ◀─────┘ + ▼ + HTTP response ◀── the response envelope your script returned +``` + +## What you can build + +A script can do a lot more than return JSON. Through the **SDK** it can store data, send mail, call +other services, react to events, and run on a schedule: + +| Capability | SDK namespace | Backed by | +|---|---|---| +| Key/value storage | `kv` | Postgres (JSONB) | +| Document storage with queries | `docs` | Postgres (JSONB) | +| Blob / file storage | `files` | Filesystem + Postgres metadata | +| Outbound HTTP requests | `http` | reqwest, with an SSRF guard | +| Send & receive email | `email` | SMTP relay / inbound webhook | +| End-user accounts & login | `users` | Postgres + Argon2id | +| Pub/sub & realtime (SSE) | `pubsub` | Postgres `LISTEN/NOTIFY` | +| Durable job queues | `queue` | Postgres | +| Encrypted secrets | `secrets` | Postgres (AES-256-GCM) | +| Call other scripts | `invoke` | in-process re-entry | +| Retry with backoff | `retry` | — | +| Scheduled / event triggers | (triggers, not an SDK call) | dispatcher | + +Plus a **standard library** for the everyday glue: `json`, `base64`, `hex`, `url`, `regex`, `random`, +`time`. See the [SDK reference](reference/sdk/overview.md) for the full surface. + +## Three ways to drive PiCloud + +Everything below is just a client of the same HTTP control plane — pick whichever fits: + +1. **The dashboard** — a web UI at `/admin` with a code editor, route binder, log viewer, and + management screens for every resource. Best for exploring and for editing scripts. +2. **The `pic` CLI** — a developer command-line client. Best for scripting deploys, CI, and quick + inspection. See the [CLI reference](reference/cli/pic.md). +3. **Plain HTTP** (`curl`, any language) — the underlying REST API. Best when you want no extra + tooling, or to integrate from your own code. See the [HTTP API reference](reference/rest-api/overview.md). + +This guide shows all three side by side throughout. + +## Where to start + +> **New here?** → [Quickstart](guide/quickstart.md) (zero to a live endpoint in ~10 minutes) → +> [Core concepts](guide/concepts.md) → pick a [tutorial](examples/index.md). +> +> **Want to build something concrete?** → the [tutorials](examples/index.md) are five complete, +> copy-pasteable example apps, each exercising a different slice of the platform. +> +> **Looking something up?** → the [SDK](reference/sdk/overview.md), +> [HTTP API](reference/rest-api/overview.md), and [CLI](reference/cli/pic.md) references are +> organized for lookup. Use the search box (top-left) for anything specific. +> +> **Running it in production?** → [Deployment](deploy/docker-compose.md) and the +> [security chapter](operations/security.md). + +## A note on versions + +This guide documents **PiCloud product `1.1.9`**. PiCloud tracks five independent version numbers, all +returned by [`GET /version`](reference/rest-api/overview.md#get-version): + +| Surface | Value | Means | +|---|---|---| +| `product` | `1.1.9` | the release you're running | +| `sdk` | `1.10` | the script-visible SDK, in **`major.minor`** form — this is the *tenth minor* of SDK v1, **not** "1.1.0" | +| `api` | `1` | the HTTP API major version, in the URL as `/api/v1/...` | +| `schema` | `44` | the database migration number | +| `wire` | `1` | the inter-node protocol (reserved; cluster mode is a future release) | + +Your scripts can read the SDK version at runtime as `ctx.sdk_version` (the string `"1.10"`). Do not +confuse `sdk` with `product`. [More on versioning →](guide/concepts.md#the-five-version-surfaces) + +## Glossary + +These terms recur throughout the guide: + +- **App** — a tenant/namespace. Owns scripts, routes, domains, and all data. Apps are isolated from + each other; a script can only ever touch its own app's data. See [Core concepts](guide/concepts.md#apps). +- **Script** — a Rhai program. Two kinds: an **endpoint** (runs on requests/events) or a **module** + (a library of `fn`/`const` that other scripts `import`). +- **Route** — a binding from an HTTP method + host + path to a script. One script can have many routes. +- **Trigger** — a binding that fires a script on an *event* instead of an HTTP request: a KV/docs/files + mutation, a published message, a queue message, a cron tick, or inbound email. +- **Event** — the thing that fired a triggered script; surfaced to the script as `ctx.event`. +- **Dispatch mode** — `sync` (the caller waits for the script's response) or `async` (the platform + returns `202 Accepted` immediately and runs the script in the background). +- **Collection** — a named bucket inside a storage service. The identity of a stored item is the tuple + `(app, collection, key)`. Collections are mandatory. +- **Response envelope** — the value an endpoint script returns to shape the HTTP response: a map with + `statusCode`, optional `headers`, and `body`. +- **Principal** — the authenticated identity behind a control-plane request: an **admin user** or an + **API key**. Distinct from an **app user** (an end-user of *your* app, managed by the `users` SDK). +- **Instance role** — an admin's platform-wide role: `owner`, `admin`, or `member`. +- **App role** — a member's per-app role: `app_admin`, `editor`, or `viewer`. +- **Dead letter** — a record of a triggered execution that exhausted its retries; you can inspect and + replay it. + +## How to read the code examples + +- Rhai snippets are shown as ```rhai``` blocks; they are the *body* of a script. +- HTTP examples use `curl` against a local dev instance on port `18080` (the dashboard-fronted stack + uses `8000` — adjust to your setup; see [Deployment](deploy/docker-compose.md)). +- CLI examples use the `pic` binary. +- Wherever a guide says a request "returns X", that output was produced by actually running it. diff --git a/docs/dev-guide/src/operations/best-practices.md b/docs/dev-guide/src/operations/best-practices.md new file mode 100644 index 0000000..ccab24b --- /dev/null +++ b/docs/dev-guide/src/operations/best-practices.md @@ -0,0 +1,99 @@ +# Best practices + +Patterns that hold up as your scripts move from demo to production. + +## Sync vs async dispatch {#sync-vs-async} + +- Use **`sync`** when the caller needs the result *now* and the work is fast (a lookup, a small + computation, a redirect). +- Use **`async`** (route or trigger) for anything slow or fire-and-forget — webhooks you don't need to + answer in detail, fan-out, outbound calls, report generation. The client gets `202` immediately and + isn't blocked by your processing. +- Don't do slow work (a chain of `http` calls, big aggregations) on a `sync` route — you'll hold the + connection and burn a concurrency permit. Enqueue it and return `202`. + +## Idempotent consumers {#idempotent-consumers} + +Queue messages and triggers can fire **more than once** (a retry after a partial success, a +visibility-timeout expiry). Make consumers idempotent: + +```rhai +let msg = ctx.event.queue.message; +let done = kv::collection("processed"); +if done.has(msg.id) { return; } // already handled — skip +// ... do the work ... +done.set(msg.id, time::now_ms()); +``` + +`ctx.event.queue.attempt` (and `ctx.event.dead_letter.attempts`) tell you when you're on a retry. + +## Retries & dead-letters + +- Wrap genuinely transient calls (`http`, `invoke`) in [`retry::run`](../reference/sdk/composition.md#retry) + with a bounded `max_attempts` and exponential backoff. Use `retry::on_codes` so you only retry the + failures worth retrying (e.g. `502/503/504`), not deterministic ones (`400`). +- Don't stack retries blindly: the trigger's own `retry_max_attempts` *and* an in-script `retry::run` + multiply. Pick one layer as primary. +- Let exhausted work become a **dead-letter**, then alert on the count and replay after a fix — rather + than retrying forever. A [`dead_letter` trigger](../reference/rest-api/triggers.md) can automate the + alert/replay. + +## Data modeling + +- **`kv`** for simple keyed values, counters, flags, caches. **`docs`** when you need to *find by + field*. **`files`** for blobs. **`secrets`** for credentials. +- Pick **collection names** deliberately — `(app, collection, key)` is the identity; one collection per + logical entity (`users`, `orders`) keeps listing and triggers clean. +- `docs.update` **replaces** the whole `data` map — read-modify-write to change one field. +- Respect the [size caps](../reference/config/env-vars.md#data-plane-size-caps) (256 KiB for + kv/docs/queue/pubsub). If a value is growing toward the cap, you're probably modeling it wrong (split + it, or use `files`). + +## Pagination + +List endpoints (`kv.list`, `docs.list`, admin lists) are **keyset/cursor** paginated. Loop until +`next_cursor` is `()`: + +```rhai +let c = docs::collection("orders"); +let cursor = (); +loop { + let page = c.list(#{ cursor: cursor, limit: 100 }); + for doc in page.docs { /* ... */ } + cursor = page.next_cursor; + if cursor == () { break; } +} +``` + +Don't assume one `list()` returns everything. + +## Script organization + +- Factor shared logic into a **`module`** and `import` it — don't copy-paste across endpoints. +- One script can serve **several routes** and branch on `ctx.request.method` / `params` (see the + [TODO API](../examples/todo-api.md)); or split per concern. Either is fine — optimize for readability. +- Validate `ctx.request.body` shape early and return a clean `400`, rather than letting a missing field + throw a `502`. + +## Logging & debugging + +- `log::` output is persisted for **async/triggered** runs — use it freely there and read it with + `pic logs --source `. +- For **synchronous HTTP** runs, `log::` is **not** persisted (a hot-path optimization). During + development, fold diagnostics into the response, or test the logic via an async path / `execute`. +- Filter logs by `--source` (`http`, `queue`, `cron`, …) to separate request handling from background + work. + +## Concurrency & limits + +- The instance caps concurrent executions (`PICLOUD_MAX_CONCURRENT_EXECUTIONS`); past it, callers get + `503 Retry-After: 1`. Build clients that honor `Retry-After`, and keep sync handlers quick so permits + free up fast. +- Lower a script's `timeout_seconds`/sandbox limits if it should be cheap — it fails fast instead of + hogging a permit. + +## Versioning + +- Read `ctx.sdk_version` if you want to feature-detect, but remember it's `major.minor` (`"1.10"`). + Minor bumps only *add*; your scripts keep working across them. +- Don't hardcode assumptions about the `product` version into scripts. diff --git a/docs/dev-guide/src/operations/security.md b/docs/dev-guide/src/operations/security.md new file mode 100644 index 0000000..d275cee --- /dev/null +++ b/docs/dev-guide/src/operations/security.md @@ -0,0 +1,105 @@ +# Security + +PiCloud gives you isolation and sandboxing by default, but several things are *your* responsibility. +Read this once before you put a public route online. + +## The data plane is public {#the-data-plane-is-public} + +**This is the most important point in the whole guide.** The routes you create are public, and a +request to one runs your script with **full authority over its app's data** — it can read every secret, +every KV key, every document, every user. PiCloud puts **no authentication in front of your routes**. + +The [capability/role model](../reference/config/capabilities.md) governs the *control plane* (the admin +API) — not your routes. So: + +- If a route must be restricted, **the script enforces it** — check a bearer token with + `users::verify`, compare a shared secret, etc. (see the [TODO API](../examples/todo-api.md) and + [webhook](../examples/webhook-receiver.md) tutorials). +- Treat anything a public script can read as reachable by anyone who can hit the route. Don't put a + secret in a collection a public script reads unless you mean to expose it. +- A script that calls `users::find_by_email` from an *unauthenticated* context is refused — but that's + one specific guard, not a general one. + +## Cross-app isolation + +Apps are hard-isolated: every SDK call derives the app from the server-side execution context, and +there is **no `app_id` argument anywhere in the SDK**. A script cannot name or reach another app's +data, period. This is enforced in the platform, not by convention — you can rely on it. Cross-app +sharing does not exist in 1.1.9. + +## Secrets and the master key {#secrets-and-the-master-key} + +- Store credentials in [`secrets`](../reference/sdk/secrets.md), not `kv` — secret values are encrypted + at rest (AES-256-GCM) and never returned by the admin API/dashboard/CLI (names only). +- Set them out-of-band (`pic secrets set` reads the value from stdin) rather than hardcoding in source. +- **Critical: the master key.** Secrets are encrypted under `PICLOUD_SECRET_KEY`. **Rotating it makes + existing secrets undecryptable** (`secrets::get` returns `()`); there's no automatic re-encryption in + 1.1.9. Treat the key as durable, store it in a secret manager, and if you must rotate, re-`set` every + secret afterward. +- **Never** run production with `PICLOUD_DEV_MODE` + `PICLOUD_DEV_INSECURE_KEY` — that "key" is + world-known, so at-rest encryption is worthless. + +## SSRF — outbound HTTP {#ssrf} + +`http::*` resolves the target and **blocks private, loopback, and link-local addresses** by default, so +a script taking a user-supplied URL can't be tricked into scanning your internal network. A blocked +call throws, e.g.: + +```text +http: blocked by SSRF policy: loopback +``` + +The escape hatch `PICLOUD_HTTP_ALLOW_PRIVATE=true` disables this — **dev/test only, never production**. +If you need a script to reach an internal service in prod, give it a routable address, not a private +one. + +## User enumeration {#user-enumeration} + +The [`users`](../reference/sdk/users.md) SDK is built to resist account enumeration: + +- `users::find_by_email` **requires an authenticated principal** — anonymous/public scripts can't use + it to probe who's registered. +- `users::email_available` *is* anonymous-safe (a signup form legitimately needs it), but it is **not + throttled**. If enumeration via the signup form is a concern, rate-limit it yourself (e.g. a `kv` + counter keyed by IP). + +## Input validation & injection + +- `ctx.request.body` is parsed JSON — validate its **shape** before use (`type_of`, `contains`) so a + missing field throws a clean `400`, not a confusing `502`. +- There's no SQL to inject (you don't write SQL), but be careful with **open redirects** (validate URLs + before issuing a `302`) and with reflecting user input into emails or downstream HTTP. +- Storing user-supplied HTML and serving it later is *not* a vector here, because + [script responses are always JSON](../guide/writing-scripts.md#the-response-envelope-in-detail) — but + if you base64-ship content for a client to render, the usual XSS rules apply on the client. + +## Webhook authenticity + +The 1.1.9 script SDK has **no HMAC/hashing primitive**, so a script cannot verify a provider's HMAC +signature itself. Options: + +- Use a **shared-secret header** compared against a stored secret (the + [webhook tutorial](../examples/webhook-receiver.md) approach). +- For **inbound email**, use the built-in [email trigger](../reference/rest-api/triggers.md) with an + `inbound_secret` — the platform does the HMAC verification server-side. + +## API keys & least privilege + +- Mint [API keys](../reference/rest-api/access.md#api-keys) with the **narrowest scopes** that work, + **bind them to one app** (`--app`), and set `expires_at`. A bound key can't hold `instance:admin`. +- The raw token is shown once — store it in your secret manager. +- Revoking a key (or deactivating an admin) takes effect immediately. +- Prefer **per-app members** with `viewer`/`editor` over handing out instance `admin`. + +## Platform hardening (already done for you) + +- Caddy adds CSP, `X-Frame-Options`, `nosniff`, `Referrer-Policy`, and (in prod) HSTS to the dashboard + and admin API, plus a 12 MB request-body ceiling. User-route responses deliberately get no CSP — your + scripts own their headers. +- Login is rate-limited per IP and per username; passwords use Argon2id; sessions and API keys are + stored as hashes. +- Scripts run in a [sandbox](../guide/writing-scripts.md#sandbox-limits) with operation, memory, and + wall-clock limits, and the whole instance caps concurrency. + +See the [Production checklist](../deploy/production-checklist.md) for the deployment-time version of +this page. diff --git a/docs/dev-guide/src/operations/troubleshooting.md b/docs/dev-guide/src/operations/troubleshooting.md new file mode 100644 index 0000000..a5b0993 --- /dev/null +++ b/docs/dev-guide/src/operations/troubleshooting.md @@ -0,0 +1,106 @@ +# Troubleshooting + +Symptoms you'll actually hit, and what they mean. All error strings below are real responses from a +running instance. + +## Routing + +**`404 {"error":"no app claims host \"…\""}`** +No app claims the request's `Host`. The `default` app claims only `localhost`. For another app, claim +the host first (`pic apps domains add `) and send a matching `Host:` header. Locally: +`pic apps domains add myapp myapp.localhost` then `curl -H 'Host: myapp.localhost' …`. + +**`404 {"error":"no route matches GET /…"}`** +The host resolved to an app, but no route matches that method+path in *that app*. Check +`pic routes ls `, and that the method matches (a route with `method: null` matches any). Use +`pic routes match --app ` to see what would match. + +**My route returns 404 but I just created it.** +Are you hitting the right app's host? Routes are app-scoped. Also: routes under `/api/`, `/admin/`, +`/healthz`, `/version` are rejected at creation — Caddy never forwards those to the matcher. + +## Script errors + +**`502 {"error":"Runtime error: … (line N, position M)"}`** +Your script `throw`ew (or hit a runtime error). To return a clean client error instead, return an +envelope: `return #{ statusCode: 400, body: #{ error: "…" } };`. + +**`422 invalid script: … 'public' is a reserved keyword`** (or `loop`, `fn`, …) +Rhai reserves more words than you'd expect (`public`, `loop`, `debug`, …). Rename the variable. Note +there's no `log::debug` — use `log::trace`. + +**`Unknown property 'x' - a getter is not registered …`** +You accessed `.x` on a value that isn't a map (or whose field is named differently). Most often a +wrong [`ctx.event`](../reference/sdk/ctx-and-events.md#events) path — e.g. the queue payload is +`ctx.event.queue.message`, **not** `.payload`; the attempt count is `.attempt`, not `.attempts`. + +**`504 Gateway Timeout`** — the script exceeded `timeout_seconds`. +**`507`** — it blew the operation budget (`max_operations`). Both mean: do less, or raise the limit +(within [ceilings](../reference/config/env-vars.md#sandbox-ceilings)) at deploy time. + +**My `log::info` output isn't in the logs.** +Synchronous HTTP runs don't persist `log::` output — only async/triggered runs do. Not a bug; see +[Writing scripts → Logging](../guide/writing-scripts.md#logging). + +## SDK calls + +**`http: blocked by SSRF policy: loopback`** (or `private`) +`http::*` blocks private/loopback targets by default. You're trying to reach `localhost`/a private IP. +Use a public address, or (dev only) `PICLOUD_HTTP_ALLOW_PRIVATE=true`. [More →](security.md#ssrf). + +**`email::send` throws `NotConfigured`.** +No SMTP relay configured. Set `PICLOUD_SMTP_*`, or in dev mode use the sink at +`GET /api/v1/admin/dev/emails`. + +**A `get` returns `()` unexpectedly.** +`()` means "absent". For `secrets::get` it can also mean decryption failed — did the +`PICLOUD_SECRET_KEY` change? ([master-key caveat](security.md#secrets-and-the-master-key)). + +**A `set`/`enqueue`/`create` throws about size.** +You exceeded a [size cap](../reference/config/env-vars.md#data-plane-size-caps) (256 KiB for +kv/docs/queue/pubsub; 100 MiB for files). Remember base64 inflates ~33%. + +## API & auth + +**`401 Unauthorized` on an admin call.** Missing/invalid/expired bearer token. Re-`pic login`, or check +the `Authorization: Bearer …` header. **`403 Forbidden`** means you're authenticated but lack the +[capability](../reference/config/capabilities.md) — wrong role for that action. + +**`422 unknown scope: app:read`** (minting an API key) +That's not a real scope. The seven valid ones: `script:read`, `script:write`, `route:write`, +`domain:manage`, `log:read`, `app:admin`, `instance:admin`. + +**`422 invalid resolution: …`** (resolving a dead-letter) +Reasons are a fixed set: `ignored`, `handled_by_script`, `handler_failed` (and `replayed`, set by +`replay`). Not free text. + +**`Failed to parse the request body as JSON …`** +Malformed JSON, or (a classic) unescaped quotes when hand-building a body in the shell. Deploy scripts +from a `.rhai` file with `pic scripts deploy` instead of inlining source in a JSON string. + +## Platform + +**`503` with `Retry-After: 1` on a route.** +The instance hit its concurrency cap (`PICLOUD_MAX_CONCURRENT_EXECUTIONS`, default 32). There's no +queue — back off and retry. Long-term: raise the cap (and `PICLOUD_DB_MAX_CONNECTIONS` with it), or +move slow work to `async`. + +**Dashboard dev server can't reach the API.** +Vite proxies `/api` to `http://127.0.0.1:18080` by default — run `picloud` on `18080` or set +`PICLOUD_API`. [More →](../deploy/bare-binary.md#running-the-dashboard-in-dev). + +**Server won't start.** +- Missing `DATABASE_URL` → it aborts. Set it. +- `PICLOUD_DEV_MODE=true` *alone* aborts — also set `PICLOUD_DEV_INSECURE_KEY=i-understand-this-is-insecure`, + or provide a real `PICLOUD_SECRET_KEY`. +- `docker compose up` errors on unset `PICLOUD_ADMIN_USERNAME`/`PICLOUD_ADMIN_PASSWORD` (compose makes + them mandatory). + +**Locked out — forgot the admin password.** +`docker compose exec picloud picloud admin reset-password ` (or run the binary's subcommand +directly). [More →](../reference/cli/server-admin.md). + +## Still stuck? + +Check the server logs (`docker compose logs -f picloud` or stdout), bump `RUST_LOG=debug,picloud=debug`, +and read the execution log for the script (`pic logs ` / the dashboard **Executions** tab). diff --git a/docs/dev-guide/src/reference/cli/pic.md b/docs/dev-guide/src/reference/cli/pic.md new file mode 100644 index 0000000..e99aaf7 --- /dev/null +++ b/docs/dev-guide/src/reference/cli/pic.md @@ -0,0 +1,97 @@ +# The `pic` CLI + +`pic` is the PiCloud command-line client — a thin, scriptable wrapper over the same +[HTTP API](../rest-api/overview.md) the dashboard uses. It's the most ergonomic way to deploy scripts, +inspect logs, and automate from CI. + +## Install & authenticate + +`pic` is the `picloud-cli` crate. Build it from the workspace: + +```sh +cargo build -p picloud-cli --release # binary at target/release/pic +``` + +Authenticate once; the token is saved to `~/.config/picloud/credentials`: + +```sh +# interactive (prompts for password) +pic login --url http://localhost:8000 --username admin + +# non-interactive (CI): password from stdin +printf "$ADMIN_PASSWORD" | pic login --url http://localhost:8000 --username admin --password-stdin + +# or authenticate with a long-lived API key instead of a session +printf 'pic_…' | pic login --url http://localhost:8000 --token - +``` + +`PICLOUD_URL` and `PICLOUD_TOKEN` environment variables work too (handy in CI). + +## Output format + +Global `--output` flag: `tsv` (default — pipe-friendly, columnar) or `json` (`jq`-ready). + +```sh +pic whoami # tsv table +pic --output json apps ls # JSON array +``` + +## Command map + +| Group | Commands | +|---|---| +| Auth | `login`, `logout`, `whoami` | +| Apps | `apps ls`, `apps create `, `apps show `, `apps delete [--force]`, `apps domains {ls,add,rm}` | +| Scripts | `scripts ls [--app]`, `scripts deploy --app `, `scripts invoke `, `scripts delete ` (also top-level `deploy`, `invoke`) | +| Routes | `routes ls `, `routes create --script … --path …`, `routes rm `, `routes check`, `routes match` | +| Logs | `logs [--limit] [--source]` | +| Triggers | `triggers ls`, `triggers create-{kv,cron,docs,files,pubsub,queue,email,dead-letter}`, `triggers create-from-json`, `triggers rm` | +| Topics | `topics {ls,create,update,rm}` | +| Secrets | `secrets {ls,set,rm}` (value via stdin) | +| Queues | `queues {ls,show}` | +| KV | `kv {ls,get}` | +| Files | `files {ls,get,rm}` | +| Dead-letters | `dead-letters {count,ls,show,replay,resolve}` | +| App users | `users {ls,show,reset-password,revoke-sessions}` | +| Members | `members {ls,add,set,rm}` | +| Admins (instance) | `admins {ls,create,show,set,rm}` | +| API keys | `api-keys {mint,ls,rm}` | + +Each resource page in the [HTTP API reference](../rest-api/overview.md) lists the matching `pic` +commands. Run `pic --help` for full flags. + +## Worked examples + +```sh +# Deploy a script and bind a route +pic scripts deploy ./shorten.rhai --app links --name shorten +SID=$(pic --output json scripts ls --app links | jq -r '.[]|select(.name=="shorten").id') +pic routes create --script $SID --path /shorten --method POST + +# A parameterized route, async dispatch +pic routes create --script $SID --path '/r/:code' --path-kind param --method GET +pic routes create --script $SID --path /ingest --method POST --dispatch async + +# Module script, sandbox override, custom timeout +pic scripts deploy ./fmt.rhai --app links --kind module +pic scripts deploy ./heavy.rhai --app links --timeout 60 --sandbox max_operations=5000000 + +# Secrets (value never hits shell history) +printf 'sk_live_…' | pic secrets set --app links stripe_key + +# Mint a scoped, app-bound, expiring API key +pic api-keys mint deploy-bot --scope script:write --scope route:write --app links --expires 30d + +# Tail background-execution logs +pic logs $SID --source queue --limit 20 +``` + +## Notes & gotchas + +- Passwords and API-key values are **never** accepted inline (they'd leak into shell history / `ps`) — + always stdin or interactive prompt. +- `routes create` defaults: host `*` (any), path-kind `exact`, dispatch `sync`. Trigger `create-*` + wrappers default dispatch to `async`. +- `--app` accepts a slug or a UUID anywhere. +- `pic` only ever talks to the HTTP API — it has no special powers the API doesn't. If something works + in `pic` it works in `curl`, and vice versa. diff --git a/docs/dev-guide/src/reference/cli/server-admin.md b/docs/dev-guide/src/reference/cli/server-admin.md new file mode 100644 index 0000000..9c53f42 --- /dev/null +++ b/docs/dev-guide/src/reference/cli/server-admin.md @@ -0,0 +1,42 @@ +# Server admin commands + +Separate from the `pic` developer CLI, the **`picloud` server binary** carries a tiny set of +operator/recovery subcommands you run directly on the host (no auth — they touch the database through +`DATABASE_URL`). These are for out-of-band situations where you can't log in. + +## Bootstrap the first admin (env vars) + +On a **fresh install** (empty `admin_users` table), the server seeds the first admin — always as +`owner` — from environment variables: + +| Variable | Notes | +|---|---| +| `PICLOUD_ADMIN_USERNAME` | required | +| `PICLOUD_ADMIN_PASSWORD_HASH` | preferred — a pre-computed Argon2id PHC string, so the raw password never lands in env/compose | +| `PICLOUD_ADMIN_PASSWORD` | fallback — raw password, hashed on first boot | + +If both the hash and the raw password are set, the hash wins (and a warning is logged). **These are +read only when no admin exists yet** — on a database that already has an admin, they're ignored (with a +warning). After that, manage admins via [`pic admins`](../rest-api/access.md#admin-users-instance) or +the recovery command below. + +## `picloud admin reset-password` + +Reset (and reactivate) an admin's password without logging in — the escape hatch when you've locked +yourself out: + +```sh +# interactive: prompts for a new password on stdin +picloud admin reset-password admin + +# non-interactive: supply a pre-computed Argon2id PHC hash +picloud admin reset-password admin --password-hash '$argon2id$v=19$m=…' +``` + +It re-activates a deactivated admin and **drops all of that admin's sessions** (forcing re-login). It +needs the same `DATABASE_URL` (and master key) the server normally runs with. Run it on the host, e.g. +inside the container: `docker compose exec picloud picloud admin reset-password admin`. + +> This is the *only* subcommand on the server binary besides running the server itself. Everything +> else — creating more admins, changing roles, minting keys — goes through the authenticated API +> (`pic` or `curl`). diff --git a/docs/dev-guide/src/reference/config/capabilities.md b/docs/dev-guide/src/reference/config/capabilities.md new file mode 100644 index 0000000..4aa93fe --- /dev/null +++ b/docs/dev-guide/src/reference/config/capabilities.md @@ -0,0 +1,73 @@ +# Capabilities & roles + +Every control-plane action requires a **capability**. A request's principal is granted capabilities by +its **instance role** and (for members) its per-app **app roles**. This page is the authoritative +mapping. + +## Instance roles + +| Role | Grants | +|---|---| +| `owner` | **everything** — every capability on every app, plus instance settings | +| `admin` | every capability on every app (implicit `app_admin` everywhere) + instance user management; **not** owner-only instance settings | +| `member` | **no** instance authority; only the capabilities its app roles grant, on apps it's a member of | + +So owners and admins never need an explicit app membership. Members start with nothing until granted an +app role. + +## App roles + +App roles form a strict chain — each includes the one before it: **viewer ⊂ editor ⊂ app_admin**. + +| App role | Adds | +|---|---| +| `viewer` | read scripts, routes, logs, KV, docs, files, secrets (names), app-users | +| `editor` | + write scripts & routes; write KV/docs/files; publish pubsub; enqueue; write secrets; send email; manage app-users; `invoke` | +| `app_admin` | + manage domains, triggers, topics, dead-letters; admin app-user actions; app settings & delete | + +## Capability reference + +Instance-level: + +| Capability | Required role | +|---|---| +| `InstanceCreateApp` | owner / admin | +| `InstanceManageUsers` | owner / admin | +| `InstanceManageSettings` | owner only | + +App-level (each is parameterized by the target app; a member needs the listed app role *on that app*): + +| Capability | viewer | editor | app_admin | Used by | +|---|:--:|:--:|:--:|---| +| `AppRead` | ✓ | ✓ | ✓ | read app/scripts/routes | +| `AppLogRead` | ✓ | ✓ | ✓ | read execution logs | +| `AppKvRead` / `AppDocsRead` / `AppFilesRead` / `AppSecretsRead` / `AppUsersRead` | ✓ | ✓ | ✓ | admin data views | +| `AppWriteScript` | | ✓ | ✓ | create/update scripts | +| `AppWriteRoute` | | ✓ | ✓ | manage routes | +| `AppKvWrite` / `AppDocsWrite` / `AppFilesWrite` | | ✓ | ✓ | (data-plane writes) | +| `AppHttpRequest` | | ✓ | ✓ | outbound `http` | +| `AppPubsubPublish` | | ✓ | ✓ | `pubsub::publish_durable` | +| `AppQueueEnqueue` | | ✓ | ✓ | `queue::enqueue` | +| `AppSecretsWrite` | | ✓ | ✓ | set/delete secrets | +| `AppEmailSend` | | ✓ | ✓ | `email::send` | +| `AppUsersWrite` | | ✓ | ✓ | manage app-users | +| `AppInvoke` | | ✓ | ✓ | `invoke()` | +| `AppManageDomains` | | | ✓ | domain claims | +| `AppManageTriggers` | | | ✓ | triggers | +| `AppTopicManage` | | | ✓ | topics | +| `AppDeadLetterManage` | | | ✓ | dead-letters | +| `AppUsersAdmin` | | | ✓ | invitations, resets | +| `AppAdmin` | | | ✓ | app settings, delete, members | + +## API-key scopes + +An [API key](../rest-api/access.md#api-keys) carries a subset of capabilities as **scopes**. The seven +valid scopes: `script:read`, `script:write`, `route:write`, `domain:manage`, `log:read`, `app:admin`, +`instance:admin`. A key bound to one app cannot hold `instance:admin`. + +## A crucial caveat: the data plane runs with full app authority + +The capability model above governs the **control plane** (the admin API). It does **not** gate your +scripts' routes. A public route runs its script with **full authority over its own app's data** — it +can read every secret and every record. PiCloud puts no auth in front of your routes. If a route must +be restricted, the script enforces it. See [Security](../../operations/security.md#the-data-plane-is-public). diff --git a/docs/dev-guide/src/reference/config/env-vars.md b/docs/dev-guide/src/reference/config/env-vars.md new file mode 100644 index 0000000..90b26f5 --- /dev/null +++ b/docs/dev-guide/src/reference/config/env-vars.md @@ -0,0 +1,151 @@ +# Environment variables + +The `picloud` server is configured entirely through environment variables. This is the complete list, +grouped by area. Only `DATABASE_URL` and a master key are strictly required to boot. + +## Required to boot + +| Variable | Default | Purpose | +|---|---|---| +| `DATABASE_URL` | — | **Required.** Postgres connection string. | +| `PICLOUD_SECRET_KEY` | — | Master encryption key, base64 of 32 bytes. Required **unless** dev mode is acknowledged (below). Encrypts secrets and realtime keys at rest. | + +On a **fresh** database you also need the [bootstrap admin](#bootstrap-admin) vars. After that, only +the two above are required. + +## Dev mode + +| Variable | Default | Purpose | +|---|---|---| +| `PICLOUD_DEV_MODE` | `false` | Enables local-dev conveniences (in-memory email sink). **Never in production.** | +| `PICLOUD_DEV_INSECURE_KEY` | — | Set to the literal `i-understand-this-is-insecure` to boot without `PICLOUD_SECRET_KEY`, using a deterministic, world-known dev key. `PICLOUD_DEV_MODE=true` **alone aborts** — you must also set this. Never in production. | + +## Network & process + +| Variable | Default | Purpose | +|---|---|---| +| `PICLOUD_BIND` | `0.0.0.0:8080` | HTTP listen address. (In Compose, picloud binds 8080 *inside* the network; Caddy publishes 8000.) | +| `PICLOUD_PUBLIC_BASE_URL` | `http://localhost:8000` | Public origin reported by `/version` and used to render URLs. | +| `PICLOUD_DB_MAX_CONNECTIONS` | `32` | Postgres pool size (matched to the execution cap). | +| `PICLOUD_MAX_CONCURRENT_EXECUTIONS` | `32` | Global cap on simultaneous script runs. Overflow → `503` + `Retry-After: 1`, no queue. | +| `PICLOUD_SESSION_TTL_HOURS` | `24` | Admin session sliding-window lifetime. | +| `RUST_LOG` | `info` | `tracing` filter, e.g. `info,picloud=debug`. | +| `PICLOUD_CONFIG_DIR` | platform default | Where the `pic` CLI stores credentials. | + +## Bootstrap admin {#bootstrap-admin} + +Read only on a fresh install (empty `admin_users`); ignored afterward. See +[Server admin commands](../cli/server-admin.md). + +| Variable | Purpose | +|---|---| +| `PICLOUD_ADMIN_USERNAME` | first admin's username (seeded as `owner`) | +| `PICLOUD_ADMIN_PASSWORD_HASH` | preferred: Argon2id PHC string | +| `PICLOUD_ADMIN_PASSWORD` | fallback: raw password | + +## Data-plane size caps + +Exceeding any cap throws in the script. All in bytes. + +| Variable | Default | Caps | +|---|---|---| +| `PICLOUD_KV_MAX_VALUE_BYTES` | `262144` (256 KiB) | `kv::...set` value | +| `PICLOUD_DOCS_MAX_VALUE_BYTES` | `262144` | `docs` document | +| `PICLOUD_PUBSUB_MAX_MESSAGE_BYTES` | `262144` | `pubsub::publish_durable` message | +| `PICLOUD_QUEUE_MAX_PAYLOAD_BYTES` | `262144` | `queue::enqueue` message | +| `PICLOUD_SECRET_MAX_VALUE_BYTES` | `262144` | `secrets::set` value | +| `PICLOUD_FILES_MAX_FILE_SIZE_BYTES` | `104857600` (100 MiB) | `files::...create/update` blob | +| `PICLOUD_EMAIL_MAX_MESSAGE_BYTES` | — | inbound/outbound email size | +| `PICLOUD_EMAIL_MAX_RECIPIENTS` | — | recipients per send | +| `PICLOUD_HTTP_MAX_REQUEST_BODY_BYTES` | — | outbound `http` request body | +| `PICLOUD_HTTP_MAX_RESPONSE_BODY_BYTES` | — | outbound `http` response body read | + +## Sandbox ceilings + +Upper bounds on per-script [sandbox overrides](../../guide/writing-scripts.md#sandbox-limits). An +override above the ceiling is rejected at deploy time. Defaults are the conservative built-ins. + +| Variable | Default ceiling | +|---|---| +| `PICLOUD_SANDBOX_MAX_OPERATIONS` | 10,000,000 | +| `PICLOUD_SANDBOX_MAX_STRING_SIZE` | 1 MiB | +| `PICLOUD_SANDBOX_MAX_ARRAY_SIZE` | 100,000 | +| `PICLOUD_SANDBOX_MAX_MAP_SIZE` | 100,000 | +| `PICLOUD_SANDBOX_MAX_CALL_LEVELS` | 128 | +| `PICLOUD_SANDBOX_MAX_EXPR_DEPTH` | 128 | + +Two related platform-level depth bounds (not per-script overridable): + +| Variable | Default | Purpose | +|---|---|---| +| `PICLOUD_MAX_TRIGGER_DEPTH` | `8` | max trigger fan-out / `invoke` re-entry depth | +| `PICLOUD_MODULE_IMPORT_DEPTH_MAX` | `8` | max `import` chain depth | + +## Outbound HTTP (`http` SDK) + +| Variable | Default | Purpose | +|---|---|---| +| `PICLOUD_HTTP_ALLOW_PRIVATE` | `false` | **Dev/test only.** `true` disables the SSRF deny-list (allows requests to private/loopback IPs). Never production. | + +## Email / SMTP + +Configure a relay to make `email::send` work; without it, sends throw `NotConfigured` (or hit the dev +sink in dev mode). + +| Variable | Purpose | +|---|---| +| `PICLOUD_SMTP_HOST` / `PICLOUD_SMTP_PORT` | relay address | +| `PICLOUD_SMTP_USER` / `PICLOUD_SMTP_PASSWORD` | relay auth | +| `PICLOUD_SMTP_TLS` | TLS mode | +| `PICLOUD_SMTP_TIMEOUT_SECS` | send timeout | + +## Triggers, dispatcher & background workers + +| Variable | Default | Purpose | +|---|---|---| +| `PICLOUD_DISPATCHER_ASYNC_EXEC_TIMEOUT_SEC` | `120` | per-message executor budget for async/triggered runs | +| `PICLOUD_DISPATCHER_TICK_INTERVAL_MS` | — | dispatcher poll interval | +| `PICLOUD_CRON_TICK_INTERVAL_MS` | — | cron scheduler resolution | +| `PICLOUD_QUEUE_RECLAIM_INTERVAL_MS` | — | how often expired claims are reclaimed | +| `PICLOUD_QUEUE_DEFAULT_VISIBILITY_TIMEOUT_SECS` | — | default queue claim lease | +| `PICLOUD_TRIGGER_RETRY_MAX_ATTEMPTS` | `3` | default trigger retry attempts | +| `PICLOUD_TRIGGER_RETRY_BACKOFF` | `exponential` | default backoff shape | +| `PICLOUD_TRIGGER_RETRY_BASE_MS` | `1000` | default backoff base | +| `PICLOUD_TRIGGER_RETRY_JITTER_PCT` | — | retry jitter | +| `PICLOUD_DEAD_LETTER_RETENTION_DAYS` | `30` | how long dead-letters are kept | +| `PICLOUD_ABANDONED_EXECUTIONS_RETENTION_DAYS` | — | sweep window for stale in-flight executions | + +## Realtime, subscriber tokens & caches + +| Variable | Default | Purpose | +|---|---|---| +| `PICLOUD_REALTIME_HEARTBEAT_SEC` | `30` | SSE heartbeat interval | +| `PICLOUD_REALTIME_BROADCAST_CAPACITY` | — | per-topic broadcast buffer | +| `PICLOUD_SUBSCRIBER_TOKEN_TTL_DEFAULT_SEC` / `_MIN_SEC` / `_MAX_SEC` | — | bounds for `pubsub::subscriber_token` TTLs | +| `PICLOUD_SCRIPT_CACHE_SIZE` / `PICLOUD_MODULE_CACHE_SIZE` | — | in-memory cache sizes | + +## Files storage + +| Variable | Default | Purpose | +|---|---|---| +| `PICLOUD_FILES_ROOT` | `./data` | filesystem root for `files` blobs | +| `PICLOUD_FILES_ORPHAN_SWEEP_INTERVAL_SEC` / `PICLOUD_FILES_ORPHAN_TMP_TTL_SEC` | — | cleanup of stale temp blobs | + +## App-user token lifetimes (`users` SDK) + +| Variable | Purpose | +|---|---| +| `PICLOUD_APP_USER_SESSION_TTL_HOURS` / `_ABSOLUTE_HOURS` | sliding & absolute session lifetime | +| `PICLOUD_APP_USER_VERIFICATION_TTL_HOURS` | email-verification token TTL | +| `PICLOUD_APP_USER_PASSWORD_RESET_TTL_HOURS` | password-reset token TTL | +| `PICLOUD_APP_USER_INVITATION_TTL_DAYS` | invitation TTL | + +## `pic` CLI + +| Variable | Purpose | +|---|---| +| `PICLOUD_URL` | default server URL for `pic` | +| `PICLOUD_TOKEN` | bearer token for `pic` (CI) | + +> Values shown as "—" have an internal default that's safe to leave unset; consult the running +> instance or release notes if you need the exact number for capacity planning. diff --git a/docs/dev-guide/src/reference/rest-api/access.md b/docs/dev-guide/src/reference/rest-api/access.md new file mode 100644 index 0000000..c3f354d --- /dev/null +++ b/docs/dev-guide/src/reference/rest-api/access.md @@ -0,0 +1,84 @@ +# Members, admins & API keys + +Three ways identity attaches to the control plane. See +[Capabilities & roles](../config/capabilities.md) for the full permission model. + +## App members + +Grant other admin users a per-app role. Under `/api/v1/admin/apps/{id_or_slug}/members`, capability +**`AppAdmin`**. + +| Method & path | Notes | +|---|---| +| `GET …/members` | list members + roles | +| `POST …/members` | grant `{user_id, role}` (`409` if already a member) | +| `PATCH …/members/{user_id}` | change `{role}` | +| `DELETE …/members/{user_id}` | remove | + +Roles: `viewer` (read scripts/logs/routes/data), `editor` (+ write scripts/routes/KV/docs/files/…), +`app_admin` (+ members, domains, triggers, topics, dead-letters). Instance `owner`/`admin` implicitly +have full access to every app, so removing the last explicit member can't orphan an app. **CLI:** +`pic members ls --app demo`, `pic members add --app demo --user --role editor`, +`pic members set --app demo --user --role app_admin`, `pic members rm --app demo --user `. + +## Admin users (instance) + +The platform accounts. Under `/api/v1/admin/admins`, capability **`InstanceManageUsers`** (owner/admin). + +| Method & path | Notes | +|---|---| +| `GET /api/v1/admin/admins` | list | +| `POST /api/v1/admin/admins` | create `{username, password, email?, instance_role?}` (default role `admin`) | +| `GET /api/v1/admin/admins/{id}` | one | +| `PATCH /api/v1/admin/admins/{id}` | update `{username?, email?, password?, is_active?, instance_role?}` | +| `DELETE /api/v1/admin/admins/{id}` | delete | + +Instance roles: `owner` (everything, including instance settings), `admin` (everything except +owner-only settings), `member` (no instance powers; only the apps they're a member of). Deactivating an +admin also expires their API keys and sessions; you can't deactivate the last active admin. The first +admin is seeded from env vars on a fresh install (see +[Server admin commands](../cli/server-admin.md)). **CLI:** `pic admins ls`, +`pic admins create alice --password - --instance-role admin`, `pic admins set --active false`. + +## API keys + +Long-lived bearer tokens for non-interactive use (CI, scripts). Under `/api/v1/admin/api-keys`; any +authenticated principal manages **their own** keys. + +| Method & path | Notes | +|---|---| +| `GET /api/v1/admin/api-keys` | list the caller's keys (no token shown) | +| `POST /api/v1/admin/api-keys` | mint `{name, scopes, app_id?, expires_at?}` | +| `DELETE /api/v1/admin/api-keys/{id}` | revoke (takes effect immediately) | + +```sh +curl -X POST $PICLOUD/api/v1/admin/api-keys -H "Authorization: Bearer $TOKEN" \ + -H 'Content-Type: application/json' -d '{"name":"ci","scopes":["script:read","log:read"]}' +``` +```json +{"id":"…","prefix":"D3ZBPTRT","name":"ci","scopes":["script:read","log:read"], + "app_id":null,"expires_at":null,"last_used_at":null,"created_at":"…Z", + "raw_token":"pic_D3ZBPTRT4AKU3VUCJM5PR2WWEJWV35C6YBLBPJJTYLURD6OW4HUQ"} +``` + +The full `raw_token` is shown **exactly once**, on mint. Store it then; afterward only the `prefix` is +visible. + +**Valid scopes** (exactly these seven): + +| Scope | Grants | +|---|---| +| `script:read` | read scripts | +| `script:write` | create/update scripts | +| `route:write` | manage routes | +| `domain:manage` | manage domains | +| `log:read` | read execution logs | +| `app:admin` | full app administration | +| `instance:admin` | instance-wide administration | + +> Common mistake: `app:read`, `app:write`, `instance:write` are **not** scopes and are rejected (`422`). + +Bind a key to one app with `"app_id"`; a bound key then can't carry `instance:admin` (rejected). Use +`"expires_at"` (RFC 3339) for short-lived keys. Present a key just like a session token: +`Authorization: Bearer pic_…`. **CLI:** `pic api-keys mint ci --scope script:read --scope log:read` +(optionally `--app demo`, `--expires 30d`), `pic api-keys ls`, `pic api-keys rm `. diff --git a/docs/dev-guide/src/reference/rest-api/app-users.md b/docs/dev-guide/src/reference/rest-api/app-users.md new file mode 100644 index 0000000..dafab09 --- /dev/null +++ b/docs/dev-guide/src/reference/rest-api/app-users.md @@ -0,0 +1,43 @@ +# App users + +These are the **administrative** endpoints over your app's end-users — the people who register through +your scripts' [`users::*`](../sdk/users.md) calls. Operators use them to inspect users, reset +passwords, revoke sessions, and manage invitations. + +> **These are not signup/login endpoints.** End-users never call these. Registration and login happen +> in *your* scripts via the `users` SDK, on routes you define. There is no built-in `/signup` or +> `/login`. See the [TODO API tutorial](../../examples/todo-api.md). + +Under `/api/v1/admin/apps/{id_or_slug}`, capabilities `AppUsersRead` (reads), `AppUsersWrite` +(create/update), `AppUsersAdmin` (delete, reset-password, revoke-sessions). + +| Method & path | Notes | +|---|---| +| `GET …/users` | list end-users | +| `POST …/users` | create `{email, password, display_name?}` (`password` is **required**) | +| `GET …/users/{user_id}` | one user | +| `PATCH …/users/{user_id}` | update `{display_name?}` (only `display_name` is patchable here) | +| `DELETE …/users/{user_id}` | delete | +| `POST …/users/{user_id}/reset-password` | mint/apply a reset (returns a one-shot token) | +| `POST …/users/{user_id}/revoke-sessions` | invalidate all of the user's sessions | +| `GET …/invitations` | list invitations | +| `POST …/invitations` | create `{email, display_name?, roles?, template?}` | +| `DELETE …/invitations/{invite_id}` | revoke an invitation | + +(There is no per-user `…/users/{user_id}/invitations` endpoint — invitations are managed at the app +level via `…/invitations`. Roles aren't set through the create/patch body here; manage them from a +script with [`users::add_role`](../sdk/users.md#roles).) + +```sh +curl -X POST $PICLOUD/api/v1/admin/apps/$APP/users -H "Authorization: Bearer $TOKEN" \ + -H 'Content-Type: application/json' -d '{"email":"u1@example.com","password":"hunter2pass"}' +``` +```json +{"id":"4fd7df09-…","app_id":"…","email":"u1@example.com","display_name":null, + "email_verified_at":null,"last_login_at":null,"created_at":"…Z","updated_at":"…Z","roles":[]} +``` + +Passwords are hashed with Argon2id; only hashes are stored — operators cannot read them. **CLI:** +`pic users ls --app demo`, `pic users show --app demo `, +`pic users reset-password --app demo ` (prints a one-shot token), +`pic users revoke-sessions --app demo `. diff --git a/docs/dev-guide/src/reference/rest-api/apps.md b/docs/dev-guide/src/reference/rest-api/apps.md new file mode 100644 index 0000000..ad56309 --- /dev/null +++ b/docs/dev-guide/src/reference/rest-api/apps.md @@ -0,0 +1,62 @@ +# Apps & domains + +Apps are tenants; domains route hosts to apps. See [Core concepts](../../guide/concepts.md#apps). + +## Apps + +| Method & path | Capability | Notes | +|---|---|---| +| `GET /api/v1/admin/apps` | authenticated | lists apps the caller can see (members: only theirs) | +| `POST /api/v1/admin/apps` | `InstanceCreateApp` (owner/admin) | create | +| `GET /api/v1/admin/apps/{id_or_slug}` | `AppRead` | one app | +| `PATCH /api/v1/admin/apps/{id_or_slug}` | `AppAdmin` | update name/description/slug | +| `DELETE /api/v1/admin/apps/{id_or_slug}` | `AppAdmin` | delete (cascades scripts, routes, data) | +| `POST /api/v1/admin/apps/{id_or_slug}/slug:check` | `AppAdmin` | `{"new_slug":"…"}` → `{ok, conflict_kind?, current_app?, reason?}` | + +Create: + +```sh +curl -X POST $PICLOUD/api/v1/admin/apps -H "Authorization: Bearer $TOKEN" \ + -H 'Content-Type: application/json' \ + -d '{"slug":"demo","name":"Demo App","description":"…"}' +``` +```json +{"id":"41f42060-…","slug":"demo","name":"Demo App","description":null, + "created_at":"2026-…Z","updated_at":"2026-…Z"} +``` + +**Slug rules:** `^[a-z0-9][a-z0-9-]{0,62}$`, and not one of the reserved words (`new`, `api`, `admin`, +`apps`, `login`, `healthz`, `version`, …). Renaming a slug records the old one for redirects; reclaim a +released slug with `"force_takeover": true` in the create/patch body. + +## Domains + +A non-`default` app's routes only match once the app **claims** the request `Host`. + +| Method & path | Capability | +|---|---| +| `GET /api/v1/admin/apps/{id_or_slug}/domains` | `AppRead` | +| `POST /api/v1/admin/apps/{id_or_slug}/domains` | `AppManageDomains` | +| `DELETE /api/v1/admin/apps/{id_or_slug}/domains/{domain_id}` | `AppManageDomains` | + +```sh +curl -X POST $PICLOUD/api/v1/admin/apps/demo/domains -H "Authorization: Bearer $TOKEN" \ + -H 'Content-Type: application/json' -d '{"pattern":"demo.localhost"}' +``` +```json +{"id":"6ff04d56-…","app_id":"41f42060-…","pattern":"demo.localhost", + "shape":"exact","shape_key":"exact:demo.localhost","created_at":"2026-…Z"} +``` + +**Pattern shapes** (use `{name}`, never `:name`, in domains): + +- exact — `app.example.com` +- wildcard — `*.example.com` +- parameterized — `{tenant}.example.com` (the segment is captured into a route param) + +The most specific claim wins. Claiming a host another app already holds (in the same shape) returns +`409`. The `default` app ships claiming `localhost`. Locally, claim `something.localhost` and test with +`curl -H 'Host: something.localhost' $PICLOUD/...`. + +**CLI:** `pic apps create demo --name "Demo App"`, `pic apps domains add demo demo.localhost`, +`pic apps ls`, `pic apps show demo`. diff --git a/docs/dev-guide/src/reference/rest-api/data-admin.md b/docs/dev-guide/src/reference/rest-api/data-admin.md new file mode 100644 index 0000000..c7a301f --- /dev/null +++ b/docs/dev-guide/src/reference/rest-api/data-admin.md @@ -0,0 +1,61 @@ +# Secrets, KV & files (admin views) + +These are the operator-facing views over data your scripts write through the +[`secrets`](../sdk/secrets.md), [`kv`](../sdk/storage.md#kv), and [`files`](../sdk/storage.md#files) +SDKs. They are read-mostly: you *write* through scripts, and inspect (and prune) here. + +## Secrets {#secrets} + +Under `/api/v1/admin/apps/{app_id}/secrets`. Values are **never** returned — only names. + +| Method & path | Capability | Notes | +|---|---|---| +| `GET …/secrets?cursor=&limit=` | `AppSecretsRead` | list names | +| `POST …/secrets` | `AppSecretsWrite` | set/upsert `{name, value}` | +| `DELETE …/secrets/{name}` | `AppSecretsWrite` | delete | + +```sh +curl -X POST $PICLOUD/api/v1/admin/apps/$APP/secrets -H "Authorization: Bearer $TOKEN" \ + -H 'Content-Type: application/json' -d '{"name":"demo_key","value":"s3cr3t"}' + +curl $PICLOUD/api/v1/admin/apps/$APP/secrets -H "Authorization: Bearer $TOKEN" +# {"secrets":[{"name":"demo_key","updated_at":"2026-…Z"}],"next_cursor":null} +``` + +There is no update verb — `POST` upserts. **CLI:** `printf 's3cr3t' | pic secrets set --app demo +demo_key`, `pic secrets ls --app demo`, `pic secrets rm --app demo demo_key` (value read from stdin so +it never hits shell history). + +## KV {#kv} + +Under `/api/v1/admin/apps/{app_id}/kv`. Read-only (writes go through `kv::...set`). + +| Method & path | Capability | Notes | +|---|---|---| +| `GET …/kv?collection=&cursor=&limit=` | `AppKvRead` | list keys in a collection | +| `GET …/kv/{collection}/{key}` | `AppKvRead` | one value | + +```sh +curl "$PICLOUD/api/v1/admin/apps/$APP/kv?collection=counters" -H "Authorization: Bearer $TOKEN" +# {"keys":[…],"next_cursor":null} +curl $PICLOUD/api/v1/admin/apps/$APP/kv/counters/home -H "Authorization: Bearer $TOKEN" +# {"value":3} +``` + +**CLI:** `pic kv ls --app demo --collection counters`, `pic kv get --app demo --collection counters home`. + +## Files {#files} + +Under `/api/v1/admin/apps/{app_id}/files`. + +| Method & path | Capability | Notes | +|---|---|---| +| `GET …/files?collection=&cursor=&limit=` | `AppFilesRead` | list metadata | +| `GET …/files/{collection}/{file_id}` | `AppFilesRead` | download the bytes (sets `Content-Type`, `Content-Disposition`, `Content-Length`) | +| `DELETE …/files/{collection}/{file_id}` | `AppFilesWrite` | delete | + +**CLI:** `pic files ls --app demo --collection avatars`, `pic files get --app demo --collection avatars +--id --out a.png`, `pic files rm --app demo --collection avatars --id `. + +Files live on disk under `PICLOUD_FILES_ROOT` (default `./data`) at +`/files////`; metadata is in Postgres. diff --git a/docs/dev-guide/src/reference/rest-api/overview.md b/docs/dev-guide/src/reference/rest-api/overview.md new file mode 100644 index 0000000..19aa7b1 --- /dev/null +++ b/docs/dev-guide/src/reference/rest-api/overview.md @@ -0,0 +1,106 @@ +# HTTP API — overview & authentication + +The control plane is a REST API under `/api/v1/`. The dashboard and the `pic` CLI are both just clients +of it; anything they do, you can do with `curl`. The **data plane** (your scripts' routes, +`/execute/{id}`, `/healthz`, `/version`) is separate and covered where relevant. + +Base URL in this reference: `$PICLOUD` (e.g. `http://localhost:8000` for the Compose stack, or +`http://localhost:18080` for a bare binary). + +## Authentication + +Every `/api/v1/admin/*` endpoint (except `auth/login`) requires a **bearer token**: + +``` +Authorization: Bearer +``` + +A token is either a **session token** (from `auth/login`) or an **API key** (`pic_…`, minted via +[api-keys](access.md#api-keys)). There is no cookie auth. + +### Log in + +```sh +curl -X POST $PICLOUD/api/v1/admin/auth/login \ + -H 'Content-Type: application/json' \ + -d '{"username":"admin","password":"…"}' +``` +```json +{"user":{"id":"…","username":"admin","instance_role":"owner","email":null}, + "token":"V-f-0ey3eEcF…","expires_at":"2026-06-18T19:49:39Z"} +``` + +| Endpoint | Auth | Purpose | +|---|---|---| +| `POST /api/v1/admin/auth/login` | none | exchange username+password for a session token | +| `POST /api/v1/admin/auth/logout` | optional | revoke the current session (idempotent → `204`) | +| `GET /api/v1/admin/auth/me` | bearer | the principal the token resolves to | + +Sessions slide: each authenticated request extends the TTL (`PICLOUD_SESSION_TTL_HOURS`, default 24). +Login is rate-limited per IP and per username. + +## Principals, roles & capabilities + +The token resolves to a **principal** with an **instance role** (`owner` > `admin` > `member`). +Members additionally hold per-app roles. Each endpoint requires a **capability**; the full mapping is +in [Capabilities & roles](../config/capabilities.md). A request that authenticates but lacks the +capability gets `403`. + +## Conventions + +- **Content type:** request and response bodies are JSON (`Content-Type: application/json`). +- **IDs** are UUID strings. Apps also accept their **slug** wherever an id is taken (`{id_or_slug}`). +- **Update verb is `PATCH`** everywhere — send only the fields you want to change — **except scripts, + which use `PUT`** (full update). +- **Two endpoints use a `:verb` suffix** (not a sub-path): `POST /routes:check`, `POST /routes:match`, + `POST /apps/{id_or_slug}/slug:check`. +- **Pagination** is keyset/cursor based: list endpoints return a `next_cursor` (`null` when exhausted); + pass it back as `?cursor=…`. Some accept `?limit=`. + +## Errors + +Errors are JSON with an `error` string and an appropriate status: + +```json +{"error":"no route matches GET /nope"} +``` + +| Status | Meaning | +|---|---| +| `400` | malformed request / bad JSON | +| `401` | missing or invalid token | +| `403` | authenticated but missing the capability | +| `404` | resource not found (or, on the data plane, no app/route matched) | +| `409` | conflict (duplicate, state violation) | +| `422` | validation error (bad Rhai, route conflict, invalid scope, …) | +| `429` | rate-limited (login) | +| `503` | overloaded — execution gate full, `Retry-After: 1` (data plane) | + +## `GET /version` and `GET /healthz` + +Public, unauthenticated: + +```sh +curl $PICLOUD/healthz # ok +curl $PICLOUD/version +``` +```json +{"api":1,"product":"1.1.9","public_base_url":"http://localhost:8000","schema":44,"sdk":"1.10","wire":1} +``` + +`/healthz` is a liveness probe (the literal string `ok`). `/version` reports the +[five version surfaces](../../guide/concepts.md#the-five-version-surfaces) plus the configured public +base URL. + +## Map of the API + +| Area | Page | +|---|---| +| Apps, domains | [Apps & domains](apps.md) | +| Scripts, routes, logs | [Scripts, routes & logs](scripts.md) | +| Triggers | [Triggers](triggers.md) | +| Topics, realtime SSE | [Topics & realtime](topics.md) | +| Secrets, KV, files (admin views) | [Secrets, KV & files](data-admin.md) | +| Queues, dead-letters | [Queues & dead-letters](queues.md) | +| App end-users | [App users](app-users.md) | +| Members, admin users, API keys | [Members, admins & API keys](access.md) | diff --git a/docs/dev-guide/src/reference/rest-api/queues.md b/docs/dev-guide/src/reference/rest-api/queues.md new file mode 100644 index 0000000..c6e6b4f --- /dev/null +++ b/docs/dev-guide/src/reference/rest-api/queues.md @@ -0,0 +1,46 @@ +# Queues & dead-letters + +Operator views over the durable [`queue`](../sdk/messaging.md#queue) system and the +[dead-letters](../sdk/composition.md#dead-letters) that failed executions produce. + +## Queues + +Under `/api/v1/admin/apps/{app_id}/queues`, capability **`AppLogRead`** (read-only inspection of +operational state). Enqueue from scripts; consume via a `queue` trigger. + +| Method & path | Notes | +|---|---| +| `GET …/queues` | list queues with depth counts | +| `GET …/queues/{queue_name}` | one queue's stats + its registered consumer | + +A summary carries `queue_name`, `total`, `pending`, `claimed`. **CLI:** `pic queues ls --app demo`, +`pic queues show --app demo emails.send`. + +## Dead-letters + +Under `/api/v1/admin/apps/{app_id}/dead_letters`, capability **`AppDeadLetterManage`** (app_admin+). +A dead-letter is written when a triggered execution exhausts its retries. + +| Method & path | Notes | +|---|---| +| `GET …/dead_letters?unresolved=true&limit=&offset=` | list | +| `GET …/dead_letters/count?unresolved=true` | bare count (cheap; for alerting) | +| `GET …/dead_letters/{dl_id}` | one row, full payload + error | +| `POST …/dead_letters/{dl_id}/replay` | re-enqueue the original event; marks the row resolved `replayed` → `204 No Content` | +| `POST …/dead_letters/{dl_id}/resolve` | close without replaying (reason: one of `ignored`, `handled_by_script`, `handler_failed`) → `204 No Content` | + +```sh +curl "$PICLOUD/api/v1/admin/apps/$APP/dead_letters?unresolved=true" -H "Authorization: Bearer $TOKEN" +curl -X POST $PICLOUD/api/v1/admin/apps/$APP/dead_letters/$DL/replay -H "Authorization: Bearer $TOKEN" +``` + +A `dead_letter` [trigger](triggers.md) can run a script automatically when a dead-letter appears (to +alert, or to `dead_letters::replay`/`resolve` programmatically). Retention defaults to 30 days +(`PICLOUD_DEAD_LETTER_RETENTION_DAYS`), swept periodically. + +**CLI:** `pic dead-letters count --app demo`, `pic dead-letters ls --app demo --unresolved`, +`pic dead-letters show --app demo `, `pic dead-letters replay --app demo `, +`pic dead-letters resolve --app demo --reason "handled manually"`. + +The [webhook tutorial](../../examples/webhook-receiver.md#dead-letters) walks the full failure → replay +loop. diff --git a/docs/dev-guide/src/reference/rest-api/scripts.md b/docs/dev-guide/src/reference/rest-api/scripts.md new file mode 100644 index 0000000..69c1a9d --- /dev/null +++ b/docs/dev-guide/src/reference/rest-api/scripts.md @@ -0,0 +1,101 @@ +# Scripts, routes & logs + +## Scripts + +| Method & path | Capability | Notes | +|---|---|---| +| `GET /api/v1/admin/scripts?app=` | `AppRead` | list (filter by app); returns a plain array, not paginated | +| `POST /api/v1/admin/scripts` | `AppWriteScript` | create | +| `GET /api/v1/admin/scripts/{id}` | `AppRead` | one script (with `source`) | +| `PUT /api/v1/admin/scripts/{id}` | `AppWriteScript` | **full update** (note: PUT, not PATCH) | +| `DELETE /api/v1/admin/scripts/{id}` | `AppAdmin` | delete | +| `GET /api/v1/admin/scripts/{id}/logs?limit=&cursor=&source=` | `AppLogRead` | execution logs (below) | + +Create: + +```sh +curl -X POST $PICLOUD/api/v1/admin/scripts -H "Authorization: Bearer $TOKEN" \ + -H 'Content-Type: application/json' -d '{ + "app_id":"41f42060-…", + "name":"greet", + "source":"return #{ statusCode: 200, body: #{ ok: true } };", + "kind":"endpoint" + }' +``` + +Fields: `app_id` (required), `name` (required, unique within the app), `source` (required), optional +`description`, `kind` (`"endpoint"` default | `"module"`), `timeout_seconds`, `memory_limit_mb`, and a +`sandbox` object of [overrides](../../guide/writing-scripts.md#sandbox-limits). New scripts default to +`timeout_seconds: 30`, `memory_limit_mb: 256`. + +The response is the stored script (with `id`, `version`, timestamps). The source is validated at create +time: it must parse as Rhai, and a `module` may contain only `fn`/`const`. Invalid source → `422`. + +## Routes + +| Method & path | Capability | Notes | +|---|---|---| +| `GET /api/v1/admin/scripts/{id}/routes` | `AppRead` | routes bound to a script | +| `POST /api/v1/admin/scripts/{id}/routes` | `AppWriteRoute` | bind a route | +| `DELETE /api/v1/admin/routes/{route_id}` | `AppWriteRoute` | unbind | +| `POST /api/v1/admin/routes:check` | `AppRead` | conflict pre-check (note the `:` suffix) | +| `POST /api/v1/admin/routes:match` | `AppRead` | what would a URL match? | + +Bind: + +```sh +curl -X POST $PICLOUD/api/v1/admin/scripts/$SID/routes -H "Authorization: Bearer $TOKEN" \ + -H 'Content-Type: application/json' -d '{ + "host_kind":"any", "host":"", + "path_kind":"param", "path":"/users/:id", + "method":"GET", "dispatch_mode":"sync" + }' +``` +```json +{"id":"…","app_id":"…","script_id":"…","host_kind":"any","host":"","host_param_name":null, + "path_kind":"param","path":"/users/:id","method":null,"dispatch_mode":"sync","created_at":"…Z"} +``` + +Fields: + +- `host_kind`: `any` (default) | `strict` (exact `host`) | `wildcard` (`*.host`, with optional + `host_param_name` to capture the subdomain into a param); +- `path_kind`: `exact` | `param` (`/users/:id`) | `prefix` (`/files/*`, tail → `ctx.request.rest`); +- `method`: `GET`/`POST`/… or omit (`null`) to match any; +- `dispatch_mode`: `sync` (default) | `async` (→ `202 Accepted`, background run). + +Routes under `/api/`, `/admin/`, `/healthz`, `/version` are rejected. Conflicts are detected within the +app only: + +```sh +curl -X POST $PICLOUD/api/v1/admin/routes:check -H "Authorization: Bearer $TOKEN" \ + -H 'Content-Type: application/json' \ + -d '{"app_id":"…","host_kind":"any","host":"","path_kind":"exact","path":"/hello"}' +# {"ok":false,"conflicting_route":{…},"conflict_reason":"IdenticalExact"} +``` + +## Execution logs + +```sh +curl "$PICLOUD/api/v1/admin/scripts/$SID/logs?limit=20&source=http" -H "Authorization: Bearer $TOKEN" +``` + +Each entry: + +```json +{"id":"…","app_id":"…","script_id":"…","request_id":"…", + "request_path":"/count","request_headers":{…},"request_body":null, + "response_code":200,"response_body":null, + "script_logs":[],"duration_ms":5,"status":"success","source":"http","created_at":"…Z"} +``` + +- `?source=` filters by origin: `http`, `kv`, `docs`, `files`, `cron`, `queue`, `pubsub`, `email`, + `invoke`, `dead_letter`. Omit for all. +- `?cursor=` is the keyset cursor (`_`) from the previous page; `?limit=` defaults to 50 + (max 200). +- **`script_logs` and `response_body` are populated for async/triggered runs, but empty/null for + synchronous HTTP runs** — see [Writing scripts → Logging](../../guide/writing-scripts.md#logging). + +**CLI:** `pic scripts deploy file.rhai --app demo`, `pic scripts ls --app demo`, +`pic routes create --script $SID --path /users/:id --path-kind param --method GET`, +`pic routes check --app demo --path /hello`, `pic logs $SID --source http`. diff --git a/docs/dev-guide/src/reference/rest-api/topics.md b/docs/dev-guide/src/reference/rest-api/topics.md new file mode 100644 index 0000000..eea73e8 --- /dev/null +++ b/docs/dev-guide/src/reference/rest-api/topics.md @@ -0,0 +1,57 @@ +# Topics & realtime + +Scripts publish to topics with [`pubsub::publish_durable`](../sdk/messaging.md#pubsub) — that needs no +registration. **Registering a topic** is only needed to let *external* clients subscribe over +Server-Sent Events (SSE) at `/realtime/topics/{topic}`. + +## Topic registry + +Under `/api/v1/admin/apps/{app_id}/topics`, capability **`AppTopicManage`** (app_admin+). + +| Method & path | Notes | +|---|---| +| `GET …/topics` | list | +| `POST …/topics` | register `{name, external_subscribable?, auth_mode?}` | +| `PATCH …/topics/{name}` | update external/auth settings | +| `DELETE …/topics/{name}` | unregister (disconnects live subscribers) | + +`name` is a concrete topic (no wildcards). `auth_mode` controls what an external subscriber must +present: + +| `auth_mode` | Subscriber must… | +|---|---| +| `public` | nothing — anyone can subscribe | +| `token` | present an HMAC subscriber token from [`pubsub::subscriber_token`](../sdk/messaging.md#pubsub) | +| `session` | present an app-user session token from [`users::login`](../sdk/users.md) | + +## Subscribing (SSE) + +```sh +# public topic +curl -N $PICLOUD/realtime/topics/orders.created + +# token / session topic — pass the token as a query param or bearer header +curl -N "$PICLOUD/realtime/topics/orders.created?token=$SUBTOKEN" +curl -N $PICLOUD/realtime/topics/orders.created -H "Authorization: Bearer $SUBTOKEN" +``` + +The stream emits each published message as an SSE `data:` event, plus periodic heartbeats +(`PICLOUD_REALTIME_HEARTBEAT_SEC`, default 30). The browser equivalent is `new EventSource(url)`. + +Minting a subscriber token inside a script (for a `token`-mode topic): + +```rhai +let t = pubsub::subscriber_token(["orders.created"], 3600); // valid 1h +return #{ statusCode: 200, body: #{ token: t } }; +``` + +## CLI + +```sh +pic topics create --app demo orders.created --external --auth-mode token # --app is a flag; name is positional +pic topics ls --app demo +pic topics update --app demo orders.created --auth-mode session +pic topics rm --app demo orders.created +``` + +The [file-upload tutorial](../../examples/file-upload.md) wires a publish → SSE subscriber end to end. diff --git a/docs/dev-guide/src/reference/rest-api/triggers.md b/docs/dev-guide/src/reference/rest-api/triggers.md new file mode 100644 index 0000000..41b95c7 --- /dev/null +++ b/docs/dev-guide/src/reference/rest-api/triggers.md @@ -0,0 +1,75 @@ +# Triggers + +Triggers fire a script on an event instead of an HTTP request. See +[Core concepts](../../guide/concepts.md#triggers-and-events) for the model and +[`ctx.event`](../sdk/ctx-and-events.md#events) for what each kind delivers. + +All endpoints are under `/api/v1/admin/apps/{app_id}/triggers` and require **`AppManageTriggers`** +(app_admin+). + +| Method & path | Notes | +|---|---| +| `GET …/triggers` | list (`{ "triggers": [ … ] }`) | +| `POST …/triggers/{kind}` | create — **one sub-path per kind** (below) | +| `DELETE …/triggers/{trigger_id}` | delete | + +There is **no update verb** — to change a trigger, delete and recreate. The eight kinds (each its own +`POST` sub-path): `kv`, `docs`, `files`, `pubsub`, `cron`, `queue`, `email`, `dead_letter`. + +## Common fields + +Every create body takes `script_id` plus optional dispatch/retry settings: + +- `dispatch_mode`: `sync` | `async` (default `async`); +- `retry_max_attempts`, `retry_backoff` (`exponential` | `linear` | `fixed`), `retry_base_ms` — all + optional, defaulting from instance config (`3` / `exponential` / `1000ms`). + +A created trigger looks like: + +```json +{"id":"f3214f88-…","app_id":"…","script_id":"…","kind":"cron","enabled":true, + "dispatch_mode":"async","retry_max_attempts":3,"retry_backoff":"exponential", + "retry_base_ms":1000,"registered_by_principal":"…","created_at":"…Z","updated_at":"…Z"} +``` + +## Per-kind bodies + +| Kind | `POST …/triggers/` body (besides `script_id`) | +|---|---| +| `kv` | `collection_glob`, optional `ops` (`["insert","update","delete"]`; empty = any) | +| `docs` | `collection_glob`, optional `ops` | +| `files` | `collection_glob`, optional `ops` | +| `pubsub` | `topic_glob` | +| `cron` | `schedule` (6-field cron, with seconds), optional `timezone` (IANA, default UTC) | +| `queue` | `queue_name`, optional `visibility_timeout_secs` (≥ 30) | +| `email` | optional `from_glob`; an `inbound_secret` for HMAC verification | +| `dead_letter` | optional `source`/`trigger_id`/`script_id` filters (omit = every DL in the app) | + +Example — a cron trigger every hour on the hour: + +```sh +curl -X POST $PICLOUD/api/v1/admin/apps/$APP/triggers/cron -H "Authorization: Bearer $TOKEN" \ + -H 'Content-Type: application/json' \ + -d '{"script_id":"'$SID'","schedule":"0 0 * * * *","timezone":"Europe/Berlin"}' +``` + +Globs match collection/topic names: `*` (all), `users`, `events_*`, `orders.*`. + +## CLI + +The CLI has per-kind wrappers and a JSON escape hatch: + +```sh +pic triggers create-cron --app demo --script $SID --schedule "0 0 * * * *" +pic triggers create-kv --app demo --script $SID --collection 'users' --op insert +pic triggers create-queue --app demo --script $SID --queue emails.send +pic triggers create-pubsub --app demo --script $SID --topic 'orders.*' +pic triggers create-email --app demo --script $SID --inbound-secret +pic triggers ls --app demo +pic triggers rm --app demo +# advanced retry/dispatch tuning beyond the wrappers: +pic triggers create-from-json --app demo --kind docs --body @trigger.json +``` + +(CLI wrappers default `dispatch` to `async`.) Inbound email also requires the platform's inbound +webhook to be reachable; see [Email](../sdk/email.md). diff --git a/docs/dev-guide/src/reference/sdk/composition.md b/docs/dev-guide/src/reference/sdk/composition.md new file mode 100644 index 0000000..a062ee2 --- /dev/null +++ b/docs/dev-guide/src/reference/sdk/composition.md @@ -0,0 +1,91 @@ +# Composition: invoke, retry, dead_letters + +Three small namespaces for composing scripts and handling failure. + +--- + +## `invoke` — call another script {#invoke} + +*Since SDK 1.10.* Synchronously call another script **in the same app** and get its return value, or +fire it off asynchronously. Like a function call across script boundaries. + +```rhai +// Synchronous: runs the target now, returns its value. +let result = invoke("/internal/price", #{ sku: "abc" }); // by route path +let result = invoke("price_worker", #{ sku: "abc" }); // by script name +let result = invoke("01HX…uuid…", #{ sku: "abc" }); // by script id (a UUID string) + +// Asynchronous: enqueue it, get an execution id back immediately. +let exec_id = invoke_async("/internal/price", #{ sku: "abc" }); +``` + +| Function | Signature | Returns | +|---|---|---| +| `invoke(target, args)` | `(string, dynamic)` | the callee's return value | +| `invoke_async(target, args)` | `(string, dynamic)` | the new execution id (string) | + +- **Target is a string**, resolved by a simple heuristic: starts with `/` → a **route path** (matched + through the app's route trie); a 36-character UUID → a **script id**; anything else → a **script + name**. (There is no `script_id(...)` constructor in 1.1.9 — just pass the string.) +- **Same-app only** — a cross-app target throws. The callee inherits the caller's principal and runs + in the same engine. `ctx.invocation_type` is `"function"` in the callee. +- **Args** become the callee's `ctx.request.body`. Closures can't be passed. +- **Depth-limited** — re-entrant `invoke` chains are capped (default 8) to prevent runaway recursion; + exceeding it throws. + +--- + +## `retry` — retry with backoff {#retry} + +*Since SDK 1.10.* Wrap a fallible closure in a retry loop. Useful around `http`, `invoke`, or any call +that can fail transiently. + +```rhai +let policy = retry::policy(#{ + max_attempts: 3, // 1–20, default 3 + backoff: "exponential", // "exponential" | "linear" | "constant", default exponential + base_ms: 500, // 1–60000, default 500 + jitter_pct: 20 // 0–100, default 20 +}); + +// Optionally only retry on certain error substrings: +let policy = retry::on_codes(policy, ["http: 503", "http: 504"]); + +let value = retry::run(policy, || http::post(url, payload)); +``` + +| Function | Signature | Returns | +|---|---|---| +| `retry::policy(opts)` | `(map)` | a `Policy` (all fields clamped to their ranges) | +| `retry::on_codes(policy, codes)` | `(Policy, array of string)` | a `Policy` that retries only when the error string contains one of `codes` (empty = retry on any throw) | +| `retry::run(policy, closure)` | `(Policy, Fn)` | the closure's value on success; re-throws the last error after the final attempt | + +Backoff between attempts: `constant` = `base_ms`; `linear` = `base_ms × attempt`; `exponential` = +`base_ms × 2^(attempt-1)`. Jitter is applied deterministically as a ± fraction. The sleep is safe to +perform inside a script (it runs on the blocking worker, not an async reactor). + +> The shipped name is **`retry::run`** (not `with`/`call`, which are Rhai reserved words). + +--- + +## `dead_letters` — handle exhausted retries {#dead-letters} + +*Since SDK 1.2.* When a triggered execution exhausts its retries, the platform writes a **dead-letter** +row. From a script (typically a `dead_letter`-trigger handler — see +[`ctx.event`](ctx-and-events.md#events)) you can act on it: + +```rhai +dead_letters::replay(id); // re-enqueue the original event; marks the row resolved "replayed" +dead_letters::resolve(id, "ignored"); // close the row without replaying, with a reason +``` + +| Function | Signature | Returns | +|---|---|---| +| `dead_letters::replay(id)` | `(string)` | `()` | +| `dead_letters::resolve(id, reason)` | `(string, string)` | `()` | + +IDs are UUID strings. `reason` must be one of the fixed set — `ignored`, `handled_by_script`, +`handler_failed` (or `replayed`, which `replay` sets for you) — not free text. Operators do the same from the dashboard, `pic dead-letters`, or +[the dead-letters API](../rest-api/queues.md#dead-letters). (Listing dead letters from a script is not +in 1.1.9 — use the admin surface.) The +[webhook tutorial](../../examples/webhook-receiver.md#dead-letters) demonstrates the failure→replay loop. diff --git a/docs/dev-guide/src/reference/sdk/ctx-and-events.md b/docs/dev-guide/src/reference/sdk/ctx-and-events.md new file mode 100644 index 0000000..76247ef --- /dev/null +++ b/docs/dev-guide/src/reference/sdk/ctx-and-events.md @@ -0,0 +1,106 @@ +# The execution context & events + +Every script runs with a global `ctx` map. For HTTP invocations it describes the request; for triggered +invocations it *also* carries `ctx.event` describing what fired the script. + +## `ctx` fields + +| Field | Type | Present | +|---|---|---| +| `ctx.sdk_version` | string (`"1.10"`) | always | +| `ctx.execution_id` | string (UUID) | always | +| `ctx.script_id` | string (UUID) | always | +| `ctx.script_name` | string | always | +| `ctx.request_id` | string (UUID) | always | +| `ctx.invocation_type` | string | always — `"http"`, `"function"` (an `invoke()` call), or `"scheduled"` | +| `ctx.request` | map | always (synthetic for non-HTTP invocations) | +| `ctx.event` | map | **only when triggered** | + +## `ctx.request` + +| Field | Type | Notes | +|---|---|---| +| `path` | string | request path | +| `method` | string | uppercased verb | +| `headers` | map | **lowercased** keys | +| `body` | dynamic | JSON-parsed when JSON; string otherwise; `()` if empty | +| `params` | map | `:name` route captures | +| `query` | map | query-string params | +| `rest` | string | tail captured by a `prefix` route | + +```rhai +let id = ctx.request.params.id; // from /users/:id +let page = ctx.request.query.page; // from ?page=2 +let token = ctx.request.headers["authorization"]; +let rest = ctx.request.rest; // from a /files/* route +``` + +> When a script is reached through `POST /api/v1/execute/{id}` (execute-by-id) or `invoke()`, there is +> no route match, so `params` and `rest` are empty. Pass everything you need in the body. + +## Events + +A triggered script reads `ctx.event`. The map always has a `source` discriminant; the rest depends on +the trigger kind. Guard with `if "event" in ctx { ... }` if a script serves both HTTP and triggers. + +### `source: "kv"` — KV mutation +```rhai +// ctx.event = #{ source:"kv", op:"insert"|"update"|"delete", +// kv: #{ collection, key, value } } // value is () on delete +let key = ctx.event.kv.key; +``` + +### `source: "docs"` — document mutation +```rhai +// ctx.event = #{ source:"docs", op:"insert"|"update"|"delete", +// docs: #{ collection, id, data, prev_data } } +// `prev_data` is the pre-change document on update/delete (change-data-capture); () on insert. +``` + +### `source: "files"` — blob mutation +```rhai +// ctx.event = #{ source:"files", op:"insert"|"update"|"delete", +// files: #{ collection, id, name, content_type, size, checksum, prev } } +// Metadata only — never the bytes. Fetch them with files::collection(c).get(id) if needed. +``` + +### `source: "pubsub"` — message published +```rhai +// ctx.event = #{ source:"pubsub", op:"publish", +// pubsub: #{ topic, message, published_at } } +let payload = ctx.event.pubsub.message; // the published value (note: `message`, not `payload`) +``` + +### `source: "queue"` — message claimed +```rhai +// ctx.event = #{ source:"queue", op:"receive", +// queue: #{ queue_name, message, enqueued_at, attempt, message_id } } +let payload = ctx.event.queue.message; // the enqueued value (note: `message`, not `payload`) +let n = ctx.event.queue.attempt; // 1 on first delivery; >1 on a retry — use it to be idempotent +``` + +### `source: "cron"` — schedule tick +```rhai +// ctx.event = #{ source:"cron", op:"tick", +// cron: #{ schedule, timezone, scheduled_at, fired_at } } +// scheduled_at / fired_at are RFC 3339 strings. +``` + +### `source: "email"` — inbound mail +```rhai +// ctx.event = #{ source:"email", op:"receive", +// email: #{ from, to:[...], cc:[...], subject, text, html, received_at, message_id } } +// text / html are () when absent. +``` + +### `source: "dead_letter"` — another trigger gave up +```rhai +// ctx.event = #{ source:"dead_letter", +// dead_letter: #{ id, original, attempts, last_error, +// trigger_id, script_id, first_attempt_at, last_attempt_at } } +// `original` is the event that failed. Use it to alert, or call dead_letters::replay / resolve. +``` + +See [Triggers](../rest-api/triggers.md) to register a trigger, and the +[webhook](../../examples/webhook-receiver.md) and [scheduled-report](../../examples/scheduled-report.md) +tutorials for working consumers. diff --git a/docs/dev-guide/src/reference/sdk/email.md b/docs/dev-guide/src/reference/sdk/email.md new file mode 100644 index 0000000..df17cee --- /dev/null +++ b/docs/dev-guide/src/reference/sdk/email.md @@ -0,0 +1,50 @@ +# Email + +*Since SDK 1.8.* Send outbound email through a configured SMTP relay. Receiving email is a +[trigger](../rest-api/triggers.md) (`email` kind), surfaced as `ctx.event` with `source: "email"`. + +```rhai +email::send(#{ + to: "alice@example.com", + from: "alerts@myapp.com", + subject: "Build finished", + text: "Your deploy completed successfully." +}); + +email::send_html(#{ + to: ["alice@x.com", "bob@y.com"], + cc: ["ops@x.com"], + bcc: ["audit@x.com"], + from: "alerts@myapp.com", + reply_to: "support@myapp.com", // optional; defaults to `from` + subject: "Weekly report", + text: "Plain-text fallback for non-HTML clients.", + html: "

Weekly report

…

" +}); +``` + +| Function | Required fields | Optional fields | +|---|---|---| +| `email::send(opts)` | `to`, `from`, `subject`, `text` | `cc`, `bcc`, `reply_to` | +| `email::send_html(opts)` | `to`, `from`, `subject`, `text`, `html` | `cc`, `bcc`, `reply_to` | + +- `to` / `cc` / `bcc` accept a single string or an array of strings. +- `email::send` is plain-text only; any `html` key is ignored. `email::send_html` requires a non-empty + `html` and sends multipart (HTML + the `text` fallback). +- Both **throw** on failure. + +## Configuration & dev mode + +- Sending requires an SMTP relay configured via `PICLOUD_SMTP_*` env vars. With no relay configured, + `email::send` throws `NotConfigured`. +- **Except in dev mode:** when `PICLOUD_DEV_MODE=true` and no relay is set, sends succeed into an + **in-memory dev sink** — the last 100 messages are readable at `GET /api/v1/admin/dev/emails` + (instance owner/admin only). This route exists *only* in that mode. Great for testing the + [scheduled-report tutorial](../../examples/scheduled-report.md) without a real mail server. + +## Rate limiting + +There is **no built-in rate limiting** on `email::send`. If a public route can trigger a send, throttle +it yourself (e.g. a `kv` counter keyed by sender/IP) — see +[Security](../../operations/security.md). The `users` SDK's verification/reset/invite helpers send +mail too; see [Users & auth](users.md). diff --git a/docs/dev-guide/src/reference/sdk/http.md b/docs/dev-guide/src/reference/sdk/http.md new file mode 100644 index 0000000..c9948bb --- /dev/null +++ b/docs/dev-guide/src/reference/sdk/http.md @@ -0,0 +1,77 @@ +# Outbound HTTP + +*Since SDK 1.5.* The `http` namespace makes outbound requests from a script — call third-party APIs, +post to webhooks, fetch data. It is `fetch`-style: non-2xx responses are returned, not thrown; only +transport-level failures (DNS, TLS, timeout, SSRF block, size) throw. + +```rhai +let r = http::get("https://api.example.com/users/42"); +if r.status == 200 { + let user = r.body; // parsed JSON (because the response was application/json) +} + +let r = http::post("https://api.example.com/orders", #{ item: "sku-1", qty: 2 }); +let r = http::post(url, #{ item: 1 }, #{ headers: #{ "Authorization": "Bearer …" }, timeout_ms: 5000 }); +``` + +## Functions + +| Function | Arities | +|---|---| +| `http::get(url)` / `http::get(url, opts)` | bodyless | +| `http::head(url)` / `http::head(url, opts)` | bodyless | +| `http::post(url)` / `(url, body)` / `(url, body, opts)` | body | +| `http::put` / `http::patch` / `http::delete` | same as `post` | +| `http::post_form(url, form)` / `(url, form, opts)` | form-encoded body from a map | +| `http::request(method, url)` / `(…, body)` / `(…, body, opts)` | any verb | + +## Body dispatch (positional `body` arg) + +| Rhai value | Sent as | +|---|---| +| map / array | JSON, `Content-Type: application/json` | +| string | raw bytes, `Content-Type: text/plain` | +| `()` | no body | + +`GET`/`HEAD` ignore any body. `post_form` always sends `application/x-www-form-urlencoded`. + +## Options map + +| Key | Type | Default | Notes | +|---|---|---|---| +| `headers` | map (string→string) | — | header names lowercased internally | +| `timeout_ms` | int | 30000 | max 60000 | +| `follow_redirects` | bool | true | | +| `max_redirects` | int | 5 | max 10 | + +Any other key throws ("unknown option key") — so a typo fails loudly. + +## Response map + +```rhai +#{ + status: 200, // int + headers: #{ ... }, // lowercased keys + body: ..., // parsed JSON if the response is application/json and parses; () if empty; else the raw string + body_raw: "..." // always the raw response text +} +``` + +## Security: the SSRF guard + +`http::*` resolves the target and **blocks requests to private/loopback/link-local addresses** by +default, so a script (especially a public one taking a user-supplied URL) can't be tricked into probing +your internal network. This is on unless an operator sets `PICLOUD_HTTP_ALLOW_PRIVATE=true` (dev/test +only — see [Security](../../operations/security.md#ssrf)). + +```rhai +// Robust outbound call: bounded timeout, retried on transient upstream errors. +let policy = retry::on_codes( + retry::policy(#{ max_attempts: 3, backoff: "exponential", base_ms: 300 }), + ["http: 502", "http: 503", "http: 504"] +); +let r = retry::run(policy, || http::post(url, payload, #{ timeout_ms: 5000 })); +``` + +See [Composition](composition.md) for `retry`. The +[webhook tutorial](../../examples/webhook-receiver.md) uses `http` + `retry` against a downstream. diff --git a/docs/dev-guide/src/reference/sdk/messaging.md b/docs/dev-guide/src/reference/sdk/messaging.md new file mode 100644 index 0000000..ecffb2a --- /dev/null +++ b/docs/dev-guide/src/reference/sdk/messaging.md @@ -0,0 +1,73 @@ +# Messaging: pubsub, queue + +Two ways for parts of your app (and outside clients) to communicate asynchronously. + +- **`pubsub`** — fan-out. A published message is delivered to *every* matching subscriber (triggers, + and external SSE clients). Fire-and-forget; no per-consumer durability guarantee beyond delivery. +- **`queue`** — work distribution. An enqueued message is delivered to *one* consumer, with retries, + attempt tracking, and dead-lettering. Use it for jobs that must complete. + +--- + +## `pubsub` — publish/subscribe {#pubsub} + +*Since SDK 1.6 (`publish_durable`), 1.7 (`subscriber_token`).* + +```rhai +pubsub::publish_durable("order.created", #{ id: 42, total: 1999 }); +pubsub::publish_durable("metrics.tick", 1); +``` + +| Function | Signature | Returns | +|---|---|---| +| `pubsub::publish_durable(topic, message)` | `(string, dynamic)` | `()` — throws over `PICLOUD_PUBSUB_MAX_MESSAGE_BYTES` (256 KiB) | +| `pubsub::subscriber_token(topics)` | `(array of string)` | an HMAC-signed token (string) | +| `pubsub::subscriber_token(topics, ttl_secs)` | `(array, int\|())` | same, custom TTL | + +- **Who receives a message:** any `pubsub` [trigger](../rest-api/triggers.md) whose topic glob matches + (runs a script), and any external SSE client subscribed to the topic (if it's a registered, + externally-subscribable [topic](../rest-api/topics.md)). +- **`subscriber_token`** mints a token an outside browser/client presents to subscribe over SSE at + `/realtime/topics/{topic}`. Only needed for `secret`/`token`-mode topics. See + [Topics & realtime](../rest-api/topics.md). +- Message encoding follows the [standard rules](overview.md#value-encoding) — blobs become base64, + `()` becomes null, closures are rejected. + +Publishing needs no topic registration; *external subscription* does. + +--- + +## `queue` — durable job queue {#queue} + +*Since SDK 1.10.* A producer + inspection API. Consumption is wired by registering a **`queue` +trigger** that runs a script per claimed message (`ctx.event.queue`). + +```rhai +queue::enqueue("emails.send", #{ to: "a@b.com", template: "welcome" }); +queue::enqueue("emails.send", msg, #{ delay_ms: 60000, max_attempts: 5 }); + +let total = queue::depth("emails.send"); // all rows +let waiting = queue::depth_pending("emails.send"); // currently claimable +``` + +| Function | Signature | Returns | +|---|---|---| +| `queue::enqueue(name, message)` | `(string, dynamic)` | `()` — throws over `PICLOUD_QUEUE_MAX_PAYLOAD_BYTES` (256 KiB); closures rejected | +| `queue::enqueue(name, message, opts)` | `(string, dynamic, map)` | `()` — `opts.delay_ms` (int), `opts.max_attempts` (int) | +| `queue::depth(name)` | `(string)` | `int` (total) | +| `queue::depth_pending(name)` | `(string)` | `int` (claimable now) | + +The consumer side: + +1. Deploy a consumer script that reads `ctx.event.queue.message` (the enqueued value). +2. Register a queue trigger: `pic triggers create-queue --app --script --queue emails.send`. +3. On failure (the script throws), the message is retried up to `max_attempts`; after that it becomes + a [dead letter](composition.md#dead-letters) you can inspect and replay. + +There is no script-level peek/dequeue/purge — consumption is the trigger's job. Inspect queues from the +dashboard, `pic queues`, or [the queues API](../rest-api/queues.md). The +[webhook tutorial](../../examples/webhook-receiver.md) builds a full producer→queue→consumer flow. + +> **Make consumers idempotent.** A message can be delivered more than once (retry after a partial +> success, visibility-timeout expiry). Use `ctx.event.queue.attempt` and a dedup key. See +> [Best practices](../../operations/best-practices.md#idempotent-consumers). diff --git a/docs/dev-guide/src/reference/sdk/overview.md b/docs/dev-guide/src/reference/sdk/overview.md new file mode 100644 index 0000000..0ff3c5c --- /dev/null +++ b/docs/dev-guide/src/reference/sdk/overview.md @@ -0,0 +1,96 @@ +# SDK reference — overview + +The SDK is the set of functions your Rhai scripts can call. It comes in two layers: + +- **Services** — stateful, app-scoped capabilities backed by the platform: `kv`, `docs`, `files`, + `http`, `email`, `users`, `pubsub`, `queue`, `secrets`, `invoke`, `retry`, `dead_letters`, and + `log`. These appear and disappear with releases (each is tagged with the SDK minor that added it). +- **Standard library** — pure, stateless helpers: `json`, `base64`, `hex`, `url`, `regex`, `random`, + `time`. See [Standard library](stdlib.md). + +This page covers the conventions that hold across *every* service. The per-service pages give exact +signatures. + +## Namespacing and the handle pattern + +Functions live in namespaces, called with `::`: + +```rhai +let id = random::uuid(); +log::info("hi"); +``` + +Storage services are **collection-scoped**: you first get a *handle* to a named collection, then call +methods on it. This is intentional — it makes the `(app, collection, key)` identity explicit. + +```rhai +let users = docs::collection("users"); // handle +let id = users.create(#{ name: "Ada" }); // method on the handle +let doc = users.get(id); +``` + +So it's `kv::collection("x").get(k)`, **not** `kv::get("x", k)`. Collections are mandatory; a +collection name may not be empty. + +## App scoping is automatic and non-negotiable + +Every service call resolves the app from the server-side execution context. **Nothing your script +passes can change which app's data it touches** — there is no `app_id` argument anywhere in the SDK. +This is the isolation boundary described in [Core concepts](../../guide/concepts.md#apps); treat it as +a hard guarantee, not a convention. + +## Error conventions + +Uniform across all services (also covered in [Writing scripts](../../guide/writing-scripts.md#errors)): + +| Situation | Behavior | +|---|---| +| Real failure (DB error, payload too large, authorization denied) | **throws** — catch with `try/catch`, or let it become a 502 | +| Item not found / no result | returns **`()`** — test `== ()` | +| Existence / was-present check | returns a **`bool`** | + +## Value encoding + +Values you store or send are JSON-encoded on the wire: + +- Maps, arrays, strings, numbers, booleans round-trip cleanly. +- `()` (Rhai unit) becomes JSON `null`. +- **Blobs** (byte arrays, e.g. from `base64::decode`) are base64-encoded inside JSON payloads + (`pubsub`, `queue`). `files` stores raw bytes directly. +- **Function pointers / closures cannot be serialized** and are rejected by `queue`, `invoke`, etc. + +## Size caps + +Several services cap payload size to protect Postgres. Defaults (all tunable, see +[env vars](../config/env-vars.md)): + +| Service | Env var | Default | +|---|---|---| +| `kv` value | `PICLOUD_KV_MAX_VALUE_BYTES` | 256 KiB | +| `docs` document | `PICLOUD_DOCS_MAX_VALUE_BYTES` | 256 KiB | +| `pubsub` message | `PICLOUD_PUBSUB_MAX_MESSAGE_BYTES` | 256 KiB | +| `queue` message | `PICLOUD_QUEUE_MAX_PAYLOAD_BYTES` | 256 KiB | +| `files` blob | `PICLOUD_FILES_MAX_FILE_SIZE_BYTES` | 100 MiB | + +Exceeding a cap **throws**. For `kv`/`queue` the size is checked *before* authorization so a public +script can't be used to hammer the database. + +## The full surface at a glance + +| Namespace | Since SDK | Page | +|---|---|---| +| `log` | 1.0 | [Writing scripts](../../guide/writing-scripts.md#logging) | +| `kv` | 1.2 | [Storage](storage.md#kv) | +| `docs` | 1.3 | [Storage](storage.md#docs) | +| `files` | 1.6 | [Storage](storage.md#files) | +| `http` | 1.5 | [Outbound HTTP](http.md) | +| `pubsub` | 1.6 / 1.7 | [Messaging](messaging.md#pubsub) | +| `secrets`, `email` | 1.8 | [Secrets](secrets.md), [Email](email.md) | +| `users` | 1.9 | [Users & auth](users.md) | +| `queue`, `invoke`, `retry` | 1.10 | [Messaging](messaging.md#queue), [Composition](composition.md) | +| `dead_letters` | 1.2 | [Composition](composition.md#dead-letters) | +| `json` `base64` `hex` `url` `regex` `random` `time` | 1.0 | [Standard library](stdlib.md) | + +Many services also have an **admin/inspection** side on the HTTP API and the `pic` CLI (e.g. browse KV, +download files, read secret names). Those are read-mostly; *writing* data is the script's job. Each SDK +page links to its admin counterpart. diff --git a/docs/dev-guide/src/reference/sdk/secrets.md b/docs/dev-guide/src/reference/sdk/secrets.md new file mode 100644 index 0000000..64cbf00 --- /dev/null +++ b/docs/dev-guide/src/reference/sdk/secrets.md @@ -0,0 +1,51 @@ +# Secrets + +*Since SDK 1.8.* Per-app encrypted key/value storage for credentials — API keys, signing secrets, DB +passwords for upstreams. Values are encrypted at rest with AES-256-GCM under the process master key +(`PICLOUD_SECRET_KEY`). Unlike `kv`, secrets are **not** collection-scoped: a secret is just a name +within the app. + +```rhai +secrets::set("stripe_key", "sk_live_…"); +secrets::set("oauth", #{ client_id: "abc", client_secret: "xyz" }); // any JSON value + +let key = secrets::get("stripe_key"); // the value, or () if missing +let was = secrets::delete("stripe_key"); // bool +let page = secrets::list(#{ cursor: (), limit: 100 }); +// page = #{ names: [...], next_cursor: () | "cursor" } — names only, never values +``` + +| Function | Signature | Returns | +|---|---|---| +| `secrets::set(name, value)` | `(string, dynamic)` | `()` — upserts (overwrites if present) | +| `secrets::get(name)` | `(string)` | the value, or `()` (also `()` if decryption fails) | +| `secrets::delete(name)` | `(string)` | `bool` | +| `secrets::list(#{ cursor?, limit? })` | | `#{ names, next_cursor }` | + +Strings round-trip as strings; other types round-trip via JSON. + +## Why use secrets instead of `kv`? + +- **Encrypted at rest.** A database dump leaks `kv` values in the clear, but secret values are + ciphertext. +- **Redacted everywhere.** The admin API, dashboard, and `pic secrets` only ever show secret *names* — + values never leave the server after `set`. + +## Setting secrets out-of-band + +You usually don't want to hardcode a secret in a script. Set it from the CLI (value read from stdin so +it never lands in shell history) or the dashboard: + +```sh +printf 'sk_live_…' | pic secrets set --app myapp stripe_key +pic secrets ls --app myapp +``` + +Then read it at runtime: `let k = secrets::get("stripe_key");`. See +[the data-admin API](../rest-api/data-admin.md#secrets) and the +[webhook tutorial](../../examples/webhook-receiver.md), which verifies an HMAC using a stored secret. + +> **Master-key caveat:** secrets are encrypted under `PICLOUD_SECRET_KEY`. **Rotating that key makes +> existing secrets undecryptable** — `secrets::get` will return `()` for them. There is no automatic +> re-encryption in 1.1.9; rotate deliberately and re-`set` your secrets afterward. See +> [Security](../../operations/security.md#secrets-and-the-master-key). diff --git a/docs/dev-guide/src/reference/sdk/stdlib.md b/docs/dev-guide/src/reference/sdk/stdlib.md new file mode 100644 index 0000000..0df5e15 --- /dev/null +++ b/docs/dev-guide/src/reference/sdk/stdlib.md @@ -0,0 +1,100 @@ +# Standard library + +*Since SDK 1.0.* Pure, stateless helpers for everyday glue. No app scope, no I/O, no failure modes +beyond bad input (which throws). For the deep design notes see the engineering reference +`docs/stdlib-reference.md` in the repo; this page is the working reference. + +> **Heads-up:** `ctx.request.body` is already parsed when the request is JSON — don't `json::parse` it +> again. Use `json::parse` only on raw strings (e.g. an `http` response's `body_raw`). + +## `json` — parse / stringify + +| Function | Signature | Returns | +|---|---|---| +| `json::parse(s)` | `(string)` | a Rhai value (map/array/scalar/`()` for null) — throws on invalid JSON | +| `json::stringify(v)` | `(dynamic)` | compact JSON string | +| `json::stringify_pretty(v)` | `(dynamic)` | 2-space-indented JSON | + +## `base64` — standard & URL-safe + +Encoders accept a string or a blob; decoders return a blob. + +| Function | Notes | +|---|---| +| `base64::encode(input)` | standard alphabet, padded | +| `base64::decode(s)` | → blob; throws on invalid | +| `base64::encode_url(input)` | URL-safe alphabet, **no** padding | +| `base64::decode_url(s)` | → blob; throws on invalid | + +```rhai +let bytes = base64::decode(ctx.request.body.b64); // → Blob, e.g. for files::create +let token = base64::encode_url(random::bytes(32)); +``` + +## `hex` + +| Function | Notes | +|---|---| +| `hex::encode(input)` | lowercase hex (string or blob in) | +| `hex::decode(s)` | → blob; case-insensitive; throws on invalid | + +## `url` — percent-encoding + +| Function | Notes | +|---|---| +| `url::encode(s)` | percent-encode one component | +| `url::decode(s)` | percent-decode; throws on invalid UTF-8 | +| `url::encode_query(map)` | build `k1=v1&k2=v2` (keys & values encoded, keys sorted) | + +## `regex` — non-backtracking regular expressions + +Linear-time engine (no catastrophic backtracking). Patterns are cached. Use backticks for literal +backslashes: `` regex::find_all(`\d+`, text) ``. + +| Function | Returns | +|---|---| +| `regex::is_match(pattern, text)` | `bool` | +| `regex::find(pattern, text)` | first match string, or `()` | +| `regex::find_all(pattern, text)` | array of match strings | +| `regex::replace(pattern, text, repl)` | replace first | +| `regex::replace_all(pattern, text, repl)` | replace all | +| `regex::split(pattern, text)` | array | +| `regex::captures(pattern, text)` | `[full, g1, g2, …]`, or `()` | + +Invalid patterns throw. + +## `random` — cryptographically secure + +Backed by the OS CSPRNG; safe for tokens and IDs. + +| Function | Returns | +|---|---| +| `random::int(min, max)` | uniform int in `[min, max]` (throws if `min > max`) | +| `random::float()` | uniform in `[0.0, 1.0)` | +| `random::bytes(n)` | blob of `n` bytes (`n ∈ [0, 65536]`) | +| `random::string(n)` | `n` alphanumeric chars (`n ∈ [0, 4096]`) | +| `random::uuid()` | UUID v4 string | + +```rhai +let code = random::string(7); // e.g. a short URL slug +let id = random::uuid(); +``` + +## `time` — UTC, milliseconds + +Canonical unit is `i64` milliseconds since the Unix epoch; ISO 8601 / RFC 3339 strings for I/O. All +UTC. + +| Function | Returns | +|---|---| +| `time::now()` | current time as ISO 8601 string (with ms) | +| `time::now_ms()` | current ms since epoch | +| `time::parse(iso)` | ms since epoch (throws on bad input) | +| `time::format(ms)` | ISO 8601 string | +| `time::add_seconds(ms, secs)` | `ms + secs*1000` (throws on overflow) | +| `time::diff_seconds(a_ms, b_ms)` | `(b_ms - a_ms)/1000`, truncated | + +```rhai +let now = time::now_ms(); +let deadline = time::add_seconds(now, 3600); // one hour from now +``` diff --git a/docs/dev-guide/src/reference/sdk/storage.md b/docs/dev-guide/src/reference/sdk/storage.md new file mode 100644 index 0000000..0aa8c47 --- /dev/null +++ b/docs/dev-guide/src/reference/sdk/storage.md @@ -0,0 +1,117 @@ +# Storage: kv, docs, files + +Three collection-scoped storage services. All share the [handle pattern](overview.md#namespacing-and-the-handle-pattern) +and [error conventions](overview.md#error-conventions). The identity of any item is the tuple +`(app, collection, key/id)` — collections are mandatory and app scoping is automatic. + +Each also has a read-mostly admin surface: [Secrets, KV & files](../rest-api/data-admin.md) on the +HTTP API and `pic kv` / `pic files` on the CLI. + +--- + +## `kv` — key/value store {#kv} + +*Since SDK 1.2.* Simple string-keyed JSON values. Best for counters, flags, small lookups, caches. + +```rhai +let c = kv::collection("settings"); + +c.set("theme", "dark"); // value can be any JSON-serializable value +let t = c.get("theme"); // "dark", or () if absent +let exists = c.has("theme"); // bool +let was = c.delete("theme"); // bool — was it present? +let page = c.list(); // #{ keys: [...], next_cursor: () | "cursor" } +``` + +| Method | Signature | Returns | +|---|---|---| +| `kv::collection(name)` | `(string)` | a `KvHandle` | +| `.set(key, value)` | `(string, dynamic)` | `()` — throws if value exceeds `PICLOUD_KV_MAX_VALUE_BYTES` (256 KiB) | +| `.get(key)` | `(string)` | the value, or `()` if absent | +| `.has(key)` | `(string)` | `bool` (cheap; no deserialization) | +| `.delete(key)` | `(string)` | `bool` (was-present) | +| `.list()` / `.list(cursor)` / `.list(cursor, limit)` | | `#{ keys, next_cursor }` | + +Pagination: `list()` returns a page plus `next_cursor` (`()` when exhausted); pass it back to +`list(cursor)` for the next page. + +--- + +## `docs` — document store {#docs} + +*Since SDK 1.3.* Auto-IDed JSON documents with a query filter. Best for entities (users, posts, +orders) where you want to find by field. + +```rhai +let posts = docs::collection("posts"); + +let id = posts.create(#{ title: "Hi", tag: "intro", views: 0 }); // returns the new id (string) +let doc = posts.get(id); // envelope, or () if missing +let hits = posts.find(#{ tag: "intro" }); // array of envelopes +let one = posts.find_one(#{ tag: "intro" }); +posts.update(id, #{ title: "Hi", tag: "intro", views: 1 }); // replaces `data` +let was = posts.delete(id); // bool +let page = posts.list(); // #{ docs: [...], next_cursor: () | "cursor" } +``` + +Every read returns an **envelope**, with your fields under `data`: + +```rhai +#{ id: "…", data: #{ title: "Hi", tag: "intro", views: 1 }, + created_at: "2026-…Z", updated_at: "2026-…Z" } +``` + +| Method | Signature | Returns | +|---|---|---| +| `docs::collection(name)` | `(string)` | a `DocsHandle` | +| `.create(data)` | `(map)` | new id (string) — throws if over `PICLOUD_DOCS_MAX_VALUE_BYTES` (256 KiB) | +| `.get(id)` | `(string)` | envelope, or `()` | +| `.find(filter)` | `(map)` | array of envelopes | +| `.find_one(filter)` | `(map)` | one envelope, or `()` | +| `.update(id, data)` | `(string, map)` | `()` | +| `.delete(id)` | `(string)` | `bool` | +| `.list()` / `.list(#{ cursor, limit })` | | `#{ docs, next_cursor }` | + +The `find` filter is a map of field equality matches (`#{ tag: "intro", published: true }` matches docs +where both hold). It is a deliberately small subset for 1.1.9; richer query operators are roadmap. + +> `update(id, data)` **replaces** the whole `data` map. To change one field, `get` it, mutate, and +> `update` with the full map. + +--- + +## `files` — blob storage {#files} + +*Since SDK 1.6.* Binary objects (images, uploads, generated artifacts) stored on the filesystem under +`PICLOUD_FILES_ROOT`, with metadata in Postgres. Bytes are passed as Rhai **blobs**. + +```rhai +let bucket = files::collection("avatars"); + +// data must be a Blob — e.g. base64::decode(ctx.request.body.b64) +let id = bucket.create(#{ name: "a.png", content_type: "image/png", data: bytes }); +let meta = bucket.head(id); // metadata map, or () (no bytes) +let bytes = bucket.get(id); // the Blob, or () +bucket.update(id, #{ data: new_bytes }); // data required; name/content_type optional +let was = bucket.delete(id); // bool +let page = bucket.list(); // #{ files: [...], next_cursor: () | "cursor" } +``` + +| Method | Signature | Returns | +|---|---|---| +| `files::collection(name)` | `(string)` | a `FilesHandle` | +| `.create(#{ name, content_type, data })` | | new id (string) — `data` is a Blob; throws over `PICLOUD_FILES_MAX_FILE_SIZE_BYTES` (100 MiB) | +| `.head(id)` | `(string)` | metadata `#{ id, collection, name, content_type, size, checksum, created_at, updated_at }`, or `()` | +| `.get(id)` | `(string)` | the Blob, or `()` | +| `.update(id, #{ data, name?, content_type? })` | | `()` | +| `.delete(id)` | `(string)` | `bool` | +| `.list()` / `.list(cursor)` / `.list(cursor, limit)` / `.list(#{ cursor, limit })` | | `#{ files, next_cursor }` | + +Reads are checksum-verified (SHA-256). **Serving the bytes back:** a script response is always JSON, so +you can't stream a raw blob through the response body (a blob body serializes to a hex JSON string). +Return base64 in JSON for the client to decode, or use the +[admin files endpoint](../rest-api/data-admin.md#files) for true raw bytes — see the +[file-upload tutorial](../../examples/file-upload.md). + +> Mutating any of these services can fire a [trigger](../rest-api/triggers.md) (`kv`/`docs`/`files` +> kinds), letting another script react to the change. diff --git a/docs/dev-guide/src/reference/sdk/users.md b/docs/dev-guide/src/reference/sdk/users.md new file mode 100644 index 0000000..ba9fe92 --- /dev/null +++ b/docs/dev-guide/src/reference/sdk/users.md @@ -0,0 +1,87 @@ +# Users & auth + +*Since SDK 1.9.* The `users` namespace manages **your app's end-users** — registration, login, +sessions, email verification, password reset, invitations, and per-user roles. Passwords are hashed +with Argon2id; sessions are opaque tokens. + +> **This is the only way to do app-user auth — PiCloud ships no `/signup` or `/login` HTTP endpoints.** +> You write scripts that call `users::*` and bind them to *your own* routes (e.g. `POST /auth/signup`). +> The [TODO API tutorial](../../examples/todo-api.md) builds the complete flow. (Distinct from +> [admin users](../rest-api/access.md), who manage the platform.) + +## The user map + +Reads return: + +```rhai +#{ id, email, display_name, // display_name is () if unset + email_verified_at, last_login_at, // RFC 3339 or () + created_at, updated_at, + roles: ["editor", ...] } +``` + +## CRUD + +| Function | Signature | Returns | +|---|---|---| +| `users::create(#{ email, password, display_name? })` | | the user map — throws if email taken | +| `users::get(id)` | `(string)` | user map, or `()` | +| `users::find_by_email(email)` | `(string)` | user map or `()` — **requires an authenticated principal** | +| `users::email_available(email)` | `(string)` | `bool` — anonymous-safe pre-check | +| `users::update(id, #{ display_name? })` | | updated user map | +| `users::delete(id)` | `(string)` | `bool` | +| `users::list(#{ "$limit"?, cursor? })` | | `#{ users: [...], next_cursor }` — `limit` is accepted as an alias for `$limit` | + +> **Enumeration:** `find_by_email` is blocked for anonymous (public) scripts to stop attackers probing +> who's registered; call it only from authenticated paths. `email_available` *is* anonymous-safe (a +> signup form needs it) but is unthrottled — rate-limit it yourself if abuse is a concern. See +> [Security](../../operations/security.md#user-enumeration). + +## Authentication + +```rhai +let token = users::login(email, password); // session token string, or () on bad credentials +let user = users::verify(token); // user map (and bumps the sliding TTL), or () if invalid/expired +users::logout(token); // invalidate +``` + +A typical gated route: + +```rhai +let tok = ctx.request.headers["authorization"].sub_string(7); // strip "Bearer " +let me = users::verify(tok); +if me == () { return #{ statusCode: 401, body: #{ error: "unauthorized" } }; } +// ... me.id, me.roles available +``` + +## Email-tied flows + +These send mail (need SMTP configured, or dev mode), templated by the options you pass: + +| Function | Purpose | +|---|---| +| `users::send_verification_email(id, #{ link_base, from, subject, body_template })` | send a verify link | +| `users::verify_email(token)` | mark verified — returns user map or `()` | +| `users::request_password_reset(email, #{ link_base, from, subject, body_template })` | send reset link | +| `users::complete_password_reset(token, new_password)` | apply — user map or `()` | +| `users::invite(email, #{ link_base?, from?, subject?, body_template?, display_name?, roles? })` | invite | +| `users::accept_invite(token, password)` / `(token, password, display_name)` | accept — returns a session token or `()` | + +`body_template` / `link_base` are required only when email sending is configured; the platform builds +the link as `link_base` + the one-time token. + +## Roles + +Per-app, string-tagged. Define whatever vocabulary your app needs (`"editor"`, `"pro"`, …). + +| Function | Returns | +|---|---| +| `users::add_role(id, role)` | `()` | +| `users::remove_role(id, role)` | `bool` (was-present) | +| `users::has_role(id, role)` | `bool` | + +## Admin & CLI side + +Operators can list/inspect app users, mint a reset token, and revoke sessions via +[the app-users API](../rest-api/app-users.md) and `pic users`. They cannot read passwords (only hashes +are stored).