PiCloud/CLAUDE.md

# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project

**PiCloud** is a self-hosted, event-driven serverless compute platform. Users upload Rhai scripts, get HTTP endpoints. Optimized for solo-dev / consumer hardware (single node MVP, multi-node cluster in v1.3+).

Authoritative design: [serverless_cloud_blueprint.md](serverless_cloud_blueprint.md). The blueprint is a living document — when architecture decisions are made in conversation that contradict it, treat the latest decision as truth and update the blueprint.

**v1.1.x — SDK foundation + services — is complete.** The SDK shape (handle pattern, `::` namespaces, `Services`/`SdkCallCx`; see [docs/sdk-shape.md](docs/sdk-shape.md), stdlib at [docs/stdlib-reference.md](docs/stdlib-reference.md)) fixed in v1.1.0, then KV, docs, modules, HTTP, cron, files, pub/sub, email, users, and durable queues + `invoke()` filled it in through **v1.1.9** — blueprint §12 has the table. Earlier groundwork: blueprint Phase 3 (admin auth, multi-app scoping, Phase 3.5 capability gating — `manager-core::authz::{can, require, Capability}`, migration `0006_users_authz.sql`).

**Current focus: v1.2 _Hierarchies_ — groups + the declarative project tool** ([docs/design/groups-and-project-tool.md](docs/design/groups-and-project-tool.md)). That doc's §11 uses its own **Phase 1–6 numbering, distinct from the blueprint product-phase numbering above — do not conflate them** (its "Phase 3" = group-inherited config, not admin auth). Implemented on `feat/groups-*` branches: §11 Phase 1 (declarative `pic plan`/`apply`/`prune` + env overlays), Phase 2 (single-parent groups tree + hierarchy-aware RBAC), Phase 3 (group-inherited, env-scoped `vars` + secrets resolved **live** via a recursive CTE — no materialized cache), Phase 4-lite (group-owned **endpoint** scripts: `scripts` polymorphic owner in `0050_group_scripts.sql`, `get_by_name_inherited`/`is_invocable_by_app` chain resolution, inherited `invoke()` + declarative route/trigger binding — all **live**, no body materialization), Phase 5 (the **declarative project tool maps onto the group tree**: the reconcile engine generalized to `ApplyOwner{App|Group}`, a `[group]` manifest kind, and a single atomic **tree apply** — `pic plan/apply --dir` reconciles a whole directory tree of `picloud.toml` nodes in one Postgres transaction, groups-before-apps so an app route can bind a group script created in the same tx; the bound token folds in each group's `structure_version`. Multi-repo single-owner/attach-point and per-env approval gating are deferred; groups pre-exist), Phase 4b (group **modules** + the **lexical (sealed-by-default) import resolver**, §5.5: owner-polymorphic `ModuleScript`, origin-rooted `ModuleSource::resolve` walking the importing node's chain, `ExecRequest.script_owner` threaded from every dispatch + `invoke()` site, `_source`-driven lexical chaining in `PicloudModuleResolver` with the compiled-module cache re-keyed by `ScriptId`, group modules/imports allowed, single-node dangling-import `plan` check — an inherited group script's imports **seal to the group**, a leaf can't shadow them), §5.5 **extension points** (opt-in polymorphism — **§5.5 now complete**: marker table `0051_extension_points.sql` (owner-polymorphic, CASCADE — structurally a `secrets` name; default body = a co-located `kind=module` script), `ModuleSource::resolve_policy` with **nearest-declaration-kind-wins** — a concrete module resolves lexically, an EP marker resolves **dynamically against the inheriting app** (its override else the default body up-chain), `NoProvider` is a hard error; declarative-only authoring via the `[app]`/`[group]` manifest key `extension_points = [...]`, reconcile mirrors `secrets`, single-node no-provider `plan` check, read-only `pic extension-points ls` + `pull` round-trip — the app can **override** a group default, the deliberate inverse of the Phase 4b sealed import). Next: §11.6 (group-level collections, v1.3) or multi-node cluster mode.

**Data-model invariant:** app-owned data-plane tables (KV, docs, files, …) start with `app_id UUID NOT NULL REFERENCES apps(id) ON DELETE CASCADE`; the group-inheritable tables — _config_ (`vars`, `secrets`) and now group-owned _code_ (`scripts`, `0050`) — instead carry a **polymorphic owner**: nullable `group_id` and `app_id` with an exactly-one CHECK and per-owner partial-unique indexes (config is `ON DELETE CASCADE`, scripts `RESTRICT` — code is not data). Inheritance resolves **live** down `apps.group_id → groups.parent_id` via `CHAIN_LEVELS_CTE` (no materialized view); nearest-owner-wins with an app's own row shadowing the inherited one (CoW). Every Rhai SDK call resolves its app from `cx.app_id`, never a script-passed arg, and a group script always runs under the *inheriting* app's `cx.app_id` (the cross-app isolation boundary).

## Three-Service Architecture

The platform splits into three logical services, each backed by a `*-core` library crate so the same logic runs in single-process MVP mode and split-process cluster mode:

| Service | Role | Library crate |
|---------|------|---------------|
| **Manager** | Control plane: script CRUD, scheduling/cron, dashboard backend, config. Single-writer to Postgres. | `manager-core` |
| **Orchestrator** | Per-node ingress: receives HTTP (later SMTP, queue) events, resolves script, dispatches to local executor. Stateless. | `orchestrator-core` |
| **Executor** | Per-node compute: runs Rhai scripts in a sandboxed engine. Stateless. | `executor-core` |

In MVP, all three run in one process (`picloud` binary). In cluster mode, each runs as its own binary on each node, with one manager total and one orchestrator + executor per node.

**Key boundary:** the orchestrator never imports `executor-core` directly — it depends on an `ExecutorClient` trait. The local impl calls `executor-core` in-process; the remote impl is an HTTP client. Same pattern keeps cluster mode a swap, not a rewrite.

## Path Scheme

Versioned API surfaces live under `/api/v{N}/...`. See [docs/versioning.md](docs/versioning.md) for the full scheme.

- `/api/v1/admin/*` — manager (control plane: script CRUD, routes CRUD + check + match, logs, config; apps CRUD once Phase 3b lands)
- `/api/v1/execute/{id}` — orchestrator (data plane: invoke a script by ID, always-available bypass)
- `/admin/*` — dashboard SPA (SvelteKit, `paths.base = '/admin'`)
- `/healthz` — liveness (string `"ok"`)
- `/version` — every compatibility-surface version + `public_base_url` (JSON)
- **everything else** — orchestrator's user-route matcher: user scripts bind to arbitrary paths via `POST /api/v1/admin/scripts/{id}/routes`; if no route matches, picloud returns 404 with a JSON error.

Reserved path prefixes (rejected at route creation): `/api/`, `/admin/`, `/healthz`, `/version`.

Caddy fronts everything. Same Caddyfile shape works for single-node and cluster — only upstream targets change.

**Param syntax convention:** route paths use `:name` (e.g., `/users/:id`); domains (once apps land) use `{name}` (e.g., `{tenant}.example.com`). These are deliberately distinct — never use `:` in a domain context or `{}` in a route-path context.

**Two-phase dispatch (Phase 3b onward):** the orchestrator first resolves `Host` → app (most-specific domain claim wins), then runs that app's route trie. The route matcher itself is unchanged and never sees other apps' routes.

## Tech Stack

- **Rust 1.92+** workspace, pinned via `rust-toolchain.toml`
- **Axum** for HTTP, **Tokio** async, **sqlx** for Postgres
- **Rhai** embedded scripting (in `executor-core`)
- **PostgreSQL 15+** with `pgcrypto`. v1.1+ data-plane tables use JSONB for value columns (hstore was considered for KV and rejected — see blueprint §8.1).
- **SvelteKit** dashboard, static adapter, CodeMirror 6 for the script editor
- **Caddy 2** reverse proxy (auto-HTTPS in prod)
- **Docker Compose** for dev and single-node prod

## Common Commands

```sh
# Rust workspace
cargo check --workspace
cargo test --workspace
cargo fmt --all
cargo clippy --all-targets --all-features -- -D warnings

# Run the all-in-one binary (MVP entry point)
cargo run -p picloud

# Run a single test
cargo test -p executor-core sandbox::tests::respects_operation_budget

# Dashboard (from dashboard/)
npm run dev
npm run build
npm run check

# Full stack (once docker-compose.yml exists)
docker compose up
docker compose down -v   # reset Postgres data
```

## Workspace Layout

```
crates/
  shared/                 # cross-cutting types (Script, IDs, error enum, db pool)
  executor-core/          # Rhai engine, sandbox, ctx, log, SDK
  orchestrator-core/      # event ingress + ExecutorClient trait + dispatch
  manager-core/           # control plane: repos, scheduler, config
  picloud/                # ★ MVP all-in-one binary
  picloud-manager/        # cluster mode binary (skeleton)
  picloud-orchestrator/   # cluster mode binary (skeleton)
  picloud-executor/       # cluster mode binary (skeleton)
dashboard/                # SvelteKit
caddy/                    # Caddyfile, Caddyfile.prod
docker/                   # Dockerfiles
docs/
  git-workflow.md         # trunk-based workflow
  architecture.md         # (TBD)
```

## Working Rules

- **Honor the three-service boundary.** Don't reach across `*-core` crates *for behavior*. If `orchestrator-core` needs to invoke logic from `manager-core`, define a trait in `shared` and inject the impl — keep implementations decoupled. **Transport DTOs are not behavior**: types like `ExecRequest` / `ExecResponse` / `ExecError` represent values produced or consumed across the wire, and depending on the originating crate's type definitions is fine. The bright line is "don't call across crates," not "don't import types." When in doubt: if the imported item is a `struct`/`enum`/`type alias` with no methods (or only data-shape methods), it's a DTO and crossing is fine; if it's a trait, function, or service, define the abstraction in `shared` and inject.
- **`executor-core` has no Postgres dependency.** Data-plane services (kv, docs, users — v1.1+) come in via injected `ServiceProvider` traits.
- **Database writes only from `manager-core`.** `orchestrator-core` reads scripts (cached); `executor-core` doesn't touch the DB.
- **Stateful SDK services use the handle pattern + `SdkCallCx`.** Collection-scoped surfaces look like `kv::collection("x").get(k)`, not `kv::get("x", k)`. Every service trait method takes `&SdkCallCx` and **MUST** derive `app_id` from `cx.app_id` — never trust a script-passed `app_id`. That is the cross-app isolation boundary. See [docs/sdk-shape.md](docs/sdk-shape.md).
- **MVP builds only the `picloud` all-in-one binary.** The three split binaries exist as skeletons so the crate boundaries stay honest; flesh them out only when cluster mode is being implemented.
- **Trunk-based dev.** See [docs/git-workflow.md](docs/git-workflow.md). No long-lived branches. Feature flags for incomplete work.

## Runtime configuration

Environment variables consumed by the `picloud` binary:

| Variable | Default | Purpose |
|---|---|---|
| `PICLOUD_BIND` | `0.0.0.0:8080` | HTTP listen address. Port 8080 is owned by another process on this host — override locally. |
| `PICLOUD_MAX_CONCURRENT_EXECUTIONS` | `32` | Global concurrency cap on data-plane script executions. Overflow returns HTTP 503 with `Retry-After: 1` immediately (no queue). |
| `DATABASE_URL` | — | Required. Postgres connection string. |
| `PICLOUD_SECRET_KEY` | — | Master encryption key (base64). Required at startup unless dev mode is acknowledged (below). |
| `PICLOUD_DEV_MODE` | `false` | `true` enables local-dev conveniences. Without `PICLOUD_SECRET_KEY` it ALSO requires the acknowledgement var below — `PICLOUD_DEV_MODE=true` alone aborts at startup. Also: when no SMTP relay is configured, `email::send` switches from disabled (`NotConfigured`) to an **in-memory dev sink** — sends succeed and the last 100 messages are readable at `GET /api/v1/admin/dev/emails` (instance Owner/Admin only; route exists only in this mode). Never in production. |
| `PICLOUD_DEV_INSECURE_KEY` | — | Set to the literal `i-understand-this-is-insecure` to let dev mode boot without `PICLOUD_SECRET_KEY`, using a deterministic, world-known dev master key. Never set in production — it would encrypt everything with a public value. |
| `PICLOUD_DB_MAX_CONNECTIONS` | `32` | Postgres pool size. Matched to `PICLOUD_MAX_CONCURRENT_EXECUTIONS` so the data plane can't starve background workers. |
| `PICLOUD_SESSION_TTL_HOURS` | `24` | Sliding-window session lifetime. |
| `PICLOUD_SANDBOX_MAX_*` | conservative defaults | Per-knob admin ceilings on Rhai sandbox overrides. See `manager-core::sandbox::SandboxCeiling`. |
| `PICLOUD_FILES_ROOT` | `./data` | Filesystem root for `files::*` blob storage (v1.1.5). Bytes live at `<root>/files/<app_id>/<collection>/<id[0:2]>/<id>`; metadata in Postgres. |
| `PICLOUD_FILES_MAX_FILE_SIZE_BYTES` | `104857600` (100 MB) | Per-file hard size cap for `files::*` (v1.1.5). Per-app quotas deferred to v1.2. |
| `PICLOUD_KV_MAX_VALUE_BYTES` | `262144` (256 KB) | Per-key JSON-encoded value cap for `kv::set`. Rejects oversized payloads before authz so anonymous public scripts can't DoS Postgres JSONB columns. |
| `PICLOUD_DOCS_MAX_VALUE_BYTES` | `262144` (256 KB) | Per-document JSON-encoded data cap for `docs::create`/`update`. |
| `PICLOUD_PUBSUB_MAX_MESSAGE_BYTES` | `262144` (256 KB) | Per-message JSON-encoded payload cap for `pubsub::publish_durable`. Prevents one publish from amplifying into N outbox rows × M MB. |
| `PICLOUD_QUEUE_MAX_PAYLOAD_BYTES` | `262144` (256 KB) | Per-message JSON-encoded payload cap for `queue::enqueue`. |

## Out of MVP

Queue triggers, cron triggers, SMTP ingress, KV / docs / email / users / HTTP SDKs in scripts, interceptors, workflows, function-to-function `invoke()`, secrets, metrics dashboard. All deferred to v1.1+ per the blueprint. Don't pre-build for them — but don't make decisions that close the door on them either.

**Pulled forward to Phase 3 (pre-v1.1):** admin auth, multi-app scoping. Cross-app data sharing (export/import) stays at v1.3+; the initial cut enforces strict isolation. See blueprint §11.5.