PiCloud/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project

PiCloud is a self-hosted, event-driven serverless compute platform. Users upload Rhai scripts, get HTTP endpoints. Optimized for solo-dev / consumer hardware (single node MVP, multi-node cluster in v1.3+).

Authoritative design: serverless_cloud_blueprint.md. The blueprint is a living document — when architecture decisions are made in conversation that contradict it, treat the latest decision as truth and update the blueprint.

v1.1.x — SDK foundation + services — is complete. The SDK shape (handle pattern, :: namespaces, Services/SdkCallCx; see docs/sdk-shape.md, stdlib at docs/stdlib-reference.md) fixed in v1.1.0, then KV, docs, modules, HTTP, cron, files, pub/sub, email, users, and durable queues + invoke() filled it in through v1.1.9 — blueprint §12 has the table. Earlier groundwork: blueprint Phase 3 (admin auth, multi-app scoping, Phase 3.5 capability gating — manager-core::authz::{can, require, Capability}, migration 0006_users_authz.sql).

Current focus: v1.2 Hierarchies — groups + the declarative project tool (docs/design/groups-and-project-tool.md). That doc's §11 uses its own Phase 1–6 numbering, distinct from the blueprint product-phase numbering above — do not conflate them (its "Phase 3" = group-inherited config, not admin auth). Implemented on feat/groups-* branches: §11 Phase 1 (declarative pic plan/apply/prune + env overlays), Phase 2 (single-parent groups tree + hierarchy-aware RBAC), Phase 3 (group-inherited, env-scoped vars + secrets resolved live via a recursive CTE — no materialized cache), Phase 4-lite (group-owned endpoint scripts: scripts polymorphic owner in 0050_group_scripts.sql, get_by_name_inherited/is_invocable_by_app chain resolution, inherited invoke() + declarative route/trigger binding — all live, no body materialization), Phase 5 (the declarative project tool maps onto the group tree: the reconcile engine generalized to ApplyOwner{App|Group}, a [group] manifest kind, and a single atomic tree apply — pic plan/apply --dir reconciles a whole directory tree of picloud.toml nodes in one Postgres transaction, groups-before-apps so an app route can bind a group script created in the same tx; the bound token folds in each group's structure_version. Multi-repo single-owner/attach-point and per-env approval gating are deferred; groups pre-exist), Phase 4b (group modules + the lexical (sealed-by-default) import resolver, §5.5: owner-polymorphic ModuleScript, origin-rooted ModuleSource::resolve walking the importing node's chain, ExecRequest.script_owner threaded from every dispatch + invoke() site, _source-driven lexical chaining in PicloudModuleResolver with the compiled-module cache re-keyed by ScriptId, group modules/imports allowed, single-node dangling-import plan check — an inherited group script's imports seal to the group, a leaf can't shadow them), §5.5 extension points (opt-in polymorphism — §5.5 now complete: marker table 0051_extension_points.sql (owner-polymorphic, CASCADE — structurally a secrets name; default body = a co-located kind=module script), ModuleSource::resolve_policy with nearest-declaration-kind-wins — a concrete module resolves lexically, an EP marker resolves dynamically against the inheriting app (its override else the default body up-chain), NoProvider is a hard error; declarative-only authoring via the [app]/[group] manifest key extension_points = [...], reconcile mirrors secrets, single-node no-provider plan check, read-only pic extension-points ls + pull round-trip — the app can override a group default, the deliberate inverse of the Phase 4b sealed import), §11.6 group-level collections — KV + DOCS + FILES slices (full cross-app shared read/write: a group declares a collection shared via the [group] manifest collections = [...] → owner-polymorphic marker 0052_group_collections.sql with a kind discriminator + a per-kind group-keyed store: 0053_group_kv_entries.sql (kind='kv'), 0054_group_docs.sql (kind='docs', the queryable-JSON store), and 0055_group_files.sql (kind='files', blob metadata in Postgres + bytes on disk under <root>/files/groups/<group_id>/..., a groups/ infix disjoint from the per-app files/<app_id>/ subtree so the existing recursive orphan sweeper covers both with zero change) — no app_id, a shared row belongs to the group; CASCADE on group delete, an app delete leaves the data. Scripts use the explicit kv::shared_collection("name") / docs::shared_collection("name") / files::shared_collection("name") handles (shared alone is a Rhai reserved word); GroupKv/GroupDocs/GroupFilesServiceImpl resolve the owning group from cx.app_id's ancestor chain filtered by kind (nearest-wins) — that walk is the isolation boundary, a foreign app gets CollectionNotShared; a kv, a docs, and a files collection of the same name are distinct stores. The docs slice reuses the docs_filter DSL — build_find_query generalized on its owner column (docs/app_id vs group_docs/group_id, both literals); the files slice likewise generalized the atomic-write + checksum-on-read path helpers on an owner-relative dir (one source for the security-sensitive disk mechanics). Reads open to any subtree script (anonymous incl. — the declaration is the grant), writes require an authenticated editor+ on the owning group (GroupKvRead/Write, GroupDocsRead/Write, GroupFilesRead/Write, script_gate_require_principal fails closed on anon). Declarative authoring is the string-or-table form collections = ["catalog", { name = "articles", kind = "docs" }, { name = "assets", kind = "files" }] (bare string = kv); reconcile keys markers by (name, kind); read-only pic collections ls --group shows a kind column. Deferred: shared-collection triggers (the "group trigger has no app to watch" problem), topics/queue shared collections (same trigger-centric gap), per-group quotas, an operator admin API for shared blobs (scripts use the SDK; pic collections ls shows the marker)), and §4.5 group TRIGGER templates (live, event kinds — a [group] declares a [[triggers.kv|docs|files|pubsub]] template binding a group-owned handler; triggers gained a polymorphic owner 0056_group_triggers.sql mirroring 0050; the dispatcher's list_matching_kv/docs/files + the pubsub publish fan-out prepend CHAIN_LEVELS_CTE + JOIN chain c ON (t.app_id = c.app_owner OR t.group_id = c.group_owner) so a descendant app's event matches its own triggers plus ancestor-group templates in one query, the handler running under the firing app_id — the chain walk is the isolation boundary, a sibling-subtree app never matches; stateful kinds cron/queue/email rejected on a group (need materialization), route templates + per-app opt-out deferred; read-only pic triggers ls --group). Next: multi-node cluster mode, or extend templates to routes / the stateful kinds, or multi-repo ownership.

Data-model invariant: app-owned data-plane tables (KV, docs, files, …) start with app_id UUID NOT NULL REFERENCES apps(id) ON DELETE CASCADE; the group-inheritable tables — config (vars, secrets) and now group-owned code (scripts, 0050) — instead carry a polymorphic owner: nullable group_id and app_id with an exactly-one CHECK and per-owner partial-unique indexes (config is ON DELETE CASCADE, scripts RESTRICT — code is not data). Inheritance resolves live down apps.group_id → groups.parent_id via CHAIN_LEVELS_CTE (no materialized view); nearest-owner-wins with an app's own row shadowing the inherited one (CoW). Every Rhai SDK call resolves its app from cx.app_id, never a script-passed arg, and a group script always runs under the inheriting app's cx.app_id (the cross-app isolation boundary).

Three-Service Architecture

The platform splits into three logical services, each backed by a *-core library crate so the same logic runs in single-process MVP mode and split-process cluster mode:

Service	Role	Library crate
Manager	Control plane: script CRUD, scheduling/cron, dashboard backend, config. Single-writer to Postgres.	manager-core
Orchestrator	Per-node ingress: receives HTTP (later SMTP, queue) events, resolves script, dispatches to local executor. Stateless.	orchestrator-core
Executor	Per-node compute: runs Rhai scripts in a sandboxed engine. Stateless.	executor-core

In MVP, all three run in one process (picloud binary). In cluster mode, each runs as its own binary on each node, with one manager total and one orchestrator + executor per node.

Key boundary: the orchestrator never imports executor-core directly — it depends on an ExecutorClient trait. The local impl calls executor-core in-process; the remote impl is an HTTP client. Same pattern keeps cluster mode a swap, not a rewrite.

Path Scheme

Versioned API surfaces live under /api/v{N}/.... See docs/versioning.md for the full scheme.

• /api/v1/admin/* — manager (control plane: script CRUD, routes CRUD + check + match, logs, config; apps CRUD once Phase 3b lands)
• /api/v1/execute/{id} — orchestrator (data plane: invoke a script by ID, always-available bypass)
• /admin/* — dashboard SPA (SvelteKit, paths.base = '/admin')
• /healthz — liveness (string "ok")
• /version — every compatibility-surface version + public_base_url (JSON)
• everything else — orchestrator's user-route matcher: user scripts bind to arbitrary paths via POST /api/v1/admin/scripts/{id}/routes; if no route matches, picloud returns 404 with a JSON error.

Reserved path prefixes (rejected at route creation): /api/, /admin/, /healthz, /version.

Caddy fronts everything. Same Caddyfile shape works for single-node and cluster — only upstream targets change.

Param syntax convention: route paths use :name (e.g., /users/:id); domains (once apps land) use {name} (e.g., {tenant}.example.com). These are deliberately distinct — never use : in a domain context or {} in a route-path context.

Two-phase dispatch (Phase 3b onward): the orchestrator first resolves Host → app (most-specific domain claim wins), then runs that app's route trie. The route matcher itself is unchanged and never sees other apps' routes.

Tech Stack

• Rust 1.92+ workspace, pinned via rust-toolchain.toml
• Axum for HTTP, Tokio async, sqlx for Postgres
• Rhai embedded scripting (in executor-core)
• PostgreSQL 15+ with pgcrypto. v1.1+ data-plane tables use JSONB for value columns (hstore was considered for KV and rejected — see blueprint §8.1).
• SvelteKit dashboard, static adapter, CodeMirror 6 for the script editor
• Caddy 2 reverse proxy (auto-HTTPS in prod)
• Docker Compose for dev and single-node prod

Common Commands

# Rust workspace
cargo check --workspace
cargo test --workspace
cargo fmt --all
cargo clippy --all-targets --all-features -- -D warnings

# Run the all-in-one binary (MVP entry point)
cargo run -p picloud

# Run a single test
cargo test -p executor-core sandbox::tests::respects_operation_budget

# Dashboard (from dashboard/)
npm run dev
npm run build
npm run check

# Full stack (once docker-compose.yml exists)
docker compose up
docker compose down -v   # reset Postgres data

Workspace Layout

crates/
  shared/                 # cross-cutting types (Script, IDs, error enum, db pool)
  executor-core/          # Rhai engine, sandbox, ctx, log, SDK
  orchestrator-core/      # event ingress + ExecutorClient trait + dispatch
  manager-core/           # control plane: repos, scheduler, config
  picloud/                # ★ MVP all-in-one binary
  picloud-manager/        # cluster mode binary (skeleton)
  picloud-orchestrator/   # cluster mode binary (skeleton)
  picloud-executor/       # cluster mode binary (skeleton)
dashboard/                # SvelteKit
caddy/                    # Caddyfile, Caddyfile.prod
docker/                   # Dockerfiles
docs/
  git-workflow.md         # trunk-based workflow
  architecture.md         # (TBD)

Working Rules

• Honor the three-service boundary. Don't reach across *-core crates for behavior. If orchestrator-core needs to invoke logic from manager-core, define a trait in shared and inject the impl — keep implementations decoupled. Transport DTOs are not behavior: types like ExecRequest / ExecResponse / ExecError represent values produced or consumed across the wire, and depending on the originating crate's type definitions is fine. The bright line is "don't call across crates," not "don't import types." When in doubt: if the imported item is a struct/enum/type alias with no methods (or only data-shape methods), it's a DTO and crossing is fine; if it's a trait, function, or service, define the abstraction in shared and inject.
• executor-core has no Postgres dependency. Data-plane services (kv, docs, users — v1.1+) come in via injected ServiceProvider traits.
• Database writes only from manager-core. orchestrator-core reads scripts (cached); executor-core doesn't touch the DB.
• Stateful SDK services use the handle pattern + SdkCallCx. Collection-scoped surfaces look like kv::collection("x").get(k), not kv::get("x", k). Every service trait method takes &SdkCallCx and MUST derive app_id from cx.app_id — never trust a script-passed app_id. That is the cross-app isolation boundary. See docs/sdk-shape.md.
• MVP builds only the picloud all-in-one binary. The three split binaries exist as skeletons so the crate boundaries stay honest; flesh them out only when cluster mode is being implemented.
• Trunk-based dev. See docs/git-workflow.md. No long-lived branches. Feature flags for incomplete work.

Runtime configuration

Environment variables consumed by the picloud binary:

Variable	Default	Purpose
PICLOUD_BIND	0.0.0.0:8080	HTTP listen address. Port 8080 is owned by another process on this host — override locally.
PICLOUD_MAX_CONCURRENT_EXECUTIONS	32	Global concurrency cap on data-plane script executions. Overflow returns HTTP 503 with Retry-After: 1 immediately (no queue).
DATABASE_URL	—	Required. Postgres connection string.
PICLOUD_SECRET_KEY	—	Master encryption key (base64). Required at startup unless dev mode is acknowledged (below).
PICLOUD_DEV_MODE	false	true enables local-dev conveniences. Without PICLOUD_SECRET_KEY it ALSO requires the acknowledgement var below — PICLOUD_DEV_MODE=true alone aborts at startup. Also: when no SMTP relay is configured, email::send switches from disabled (NotConfigured) to an in-memory dev sink — sends succeed and the last 100 messages are readable at GET /api/v1/admin/dev/emails (instance Owner/Admin only; route exists only in this mode). Never in production.
PICLOUD_DEV_INSECURE_KEY	—	Set to the literal i-understand-this-is-insecure to let dev mode boot without PICLOUD_SECRET_KEY, using a deterministic, world-known dev master key. Never set in production — it would encrypt everything with a public value.
PICLOUD_DB_MAX_CONNECTIONS	32	Postgres pool size. Matched to PICLOUD_MAX_CONCURRENT_EXECUTIONS so the data plane can't starve background workers.
PICLOUD_SESSION_TTL_HOURS	24	Sliding-window session lifetime.
PICLOUD_SANDBOX_MAX_*	conservative defaults	Per-knob admin ceilings on Rhai sandbox overrides. See manager-core::sandbox::SandboxCeiling.
PICLOUD_FILES_ROOT	./data	Filesystem root for files::* blob storage (v1.1.5). Bytes live at <root>/files/<app_id>/<collection>/<id[0:2]>/<id>; metadata in Postgres.
PICLOUD_FILES_MAX_FILE_SIZE_BYTES	104857600 (100 MB)	Per-file hard size cap for files::* (v1.1.5). Per-app quotas deferred to v1.2.
PICLOUD_KV_MAX_VALUE_BYTES	262144 (256 KB)	Per-key JSON-encoded value cap for kv::set. Rejects oversized payloads before authz so anonymous public scripts can't DoS Postgres JSONB columns.
PICLOUD_DOCS_MAX_VALUE_BYTES	262144 (256 KB)	Per-document JSON-encoded data cap for docs::create/update.
PICLOUD_PUBSUB_MAX_MESSAGE_BYTES	262144 (256 KB)	Per-message JSON-encoded payload cap for pubsub::publish_durable. Prevents one publish from amplifying into N outbox rows × M MB.
PICLOUD_QUEUE_MAX_PAYLOAD_BYTES	262144 (256 KB)	Per-message JSON-encoded payload cap for queue::enqueue.

Out of MVP

Queue triggers, cron triggers, SMTP ingress, KV / docs / email / users / HTTP SDKs in scripts, interceptors, workflows, function-to-function invoke(), secrets, metrics dashboard. All deferred to v1.1+ per the blueprint. Don't pre-build for them — but don't make decisions that close the door on them either.

Pulled forward to Phase 3 (pre-v1.1): admin auth, multi-app scoping. Cross-app data sharing (export/import) stays at v1.3+; the initial cut enforces strict isolation. See blueprint §11.5.