51f14fa2b1
Closes the gaps and the one security finding from the second end-to-end
CLI test (E2E_STASH_REPORT.md), plus the H1 boot-regression found while
re-reviewing those fixes.
Security
- S6: reserved-path validation (`check_reserved`) now case-folds before
comparing, so `/API/v2/x`, `/HEALTHZ`, `/Admin/x` are rejected like
their lowercase forms. Request-time matching stays case-sensitive.
- S10: "public route != public data" callout in sdk-shape.md (script_gate
skips authz when the principal is anonymous).
Observability / features
- G1: trigger executions now write `execution_logs`. Migration 0043 adds
a `source` column (CHECK mirrors ExecutionSource/OutboxSourceKind,
DEFAULT 'http' backfills history); a shared `build_execution_log` helper
in executor-core; dispatcher logging for outbox triggers + queue
consumers (skips sync-HTTP rows the orchestrator already logs). `pic
logs` gains a source column + `--source` filter.
- G5: dev-only in-memory email capture under PICLOUD_DEV_MODE with no SMTP
(email::send succeeds locally), readable at GET /api/v1/admin/dev/emails
(Owner/Admin only; route mounted only in capture mode).
- G6: generalized the Rhai in-place-mutation footgun note (trim/replace/
make_upper/make_lower/crop/truncate/pad return ()).
- G2/G3/G4 (CLI): `pic members`, `pic files`, `pic queues`, read-only
`pic kv` (+ new kv_api.rs); `pic deploy --timeout/--memory/--kind/
--sandbox`; first-class `pic triggers create-{docs,files,pubsub,queue,
email}` wrappers. All new client path segments percent-encoded via seg().
H1 regression fix (found in re-review)
- The S6 change also runs in `compile_routes`, which compiles every stored
route at boot and on each route CRUD. A single stored route the new
validation rejects (creatable while the S6 gap existed) made the whole
compile Err and aborted startup. `compile_routes` is now lenient: it
skips an un-compilable row with a warning instead of bricking boot
(route creation still validates separately). Migration 0044 sweeps
pre-existing reserved-path routes on upgrade (WHERE mirrors
check_reserved exactly). Added regression tests for both.
Verified: cargo fmt, clippy --all-targets --all-features -D warnings, the
schema_snapshot test, and the new S6/lenient-compile unit tests all pass;
boot-resilience and G1/G5 confirmed live.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
137 lines
10 KiB
Markdown
137 lines
10 KiB
Markdown
# CLAUDE.md
|
||
|
||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||
|
||
## Project
|
||
|
||
**PiCloud** is a self-hosted, event-driven serverless compute platform. Users upload Rhai scripts, get HTTP endpoints. Optimized for solo-dev / consumer hardware (single node MVP, multi-node cluster in v1.3+).
|
||
|
||
Authoritative design: [serverless_cloud_blueprint.md](serverless_cloud_blueprint.md). The blueprint is a living document — when architecture decisions are made in conversation that contradict it, treat the latest decision as truth and update the blueprint.
|
||
|
||
**Current focus (Phase 4, v1.1.0):** SDK foundation + stdlib utilities — the shape every v1.1.x service module hangs off, see [docs/sdk-shape.md](docs/sdk-shape.md). Stdlib reference at [docs/stdlib-reference.md](docs/stdlib-reference.md). Subsequent v1.1.x releases (KV in v1.1.1, docs in v1.1.2, …) fill it in; see blueprint §12 for the full table. Phase 3 shipped end-to-end: admin auth, multi-app scoping, and Phase 3.5 capability gating (`manager-core::authz::{can, require, Capability}` + migration `0006_users_authz.sql`). Every v1.1+ table starts with `app_id UUID NOT NULL REFERENCES apps(id) ON DELETE CASCADE` and every Rhai SDK call resolves its app from the execution context.
|
||
|
||
## Three-Service Architecture
|
||
|
||
The platform splits into three logical services, each backed by a `*-core` library crate so the same logic runs in single-process MVP mode and split-process cluster mode:
|
||
|
||
| Service | Role | Library crate |
|
||
|---------|------|---------------|
|
||
| **Manager** | Control plane: script CRUD, scheduling/cron, dashboard backend, config. Single-writer to Postgres. | `manager-core` |
|
||
| **Orchestrator** | Per-node ingress: receives HTTP (later SMTP, queue) events, resolves script, dispatches to local executor. Stateless. | `orchestrator-core` |
|
||
| **Executor** | Per-node compute: runs Rhai scripts in a sandboxed engine. Stateless. | `executor-core` |
|
||
|
||
In MVP, all three run in one process (`picloud` binary). In cluster mode, each runs as its own binary on each node, with one manager total and one orchestrator + executor per node.
|
||
|
||
**Key boundary:** the orchestrator never imports `executor-core` directly — it depends on an `ExecutorClient` trait. The local impl calls `executor-core` in-process; the remote impl is an HTTP client. Same pattern keeps cluster mode a swap, not a rewrite.
|
||
|
||
## Path Scheme
|
||
|
||
Versioned API surfaces live under `/api/v{N}/...`. See [docs/versioning.md](docs/versioning.md) for the full scheme.
|
||
|
||
- `/api/v1/admin/*` — manager (control plane: script CRUD, routes CRUD + check + match, logs, config; apps CRUD once Phase 3b lands)
|
||
- `/api/v1/execute/{id}` — orchestrator (data plane: invoke a script by ID, always-available bypass)
|
||
- `/admin/*` — dashboard SPA (SvelteKit, `paths.base = '/admin'`)
|
||
- `/healthz` — liveness (string `"ok"`)
|
||
- `/version` — every compatibility-surface version + `public_base_url` (JSON)
|
||
- **everything else** — orchestrator's user-route matcher: user scripts bind to arbitrary paths via `POST /api/v1/admin/scripts/{id}/routes`; if no route matches, picloud returns 404 with a JSON error.
|
||
|
||
Reserved path prefixes (rejected at route creation): `/api/`, `/admin/`, `/healthz`, `/version`.
|
||
|
||
Caddy fronts everything. Same Caddyfile shape works for single-node and cluster — only upstream targets change.
|
||
|
||
**Param syntax convention:** route paths use `:name` (e.g., `/users/:id`); domains (once apps land) use `{name}` (e.g., `{tenant}.example.com`). These are deliberately distinct — never use `:` in a domain context or `{}` in a route-path context.
|
||
|
||
**Two-phase dispatch (Phase 3b onward):** the orchestrator first resolves `Host` → app (most-specific domain claim wins), then runs that app's route trie. The route matcher itself is unchanged and never sees other apps' routes.
|
||
|
||
## Tech Stack
|
||
|
||
- **Rust 1.92+** workspace, pinned via `rust-toolchain.toml`
|
||
- **Axum** for HTTP, **Tokio** async, **sqlx** for Postgres
|
||
- **Rhai** embedded scripting (in `executor-core`)
|
||
- **PostgreSQL 15+** with `pgcrypto`. v1.1+ data-plane tables use JSONB for value columns (hstore was considered for KV and rejected — see blueprint §8.1).
|
||
- **SvelteKit** dashboard, static adapter, CodeMirror 6 for the script editor
|
||
- **Caddy 2** reverse proxy (auto-HTTPS in prod)
|
||
- **Docker Compose** for dev and single-node prod
|
||
|
||
## Common Commands
|
||
|
||
```sh
|
||
# Rust workspace
|
||
cargo check --workspace
|
||
cargo test --workspace
|
||
cargo fmt --all
|
||
cargo clippy --all-targets --all-features -- -D warnings
|
||
|
||
# Run the all-in-one binary (MVP entry point)
|
||
cargo run -p picloud
|
||
|
||
# Run a single test
|
||
cargo test -p executor-core sandbox::tests::respects_operation_budget
|
||
|
||
# Dashboard (from dashboard/)
|
||
npm run dev
|
||
npm run build
|
||
npm run check
|
||
|
||
# Full stack (once docker-compose.yml exists)
|
||
docker compose up
|
||
docker compose down -v # reset Postgres data
|
||
```
|
||
|
||
## Workspace Layout
|
||
|
||
```
|
||
crates/
|
||
shared/ # cross-cutting types (Script, IDs, error enum, db pool)
|
||
executor-core/ # Rhai engine, sandbox, ctx, log, SDK
|
||
orchestrator-core/ # event ingress + ExecutorClient trait + dispatch
|
||
manager-core/ # control plane: repos, scheduler, config
|
||
picloud/ # ★ MVP all-in-one binary
|
||
picloud-manager/ # cluster mode binary (skeleton)
|
||
picloud-orchestrator/ # cluster mode binary (skeleton)
|
||
picloud-executor/ # cluster mode binary (skeleton)
|
||
dashboard/ # SvelteKit
|
||
caddy/ # Caddyfile, Caddyfile.prod
|
||
docker/ # Dockerfiles
|
||
docs/
|
||
git-workflow.md # trunk-based workflow
|
||
architecture.md # (TBD)
|
||
```
|
||
|
||
## Working Rules
|
||
|
||
- **Honor the three-service boundary.** Don't reach across `*-core` crates *for behavior*. If `orchestrator-core` needs to invoke logic from `manager-core`, define a trait in `shared` and inject the impl — keep implementations decoupled. **Transport DTOs are not behavior**: types like `ExecRequest` / `ExecResponse` / `ExecError` represent values produced or consumed across the wire, and depending on the originating crate's type definitions is fine. The bright line is "don't call across crates," not "don't import types." When in doubt: if the imported item is a `struct`/`enum`/`type alias` with no methods (or only data-shape methods), it's a DTO and crossing is fine; if it's a trait, function, or service, define the abstraction in `shared` and inject.
|
||
- **`executor-core` has no Postgres dependency.** Data-plane services (kv, docs, users — v1.1+) come in via injected `ServiceProvider` traits.
|
||
- **Database writes only from `manager-core`.** `orchestrator-core` reads scripts (cached); `executor-core` doesn't touch the DB.
|
||
- **Stateful SDK services use the handle pattern + `SdkCallCx`.** Collection-scoped surfaces look like `kv::collection("x").get(k)`, not `kv::get("x", k)`. Every service trait method takes `&SdkCallCx` and **MUST** derive `app_id` from `cx.app_id` — never trust a script-passed `app_id`. That is the cross-app isolation boundary. See [docs/sdk-shape.md](docs/sdk-shape.md).
|
||
- **MVP builds only the `picloud` all-in-one binary.** The three split binaries exist as skeletons so the crate boundaries stay honest; flesh them out only when cluster mode is being implemented.
|
||
- **Trunk-based dev.** See [docs/git-workflow.md](docs/git-workflow.md). No long-lived branches. Feature flags for incomplete work.
|
||
|
||
## Runtime configuration
|
||
|
||
Environment variables consumed by the `picloud` binary:
|
||
|
||
| Variable | Default | Purpose |
|
||
|---|---|---|
|
||
| `PICLOUD_BIND` | `0.0.0.0:8080` | HTTP listen address. Port 8080 is owned by another process on this host — override locally. |
|
||
| `PICLOUD_MAX_CONCURRENT_EXECUTIONS` | `32` | Global concurrency cap on data-plane script executions. Overflow returns HTTP 503 with `Retry-After: 1` immediately (no queue). |
|
||
| `DATABASE_URL` | — | Required. Postgres connection string. |
|
||
| `PICLOUD_SECRET_KEY` | — | Master encryption key (base64). Required at startup unless dev mode is acknowledged (below). |
|
||
| `PICLOUD_DEV_MODE` | `false` | `true` enables local-dev conveniences. Without `PICLOUD_SECRET_KEY` it ALSO requires the acknowledgement var below — `PICLOUD_DEV_MODE=true` alone aborts at startup. Also: when no SMTP relay is configured, `email::send` switches from disabled (`NotConfigured`) to an **in-memory dev sink** — sends succeed and the last 100 messages are readable at `GET /api/v1/admin/dev/emails` (instance Owner/Admin only; route exists only in this mode). Never in production. |
|
||
| `PICLOUD_DEV_INSECURE_KEY` | — | Set to the literal `i-understand-this-is-insecure` to let dev mode boot without `PICLOUD_SECRET_KEY`, using a deterministic, world-known dev master key. Never set in production — it would encrypt everything with a public value. |
|
||
| `PICLOUD_DB_MAX_CONNECTIONS` | `32` | Postgres pool size. Matched to `PICLOUD_MAX_CONCURRENT_EXECUTIONS` so the data plane can't starve background workers. |
|
||
| `PICLOUD_SESSION_TTL_HOURS` | `24` | Sliding-window session lifetime. |
|
||
| `PICLOUD_SANDBOX_MAX_*` | conservative defaults | Per-knob admin ceilings on Rhai sandbox overrides. See `manager-core::sandbox::SandboxCeiling`. |
|
||
| `PICLOUD_FILES_ROOT` | `./data` | Filesystem root for `files::*` blob storage (v1.1.5). Bytes live at `<root>/files/<app_id>/<collection>/<id[0:2]>/<id>`; metadata in Postgres. |
|
||
| `PICLOUD_FILES_MAX_FILE_SIZE_BYTES` | `104857600` (100 MB) | Per-file hard size cap for `files::*` (v1.1.5). Per-app quotas deferred to v1.2. |
|
||
| `PICLOUD_KV_MAX_VALUE_BYTES` | `262144` (256 KB) | Per-key JSON-encoded value cap for `kv::set`. Rejects oversized payloads before authz so anonymous public scripts can't DoS Postgres JSONB columns. |
|
||
| `PICLOUD_DOCS_MAX_VALUE_BYTES` | `262144` (256 KB) | Per-document JSON-encoded data cap for `docs::create`/`update`. |
|
||
| `PICLOUD_PUBSUB_MAX_MESSAGE_BYTES` | `262144` (256 KB) | Per-message JSON-encoded payload cap for `pubsub::publish_durable`. Prevents one publish from amplifying into N outbox rows × M MB. |
|
||
| `PICLOUD_QUEUE_MAX_PAYLOAD_BYTES` | `262144` (256 KB) | Per-message JSON-encoded payload cap for `queue::enqueue`. |
|
||
|
||
## Out of MVP
|
||
|
||
Queue triggers, cron triggers, SMTP ingress, KV / docs / email / users / HTTP SDKs in scripts, interceptors, workflows, function-to-function `invoke()`, secrets, metrics dashboard. All deferred to v1.1+ per the blueprint. Don't pre-build for them — but don't make decisions that close the door on them either.
|
||
|
||
**Pulled forward to Phase 3 (pre-v1.1):** admin auth, multi-app scoping. Cross-app data sharing (export/import) stays at v1.3+; the initial cut enforces strict isolation. See blueprint §11.5.
|