PiCloud/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project

PiCloud is a self-hosted, event-driven serverless compute platform. Users upload Rhai scripts, get HTTP endpoints. Optimized for solo-dev / consumer hardware (single node MVP, multi-node cluster in v1.3+).

Authoritative design: serverless_cloud_blueprint.md. The blueprint is a living document — when architecture decisions are made in conversation that contradict it, treat the latest decision as truth and update the blueprint.

v1.1.x — SDK foundation + services — is complete. The SDK shape (handle pattern, :: namespaces, Services/SdkCallCx; see docs/sdk-shape.md, stdlib at docs/stdlib-reference.md) fixed in v1.1.0, then KV, docs, modules, HTTP, cron, files, pub/sub, email, users, and durable queues + invoke() filled it in through v1.1.9 — blueprint §12 has the table. Earlier groundwork: blueprint Phase 3 (admin auth, multi-app scoping, Phase 3.5 capability gating — manager-core::authz::{can, require, Capability}, migration 0006_users_authz.sql).

Current focus: v1.2 Hierarchies — groups + the declarative project tool (docs/design/groups-and-project-tool.md). That doc's §11 uses its own Phase 1–6 numbering, distinct from the blueprint product-phase numbering above — do not conflate them (its "Phase 3" = group-inherited config, not admin auth). Implemented on feat/groups-* branches: §11 Phase 1 (declarative pic plan/apply/prune + env overlays), Phase 2 (single-parent groups tree + hierarchy-aware RBAC), Phase 3 (group-inherited, env-scoped vars + secrets resolved live via a recursive CTE — no materialized cache), Phase 4-lite (group-owned endpoint scripts: scripts polymorphic owner in 0050_group_scripts.sql, get_by_name_inherited/is_invocable_by_app chain resolution, inherited invoke() + declarative route/trigger binding — all live, no body materialization), Phase 5 (the declarative project tool maps onto the group tree: the reconcile engine generalized to ApplyOwner{App|Group}, a [group] manifest kind, and a single atomic tree apply — pic plan/apply --dir reconciles a whole directory tree of picloud.toml nodes in one Postgres transaction, groups-before-apps so an app route can bind a group script created in the same tx; the bound token folds in each group's structure_version. The single-owner ownership claim shipped as §7 M1, the attach-point ceiling + blast-radius as §7 M2/M3, and per-env approval gating as §3 M3 — all server-authoritative (see the tail); declarative group create/reparent + structural-divergence detection shipped as §6 (reconcile_group_structure_tx/reparent_group_tx/StructureMode), so groups no longer need to pre-exist), Phase 4b (group modules + the lexical (sealed-by-default) import resolver, §5.5: owner-polymorphic ModuleScript, origin-rooted ModuleSource::resolve walking the importing node's chain, ExecRequest.script_owner threaded from every dispatch + invoke() site, _source-driven lexical chaining in PicloudModuleResolver with the compiled-module cache re-keyed by ScriptId, group modules/imports allowed, single-node dangling-import plan check — an inherited group script's imports seal to the group, a leaf can't shadow them), §5.5 extension points (opt-in polymorphism — §5.5 now complete: marker table 0051_extension_points.sql (owner-polymorphic, CASCADE — structurally a secrets name; default body = a co-located kind=module script), ModuleSource::resolve_policy with nearest-declaration-kind-wins — a concrete module resolves lexically, an EP marker resolves dynamically against the inheriting app (its override else the default body up-chain), NoProvider is a hard error; declarative-only authoring via the [app]/[group] manifest key extension_points = [...], reconcile mirrors secrets, single-node no-provider plan check, read-only pic extension-points ls + pull round-trip — the app can override a group default, the deliberate inverse of the Phase 4b sealed import), §11.6 group-level collections — KV + DOCS + FILES slices (full cross-app shared read/write: a group declares a collection shared via the [group] manifest collections = [...] → owner-polymorphic marker 0052_group_collections.sql with a kind discriminator + a per-kind group-keyed store: 0053_group_kv_entries.sql (kind='kv'), 0054_group_docs.sql (kind='docs', the queryable-JSON store), and 0055_group_files.sql (kind='files', blob metadata in Postgres + bytes on disk under <root>/files/groups/<group_id>/..., a groups/ infix disjoint from the per-app files/<app_id>/ subtree so the existing recursive orphan sweeper covers both with zero change) — no app_id, a shared row belongs to the group; CASCADE on group delete, an app delete leaves the data. Scripts use the explicit kv::shared_collection("name") / docs::shared_collection("name") / files::shared_collection("name") handles (shared alone is a Rhai reserved word); GroupKv/GroupDocs/GroupFilesServiceImpl resolve the owning group from cx.app_id's ancestor chain filtered by kind (nearest-wins) — that walk is the isolation boundary, a foreign app gets CollectionNotShared; a kv, a docs, and a files collection of the same name are distinct stores. The docs slice reuses the docs_filter DSL — build_find_query generalized on its owner column (docs/app_id vs group_docs/group_id, both literals); the files slice likewise generalized the atomic-write + checksum-on-read path helpers on an owner-relative dir (one source for the security-sensitive disk mechanics). Reads open to any subtree script (anonymous incl. — the declaration is the grant), writes require an authenticated editor+ on the owning group (GroupKvRead/Write, GroupDocsRead/Write, GroupFilesRead/Write, script_gate_require_principal fails closed on anon). Declarative authoring is the string-or-table form collections = ["catalog", { name = "articles", kind = "docs" }, { name = "assets", kind = "files" }] (bare string = kv); reconcile keys markers by (name, kind); read-only pic collections ls --group shows a kind column. Topic shared collections shipped as D2 (storeless), queue as D3 — see below), and §4.5 group TRIGGER templates (live, event kinds — a [group] declares a [[triggers.kv|docs|files|pubsub]] template binding a group-owned handler; triggers gained a polymorphic owner 0056_group_triggers.sql mirroring 0050; the dispatcher's list_matching_kv/docs/files + the pubsub publish fan-out prepend CHAIN_LEVELS_CTE + JOIN chain c ON (t.app_id = c.app_owner OR t.group_id = c.group_owner) so a descendant app's event matches its own triggers plus ancestor-group templates in one query, the handler running under the firing app_id — the chain walk is the isolation boundary, a sibling-subtree app never matches; stateful kinds cron/queue/email need materialization — see M5 below; per-app opt-out deferred; read-only pic triggers ls --group), and §4.5 group ROUTE templates (live, inherited — a [group] declares a [[routes]] template binding a group-owned endpoint; routes gained a polymorphic owner 0057_group_routes.sql mirroring 0056. Unlike triggers (per-event SQL), routes serve from the in-memory RouteTable, so the HTTP hot path can't resolve inheritance per request — instead the table rebuild expands templates into each descendant app's slice via RouteRepository::list_effective (all-apps generalization of CHAIN_LEVELS_CTE: every app × its ancestor chain ⋈ routes), and compile_effective_routes applies nearest-owner-wins shadowing (an app's own identical binding shadows the inherited template; non-identical bindings coexist under the matcher's existing precedence — a route picks one winner, unlike a fanning trigger). Because the table is a cache, inheritance is rebuilt full-live through the single rebuild_route_table chokepoint on every edge that changes it: route CRUD, apply, and tree mutations — app create/delete (apps_api) + group reparent (groups_api) — so a new app under a group serves its templates instantly. Host-claim validation is skipped for a group template (descendants serve it on their own host claim; templates use host_kind = any). The chain expansion is the isolation boundary — a sibling-subtree app never inherits (pinned by tests/group_route_templates.rs + the group_routes journey); read-only pic routes ls --group. Deferred: multi-node snapshot propagation), and §11 tail per-app opt-out (template suppression) (a descendant declines an inherited group template: an [app] declares [suppress] with triggers = [...] (handler script names) + routes = [...] (paths) — coarse by reference, not a full definition, since template row-ids churn on re-apply but a reference is stable (re-apply NoOp, may decline several templates bound to the same script/path). App-only marker 0058_template_suppressions.sql (app_id NOT NULL CASCADE, a target_kind discriminator), reconciled with the extension-point marker pattern (prunable → re-inherits). Consumed at the two resolution points: the trigger dispatch queries gain a correlated NOT EXISTS anti-join (gated to t.group_id IS NOT NULL), and compile_effective_routes drops an inherited (depth > 0) route at a suppressed path (loaded via RouteRepository::list_route_suppressions). Inheritance-only — the group_id IS NOT NULL / depth > 0 gates mean an app can only decline what it inherits, never its own or a sibling's (pinned by tests/template_suppression.rs + the suppress journey); a dangling suppress is an apply-time warning; read-only pic suppress ls --app. Trust-model consequence: group templates are advisory-by-default (run unless a descendant declines) — a footgun for compliance hooks (audit/security triggers a tenant can opt out of)), and §11 tail sealed (mandatory) group templates (closes that footgun: a [group] marks a route/event-trigger template sealed = true and the two suppression filters skip it, so a descendant's [suppress] is ignored — it fires/serves on every descendant. Column sealed BOOLEAN on triggers + routes (0059_sealed_templates.sql); the trigger anti-join gains AND t.sealed = FALSE (a sealed row is never excluded → fires through), and compile_effective_routes gates its suppression continue on !er.route.sealed. sealed lives on the shared Route DTO + manager-core Trigger (both pure data) so the apply diff sees the current value — part of the route Update comparison + the trigger identity, so toggling it re-applies. Authored per-template (sealed = true on a [[routes]]/[[triggers.kv|docs|files|pubsub]]), group-only — validate_bundle_for rejects it on an app owner (an app resource is never inherited). Sealing only strengthens the guarantee (a sealed template can't be declined; it never grants new reach — the chain walk is still the isolation boundary). The dangling-suppress warning also flags a suppression matching only sealed templates ("… is sealed — the suppression has no effect"), and pic triggers/routes ls --group show a sealed column; pinned by tests/sealed_templates.rs + the sealed journey. Deferred: multi-node snapshot propagation), and §11 tail M1 group-level suppression (template_suppressions gained a polymorphic owner 0060_group_suppressions.sql — a [group] declares [suppress] to decline a template it inherits from a higher ancestor for its whole subtree. Both filters generalized to the chain: the trigger anti-join joins the chain CTE (ts.app_id = sc.app_owner OR ts.group_id = sc.group_owner), and list_route_suppressions expands group suppressions across descendants via the all-apps app_chain CTE (compile_effective_routes unchanged). Still inheritance-only + sealed overrides; owner-polymorphic suppression_repo::list_for_owner/insert/delete; read-only pic suppress ls --group; the ineffective-suppress warning walks GROUP_CHAIN_LEVELS_CTE for a group node. Pinned by tests/group_suppression.rs + the suppress journey), and §11.6 M2 shared-collection triggers (a write to a group SHARED collection now fires a trigger — closes the "group trigger has no app to watch" gap. A group-owned trigger marked shared = true (shared BOOLEAN on triggers, 0061_shared_triggers.sql) watches the group's shared collection; the per-app list_matching_kv/docs/files add AND t.shared = FALSE, new list_matching_shared_{kv,docs,files}(owning_group,…) select shared = TRUE triggers on the owning group — the shared flag is the namespace boundary. ServiceEventEmitter gained emit_shared(cx, owning_group, event); GroupKv/Docs/FilesServiceImpl (via a with_events builder) emit on write, and the outbox emitter stamps the WRITER app_id so the handler runs under the writer. shared is authored on a group [[triggers.kv|docs|files]], is part of the diff identity, and validate_bundle_for rejects it on an app owner / a non-collection kind / an undeclared collection. Read-only shared column in pic triggers ls --group; pinned by tests/shared_triggers.rs + the shared_triggers journey. Shared pubsub triggers shipped as D2 — see below), and §11.6 M3 per-group quotas (global env-var ceilings enforced in the group write path: PICLOUD_GROUP_KV_MAX_ROWS/_DOCS_MAX_ROWS (per-group row count, new-key only), PICLOUD_GROUP_{KV,DOCS}_MAX_TOTAL_BYTES (per-group total stored bytes, projected-total check — Track A M4) + _FILES_MAX_TOTAL_BYTES (per-group total bytes); group_quota helpers + count_rows/total_bytes repo methods + QuotaExceeded/TotalBytesQuotaExceeded errors), and §11.6 M4 read-only operator admin API (group_blobs_api mirrors the per-app kv/files admin surface for a group's shared collections: GET /groups/{id}/{kv,docs,files}[/{collection}/{key|id}], authz GroupKvRead/GroupDocsRead/GroupFilesRead; the host hoists the group repos to share one instance; pic kv ls/get --group; reads-only, writes stay script-only), and §4.5 M5 stateful group trigger templates via materialization (cron + queue + email — the kinds that can't resolve live because each needs a per-app row: cron last_fired_at, the queue one-consumer advisory lock, the email sealed inbound secret. A group [[triggers.cron|queue|email]] template is materialized into an app-owned copy per descendant (materialized_from col, 0062_materialized_triggers.sql + the 0063_materialized_unique.sql partial-unique index that makes rematerialization idempotent under concurrency) by materialize::rematerialize_stateful_templates — all-apps app_chain CTE ⋈ group-owned stateful templates, a precise create/delete diff preserving cron state — run full-live at the route-rebuild chokepoints (apply single+tree, app create/delete in apps_api, group reparent in groups_api; AppsState/GroupsState gained a pool). The scheduler + list_active_queue_consumers + email_inbound_target gained AND t.app_id IS NOT NULL so a group TEMPLATE is never dispatched directly (nor invocable via its own webhook URL) — only its per-app copies are; a queue copy is skipped-with-warning when the app already fills that queue's slot. M5.5 email uses the shared-group-secret model: the template resolves its inbound_secret_ref against the group's own secret store once at apply and seals it (resolve_and_seal/insert_email_trigger_tx generalized to a SecretOwner/ScriptOwner); email secrets are now v1 AAD-bound to the SEALING OWNER (Track A M3, migration 0069_email_secret_version + seal_email/open_email; the group AAD is stable across rows, so materialization still copies the sealed bytes verbatim — the inbound path recovers the sealing group via materialized_from and opens under its AAD; a legacy v0 read path remains for pre-M3 rows). All descendant webhooks share the one group HMAC secret; an unset group secret fails apply hard. Loading a group's own set-secret names into CurrentState (was hardcoded empty) lets the plan-time email-secret check resolve against the group. Pinned by tests/stateful_templates.rs + the stateful_templates journey. Per-app (non-shared) email secrets already work — an app-owned email trigger resolves its inbound_secret_ref against the app's own secret store and seals it AAD-bound to the app (SecretOwner::App, AAD email:{app_id} in secrets_service::email_secret_aad); M5.5 generalized that original app path to groups, not the reverse, and the inbound path recovers SecretOwner::App when materialized_from is NULL. The only per-app nuance still open is deliberate: the interactive POST .../triggers/email API takes the secret inline, whereas the apply path resolves a named app secret — the sealing/AAD machinery is fully app-aware either way), and D1 the materialized column (pic triggers ls --app now shows a read-only materialized column — a copy of an M5 group stateful template reads true, distinct from a hand-authored trigger; a derived materialized bool = materialized_from IS NOT NULL threaded row→domain→API→CLI mirroring sealed/shared), and §11.6 D2 shared TOPICS + shared pub/sub triggers (a group declares a storeless topic shared collection (group_collections.kind widened to topic+queue in 0064); a [[triggers.pubsub]] shared = true handler watches it. BundleTrigger::Pubsub gained shared (part of the identity); validate_bundle_for requires the topic pattern's ROOT segment (events.* → events) be a declared kind='topic' collection, group-only. Scripts publish via the explicit pubsub::shared_topic("events").publish("created", msg) handle → GroupPubsubServiceImpl resolves the owning group (kind topic) from cx.app_id's chain, requires editor+ (GroupPubsubPublish, fails closed on anon), and fans out via PubsubRepo::fan_out_shared_publish to shared = true pubsub triggers on that group, each outbox row stamped the WRITER app_id (M2 model). The per-app fan_out_publish gained AND t.shared = FALSE — a shared trigger never fires on a per-app publish and vice-versa (the shared flag is the namespace boundary); the owning-group chain walk is the isolation boundary. Pinned by tests/shared_topics.rs + the shared_topics journey. External SSE subscription shipped (Track A M6): GET /realtime/shared/topics/{topic} streams a shared topic to external clients; RealtimeAuthority::authorize_subscribe_shared resolves the owning group from the subscriber app's chain (reads-open — the resolution IS the authorization, a foreign subtree 404s), and GroupPubsubServiceImpl::with_realtime bridges publish→broadcast after the durable fan-out. Deferred: multi-node broadcast propagation (cluster mode)), and §11.6 D3 shared durable QUEUES (a group declares a queue shared collection; any subtree app enqueues into ONE group-keyed store (group_queue_messages, 0065, mirrors queue_messages keyed by (group_id, collection); CASCADE on group delete) via queue::shared_collection("name").enqueue(...) → GroupQueueServiceImpl resolves the owning group (kind queue), requires editor+ (GroupQueueEnqueue, fails closed on anon). Consumption is by COMPETING CONSUMERS: a group [[triggers.queue]] shared = true consumer (shared threaded through BundleTrigger::Queue + identity) materializes a consumer copy per descendant app (materialize skips the M5 one-consumer-slot check for shared — each descendant intentionally gets a consumer), and all copies claim the SHARED store with FOR UPDATE SKIP LOCKED — each message delivered at-most-once across the subtree, scaling horizontally, each handler under its own cx.app_id. The dispatcher's queue arm gained a shared_group on ActiveQueueConsumer (recovered via a LEFT JOIN to the materialized copy's source template) and routes claim/ack/nack/terminal to the group store when set (q_claim/q_ack/q_nack/q_terminal helpers; a group claim normalizes to a ClaimedMessage under the consuming app); the reclaim task drains both stores. validate_bundle_for requires a shared queue on a group to name a declared kind='queue' collection; a shared queue on an app is rejected by the app-owner shared guard. Pinned by tests/group_queue.rs (competing-consumer at-most-once) + stateful_templates.rs + the shared_queues journey. Group dead-letter store shipped (Track A M2): an exhausted shared-queue message is preserved in group_dead_letters (0068_group_dead_letters) via GroupQueueRepo::dead_letter (atomic INSERT+DELETE) and is operator-visible at read-only GET /api/v1/admin/groups/{id}/dead-letters (GroupKvRead). Shared dead-letter fan-out shipped (B2): a group declares a declaratively-authored [[triggers.dead_letter]] shared = true handler; when a shared-queue message exhausts, the dispatcher's q_terminal group branch (after persisting to group_dead_letters) fans out to list_matching_shared_dead_letter(owning_group, "queue", …), each outbox row stamped the WRITER app_id (the consuming app — M2 model), so the handler runs under the consumer. The per-app list_matching_dead_letter gained AND t.shared = FALSE (the shared flag is the namespace boundary); insert_trigger_tx now accepts dead_letter (BundleTrigger::DeadLetter), and validate_bundle_for requires it group-owned + shared. Pinned by tests/shared_dead_letter.rs), and §7 multi-repo ownership M1 — the single-owner claim (the owner_project seam (0047) is now live behind a first-class projects table (0066_projects.sql, UUID pk + unique slug; owner_project FKs it ON DELETE SET NULL — un-claim, never cascade-destroy a tree). A [project] block (slug + optional name) in the repo's ROOT manifest declares identity (independent of the [app]/[group] XOR); the first apply with a new slug registers the project and claims each group node it touches. The claim runs inside the apply tx under the per-node advisory lock before the diff, so a conflict short-circuits with a 409 before any write. Pure policy decide_group_claim/decide_app_owner in apply_service (unit-tested, DB-free): unclaimed→claim, owner→no-op, foreign→conflict unless --takeover (which additionally requires GroupAdmin — ownership ⟂ RBAC, mapped to 403 vs the 409 conflict), no-project-into-a-claimed-subtree→conflict. Apps carry no owner — an app inherits ownership from its nearest claimed ancestor group (the ancestor walk, via groups.ancestors now carrying owner_project through its recursive CTE, is the isolation boundary); an unclaimed subtree stays open, so nothing changes until a repo first declares [project] (backward-compatible). ProjectRepository (read side) + tx free-fns upsert_project_tx/read_group_owner_tx/write_group_owner_tx; the claim deliberately does not bump structure_version (not a diff change → won't churn a pending bound plan). Wire: project/takeover on the apply request (both #[serde(default)] → the pre-M1 CLI stays compatible); the CLI surfaces the server's 409 message verbatim (covers StateMoved + OwnershipConflict). Visibility: pic groups ls owner column (server list_with_owner LEFT JOIN; the shared Group deserialize ignores the extra field so pic groups tree/dashboard are unaffected) + pic apply --takeover. Pinned by apply_service unit tests, tests/projects_repo.rs, and the apply_ownership journey. M2 shipped — the attach-point ceiling: a [project] parent_group = "<slug>" binds the repo under a pre-existing group; check_within_attach refuses (422 OutsideAttachPoint) any node not strictly within that subtree (a group node must be a proper descendant — you can't apply the attach point itself; an app node's group must be at-or-below it), resolved via groups.ancestors and enforced read-only before the claim in apply_owner/apply_tree; absent = instance root = no ceiling. M3 shipped — plan preview + pic projects ls: pic plan now carries the [project] and returns an ownership preview per node (claim/owned/conflict-owner-named/unclaimed, pure preview_ownership) plus, for a group node, the cross-repo blast radius (descendant apps owned by OTHER projects the change reaches, via group_blast_radius — a subtree CTE with per-group memoized nearest-claimed resolution); the attach ceiling is previewed at plan too. Read-only pic projects ls (GET /api/v1/admin/projects, list_with_counts) lists projects + owned-group counts. Pinned by the apply_ownership plan-preview case. With M3 the §7 multi-repo ownership track (M1 claim · M2 attach ceiling · M3 preview + pic projects ls) is COMPLETE. Also shipped: §6 group-tree Tier 1 (declarative group create via dir-nesting · structural-divergence detection · declarative reparent — reconcile_group_structure_tx/StructureMode, 422 StructuralDivergence) and §3 M3 the per-env approval gate, now server-authoritative (migration 0067_project_environments; the gate resolves the governing project from the target node's nearest-claimed ancestor — governing_env_policy/_tree — so omitting/spoofing [project] can't bypass it; an approved gated apply requires AppAdmin/GroupAdmin step-up + audit).

Track A (v1.2 deferred-gap closeout, migrations 0067–0069) shipped to local main: M1 hermetic approval gate · M2 shared-queue dead-letter store · M3 email-secret AAD v0→v1 · M4 per-group KV/docs byte quotas · M5 set_if compare-and-swap for KV (per-app + shared + Rhai SDK) · M6 shared-topic external SSE. Audit 2026-07-11 remediation (migration 0070, admin-session absolute cap) also shipped. With that, v1.2 Hierarchies is complete. The Workflows track then shipped too (M1–M6: DAG execution + conditional when, nested sub-workflows, durable orchestrator, workflow::start SDK + admin run API + pic workflows, dashboard DAG + run-history; migrations 0071/0072). §9.4 service interceptors are now COMPLETE (migrations 0073–0075). A [[interceptors]] block (app or group) binds a script to a (service, op, phase) marker resolved on the app chain and run via the invoke() re-entry path. Shipped across M1–M12: all six data-plane services — kv (set/delete/set_if), docs/files (create/update/delete), queue (enqueue), pubsub (publish), http (request), each per-app AND group-shared; before + after phases (phase = "before"|"after", migration 0074) — before = allow/deny + a data transform (rewrites the written value, size-capped), after = observe/audit with the write result (cannot roll back); ordered chaining ancestor→app (a group guard runs first; before ancestor→app, after app→ancestor) with an identity cycle guard + the existing re-entrancy bypass; a per-interceptor timeout (timeout_ms, migration 0075, env default PICLOUD_INTERCEPTOR_TIMEOUT_MS, tightened to at most the caller's remaining deadline); a per-execution resolve cache (N un-hooked writes → 1 chain query); the seal (a group's interceptor script resolves at the group — a descendant can't shadow it) and fail-closed verdict (only #{ allowed: true } allows). validate_bundle_for enforces a per-service allowed-ops map (every validated (service, op) has a matching runtime hook — no fail-open). Read-only pic interceptors ls --app (resolved chain view) / --group (own markers). Pinned by tests/interceptors.rs (12 journeys) + executor-core unit tests. Deferred: interceptor coverage of non-listed services/ops. Next: multi-node cluster mode (the deferred multi-node route/broadcast propagation lives there).

Write-path invariant — the transactional outbox (manager-core::atomic_write). A data-plane mutation (KV / docs / files, per-app and group-shared) and the trigger fan-out it produces commit in ONE transaction on ONE connection, via a *Writer injected into each service (with_atomic_writes(pool) in the host). The services previously wrote the row, waited for it to commit, and then asked the emitter to resolve triggers and insert outbox rows — a second transaction on a second connection, so an outbox failure left a committed row whose trigger never fired, invisible to the caller and unrecoverable by any retry. Now an emit failure rolls the write back and surfaces as an error. Three rules make it work:

• Everything inside a transaction runs on that transaction's connection. A writer that held a tx and then reached for a second pooled connection could deadlock — the pool is sized to the execution-concurrency cap, so N executions each wanting 2 connections starve. Hence outbox_event_emitter::emit_on(&mut *tx, …) and the *_on(exec, …) repo free fns (the trait methods delegate to them, so each query has one home). shared has no sqlx and must not gain it — the ServiceEventEmitter trait stays connection-free; the transactional path is entirely manager-core-internal.
• Per-group quotas need a lock, not just a transaction. Under READ COMMITTED each transaction's COUNT(*)/SUM(...) sees a snapshot without the other writers' uncommitted rows, so concurrent writers all pass the check anyway (measured: 19 rows stored against a ceiling of 5). Every group write therefore takes pg_advisory_xact_lock on a per-(group, kind) key first, serializing check-then-write. The row/byte policy itself is one shared fn, group_quota::check_group_write.
• Files order around the disk write, which cannot join a transaction: create/update write the blob first and unlink it if the tx rolls back; delete commits the metadata removal first and unlinks after (the reverse would destroy the bytes of a row a rollback keeps).

Data-model invariant: app-owned data-plane tables (KV, docs, files, …) start with app_id UUID NOT NULL REFERENCES apps(id) ON DELETE CASCADE; the group-inheritable tables — config (vars, secrets) and now group-owned code (scripts, 0050) — instead carry a polymorphic owner: nullable group_id and app_id with an exactly-one CHECK and per-owner partial-unique indexes (config is ON DELETE CASCADE, scripts RESTRICT — code is not data). Inheritance resolves live down apps.group_id → groups.parent_id via CHAIN_LEVELS_CTE (no materialized view); nearest-owner-wins with an app's own row shadowing the inherited one (CoW). Every Rhai SDK call resolves its app from cx.app_id, never a script-passed arg, and a group script always runs under the inheriting app's cx.app_id (the cross-app isolation boundary).

Three-Service Architecture

The platform splits into three logical services, each backed by a *-core library crate so the same logic runs in single-process MVP mode and split-process cluster mode:

Service	Role	Library crate
Manager	Control plane: script CRUD, scheduling/cron, dashboard backend, config. Single-writer to Postgres.	manager-core
Orchestrator	Per-node ingress: receives HTTP (later SMTP, queue) events, resolves script, dispatches to local executor. Stateless.	orchestrator-core
Executor	Per-node compute: runs Rhai scripts in a sandboxed engine. Stateless.	executor-core

In MVP, all three run in one process (picloud binary). In cluster mode, each runs as its own binary on each node, with one manager total and one orchestrator + executor per node.

Key boundary: the orchestrator never imports executor-core directly — it depends on an ExecutorClient trait. The local impl calls executor-core in-process; the remote impl is an HTTP client. Same pattern keeps cluster mode a swap, not a rewrite.

Path Scheme

Versioned API surfaces live under /api/v{N}/.... See docs/versioning.md for the full scheme.

• /api/v1/admin/* — manager (control plane: script CRUD, routes CRUD + check + match, logs, config; apps CRUD once Phase 3b lands)
• /api/v1/execute/{id} — orchestrator (data plane: invoke a script by ID, always-available bypass)
• /admin/* — dashboard SPA (SvelteKit, paths.base = '/admin')
• /healthz — liveness (string "ok")
• /version — every compatibility-surface version + public_base_url (JSON)
• everything else — orchestrator's user-route matcher: user scripts bind to arbitrary paths via POST /api/v1/admin/scripts/{id}/routes; if no route matches, picloud returns 404 with a JSON error.

Reserved path prefixes (rejected at route creation): /api/, /admin/, /healthz, /version.

Caddy fronts everything. Same Caddyfile shape works for single-node and cluster — only upstream targets change.

Param syntax convention: route paths use :name (e.g., /users/:id); domains (once apps land) use {name} (e.g., {tenant}.example.com). These are deliberately distinct — never use : in a domain context or {} in a route-path context.

Two-phase dispatch (Phase 3b onward): the orchestrator first resolves Host → app (most-specific domain claim wins), then runs that app's route trie. The route matcher itself is unchanged and never sees other apps' routes.

Tech Stack

• Rust 1.92+ workspace, pinned via rust-toolchain.toml
• Axum for HTTP, Tokio async, sqlx for Postgres
• Rhai embedded scripting (in executor-core)
• PostgreSQL 15+ with pgcrypto. v1.1+ data-plane tables use JSONB for value columns (hstore was considered for KV and rejected — see blueprint §8.1).
• SvelteKit dashboard, static adapter, CodeMirror 6 for the script editor
• Caddy 2 reverse proxy (auto-HTTPS in prod)
• Docker Compose for dev and single-node prod

Common Commands

# Rust workspace
cargo check --workspace
cargo test --workspace
cargo fmt --all
cargo clippy --all-targets --all-features -- -D warnings

# Run the all-in-one binary (MVP entry point)
cargo run -p picloud

# Run a single test
cargo test -p executor-core sandbox::tests::respects_operation_budget

# Dashboard (from dashboard/)
npm run dev
npm run build
npm run check

# Full stack (once docker-compose.yml exists)
docker compose up
docker compose down -v   # reset Postgres data

Workspace Layout

crates/
  shared/                 # cross-cutting types (Script, IDs, error enum, db pool)
  executor-core/          # Rhai engine, sandbox, ctx, log, SDK
  orchestrator-core/      # event ingress + ExecutorClient trait + dispatch
  manager-core/           # control plane: repos, scheduler, config
  picloud/                # ★ MVP all-in-one binary
  picloud-manager/        # cluster mode binary (skeleton)
  picloud-orchestrator/   # cluster mode binary (skeleton)
  picloud-executor/       # cluster mode binary (skeleton)
dashboard/                # SvelteKit
caddy/                    # Caddyfile, Caddyfile.prod
docker/                   # Dockerfiles
docs/
  git-workflow.md         # trunk-based workflow
  architecture.md         # (TBD)

Working Rules

• Honor the three-service boundary. Don't reach across *-core crates for behavior. If orchestrator-core needs to invoke logic from manager-core, define a trait in shared and inject the impl — keep implementations decoupled. Transport DTOs are not behavior: types like ExecRequest / ExecResponse / ExecError represent values produced or consumed across the wire, and depending on the originating crate's type definitions is fine. The bright line is "don't call across crates," not "don't import types." When in doubt: if the imported item is a struct/enum/type alias with no methods (or only data-shape methods), it's a DTO and crossing is fine; if it's a trait, function, or service, define the abstraction in shared and inject.
• executor-core has no Postgres dependency. Data-plane services (kv, docs, users — v1.1+) come in via injected ServiceProvider traits.
• Database writes only from manager-core. orchestrator-core reads scripts (cached); executor-core doesn't touch the DB.
• Stateful SDK services use the handle pattern + SdkCallCx. Collection-scoped surfaces look like kv::collection("x").get(k), not kv::get("x", k). Every service trait method takes &SdkCallCx and MUST derive app_id from cx.app_id — never trust a script-passed app_id. That is the cross-app isolation boundary. See docs/sdk-shape.md.
• MVP builds only the picloud all-in-one binary. The three split binaries exist as skeletons so the crate boundaries stay honest; flesh them out only when cluster mode is being implemented.
• Trunk-based dev. See docs/git-workflow.md. No long-lived branches. Feature flags for incomplete work.

Runtime configuration

Environment variables consumed by the picloud binary:

Variable	Default	Purpose
PICLOUD_BIND	0.0.0.0:8080	HTTP listen address. Port 8080 is owned by another process on this host — override locally.
PICLOUD_MAX_CONCURRENT_EXECUTIONS	32	Global concurrency cap on data-plane script executions. Overflow returns HTTP 503 with Retry-After: 1 immediately (no queue).
PICLOUD_MAX_EMISSIONS_PER_EXECUTION	1000	Per-execution fan-out ceiling: max DURABLE emissions (invoke_async + pubsub::publish_durable + queue::enqueue, incl. shared variants) one execution may make before the SDK call errors. Bounds a one-request outbox-amplification DoS. Re-entrancy aware — a synchronous invoke()/interceptor chain shares one budget; a dispatched handler starts fresh. trigger_depth bounds chain DEPTH, this bounds fan-out WIDTH.
DATABASE_URL	—	Required. Postgres connection string.
PICLOUD_SECRET_KEY	—	Master encryption key (base64). Required at startup unless dev mode is acknowledged (below).
PICLOUD_DEV_MODE	false	true enables local-dev conveniences. Without PICLOUD_SECRET_KEY it ALSO requires the acknowledgement var below — PICLOUD_DEV_MODE=true alone aborts at startup. Also: when no SMTP relay is configured, email::send switches from disabled (NotConfigured) to an in-memory dev sink — sends succeed and the last 100 messages are readable at GET /api/v1/admin/dev/emails (instance Owner/Admin only; route exists only in this mode). Never in production.
PICLOUD_DEV_INSECURE_KEY	—	Set to the literal i-understand-this-is-insecure to let dev mode boot without PICLOUD_SECRET_KEY, using a deterministic, world-known dev master key. Never set in production — it would encrypt everything with a public value.
PICLOUD_DB_MAX_CONNECTIONS	32	Postgres pool size. Matched to PICLOUD_MAX_CONCURRENT_EXECUTIONS so the data plane can't starve background workers.
PICLOUD_SESSION_TTL_HOURS	24	Sliding-window admin session lifetime (idle timeout).
PICLOUD_SESSION_ABSOLUTE_TTL_HOURS	720 (30 days)	Absolute hard cap on an admin session's lifetime (audit 2026-07-11 C1). The sliding touch bump is clamped at login_time + this, so even a continuously-used or stolen-but-warm admin token self-expires. Mirrors the data-plane app-user session cap.
PICLOUD_SMTP_BIND	— (off)	A3 native SMTP ingress. When set (e.g. 0.0.0.0:2525), a receive-only SMTP listener binds alongside the HTTP server: it resolves each RCPT TO mailbox to the app-owned email trigger that claims it (email_trigger_details.inbound_address, unique) and writes an Email outbox row the dispatcher fires — same tail as the HMAC webhook. No AUTH/STARTTLS at the listener (TLS terminates upstream — Caddy/relay); the recipient address + per-app isolation are the boundary. DATA is size-capped. Unset = the webhook path only. Distinct from the outbound relay PICLOUD_SMTP_* vars (SmtpConfig).
PICLOUD_INTERCEPTOR_TIMEOUT_MS	5000 (5s)	§9.4 M5 default per-interceptor wall-clock timeout when a [[interceptors]] marker sets no timeout_ms. Bounds ONE hook run so a runaway guard (loop {}) is denied (fail closed) rather than hanging the guarded write. The effective deadline is always tightened to at most the caller's remaining budget — a hook can never EXTEND its caller's deadline.
PICLOUD_OUTBOX_CLAIM_TIMEOUT_SEC	600 (10 min)	How long a dispatcher may hold an OUTBOX claim before the reclaim task returns the row to the queue. Without this a crash or restart mid-dispatch stranded every in-flight row permanently — the dispatcher claims a row and only clears the claim on success (delete) or failure (reschedule), and claim_due selects claimed_at IS NULL, so a process that died in between left rows nothing would ever pick up again. Every other claim-based store (queue, group queue, workflow steps) already had a reclaimer; the outbox was the gap, and since it is the universal trigger path, the loss covered kv/docs/files/cron/pubsub/email/invoke_async/dead-letter alike. The default is deliberately generous: a script may run 300s, so a claim older than twice that is abandoned rather than slow. A reclaim does not bump attempt_count (the handler never ran — same reasoning as the transient queue release). Runs on the existing PICLOUD_QUEUE_RECLAIM_INTERVAL_MS ticker.
PICLOUD_REALTIME_BROADCAST_CAPACITY	64	Per-channel SSE broadcast buffer depth (a slow consumer sees oldest events dropped).
PICLOUD_REALTIME_MAX_CHANNELS	100000	Max live SSE channels per map (per-app and per-group). A subscribe that would open a NEW channel past this is refused with 503 (audit 2026-07-11 B6), so a client naming unbounded distinct topics can't grow the broadcaster maps to OOM.
PICLOUD_SANDBOX_MAX_*	conservative defaults	Per-knob admin ceilings on Rhai sandbox overrides. See manager-core::sandbox::SandboxCeiling.
PICLOUD_FILES_ROOT	./data	Filesystem root for files::* blob storage (v1.1.5). Bytes live at <root>/files/<app_id>/<collection>/<id[0:2]>/<id>; metadata in Postgres.
PICLOUD_FILES_MAX_FILE_SIZE_BYTES	104857600 (100 MB)	Per-file hard size cap for files::* (v1.1.5). Per-app quotas deferred to v1.2.
PICLOUD_KV_MAX_VALUE_BYTES	262144 (256 KB)	Per-key JSON-encoded value cap for kv::set. Rejects oversized payloads before authz so anonymous public scripts can't DoS Postgres JSONB columns.
PICLOUD_DOCS_MAX_VALUE_BYTES	262144 (256 KB)	Per-document JSON-encoded data cap for docs::create/update.
PICLOUD_PUBSUB_MAX_MESSAGE_BYTES	262144 (256 KB)	Per-message JSON-encoded payload cap for pubsub::publish_durable. Prevents one publish from amplifying into N outbox rows × M MB.
PICLOUD_QUEUE_MAX_PAYLOAD_BYTES	262144 (256 KB)	Per-message JSON-encoded payload cap for queue::enqueue.
PICLOUD_APP_KV_MAX_ROWS	100000	Per-app ceiling on total KV keys. Checked only when a write ADDS a key (kv::set of a new key, or a set_if insert) — an update is net-zero rows and pays nothing. Deliberately not advisory-locked: kv::set is the hottest write path, and serializing an app's data plane (or scanning SUM(...) on every set) is a far worse trade than the ~32-row overshoot (bounded by PICLOUD_MAX_CONCURRENT_EXECUTIONS) that the lock would prevent. These are anti-DoS rails, not billing. Together with PICLOUD_KV_MAX_VALUE_BYTES this also bounds stored bytes (rows x value cap), so there is no separate per-app KV byte ceiling.
PICLOUD_APP_DOCS_MAX_ROWS	100000	Per-app ceiling on total docs (docs::create). Same rationale as KV — create-only, unlocked, and bounded in bytes by PICLOUD_DOCS_MAX_VALUE_BYTES.
PICLOUD_APP_FILES_MAX_TOTAL_BYTES	10737418240 (10 GiB)	Per-app ceiling on total stored blob bytes. This one IS advisory-locked and does sum bytes, unlike KV/docs: one blob may be 100 MB, so racing uploads could overshoot by gigabytes of real disk, and a file write is heavy enough that the lock + SUM are lost in the noise. Checked on the projected total (the replaced blob subtracted) on create and update — an update that skipped it would be a free bypass. Closes the anonymous disk-exhaustion path: script_gate passes for an unauthenticated principal, so a public route could previously write blobs unbounded.
PICLOUD_GROUP_KV_MAX_ROWS	100000	§11.6 M3 per-group quota: max total keys across a group's shared-KV collections. Checked only on a NEW key (kv::shared_collection().set); an update is exempt.
PICLOUD_GROUP_DOCS_MAX_ROWS	100000	§11.6 M3 per-group quota: max total docs across a group's shared-docs collections (docs::shared_collection().create).
PICLOUD_GROUP_KV_MAX_TOTAL_BYTES	268435456 (256 MiB)	§11.6 M4 per-group quota: max total stored JSON bytes across a group's shared-KV collections. Checked on the projected total (old value subtracted, new added), so a same/smaller update near the cap is still allowed.
PICLOUD_GROUP_DOCS_MAX_TOTAL_BYTES	268435456 (256 MiB)	§11.6 M4 per-group quota: max total stored JSON bytes across a group's shared-docs collections (projected-total check, as KV).
PICLOUD_GROUP_FILES_MAX_TOTAL_BYTES	10737418240 (10 GiB)	§11.6 M3 per-group quota: max total stored bytes across a group's shared-files collections (files::shared_collection().create).

Out of MVP

This section captured the original MVP cut. Most of it has since shipped in v1.1.x: queue triggers, cron triggers, inbound email (email:receive, HMAC-webhook model), KV / docs / email / users / HTTP SDKs, function-to-function invoke(), and secrets are all live (blueprint §12 Phase 4 table). The Workflows track (DAG + nested workflows) has since shipped (migrations 0071/0072). The §9.4 service-interceptor track has since shipped in full (migrations 0073–0075; before/after phases, data-transform, all six services, per-interceptor timeout, resolve cache, pic interceptors ls). A read-only metrics/observability dashboard has since shipped (A2): GET /api/v1/admin/apps/{id}/metrics?window= aggregates the existing execution_logs table (counts, error rate, latency avg/p50/p95, an hourly series) via ExecutionLogRepository::summarize_for_app + metrics_api, surfaced as the dashboard's per-app Metrics tab — no hot-path instrumentation. Still deferred: multi-node cluster mode. Don't pre-build for them — but don't make decisions that close the door on them either.

Pulled forward to Phase 3 (pre-v1.1): admin auth, multi-app scoping. The general cross-app export/import sharing model stays at v1.3+; note that v1.2 §11.6 shipped a narrower form — group-owned shared collections (KV/docs/files/topics/queues) let apps in one subtree share data through the owning group, with the ancestor-chain walk as the isolation boundary. See blueprint §11.5 + design-doc §11.6.