PiCloud/CHANGELOG.md

PiCloud Changelog

v1.1.3 — Modules (unreleased)

Real per-app Rhai module system. Scripts can import "<name>" as <alias>; other scripts in the same app as reusable libraries. The v1.0 placeholder DummyModuleResolver is replaced by a per-call PicloudModuleResolver that loads kind = 'module' scripts via a new ModuleSource trait, compiles them into Rhai modules, caches the compiled output, and enforces cross-app isolation, circular- import detection, and an import-depth limit. Two LRU AST caches (top-level script + per-module compiled module) eliminate the per-invocation compile cost; both invalidate on updated_at change.

Added

• scripts.kind column — 'endpoint' | 'module', default 'endpoint'. Endpoints handle HTTP routes / trigger events; modules are libraries imported by other scripts. The dashboard scripts list + script detail page surface the distinction as a colored badge.
• script_imports dep-graph table — populated at script save- time from the literal-path import "<name>" declarations in the source. FK-CASCADE on both columns. No admin surface in v1.1.3 (drives a v1.2+ "Used by" dashboard panel and v1.3+ cluster-mode eager invalidation).
• ModuleSource trait — lookup(&SdkCallCx, name). Postgres impl PostgresModuleSource in manager-core. app_id derived from cx.app_id (cross-app isolation boundary, mirrors KV / docs).
• PicloudModuleResolver — implements rhai::ModuleResolver. Per-call instance owns Arc<SdkCallCx>, the in-progress imports stack, the depth counter. Bridges sync resolve() to async lookup() via Handle::block_on (safe under the executor's spawn_blocking wrap). Replaces DummyModuleResolver at line 139 of executor-core::engine::build_engine.
• Module-shape validation — kind = 'module' source must contain only fn declarations, const declarations, and import statements at top level (no executable expressions). Walks ast.statements() via rhai/internals. Admin endpoint is the primary gate; the resolver re-runs the check at load time for defense in depth against DB-direct inserts.
• Per-module compiled-Module cache — LruCache<(AppId, name), (updated_at, Arc<rhai::Module>)> owned by Engine. Invalidated lazily on updated_at mismatch. Size via PICLOUD_MODULE_CACHE_SIZE (default 512).
• Top-level script AST cache — LruCache<ScriptId, (updated_at, Arc<rhai::AST>)> owned by LocalExecutorClient. Same staleness semantics. Size via PICLOUD_SCRIPT_CACHE_SIZE (default 256).
• ScriptIdentity + ExecutorClient::execute_with_identity — new method on the trait; default impl forwards to execute so RemoteExecutorClient (and future transports) keep working. LocalExecutorClient overrides it to consult the script cache and pass the resulting Arc<rhai::AST> to Engine::execute_ast.
• Engine::execute_ast — companion to execute that takes a pre-compiled AST so callers (the orchestrator) can reuse one compile across many invocations.
• Import depth limit — Limits::module_import_depth_max (default 8). Not script-overridable.
• Reserved module names — module-kind scripts cannot be named log, regex, random, time, json, base64, hex, url, kv, docs, dead_letters, http, files, pubsub, secrets, email, users, queue. Defense against author confusion with stdlib namespaces.

Changed

• Workspace version: 1.1.2 → 1.1.3.
• Rhai SDK version: 1.3 → 1.4 (additive — every v1.3 script still runs unchanged; new surface: import "<name>" as <alias>; for endpoint scripts that consume modules in the same app).
• Dashboard version: 0.8.0 → 0.9.0. Adds kind dropdown on script create + kind badges on the scripts list and detail page.
• Services bundle — grows a modules: Arc<dyn ModuleSource> field. Constructor signature becomes Services::new(kv, docs, dead_letters, events, modules).
• ScriptValidator trait — validate now returns ValidatedScript { imports: Vec<String> } so the repo can write dep-graph edges in the same transaction as the script row. New validate_module method enforces module-shape rules.
• Trigger creation tightening — POST /api/v1/admin/apps/{id}/triggers/{kv,docs,dead_letter} now load the target script and reject when (1) it doesn't exist, (2) it belongs to a different app (latent v1.1.1/v1.1.2 gap — closed in v1.1.3), or (3) it is kind = 'module'.
• Route creation — POST /api/v1/admin/scripts/{id}/routes returns 400 when the target script is kind = 'module'.

Migrations

• 0015_scripts_kind.sql — adds scripts.kind with CHECK IN ('endpoint','module'), composite index (app_id, kind), and a module-name shape CHECK (^[a-zA-Z_][a-zA-Z0-9_]{0,63}$).
• 0016_script_imports.sql — adds the dep-graph table with FK CASCADE on both columns, PK (importer, imported), and a reverse-edge index on imported_script_id.

Downgrade caveats

Rolling back v1.1.3 → v1.1.2 with module-kind scripts present strands them (no kind column means everything looks like an endpoint; modules will then succeed as route targets and immediately fail to execute meaningfully). Migration 0016_script_imports.sql is safe to drop (the table is auxiliary). 0015_scripts_kind.sql must be reversed by DROP COLUMN kind only after manually re-homing or deleting module-kind rows.

v1.1.2 — Documents (unreleased)

docs::* SDK — schemaless JSONB document storage with a first-cut query DSL — plus docs:* triggers as the second concrete kind on the v1.1.1 triggers framework. Sets the precedent for the v1.2 query DSL expansion and dead_letters::list.

Added

• Docs store — docs table keyed (app_id, collection, id) with JSONB values and a GIN-on-jsonb_path_ops index. Rhai SDK exposes the handle pattern: docs::collection(name).{create,get,find,find_one,update,delete,list}. Cursor-style pagination on list. Cross-app isolation enforced via cx.app_id (never script-passed). Document envelope shape returned by reads: #{ id, data: #{...}, created_at, updated_at } — explicit metadata + user-data separation (sets precedent for v1.2 dead_letters::list).
• Query DSL (v1.1.2 subset) — implicit equality at top level (#{ tier: "gold" }), operator-object form (#{ created_at: #{ "$gt": "..." } }), dotted field paths up to 5 levels ("user.email"), and operators $eq/$ne/$gt/$gte/ $lt/$lte/$in. Filter modifiers $sort (single field) and $limit. Unsupported operators ($or, $regex, etc.) reject with a clear v1.2-pointer error.
• Docs triggers (docs:*) — docs_trigger_details table mirrors kv_trigger_details. Admin endpoint POST /api/v1/admin/apps/{id}/triggers/docs accepts the same DTO shape as the KV endpoint with ops of DocsEventOp (create / update / delete). Dispatcher routes OutboxSourceKind::Docs through the same generic path as KV + dead-letter.
• ctx.event.docs.prev_data — change-data-capture surface for docs trigger handlers. prev_data carries the document state prior to the mutation (None for create), letting handlers see what changed. The repo reads the old row in the same SQL statement as the write so the trigger event has the prior value.
• Capability::AppDocsRead(AppId) + AppDocsWrite(AppId) — granted to Viewer / Editor respectively in the per-app role table. Same trust shape as KV's AppKvRead / AppKvWrite.

Changed

• Workspace version: 1.1.1 → 1.1.2.
• Rhai SDK version: 1.2 → 1.3 (additive — every v1.2 script still runs unchanged; new surfaces: docs::collection(name).{...}, ctx.event.docs for triggered handlers).
• Dashboard version: 0.7.0 → 0.8.0. Workspace alignment; no docs-specific UI in v1.1.2 (the dashboard's Rhai-mode hints don't list KV completions either — focused UX pass is a separate task).
• Services bundle — grows a docs: Arc<dyn DocsService> field. Constructor signature becomes Services::new(kv, docs, dead_letters, events).
• Scope mapping: API keys with script:read scope can call docs::find / get / list; script:write can call docs::create / update / delete. Same trust shape as KV — honors the seven-scope commitment from v1.1.0.

Migrations

• 0013_docs.sql — docs table + per-(app_id, collection) index + GIN-on-jsonb_path_ops index.
• 0014_docs_triggers.sql — extends triggers.kind and outbox.source_kind CHECK constraints to include 'docs'; adds docs_trigger_details table.

Downgrade caveats

Rolling a deployment back from v1.1.2 → v1.1.1 with docs-source outbox rows still queued will cause the v1.1.1 dispatcher to fail deserialising TriggerEvent::Docs (#[serde(tag = "source")] rejects unknown variants). Drain or delete outbox WHERE source_kind = 'docs' before downgrading. Trunk-only deployments don't hit this.

Known limitations

• Text-lex comparison for $gt / $gte / $lt / $lte is incorrect for unpadded numbers crossing digit-count boundaries ('10' < '9' is TRUE under any text collation). Workaround: zero-pad numeric strings. v1.2's advanced query expansion adds numeric-aware operators.
• Concurrent update()s on the same doc may both emit the pre-update prev_data (last-writer-wins). Inherited from KV's set pattern; documented for forensic-trace use cases.
• v1.1.2 has no partial-update DSL — scripts that want partial update do get + modify + update. Planned for v1.2.

v1.1.1 — Storage & Events (unreleased)

The triggers framework — KV store + universal outbox + dispatcher + NATS-style sync HTTP + per-route async dispatch + dead-letter handling + dashboard surface. Every subsequent v1.1.x service module (docs, files, pubsub, …) hangs off the dispatcher built here.

Added

• KV store — kv_entries table keyed (app_id, collection, key) with JSONB values. Rhai SDK exposes the handle pattern: kv::collection(name).{get,set,has,delete,list}. Cursor-style pagination with opaque base64 cursors. Cross-app isolation enforced via cx.app_id (never script-passed).
• Triggers framework (Layout E) — parent triggers table + per-kind detail tables (kv_trigger_details, dead_letter_trigger_details). Trigger CRUD admin endpoints (/api/v1/admin/apps/{id}/triggers/{kv,dead_letter}) + Capability::AppManageTriggers(AppId).
• Universal outbox + dispatcher — single tokio task that polls the outbox via FOR UPDATE SKIP LOCKED, routes due rows to the executor through the shared ExecutionGate. Retry with exponential backoff + ±jitter; on exhaustion, dead-letter.
• NATS-style sync HTTP via outbox — InboxRegistry (in-process oneshot map) lets the orchestrator await dispatcher delivery on every sync HTTP request. Cluster mode (v1.3+) swaps this for LISTEN/NOTIFY behind the same InboxResolver trait.
• dispatch_mode: async on routes — POST to a route with dispatch_mode = 'async' returns 202 Accepted immediately; the script runs via the dispatcher (with retries / dead-letter).
• Dead-letter handling — separate dead_letters table per design notes §4. dead_letters::{replay,resolve} Rhai SDK + admin endpoints + Capability::AppDeadLetterManage(AppId). Recursion-stop rule: dead-letter handler failures annotate the original row as resolution = 'handler_failed' and never produce a new dead-letter or retry.
• Dashboard surface for dead letters — unresolved-count red badge on the apps list + per-app page; per-app dead-letters list view at /admin/apps/{slug}/dead-letters with Replay + Mark resolved per-row actions and expandable payload detail.
• abandoned_executions table — forensic row written by the dispatcher when it tries to resolve an inbox the orchestrator already abandoned (timed out). Counter metric path reserved.
• Trigger-depth limit — cx.trigger_depth > max_trigger_depth (default 8) skips execution + logs; does NOT dead-letter (depth-exceeded means "you built a loop").
• GC sweepers — weekly retention sweeps for dead_letters (30 days) and abandoned_executions (7 days), both with FOR UPDATE SKIP LOCKED for cluster-mode safety.
• Env-overridable trigger config — TriggerConfig::from_env reads PICLOUD_MAX_TRIGGER_DEPTH, PICLOUD_TRIGGER_RETRY_*, PICLOUD_DEAD_LETTER_RETENTION_DAYS, PICLOUD_ABANDONED_EXECUTIONS_RETENTION_DAYS.

Changed

• Workspace version: 1.1.0 → 1.1.1.
• Rhai SDK version: 1.1 → 1.2 (additive — every v1.1 script still runs unchanged; new surfaces: kv::*, dead_letters::*, ctx.event for triggered handlers).
• Dashboard version: 0.6.0 → 0.7.0 for the dead-letters UI.
• Services bundle — replaces v1.1.0's no-arg Services::new() with explicit Services::new(kv, dead_letters, events). Tests use Services::default() for an all-noop bundle.
• SdkCallCx grows is_dead_letter_handler: bool and event: Option<TriggerEvent> fields.
• ExecRequest mirrors the new SdkCallCx fields and grows event for serializable trigger payload transport.
• Routes table grows dispatch_mode TEXT NOT NULL DEFAULT 'sync' (CHECK in {sync, async}).
• Schema version: 6 → 12 (migrations 0007 through 0012).

Migrations

• 0007_kv.sql — kv_entries table + index
• 0008_triggers.sql — triggers + kv_trigger_details + dead_letter_trigger_details
• 0009_outbox.sql — universal outbox table + due-row partial index
• 0010_dead_letters.sql — dead_letters table + unresolved partial index + GC index
• 0011_abandoned_executions.sql — forensic table + GC index
• 0012_routes_dispatch_mode.sql — routes.dispatch_mode column

v1.1.0 — Foundation & Standard Library

See docs/v1.1.x-design-notes.md §7 for the full v1.1.x roadmap.