diff --git a/CHANGELOG.md b/CHANGELOG.md index ead643f..77cfd02 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,108 @@ # PiCloud Changelog +## v1.1.3 — Modules (unreleased) + +Real per-app Rhai module system. Scripts can `import "" as +;` other scripts in the same app as reusable libraries. The +v1.0 placeholder `DummyModuleResolver` is replaced by a per-call +`PicloudModuleResolver` that loads `kind = 'module'` scripts via a +new `ModuleSource` trait, compiles them into Rhai modules, caches +the compiled output, and enforces cross-app isolation, circular- +import detection, and an import-depth limit. Two LRU AST caches +(top-level script + per-module compiled module) eliminate the +per-invocation compile cost; both invalidate on `updated_at` change. + +### Added + +- **`scripts.kind` column** — `'endpoint' | 'module'`, default + `'endpoint'`. Endpoints handle HTTP routes / trigger events; + modules are libraries imported by other scripts. The dashboard + scripts list + script detail page surface the distinction as a + colored badge. +- **`script_imports` dep-graph table** — populated at script save- + time from the literal-path `import ""` declarations in the + source. FK-CASCADE on both columns. No admin surface in v1.1.3 + (drives a v1.2+ "Used by" dashboard panel and v1.3+ cluster-mode + eager invalidation). +- **`ModuleSource` trait** — `lookup(&SdkCallCx, name)`. Postgres + impl `PostgresModuleSource` in manager-core. `app_id` derived from + `cx.app_id` (cross-app isolation boundary, mirrors KV / docs). +- **`PicloudModuleResolver`** — implements `rhai::ModuleResolver`. + Per-call instance owns `Arc`, the in-progress imports + stack, the depth counter. Bridges sync `resolve()` to async + `lookup()` via `Handle::block_on` (safe under the executor's + `spawn_blocking` wrap). Replaces `DummyModuleResolver` at line 139 + of `executor-core::engine::build_engine`. +- **Module-shape validation** — `kind = 'module'` source must contain + only `fn` declarations, `const` declarations, and `import` + statements at top level (no executable expressions). Walks + `ast.statements()` via `rhai/internals`. Admin endpoint is the + primary gate; the resolver re-runs the check at load time for + defense in depth against DB-direct inserts. +- **Per-module compiled-Module cache** — `LruCache<(AppId, name), + (updated_at, Arc)>` owned by `Engine`. Invalidated + lazily on `updated_at` mismatch. Size via + `PICLOUD_MODULE_CACHE_SIZE` (default 512). +- **Top-level script AST cache** — `LruCache)>` owned by `LocalExecutorClient`. Same staleness + semantics. Size via `PICLOUD_SCRIPT_CACHE_SIZE` (default 256). +- **`ScriptIdentity` + `ExecutorClient::execute_with_identity`** — + new method on the trait; default impl forwards to `execute` so + `RemoteExecutorClient` (and future transports) keep working. + `LocalExecutorClient` overrides it to consult the script cache and + pass the resulting `Arc` to `Engine::execute_ast`. +- **`Engine::execute_ast`** — companion to `execute` that takes a + pre-compiled AST so callers (the orchestrator) can reuse one + compile across many invocations. +- **Import depth limit** — `Limits::module_import_depth_max` + (default 8). Not script-overridable. +- **Reserved module names** — module-kind scripts cannot be named + `log`, `regex`, `random`, `time`, `json`, `base64`, `hex`, `url`, + `kv`, `docs`, `dead_letters`, `http`, `files`, `pubsub`, `secrets`, + `email`, `users`, `queue`. Defense against author confusion with + stdlib namespaces. + +### Changed + +- **Workspace version**: `1.1.2` → `1.1.3`. +- **Rhai SDK version**: `1.3` → `1.4` (additive — every v1.3 script + still runs unchanged; new surface: `import "" as ;` + for endpoint scripts that consume modules in the same app). +- **Dashboard version**: `0.8.0` → `0.9.0`. Adds kind dropdown on + script create + kind badges on the scripts list and detail page. +- **`Services` bundle** — grows a `modules: Arc` + field. Constructor signature becomes + `Services::new(kv, docs, dead_letters, events, modules)`. +- **`ScriptValidator` trait** — `validate` now returns + `ValidatedScript { imports: Vec }` so the repo can write + dep-graph edges in the same transaction as the script row. New + `validate_module` method enforces module-shape rules. +- **Trigger creation tightening** — `POST /api/v1/admin/apps/{id}/triggers/{kv,docs,dead_letter}` + now load the target script and reject when (1) it doesn't exist, + (2) it belongs to a different app (latent v1.1.1/v1.1.2 gap — + closed in v1.1.3), or (3) it is `kind = 'module'`. +- **Route creation** — `POST /api/v1/admin/scripts/{id}/routes` + returns 400 when the target script is `kind = 'module'`. + +### Migrations + +- `0015_scripts_kind.sql` — adds `scripts.kind` with CHECK + `IN ('endpoint','module')`, composite index `(app_id, kind)`, and + a module-name shape CHECK (`^[a-zA-Z_][a-zA-Z0-9_]{0,63}$`). +- `0016_script_imports.sql` — adds the dep-graph table with FK + CASCADE on both columns, PK `(importer, imported)`, and a + reverse-edge index on `imported_script_id`. + +### Downgrade caveats + +Rolling back v1.1.3 → v1.1.2 with module-kind scripts present +strands them (no `kind` column means everything looks like an +endpoint; modules will then succeed as route targets and immediately +fail to execute meaningfully). Migration `0016_script_imports.sql` +is safe to drop (the table is auxiliary). `0015_scripts_kind.sql` +must be reversed by `DROP COLUMN kind` only after manually re-homing +or deleting module-kind rows. + ## v1.1.2 — Documents (unreleased) `docs::*` SDK — schemaless JSONB document storage with a first-cut diff --git a/Cargo.lock b/Cargo.lock index 7bc6e3c..884bb1a 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1514,7 +1514,7 @@ checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220" [[package]] name = "picloud" -version = "1.1.2" +version = "1.1.3" dependencies = [ "anyhow", "async-trait", @@ -1540,7 +1540,7 @@ dependencies = [ [[package]] name = "picloud-cli" -version = "1.1.2" +version = "1.1.3" dependencies = [ "anyhow", "assert_cmd", @@ -1561,7 +1561,7 @@ dependencies = [ [[package]] name = "picloud-executor" -version = "1.1.2" +version = "1.1.3" dependencies = [ "anyhow", "picloud-executor-core", @@ -1573,7 +1573,7 @@ dependencies = [ [[package]] name = "picloud-executor-core" -version = "1.1.2" +version = "1.1.3" dependencies = [ "async-trait", "base64", @@ -1595,7 +1595,7 @@ dependencies = [ [[package]] name = "picloud-manager" -version = "1.1.2" +version = "1.1.3" dependencies = [ "anyhow", "picloud-manager-core", @@ -1607,7 +1607,7 @@ dependencies = [ [[package]] name = "picloud-manager-core" -version = "1.1.2" +version = "1.1.3" dependencies = [ "argon2", "async-trait", @@ -1632,7 +1632,7 @@ dependencies = [ [[package]] name = "picloud-orchestrator" -version = "1.1.2" +version = "1.1.3" dependencies = [ "anyhow", "picloud-orchestrator-core", @@ -1644,7 +1644,7 @@ dependencies = [ [[package]] name = "picloud-orchestrator-core" -version = "1.1.2" +version = "1.1.3" dependencies = [ "async-trait", "axum", @@ -1665,7 +1665,7 @@ dependencies = [ [[package]] name = "picloud-shared" -version = "1.1.2" +version = "1.1.3" dependencies = [ "async-trait", "chrono", diff --git a/Cargo.toml b/Cargo.toml index 9d8913c..b0c8c19 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -13,7 +13,7 @@ members = [ ] [workspace.package] -version = "1.1.2" +version = "1.1.3" edition = "2021" rust-version = "1.92" license = "MIT OR Apache-2.0" diff --git a/dashboard/package.json b/dashboard/package.json index 45d4ea2..c9e2142 100644 --- a/dashboard/package.json +++ b/dashboard/package.json @@ -1,6 +1,6 @@ { "name": "picloud-dashboard", - "version": "0.8.0", + "version": "0.9.0", "private": true, "type": "module", "scripts": { diff --git a/serverless_cloud_blueprint.md b/serverless_cloud_blueprint.md index 75877c0..6350070 100644 --- a/serverless_cloud_blueprint.md +++ b/serverless_cloud_blueprint.md @@ -1772,7 +1772,7 @@ if allowed { - [ ] **Debugging**: How to trace interceptor execution in logs/dashboard? ### Rhai & SDK -- [ ] **Module loading**: Can scripts `import` external Rhai modules? (probably no for MVP) +- [x] **Module loading**: Scripts can `import "" as ;` other scripts in the same app (v1.1.3 — `scripts.kind = 'module'`). Per-app, cross-app isolated, cache-invalidated on `updated_at` change. External (off-platform) modules remain out of scope. - [ ] **File system access**: Can scripts read/write to local filesystem? (no for MVP) - [ ] **Request/response sizes**: Max payload size? (set sensible default, e.g., 10MB)