feat(v1.1.3-modules): shared types, migrations, engine + resolver scaffold
Lays down the v1.1.3 plumbing:
- `ScriptKind` enum in `picloud-shared` ('endpoint' | 'module').
- `ModuleSource` trait + `ModuleScript` DTO + `NoopModuleSource` in
`picloud-shared`. Resolver lives in `executor-core`; Postgres impl
in `manager-core` (`PostgresModuleSource`).
- `Services::new` grows a fifth `modules: Arc<dyn ModuleSource>` arg.
- `ScriptValidator` returns `ValidatedScript { imports }` so the
manager can populate the dep-graph table on save. New
`validate_module` method on the trait gates module-shape rules.
- `Engine::execute_ast(&Arc<rhai::AST>, req)` lets the orchestrator's
script cache reuse compiled ASTs. `Engine::execute(&str, req)` is
preserved as a convenience that compiles inline. `Engine::compile`
exposes the AST for callers that want to cache.
- `PicloudModuleResolver` replaces `DummyModuleResolver` per-call.
Bridges Rhai's sync `ModuleResolver::resolve` to async
`ModuleSource::lookup` via `Handle::block_on`. Enforces:
- cross-app isolation (resolver captures `Arc<SdkCallCx>`),
- circular import detection (in-progress stack on the resolver),
- import depth limit (default 8 via
`Limits::module_import_depth_max`).
- Module-shape validation walks `ast.statements()` via `rhai/internals`
and accepts only `Var { CONSTANT }`, `Import`, and `Noop`. The
manager admin endpoint runs `validate_module` at save (primary
gate); resolver re-runs it at load (defense in depth).
- LRU cache `(AppId, name) -> (updated_at, Arc<Module>)` owned by
`Engine`. Size from `PICLOUD_MODULE_CACHE_SIZE` (default 512).
- Migration `0015_scripts_kind.sql` adds `scripts.kind` + composite
index + module-name shape CHECK.
- Migration `0016_script_imports.sql` adds the dep-graph table with
FK CASCADE on both columns.
- Repo: `kind` threaded through SELECT/INSERT/UPDATE. New
`count_routes_for_script` / `count_triggers_for_script` /
`list_imports` methods. `create`/`update` open a transaction and
call `replace_imports_tx` to populate the dep-graph.
- Admin endpoint: accepts `kind`; rejects reserved module names;
rejects `endpoint → module` transitions when routes / triggers
exist.
- SDK_VERSION 1.3 → 1.4.
Workspace builds; full test suite (~440 tests) green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -16,6 +16,7 @@ pub mod ids;
|
||||
pub mod inbox;
|
||||
pub mod kv;
|
||||
pub mod log_sink;
|
||||
pub mod modules;
|
||||
pub mod outbox_writer;
|
||||
pub mod route;
|
||||
pub mod sandbox;
|
||||
@@ -40,12 +41,13 @@ pub use inbox::{
|
||||
};
|
||||
pub use kv::{KvError, KvListPage, KvService, NoopKvService};
|
||||
pub use log_sink::{ExecutionLogSink, LogSinkError};
|
||||
pub use modules::{ModuleScript, ModuleSource, ModuleSourceError, NoopModuleSource};
|
||||
pub use outbox_writer::{HttpDispatchPayload, NewHttpOutbox, OutboxWriter, OutboxWriterError};
|
||||
pub use route::{DispatchMode, HostKind, PathKind, Route};
|
||||
pub use sandbox::ScriptSandbox;
|
||||
pub use script::Script;
|
||||
pub use script::{Script, ScriptKind};
|
||||
pub use sdk_cx::SdkCallCx;
|
||||
pub use services::Services;
|
||||
pub use trigger_event::{DeadLetterEventDetail, DocsEventOp, KvEventOp, TriggerEvent};
|
||||
pub use validator::{ScriptValidator, ValidationError};
|
||||
pub use validator::{ScriptValidator, ValidatedScript, ValidationError};
|
||||
pub use version::{API_VERSION, PRODUCT_VERSION, SDK_VERSION, WIRE_VERSION};
|
||||
|
||||
75
crates/shared/src/modules.rs
Normal file
75
crates/shared/src/modules.rs
Normal file
@@ -0,0 +1,75 @@
|
||||
//! `ModuleSource` — the v1.1.3 Rhai module-loading contract.
|
||||
//!
|
||||
//! The executor-core `PicloudModuleResolver` calls into this trait to
|
||||
//! load `kind = 'module'` scripts referenced by `import "<name>" as <alias>;`
|
||||
//! statements. The Postgres impl in `manager-core` reads from the
|
||||
//! `scripts` table; tests pin in-memory fakes.
|
||||
//!
|
||||
//! Implementations MUST derive `app_id` from `cx.app_id` and pass it
|
||||
//! to every backend query. The `name` argument carries only the
|
||||
//! script's name (the literal between the import quotes); the trait
|
||||
//! has no way to express a cross-app lookup. That asymmetry is the
|
||||
//! load-bearing cross-app isolation boundary — see `docs/sdk-shape.md`.
|
||||
|
||||
use async_trait::async_trait;
|
||||
use chrono::{DateTime, Utc};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use thiserror::Error;
|
||||
|
||||
use crate::{AppId, ScriptId, SdkCallCx};
|
||||
|
||||
/// A module script as returned by `ModuleSource::lookup`. Carries only
|
||||
/// the fields the resolver needs: the id (for diagnostics), the source
|
||||
/// (to compile), and `updated_at` (the cache-staleness comparator).
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct ModuleScript {
|
||||
pub script_id: ScriptId,
|
||||
pub app_id: AppId,
|
||||
pub name: String,
|
||||
pub source: String,
|
||||
pub updated_at: DateTime<Utc>,
|
||||
}
|
||||
|
||||
/// Lookup contract used by `PicloudModuleResolver`. `lookup` MUST
|
||||
/// scope by `cx.app_id`; cross-app reads must be unreachable.
|
||||
#[async_trait]
|
||||
pub trait ModuleSource: Send + Sync {
|
||||
/// Resolve a module script by `(cx.app_id, name)`. Returns `None`
|
||||
/// when no row exists, or when a row exists but its `kind` is
|
||||
/// `'endpoint'` (endpoints are never importable). The resolver
|
||||
/// surfaces `None` as `ErrorModuleNotFound` to Rhai.
|
||||
async fn lookup(
|
||||
&self,
|
||||
cx: &SdkCallCx,
|
||||
name: &str,
|
||||
) -> Result<Option<ModuleScript>, ModuleSourceError>;
|
||||
}
|
||||
|
||||
/// Failure modes surfaced from `ModuleSource::lookup`. "Not found" is
|
||||
/// not exceptional — it's `Ok(None)`.
|
||||
#[derive(Debug, Error)]
|
||||
pub enum ModuleSourceError {
|
||||
/// Backend (Postgres, network, etc.) unavailable or returned an
|
||||
/// error. The string is safe to surface to a script (Rhai wraps
|
||||
/// it in `ErrorModuleNotFound` with the module name + reason).
|
||||
#[error("module backend error: {0}")]
|
||||
Backend(String),
|
||||
}
|
||||
|
||||
/// Stub used by the executor-core test harness so engine integration
|
||||
/// tests don't need a real DB-backed source. Every lookup returns
|
||||
/// `Ok(None)` — `import "x"` always errors as "module not found"
|
||||
/// under this impl.
|
||||
#[derive(Debug, Default, Clone, Copy)]
|
||||
pub struct NoopModuleSource;
|
||||
|
||||
#[async_trait]
|
||||
impl ModuleSource for NoopModuleSource {
|
||||
async fn lookup(
|
||||
&self,
|
||||
_cx: &SdkCallCx,
|
||||
_name: &str,
|
||||
) -> Result<Option<ModuleScript>, ModuleSourceError> {
|
||||
Ok(None)
|
||||
}
|
||||
}
|
||||
@@ -3,6 +3,52 @@ use serde::{Deserialize, Serialize};
|
||||
|
||||
use crate::{AppId, ScriptId, ScriptSandbox};
|
||||
|
||||
/// Semantic role of a script (v1.1.3).
|
||||
///
|
||||
/// `Endpoint` scripts have an executable entry point — they bind to HTTP
|
||||
/// routes and act as trigger handlers. `Module` scripts are libraries of
|
||||
/// `fn`/`const` declarations imported by other scripts via Rhai's
|
||||
/// `import "<name>" as <alias>;` syntax. Modules cannot be invoked
|
||||
/// directly: route binding and trigger creation reject `Module` targets.
|
||||
///
|
||||
/// Serialized as `"endpoint"` / `"module"` so the wire shape is the
|
||||
/// same string the SQL `CHECK (kind IN ('endpoint','module'))`
|
||||
/// constraint enforces.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "lowercase")]
|
||||
pub enum ScriptKind {
|
||||
Endpoint,
|
||||
Module,
|
||||
}
|
||||
|
||||
impl Default for ScriptKind {
|
||||
fn default() -> Self {
|
||||
Self::Endpoint
|
||||
}
|
||||
}
|
||||
|
||||
impl ScriptKind {
|
||||
/// Wire / SQL representation. Inverse of `from_str`.
|
||||
#[must_use]
|
||||
pub fn as_str(self) -> &'static str {
|
||||
match self {
|
||||
Self::Endpoint => "endpoint",
|
||||
Self::Module => "module",
|
||||
}
|
||||
}
|
||||
|
||||
/// Parse the canonical wire / SQL form. Returns `None` for any
|
||||
/// other input; callers map that to a 400 / `ValidationError`.
|
||||
#[must_use]
|
||||
pub fn from_str(s: &str) -> Option<Self> {
|
||||
match s {
|
||||
"endpoint" => Some(Self::Endpoint),
|
||||
"module" => Some(Self::Module),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// A user-uploaded Rhai script and its execution configuration.
|
||||
///
|
||||
/// This is the canonical representation that flows between manager (storage),
|
||||
@@ -20,6 +66,12 @@ pub struct Script {
|
||||
pub version: i32,
|
||||
pub source: String,
|
||||
|
||||
/// `Endpoint` (default; the only kind v1.0 through v1.1.2 supported)
|
||||
/// or `Module` (v1.1.3 — imported by other scripts, never bound
|
||||
/// directly to a route or trigger).
|
||||
#[serde(default)]
|
||||
pub kind: ScriptKind,
|
||||
|
||||
pub timeout_seconds: u32,
|
||||
|
||||
/// Per-script overrides for Rhai sandbox limits. Empty = platform
|
||||
|
||||
@@ -7,9 +7,11 @@
|
||||
//! handed to `executor-core::sdk::register_all` alongside an
|
||||
//! `SdkCallCx` to wire each `::` namespace.
|
||||
//!
|
||||
//! v1.1.0 shipped this empty; v1.1.1 adds the first two service fields
|
||||
//! v1.1.0 shipped this empty; v1.1.1 added the first two service fields
|
||||
//! (`kv`, `dead_letters`) plus the `events` emitter that bound services
|
||||
//! use to publish events into the triggers outbox.
|
||||
//! use to publish events into the triggers outbox. v1.1.3 adds the
|
||||
//! `modules` field — the `ModuleSource` consulted by the per-call
|
||||
//! `PicloudModuleResolver` to load `import`ed module scripts.
|
||||
//!
|
||||
//! `#[non_exhaustive]` so adding fields is a non-breaking change for
|
||||
//! consumers that only *pattern-match* a `&Services`; only crates that
|
||||
@@ -18,8 +20,8 @@
|
||||
use std::sync::Arc;
|
||||
|
||||
use crate::{
|
||||
DeadLetterService, DocsService, KvService, NoopDeadLetterService, NoopDocsService,
|
||||
NoopEventEmitter, NoopKvService, ServiceEventEmitter,
|
||||
DeadLetterService, DocsService, KvService, ModuleSource, NoopDeadLetterService, NoopDocsService,
|
||||
NoopEventEmitter, NoopKvService, NoopModuleSource, ServiceEventEmitter,
|
||||
};
|
||||
|
||||
/// SDK service bundle. See module docs for the lifecycle and the v1.1.x
|
||||
@@ -45,6 +47,12 @@ pub struct Services {
|
||||
/// `manager-core::outbox_event_emitter` replaces v1.1.0's
|
||||
/// `NoopEventEmitter`.
|
||||
pub events: Arc<dyn ServiceEventEmitter>,
|
||||
|
||||
/// Module source (v1.1.3). The `PicloudModuleResolver` consults
|
||||
/// this to load `kind = 'module'` scripts that other scripts
|
||||
/// `import`. Backed by Postgres in the picloud binary; in-memory
|
||||
/// fakes in resolver tests.
|
||||
pub modules: Arc<dyn ModuleSource>,
|
||||
}
|
||||
|
||||
impl Services {
|
||||
@@ -57,12 +65,14 @@ impl Services {
|
||||
docs: Arc<dyn DocsService>,
|
||||
dead_letters: Arc<dyn DeadLetterService>,
|
||||
events: Arc<dyn ServiceEventEmitter>,
|
||||
modules: Arc<dyn ModuleSource>,
|
||||
) -> Self {
|
||||
Self {
|
||||
kv,
|
||||
docs,
|
||||
dead_letters,
|
||||
events,
|
||||
modules,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -78,6 +88,7 @@ impl Services {
|
||||
Arc::new(NoopDocsService),
|
||||
Arc::new(NoopDeadLetterService),
|
||||
Arc::new(NoopEventEmitter),
|
||||
Arc::new(NoopModuleSource),
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -10,8 +10,39 @@ use thiserror::Error;
|
||||
pub enum ValidationError {
|
||||
#[error("invalid script source: {0}")]
|
||||
Syntax(String),
|
||||
|
||||
/// v1.1.3: source compiled but failed the module-shape gate
|
||||
/// (top-level statements other than `fn` / `const` / `import`).
|
||||
#[error("module syntax error: {0}")]
|
||||
ModuleShape(String),
|
||||
}
|
||||
|
||||
/// Output of a successful validate. v1.1.3 carries the list of literal
|
||||
/// `import "<name>"` paths the script declares — the manager writes
|
||||
/// these into the `script_imports` dep-graph table. Endpoints may also
|
||||
/// have imports; the field is populated unconditionally.
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct ValidatedScript {
|
||||
/// Literal-path imports (in declaration order). Dynamic imports
|
||||
/// `import some_var as y;` are not captured — the resolver still
|
||||
/// honors them at runtime, but the dep graph only tracks names
|
||||
/// known at compile time.
|
||||
pub imports: Vec<String>,
|
||||
}
|
||||
|
||||
pub trait ScriptValidator: Send + Sync {
|
||||
fn validate(&self, source: &str) -> Result<(), ValidationError>;
|
||||
/// Endpoint-shape validation: parse-only syntax check. Returns the
|
||||
/// declared (literal) imports so the manager can populate the
|
||||
/// dep-graph table on save.
|
||||
fn validate(&self, source: &str) -> Result<ValidatedScript, ValidationError>;
|
||||
|
||||
/// Module-shape validation: parse + reject any top-level
|
||||
/// statement that isn't `fn` / `const` / `import`. Default impl
|
||||
/// rejects every module so non-engine validators stay simple
|
||||
/// (tests / stubs don't need to know module rules).
|
||||
fn validate_module(&self, _source: &str) -> Result<ValidatedScript, ValidationError> {
|
||||
Err(ValidationError::ModuleShape(
|
||||
"module validation not implemented by this validator".into(),
|
||||
))
|
||||
}
|
||||
}
|
||||
|
||||
@@ -27,7 +27,13 @@ pub const PRODUCT_VERSION: &str = env!("CARGO_PKG_VERSION");
|
||||
/// `docs::collection(name).{create,get,find,find_one,update,delete,list}`
|
||||
/// with the v1.1.2 query DSL subset; `ctx.event.docs` for docs-trigger
|
||||
/// handlers (carries `prev_data` change-data-capture for update/delete).
|
||||
pub const SDK_VERSION: &str = "1.3";
|
||||
///
|
||||
/// 1.4 additions (v1.1.3): `import "<name>" as <alias>;` for scripts
|
||||
/// whose corresponding module (`kind = 'module'`) lives in the same
|
||||
/// app. Cross-app imports are unreachable (the `name` argument carries
|
||||
/// no `app_id`). Modules expose `fn`/`const` declarations only;
|
||||
/// top-level statements are rejected at create-time.
|
||||
pub const SDK_VERSION: &str = "1.4";
|
||||
|
||||
/// HTTP API major version. Appears in URL paths as `/api/v{N}/...`.
|
||||
/// Bump (new integer + new URL prefix) when the request/response
|
||||
|
||||
Reference in New Issue
Block a user