fabi/PiCloud
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: end-to-end script CRUD + Rhai execution

Browse Source 
Brings the MVP feature set online: upload a Rhai script, get an HTTP
endpoint that runs it sandboxed in-process, list/update/delete it, and
have invalid sources rejected at upload time. Verified live through
Caddy with a full lifecycle (`create → list → get → execute → update
→ delete`) plus error paths (syntax error, duplicate name, deleted).

Layout — every concern lands behind the trait seam its layer owns, so
cluster-mode in v1.3+ is a swap of two impls, not a rewrite:

  * shared::ScriptValidator — manager calls into validation without
    a hard dep on executor-core; executor-core impls the trait on
    `Engine`. Pinned in shared so neither crate has to know about
    the other.

  * executor-core::Engine — real Rhai engine: sandbox limits (max
    operations / string size / map size / call depth), disabled
    `print`, blocked `import` (DummyModuleResolver), `log::trace
    /info/warn/error` registered as a static module with shared
    log-capture buffer (no `log::debug` because `debug` is a Rhai
    reserved keyword — `log::trace` covers the same need).

      - `ctx` is pushed as a Scope constant exposing
        execution_id, script_id, script_name, request_id,
        invocation_type, request.{path,headers,body}.

      - Response convention: a Map with `statusCode` is the
        structured shape (`{statusCode, headers?, body}`); any
        other return value is a 200 with the value as the body.

      - Engine::execute is now synchronous (pure compute); the
        async wrapper + wall-clock timeout live in
        LocalExecutorClient, which spawns_blocking and applies a
        300s hard ceiling regardless of per-script config.

      - 10 unit tests cover validate, exec, structured response,
        ctx exposure, log capture, op-budget enforcement, runtime
        errors, blocked imports, JSON round-tripping.

  * manager-core::repo — full sqlx CRUD over the `scripts` table,
    with proper unique-violation handling for duplicate names.
    Embedded migrations via `sqlx::migrate!` (one initial
    `0001_init.sql` for pgcrypto + scripts + execution_logs).

  * manager-core::api — `admin_router` mounts `/scripts` and
    `/scripts/{id}`. Create + Update validate source through the
    injected `ScriptValidator` before persistence. Returns proper
    422/409/404 status codes via `ApiError::IntoResponse`.

  * orchestrator-core::api — `data_plane_router` mounts
    `/execute/{id}`: resolves the script through `ScriptResolver`,
    constructs the `ExecRequest` from headers+body, awaits
    `ExecutorClient::execute(..., timeout)`, translates the
    `ExecResponse` to an axum `Response` with header passthrough.
    Maps `ExecError` variants to 422/504/502/507.

  * picloud all-in-one — opens the pool, runs migrations, builds
    one engine, nests both routers under `/api/admin` and `/api`,
    enables structured JSON tracing and graceful shutdown on
    SIGTERM. Single `PostgresScriptRepository` Arc is shared by
    the admin router (writes) and the resolver (reads).

Other changes:
  * Workspace axum bump 0.7 → 0.8 for the `{id}` path syntax
    matching the route definitions.
  * Workspace clippy: allow `needless_pass_by_value` and
    `boxed_local` to keep API ergonomics over pedantic noise.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
MechaCat02
2026-05-23 00:00:36 +02:00
parent 9efe678983
commit 4f044e7b81
19 changed files with 1272 additions and 76 deletions
Download Patch File Download Diff File Expand all files Collapse all files

321
crates/executor-core/src/engine.rs
View File

@@ -1,15 +1,26 @@
use std::collections::BTreeMap;
use std::sync::{Arc, Mutex};
use std::time::Instant;
use chrono::Utc;
use picloud_shared::{ScriptValidator, ValidationError};
use rhai::{Dynamic, Engine as RhaiEngine, EvalAltResult, Map, Module, Scope};
use serde_json::Value as Json;
use crate::sandbox::Limits;
use crate::types::{ExecError, ExecRequest, ExecResponse};
use crate::types::{
ExecError, ExecRequest, ExecResponse, ExecStats, InvocationType, LogEntry, LogLevel,
};
/// Preconfigured Rhai engine with sandbox limits applied.
///
/// One `Engine` is constructed at process startup and reused across
/// invocations. `execute` is the only entry point — it owns the per-call
/// scope and log buffer, then returns a complete `ExecResponse`.
/// invocations. `execute` is **synchronous** — it owns the per-call
/// scope and log buffer. Wall-clock timeouts and offloading off the
/// async runtime belong to the caller (orchestrator-core's
/// `LocalExecutorClient` wraps this with `spawn_blocking` + `timeout`).
pub struct Engine {
limits: Limits,
// The actual `rhai::Engine` lands with the first execution implementation.
// Keep this opaque for now so callers don't bind to it.
}
impl Engine {
@@ -23,26 +34,288 @@ impl Engine {
&self.limits
}
/// Parse-only validation, used by the manager at script-upload time
/// so syntax errors are surfaced before the first invocation.
pub fn validate(&self, _source: &str) -> Result<(), ExecError> {
// TODO(executor-core): wire `rhai::Engine::compile`
Ok(())
/// Parse-only validation. Surfaced at script-upload time so syntax
/// errors are caught before the first invocation. Same logic as the
/// `ScriptValidator` impl below but with the richer `ExecError`
/// variant; callers in the executor path use this, the manager
/// path goes through the trait.
pub fn validate(&self, source: &str) -> Result<(), ExecError> {
let engine = build_engine(self.limits, None);
engine
.compile(source)
.map(|_| ())
.map_err(|e| ExecError::Parse(e.to_string()))
}
/// Execute `source` against `req` under the configured sandbox.
///
/// `async` is part of the contract: v1.1+ SDK calls (kv, docs, http)
/// will await injected service providers from inside this method.
#[allow(clippy::unused_async)]
pub async fn execute(
&self,
_source: &str,
_req: ExecRequest,
) -> Result<ExecResponse, ExecError> {
// TODO(executor-core): wire `rhai::Engine::eval_with_scope`
Err(ExecError::Runtime(
"executor-core::Engine::execute not yet implemented".into(),
))
/// Execute `source` against `req`. Op-budget protection comes from
/// Rhai's `set_max_operations`; wall-clock enforcement is the
/// caller's responsibility.
pub fn execute(&self, source: &str, req: ExecRequest) -> Result<ExecResponse, ExecError> {
let logs: Arc<Mutex<Vec<LogEntry>>> = Arc::new(Mutex::new(Vec::new()));
let engine = build_engine(self.limits, Some(logs.clone()));
let ast = engine
.compile(source)
.map_err(|e| ExecError::Parse(e.to_string()))?;
let mut scope = Scope::new();
scope.push_constant("ctx", build_ctx_map(&req));
let started = Instant::now();
let value: Dynamic = engine
.eval_ast_with_scope(&mut scope, &ast)
.map_err(map_eval_error)?;
let duration = started.elapsed();
let logs = Arc::try_unwrap(logs).map_or_else(
|arc| arc.lock().map(|g| g.clone()).unwrap_or_default(),
|m| m.into_inner().unwrap_or_default(),
);
let (status_code, headers, body) = parse_response(value)?;
Ok(ExecResponse {
status_code,
headers,
body,
logs,
stats: ExecStats {
duration_ms: u64::try_from(duration.as_millis()).unwrap_or(u64::MAX),
operations: 0,
},
})
}
}
impl ScriptValidator for Engine {
fn validate(&self, source: &str) -> Result<(), ValidationError> {
Engine::validate(self, source).map_err(|e| ValidationError::Syntax(e.to_string()))
}
}
// ----------------------------------------------------------------------------
// Engine construction
// ----------------------------------------------------------------------------
fn build_engine(limits: Limits, logs: Option<Arc<Mutex<Vec<LogEntry>>>>) -> RhaiEngine {
let mut engine = RhaiEngine::new();
engine.set_max_operations(limits.max_operations);
engine.set_max_string_size(limits.max_string_size);
engine.set_max_array_size(limits.max_array_size);
engine.set_max_map_size(limits.max_map_size);
engine.set_max_call_levels(limits.max_call_levels);
engine.set_max_expr_depths(limits.max_expr_depth, limits.max_expr_depth);
// Reject `import` — scripts cannot pull external modules.
engine.set_module_resolver(rhai::module_resolvers::DummyModuleResolver);
// Rhai's built-in `print` and `debug` map to stdout/stderr by
// default; we never want scripts dumping there directly. Disable
// them so scripts route all output through `log::*` instead.
engine.disable_symbol("print");
if let Some(logs) = logs {
engine.register_static_module("log", build_log_module(logs).into());
}
engine
}
fn build_log_module(logs: Arc<Mutex<Vec<LogEntry>>>) -> Module {
let mut module = Module::new();
register_log_fn(&mut module, "trace", LogLevel::Trace, &logs);
register_log_fn(&mut module, "info", LogLevel::Info, &logs);
register_log_fn(&mut module, "warn", LogLevel::Warn, &logs);
register_log_fn(&mut module, "error", LogLevel::Error, &logs);
// No `log::debug` — `debug` is a Rhai reserved keyword. Use
// `log::trace` for sub-info-level diagnostics.
module
}
fn register_log_fn(
module: &mut Module,
name: &str,
level: LogLevel,
logs: &Arc<Mutex<Vec<LogEntry>>>,
) {
// Single-argument form: `log::info("message")`.
let logs_single = logs.clone();
module.set_native_fn(name, move |msg: &str| {
push_log(&logs_single, level, msg, None);
Ok::<_, Box<EvalAltResult>>(())
});
// Two-argument form: `log::info("message", #{ user: 42 })`.
let logs_struct = logs.clone();
module.set_native_fn(name, move |msg: &str, data: Dynamic| {
let json = dynamic_to_json(&data);
push_log(&logs_struct, level, msg, Some(json));
Ok::<_, Box<EvalAltResult>>(())
});
}
fn push_log(logs: &Arc<Mutex<Vec<LogEntry>>>, level: LogLevel, message: &str, data: Option<Json>) {
if let Ok(mut g) = logs.lock() {
g.push(LogEntry {
timestamp: Utc::now(),
level,
message: message.to_string(),
data,
});
}
}
// ----------------------------------------------------------------------------
// ctx construction
// ----------------------------------------------------------------------------
fn build_ctx_map(req: &ExecRequest) -> Map {
let mut ctx = Map::new();
ctx.insert("execution_id".into(), req.execution_id.to_string().into());
ctx.insert("script_id".into(), req.script_id.to_string().into());
ctx.insert("script_name".into(), req.script_name.clone().into());
ctx.insert("request_id".into(), req.request_id.to_string().into());
ctx.insert(
"invocation_type".into(),
invocation_type_str(req.invocation_type).into(),
);
let mut request = Map::new();
request.insert("path".into(), req.path.clone().into());
let mut headers = Map::new();
for (k, v) in &req.headers {
headers.insert(k.clone().into(), v.clone().into());
}
request.insert("headers".into(), headers.into());
request.insert("body".into(), json_to_dynamic(req.body.clone()));
ctx.insert("request".into(), request.into());
ctx
}
fn invocation_type_str(it: InvocationType) -> &'static str {
match it {
InvocationType::Http => "http",
InvocationType::Function => "function",
InvocationType::Scheduled => "scheduled",
}
}
// ----------------------------------------------------------------------------
// Response parsing
// ----------------------------------------------------------------------------
fn parse_response(value: Dynamic) -> Result<(u16, BTreeMap<String, String>, Json), ExecError> {
// Convention: a Map with a `statusCode` field is the structured shape.
// Anything else is treated as a 200 response with the value as body.
if value.is_map() {
if let Some(map) = value.clone().try_cast::<Map>() {
if map.contains_key("statusCode") {
return parse_structured_response(map);
}
}
}
Ok((200, BTreeMap::new(), dynamic_to_json(&value)))
}
fn parse_structured_response(map: Map) -> Result<(u16, BTreeMap<String, String>, Json), ExecError> {
let status_dyn = map
.get("statusCode")
.ok_or_else(|| ExecError::InvalidResponse("missing statusCode".into()))?;
let status_code: i64 = status_dyn
.as_int()
.map_err(|_| ExecError::InvalidResponse("statusCode must be an integer".into()))?;
let status_code = u16::try_from(status_code)
.map_err(|_| ExecError::InvalidResponse("statusCode out of HTTP range".into()))?;
let mut headers: BTreeMap<String, String> = BTreeMap::new();
if let Some(h) = map.get("headers") {
if let Some(h_map) = h.clone().try_cast::<Map>() {
for (k, v) in h_map {
headers.insert(k.to_string(), v.to_string());
}
}
}
let body = map.get("body").map_or(Json::Null, dynamic_to_json);
Ok((status_code, headers, body))
}
// ----------------------------------------------------------------------------
// Rhai ↔ serde_json bridges
// ----------------------------------------------------------------------------
fn json_to_dynamic(value: Json) -> Dynamic {
match value {
Json::Null => Dynamic::UNIT,
Json::Bool(b) => b.into(),
Json::Number(n) => {
if let Some(i) = n.as_i64() {
i.into()
} else if let Some(f) = n.as_f64() {
f.into()
} else {
n.to_string().into()
}
}
Json::String(s) => s.into(),
Json::Array(arr) => arr
.into_iter()
.map(json_to_dynamic)
.collect::<Vec<Dynamic>>()
.into(),
Json::Object(obj) => {
let mut m = Map::new();
for (k, v) in obj {
m.insert(k.into(), json_to_dynamic(v));
}
Dynamic::from(m)
}
}
}
fn dynamic_to_json(value: &Dynamic) -> Json {
if value.is_unit() {
return Json::Null;
}
if let Ok(b) = value.as_bool() {
return Json::Bool(b);
}
if let Ok(i) = value.as_int() {
return Json::Number(i.into());
}
if let Ok(f) = value.as_float() {
return serde_json::Number::from_f64(f).map_or(Json::Null, Json::Number);
}
if value.is_string() {
return Json::String(value.clone().into_string().unwrap_or_default());
}
if let Some(arr) = value.clone().try_cast::<rhai::Array>() {
return Json::Array(arr.iter().map(dynamic_to_json).collect());
}
if let Some(map) = value.clone().try_cast::<Map>() {
let mut out = serde_json::Map::new();
for (k, v) in map {
out.insert(k.to_string(), dynamic_to_json(&v));
}
return Json::Object(out);
}
// Anything else (timestamps, custom types) — best-effort string form.
Json::String(value.to_string())
}
// ----------------------------------------------------------------------------
// Error mapping
// ----------------------------------------------------------------------------
fn map_eval_error(err: Box<EvalAltResult>) -> ExecError {
match *err {
EvalAltResult::ErrorTooManyOperations(_) => ExecError::OperationBudgetExceeded,
EvalAltResult::ErrorParsing(parse_err, _) => ExecError::Parse(parse_err.to_string()),
other => ExecError::Runtime(other.to_string()),
}
}

2
crates/executor-core/src/types.rs
View File

@@ -41,7 +41,7 @@ pub struct ExecResponse {
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)]
#[serde(rename_all = "lowercase")]
pub enum LogLevel {
Debug,
Trace,
Info,
Warn,
Error,