Compare commits
9 Commits
feat/v1.1.
...
10cfde9e40
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
10cfde9e40 | ||
|
|
bb88b024d2 | ||
|
|
9d01f42d5e | ||
|
|
1a6324078c | ||
|
|
54efe61167 | ||
|
|
1d2e99e42c | ||
|
|
9e54b7f875 | ||
|
|
a685674dbf | ||
|
|
a8aab22163 |
@@ -8,7 +8,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
|
||||
|
||||
Authoritative design: [serverless_cloud_blueprint.md](serverless_cloud_blueprint.md). The blueprint is a living document — when architecture decisions are made in conversation that contradict it, treat the latest decision as truth and update the blueprint.
|
||||
|
||||
**Current focus (Phase 4, v1.1.0):** SDK foundation + stdlib utilities — the shape every v1.1.x service module hangs off, see [docs/sdk-shape.md](docs/sdk-shape.md). Subsequent v1.1.x releases (KV in v1.1.1, docs in v1.1.2, …) fill it in; see blueprint §12 for the full table. Phase 3 shipped end-to-end: admin auth, multi-app scoping, and Phase 3.5 capability gating (`manager-core::authz::{can, require, Capability}` + migration `0006_users_authz.sql`). Every v1.1+ table starts with `app_id UUID NOT NULL REFERENCES apps(id) ON DELETE CASCADE` and every Rhai SDK call resolves its app from the execution context.
|
||||
**Current focus (Phase 4, v1.1.0):** SDK foundation + stdlib utilities — the shape every v1.1.x service module hangs off, see [docs/sdk-shape.md](docs/sdk-shape.md). Stdlib reference at [docs/stdlib-reference.md](docs/stdlib-reference.md). Subsequent v1.1.x releases (KV in v1.1.1, docs in v1.1.2, …) fill it in; see blueprint §12 for the full table. Phase 3 shipped end-to-end: admin auth, multi-app scoping, and Phase 3.5 capability gating (`manager-core::authz::{can, require, Capability}` + migration `0006_users_authz.sql`). Every v1.1+ table starts with `app_id UUID NOT NULL REFERENCES apps(id) ON DELETE CASCADE` and every Rhai SDK call resolves its app from the execution context.
|
||||
|
||||
## Three-Service Architecture
|
||||
|
||||
|
||||
23
Cargo.lock
generated
23
Cargo.lock
generated
@@ -1505,7 +1505,7 @@ checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220"
|
||||
|
||||
[[package]]
|
||||
name = "picloud"
|
||||
version = "0.6.0"
|
||||
version = "1.1.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"async-trait",
|
||||
@@ -1531,7 +1531,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "picloud-cli"
|
||||
version = "0.6.0"
|
||||
version = "1.1.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"assert_cmd",
|
||||
@@ -1552,7 +1552,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "picloud-executor"
|
||||
version = "0.6.0"
|
||||
version = "1.1.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"picloud-executor-core",
|
||||
@@ -1564,10 +1564,15 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "picloud-executor-core"
|
||||
version = "0.6.0"
|
||||
version = "1.1.0"
|
||||
dependencies = [
|
||||
"base64",
|
||||
"chrono",
|
||||
"hex",
|
||||
"percent-encoding",
|
||||
"picloud-shared",
|
||||
"rand 0.8.6",
|
||||
"regex",
|
||||
"rhai",
|
||||
"serde",
|
||||
"serde_json",
|
||||
@@ -1578,7 +1583,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "picloud-manager"
|
||||
version = "0.6.0"
|
||||
version = "1.1.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"picloud-manager-core",
|
||||
@@ -1590,7 +1595,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "picloud-manager-core"
|
||||
version = "0.6.0"
|
||||
version = "1.1.0"
|
||||
dependencies = [
|
||||
"argon2",
|
||||
"async-trait",
|
||||
@@ -1614,7 +1619,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "picloud-orchestrator"
|
||||
version = "0.6.0"
|
||||
version = "1.1.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"picloud-orchestrator-core",
|
||||
@@ -1626,7 +1631,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "picloud-orchestrator-core"
|
||||
version = "0.6.0"
|
||||
version = "1.1.0"
|
||||
dependencies = [
|
||||
"async-trait",
|
||||
"axum",
|
||||
@@ -1645,7 +1650,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "picloud-shared"
|
||||
version = "0.6.0"
|
||||
version = "1.1.0"
|
||||
dependencies = [
|
||||
"async-trait",
|
||||
"chrono",
|
||||
|
||||
@@ -13,7 +13,7 @@ members = [
|
||||
]
|
||||
|
||||
[workspace.package]
|
||||
version = "0.6.0"
|
||||
version = "1.1.0"
|
||||
edition = "2021"
|
||||
rust-version = "1.92"
|
||||
license = "MIT OR Apache-2.0"
|
||||
@@ -74,6 +74,12 @@ sha2 = "0.10"
|
||||
base64 = "0.22"
|
||||
data-encoding = "2.6"
|
||||
|
||||
# Stdlib utility crates (v1.1.0 stdlib PR — registered into the
|
||||
# Rhai engine as the regex::/random::/etc. namespaces)
|
||||
regex = "1"
|
||||
hex = "0.4"
|
||||
percent-encoding = "2"
|
||||
|
||||
[workspace.lints.rust]
|
||||
unsafe_code = "forbid"
|
||||
|
||||
|
||||
@@ -18,3 +18,10 @@ tracing.workspace = true
|
||||
uuid.workspace = true
|
||||
chrono.workspace = true
|
||||
rhai.workspace = true
|
||||
|
||||
# Stdlib utility modules — see crates/executor-core/src/sdk/stdlib/.
|
||||
regex.workspace = true
|
||||
rand.workspace = true
|
||||
base64.workspace = true
|
||||
hex.workspace = true
|
||||
percent-encoding.workspace = true
|
||||
|
||||
@@ -143,6 +143,11 @@ fn build_engine(limits: Limits, logs: Option<Arc<Mutex<Vec<LogEntry>>>>) -> Rhai
|
||||
engine.register_static_module("log", build_log_module(logs).into());
|
||||
}
|
||||
|
||||
// Stateless utility modules — regex::/random::/time::/json::/base64::/
|
||||
// hex::/url::. Always registered, including in the parse-only validate
|
||||
// path, so script authors get consistent surface in both phases.
|
||||
sdk::stdlib::register_stdlib(&mut engine);
|
||||
|
||||
engine
|
||||
}
|
||||
|
||||
|
||||
@@ -13,6 +13,7 @@
|
||||
|
||||
pub mod bridge;
|
||||
pub mod cx;
|
||||
pub mod stdlib;
|
||||
|
||||
pub use bridge::{dynamic_to_json, json_to_dynamic};
|
||||
pub use cx::SdkCallCx;
|
||||
|
||||
48
crates/executor-core/src/sdk/stdlib/base64.rs
Normal file
48
crates/executor-core/src/sdk/stdlib/base64.rs
Normal file
@@ -0,0 +1,48 @@
|
||||
//! `base64::` — standard and URL-safe Base64.
|
||||
//!
|
||||
//! Two encoders are exposed: standard alphabet with padding (`encode`/
|
||||
//! `decode`) and URL-safe alphabet without padding (`encode_url`/
|
||||
//! `decode_url`). Each encoder accepts both `String` and `Blob` inputs
|
||||
//! as separate Rhai overloads; decoders always return `Blob` — the
|
||||
//! caller knows whether the original bytes were textual.
|
||||
|
||||
use base64::engine::general_purpose::{STANDARD, URL_SAFE_NO_PAD};
|
||||
use base64::Engine as _;
|
||||
use rhai::{Blob, Engine as RhaiEngine, EvalAltResult, Module};
|
||||
|
||||
pub fn register(engine: &mut RhaiEngine) {
|
||||
let mut module = Module::new();
|
||||
|
||||
module.set_native_fn("encode", |s: &str| -> Result<String, Box<EvalAltResult>> {
|
||||
Ok(STANDARD.encode(s.as_bytes()))
|
||||
});
|
||||
module.set_native_fn("encode", |b: Blob| -> Result<String, Box<EvalAltResult>> {
|
||||
Ok(STANDARD.encode(&b))
|
||||
});
|
||||
module.set_native_fn("decode", |s: &str| -> Result<Blob, Box<EvalAltResult>> {
|
||||
STANDARD
|
||||
.decode(s)
|
||||
.map_err(|e| format!("base64::decode: {e}").into())
|
||||
});
|
||||
|
||||
module.set_native_fn(
|
||||
"encode_url",
|
||||
|s: &str| -> Result<String, Box<EvalAltResult>> {
|
||||
Ok(URL_SAFE_NO_PAD.encode(s.as_bytes()))
|
||||
},
|
||||
);
|
||||
module.set_native_fn(
|
||||
"encode_url",
|
||||
|b: Blob| -> Result<String, Box<EvalAltResult>> { Ok(URL_SAFE_NO_PAD.encode(&b)) },
|
||||
);
|
||||
module.set_native_fn(
|
||||
"decode_url",
|
||||
|s: &str| -> Result<Blob, Box<EvalAltResult>> {
|
||||
URL_SAFE_NO_PAD
|
||||
.decode(s)
|
||||
.map_err(|e| format!("base64::decode_url: {e}").into())
|
||||
},
|
||||
);
|
||||
|
||||
engine.register_static_module("base64", module.into());
|
||||
}
|
||||
21
crates/executor-core/src/sdk/stdlib/hex.rs
Normal file
21
crates/executor-core/src/sdk/stdlib/hex.rs
Normal file
@@ -0,0 +1,21 @@
|
||||
//! `hex::` — hexadecimal encode/decode (lowercase output, case-
|
||||
//! insensitive input). String and Blob inputs are both accepted on
|
||||
//! encode; decode always returns `Blob`.
|
||||
|
||||
use rhai::{Blob, Engine as RhaiEngine, EvalAltResult, Module};
|
||||
|
||||
pub fn register(engine: &mut RhaiEngine) {
|
||||
let mut module = Module::new();
|
||||
|
||||
module.set_native_fn("encode", |s: &str| -> Result<String, Box<EvalAltResult>> {
|
||||
Ok(hex::encode(s.as_bytes()))
|
||||
});
|
||||
module.set_native_fn("encode", |b: Blob| -> Result<String, Box<EvalAltResult>> {
|
||||
Ok(hex::encode(&b))
|
||||
});
|
||||
module.set_native_fn("decode", |s: &str| -> Result<Blob, Box<EvalAltResult>> {
|
||||
hex::decode(s).map_err(|e| format!("hex::decode: {e}").into())
|
||||
});
|
||||
|
||||
engine.register_static_module("hex", module.into());
|
||||
}
|
||||
43
crates/executor-core/src/sdk/stdlib/json.rs
Normal file
43
crates/executor-core/src/sdk/stdlib/json.rs
Normal file
@@ -0,0 +1,43 @@
|
||||
//! `json::` — JSON parse and stringify. Reuses the bridge functions in
|
||||
//! `crate::sdk::bridge` so script-visible JSON has the same shape
|
||||
//! (numbers, maps, arrays, nulls) as `ctx.request.body` already does.
|
||||
|
||||
use rhai::{Dynamic, Engine as RhaiEngine, EvalAltResult, Module};
|
||||
|
||||
use crate::sdk::bridge::{dynamic_to_json, json_to_dynamic};
|
||||
|
||||
pub fn register(engine: &mut RhaiEngine) {
|
||||
let mut module = Module::new();
|
||||
register_parse(&mut module);
|
||||
register_stringify(&mut module);
|
||||
register_stringify_pretty(&mut module);
|
||||
engine.register_static_module("json", module.into());
|
||||
}
|
||||
|
||||
fn register_parse(module: &mut Module) {
|
||||
module.set_native_fn("parse", |s: &str| -> Result<Dynamic, Box<EvalAltResult>> {
|
||||
let value: serde_json::Value =
|
||||
serde_json::from_str(s).map_err(|e| format!("json::parse: {e}"))?;
|
||||
Ok(json_to_dynamic(value))
|
||||
});
|
||||
}
|
||||
|
||||
fn register_stringify(module: &mut Module) {
|
||||
module.set_native_fn(
|
||||
"stringify",
|
||||
|v: Dynamic| -> Result<String, Box<EvalAltResult>> {
|
||||
serde_json::to_string(&dynamic_to_json(&v))
|
||||
.map_err(|e| format!("json::stringify: {e}").into())
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
fn register_stringify_pretty(module: &mut Module) {
|
||||
module.set_native_fn(
|
||||
"stringify_pretty",
|
||||
|v: Dynamic| -> Result<String, Box<EvalAltResult>> {
|
||||
serde_json::to_string_pretty(&dynamic_to_json(&v))
|
||||
.map_err(|e| format!("json::stringify_pretty: {e}").into())
|
||||
},
|
||||
);
|
||||
}
|
||||
25
crates/executor-core/src/sdk/stdlib/mod.rs
Normal file
25
crates/executor-core/src/sdk/stdlib/mod.rs
Normal file
@@ -0,0 +1,25 @@
|
||||
//! Stateless utility modules registered once at engine build via
|
||||
//! `Engine::register_static_module`. They have no per-call state, no
|
||||
//! cross-app sensitivity, and no `SdkCallCx` — distinguishing them
|
||||
//! from stateful service modules (KV, docs, …) which hook into
|
||||
//! `sdk::register_all` instead. See [docs/sdk-shape.md](../../../../../docs/sdk-shape.md).
|
||||
|
||||
use rhai::Engine as RhaiEngine;
|
||||
|
||||
pub mod base64;
|
||||
pub mod hex;
|
||||
pub mod json;
|
||||
pub mod random;
|
||||
pub mod regex;
|
||||
pub mod time;
|
||||
pub mod url;
|
||||
|
||||
pub fn register_stdlib(engine: &mut RhaiEngine) {
|
||||
regex::register(engine);
|
||||
random::register(engine);
|
||||
time::register(engine);
|
||||
json::register(engine);
|
||||
base64::register(engine);
|
||||
hex::register(engine);
|
||||
url::register(engine);
|
||||
}
|
||||
70
crates/executor-core/src/sdk/stdlib/random.rs
Normal file
70
crates/executor-core/src/sdk/stdlib/random.rs
Normal file
@@ -0,0 +1,70 @@
|
||||
//! `random::` — CSPRNG primitives (`rand::rngs::OsRng`).
|
||||
//!
|
||||
//! Only the OS RNG is exposed. No "fast non-crypto" variant — scripts
|
||||
//! should not pick between secure and insecure entropy. Output sizes
|
||||
//! are capped to keep a single script call from blowing host memory.
|
||||
|
||||
use rand::distributions::{Alphanumeric, DistString};
|
||||
use rand::{rngs::OsRng, Rng, RngCore};
|
||||
use rhai::{Blob, Engine as RhaiEngine, EvalAltResult, Module};
|
||||
use uuid::Uuid;
|
||||
|
||||
const MAX_BYTES: i64 = 65_536;
|
||||
const MAX_STRING: i64 = 4_096;
|
||||
|
||||
pub fn register(engine: &mut RhaiEngine) {
|
||||
let mut module = Module::new();
|
||||
register_int(&mut module);
|
||||
register_float(&mut module);
|
||||
register_bytes(&mut module);
|
||||
register_string(&mut module);
|
||||
register_uuid(&mut module);
|
||||
engine.register_static_module("random", module.into());
|
||||
}
|
||||
|
||||
fn register_int(module: &mut Module) {
|
||||
module.set_native_fn(
|
||||
"int",
|
||||
|min: i64, max: i64| -> Result<i64, Box<EvalAltResult>> {
|
||||
if min > max {
|
||||
return Err(format!("random::int: min ({min}) > max ({max})").into());
|
||||
}
|
||||
Ok(OsRng.gen_range(min..=max))
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
fn register_float(module: &mut Module) {
|
||||
module.set_native_fn("float", || -> Result<f64, Box<EvalAltResult>> {
|
||||
Ok(OsRng.gen::<f64>())
|
||||
});
|
||||
}
|
||||
|
||||
fn register_bytes(module: &mut Module) {
|
||||
module.set_native_fn("bytes", |n: i64| -> Result<Blob, Box<EvalAltResult>> {
|
||||
if !(0..=MAX_BYTES).contains(&n) {
|
||||
return Err(format!("random::bytes: n must be in 0..={MAX_BYTES}, got {n}").into());
|
||||
}
|
||||
// Safe: n is non-negative and bounded by MAX_BYTES, which fits in usize.
|
||||
let len = usize::try_from(n).expect("n bounded above by MAX_BYTES");
|
||||
let mut buf = vec![0u8; len];
|
||||
OsRng.fill_bytes(&mut buf);
|
||||
Ok(buf)
|
||||
});
|
||||
}
|
||||
|
||||
fn register_string(module: &mut Module) {
|
||||
module.set_native_fn("string", |n: i64| -> Result<String, Box<EvalAltResult>> {
|
||||
if !(0..=MAX_STRING).contains(&n) {
|
||||
return Err(format!("random::string: n must be in 0..={MAX_STRING}, got {n}").into());
|
||||
}
|
||||
let len = usize::try_from(n).expect("n bounded above by MAX_STRING");
|
||||
Ok(Alphanumeric.sample_string(&mut OsRng, len))
|
||||
});
|
||||
}
|
||||
|
||||
fn register_uuid(module: &mut Module) {
|
||||
module.set_native_fn("uuid", || -> Result<String, Box<EvalAltResult>> {
|
||||
Ok(Uuid::new_v4().to_string())
|
||||
});
|
||||
}
|
||||
105
crates/executor-core/src/sdk/stdlib/regex.rs
Normal file
105
crates/executor-core/src/sdk/stdlib/regex.rs
Normal file
@@ -0,0 +1,105 @@
|
||||
//! `regex::` — non-backtracking regular expressions (Rust `regex` crate).
|
||||
//!
|
||||
//! Patterns compile per call. No cache: premature for v1.1.0, and the
|
||||
//! `regex` crate's linear-time guarantees keep per-call cost bounded.
|
||||
//! Catastrophic patterns are rejected at compile time by the crate
|
||||
//! itself; no extra defense needed.
|
||||
|
||||
use regex::Regex;
|
||||
use rhai::{Array, Dynamic, Engine as RhaiEngine, EvalAltResult, Module};
|
||||
|
||||
pub fn register(engine: &mut RhaiEngine) {
|
||||
let mut module = Module::new();
|
||||
register_is_match(&mut module);
|
||||
register_find(&mut module);
|
||||
register_find_all(&mut module);
|
||||
register_replace(&mut module);
|
||||
register_replace_all(&mut module);
|
||||
register_split(&mut module);
|
||||
register_captures(&mut module);
|
||||
engine.register_static_module("regex", module.into());
|
||||
}
|
||||
|
||||
fn compile(pattern: &str) -> Result<Regex, Box<EvalAltResult>> {
|
||||
Regex::new(pattern).map_err(|e| format!("invalid regex: {e}").into())
|
||||
}
|
||||
|
||||
fn register_is_match(module: &mut Module) {
|
||||
module.set_native_fn(
|
||||
"is_match",
|
||||
|pattern: &str, text: &str| -> Result<bool, Box<EvalAltResult>> {
|
||||
Ok(compile(pattern)?.is_match(text))
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
fn register_find(module: &mut Module) {
|
||||
module.set_native_fn(
|
||||
"find",
|
||||
|pattern: &str, text: &str| -> Result<Dynamic, Box<EvalAltResult>> {
|
||||
Ok(compile(pattern)?
|
||||
.find(text)
|
||||
.map_or(Dynamic::UNIT, |m| Dynamic::from(m.as_str().to_string())))
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
fn register_find_all(module: &mut Module) {
|
||||
module.set_native_fn(
|
||||
"find_all",
|
||||
|pattern: &str, text: &str| -> Result<Array, Box<EvalAltResult>> {
|
||||
Ok(compile(pattern)?
|
||||
.find_iter(text)
|
||||
.map(|m| Dynamic::from(m.as_str().to_string()))
|
||||
.collect())
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
fn register_replace(module: &mut Module) {
|
||||
module.set_native_fn(
|
||||
"replace",
|
||||
|pattern: &str, text: &str, replacement: &str| -> Result<String, Box<EvalAltResult>> {
|
||||
Ok(compile(pattern)?.replace(text, replacement).into_owned())
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
fn register_replace_all(module: &mut Module) {
|
||||
module.set_native_fn(
|
||||
"replace_all",
|
||||
|pattern: &str, text: &str, replacement: &str| -> Result<String, Box<EvalAltResult>> {
|
||||
Ok(compile(pattern)?
|
||||
.replace_all(text, replacement)
|
||||
.into_owned())
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
fn register_split(module: &mut Module) {
|
||||
module.set_native_fn(
|
||||
"split",
|
||||
|pattern: &str, text: &str| -> Result<Array, Box<EvalAltResult>> {
|
||||
Ok(compile(pattern)?
|
||||
.split(text)
|
||||
.map(|s| Dynamic::from(s.to_string()))
|
||||
.collect())
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
fn register_captures(module: &mut Module) {
|
||||
module.set_native_fn(
|
||||
"captures",
|
||||
|pattern: &str, text: &str| -> Result<Dynamic, Box<EvalAltResult>> {
|
||||
let re = compile(pattern)?;
|
||||
Ok(re.captures(text).map_or(Dynamic::UNIT, |caps| {
|
||||
let arr: Array = caps
|
||||
.iter()
|
||||
.map(|m| m.map_or(Dynamic::UNIT, |m| Dynamic::from(m.as_str().to_string())))
|
||||
.collect();
|
||||
Dynamic::from(arr)
|
||||
}))
|
||||
},
|
||||
);
|
||||
}
|
||||
68
crates/executor-core/src/sdk/stdlib/time.rs
Normal file
68
crates/executor-core/src/sdk/stdlib/time.rs
Normal file
@@ -0,0 +1,68 @@
|
||||
//! `time::` — UTC time. The canonical "time value" is milliseconds
|
||||
//! since the Unix epoch as `i64`. ISO 8601 strings are for parsing and
|
||||
//! display only. UTC only — no timezone support in v1.1.0 (would pull
|
||||
//! in chrono-tz, deferred until a real use case demands it).
|
||||
|
||||
use chrono::{DateTime, SecondsFormat, Utc};
|
||||
use rhai::{Engine as RhaiEngine, EvalAltResult, Module};
|
||||
|
||||
pub fn register(engine: &mut RhaiEngine) {
|
||||
let mut module = Module::new();
|
||||
register_now(&mut module);
|
||||
register_now_ms(&mut module);
|
||||
register_parse(&mut module);
|
||||
register_format(&mut module);
|
||||
register_add_seconds(&mut module);
|
||||
register_diff_seconds(&mut module);
|
||||
engine.register_static_module("time", module.into());
|
||||
}
|
||||
|
||||
fn register_now(module: &mut Module) {
|
||||
module.set_native_fn("now", || -> Result<String, Box<EvalAltResult>> {
|
||||
Ok(Utc::now().to_rfc3339_opts(SecondsFormat::Millis, true))
|
||||
});
|
||||
}
|
||||
|
||||
fn register_now_ms(module: &mut Module) {
|
||||
module.set_native_fn("now_ms", || -> Result<i64, Box<EvalAltResult>> {
|
||||
Ok(Utc::now().timestamp_millis())
|
||||
});
|
||||
}
|
||||
|
||||
fn register_parse(module: &mut Module) {
|
||||
module.set_native_fn("parse", |iso: &str| -> Result<i64, Box<EvalAltResult>> {
|
||||
DateTime::parse_from_rfc3339(iso)
|
||||
.map(|dt| dt.timestamp_millis())
|
||||
.map_err(|e| format!("time::parse: invalid ISO 8601 / RFC 3339: {e}").into())
|
||||
});
|
||||
}
|
||||
|
||||
fn register_format(module: &mut Module) {
|
||||
module.set_native_fn("format", |ms: i64| -> Result<String, Box<EvalAltResult>> {
|
||||
DateTime::<Utc>::from_timestamp_millis(ms)
|
||||
.map(|dt| dt.to_rfc3339_opts(SecondsFormat::Millis, true))
|
||||
.ok_or_else(|| format!("time::format: ms ({ms}) out of representable range").into())
|
||||
});
|
||||
}
|
||||
|
||||
fn register_add_seconds(module: &mut Module) {
|
||||
module.set_native_fn(
|
||||
"add_seconds",
|
||||
|ms: i64, secs: i64| -> Result<i64, Box<EvalAltResult>> {
|
||||
secs.checked_mul(1000)
|
||||
.and_then(|delta| ms.checked_add(delta))
|
||||
.ok_or_else(|| format!("time::add_seconds: overflow (ms={ms}, secs={secs})").into())
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
fn register_diff_seconds(module: &mut Module) {
|
||||
module.set_native_fn(
|
||||
"diff_seconds",
|
||||
|a_ms: i64, b_ms: i64| -> Result<i64, Box<EvalAltResult>> {
|
||||
b_ms.checked_sub(a_ms)
|
||||
.map(|d| d / 1000)
|
||||
.ok_or_else(|| format!("time::diff_seconds: overflow (a={a_ms}, b={b_ms})").into())
|
||||
},
|
||||
);
|
||||
}
|
||||
64
crates/executor-core/src/sdk/stdlib/url.rs
Normal file
64
crates/executor-core/src/sdk/stdlib/url.rs
Normal file
@@ -0,0 +1,64 @@
|
||||
//! `url::` — RFC 3986 percent-encoding.
|
||||
//!
|
||||
//! `encode`/`decode` operate on opaque component values; `encode_query`
|
||||
//! builds an `application/x-www-form-urlencoded`-style query string
|
||||
//! from a Rhai `Map`. Key ordering is the map's natural order (Rhai's
|
||||
//! `Map` is a `BTreeMap`, so keys come out alphabetically — fine for
|
||||
//! query strings, which RFC 3986 leaves unordered).
|
||||
|
||||
use percent_encoding::{percent_decode_str, utf8_percent_encode, AsciiSet, NON_ALPHANUMERIC};
|
||||
use rhai::{Engine as RhaiEngine, EvalAltResult, Map, Module};
|
||||
|
||||
/// RFC 3986 unreserved set: `A-Z / a-z / 0-9 / - / _ / . / ~`.
|
||||
/// Everything outside this set gets percent-encoded.
|
||||
const UNRESERVED: &AsciiSet = &NON_ALPHANUMERIC
|
||||
.remove(b'-')
|
||||
.remove(b'_')
|
||||
.remove(b'.')
|
||||
.remove(b'~');
|
||||
|
||||
pub fn register(engine: &mut RhaiEngine) {
|
||||
let mut module = Module::new();
|
||||
register_encode(&mut module);
|
||||
register_decode(&mut module);
|
||||
register_encode_query(&mut module);
|
||||
engine.register_static_module("url", module.into());
|
||||
}
|
||||
|
||||
fn register_encode(module: &mut Module) {
|
||||
module.set_native_fn("encode", |s: &str| -> Result<String, Box<EvalAltResult>> {
|
||||
Ok(utf8_percent_encode(s, UNRESERVED).to_string())
|
||||
});
|
||||
}
|
||||
|
||||
fn register_decode(module: &mut Module) {
|
||||
module.set_native_fn("decode", |s: &str| -> Result<String, Box<EvalAltResult>> {
|
||||
percent_decode_str(s)
|
||||
.decode_utf8()
|
||||
.map(std::borrow::Cow::into_owned)
|
||||
.map_err(|e| format!("url::decode: invalid UTF-8: {e}").into())
|
||||
});
|
||||
}
|
||||
|
||||
fn register_encode_query(module: &mut Module) {
|
||||
module.set_native_fn(
|
||||
"encode_query",
|
||||
|m: Map| -> Result<String, Box<EvalAltResult>> {
|
||||
let mut out = String::new();
|
||||
for (k, v) in m {
|
||||
if !out.is_empty() {
|
||||
out.push('&');
|
||||
}
|
||||
out.push_str(&utf8_percent_encode(&k, UNRESERVED).to_string());
|
||||
out.push('=');
|
||||
// Coerce values via `to_string` rather than throwing on
|
||||
// non-strings — scripts commonly pass numbers/bools here
|
||||
// and a forced cast at the call site is friction with
|
||||
// no upside.
|
||||
let value = v.to_string();
|
||||
out.push_str(&utf8_percent_encode(&value, UNRESERVED).to_string());
|
||||
}
|
||||
Ok(out)
|
||||
},
|
||||
);
|
||||
}
|
||||
382
crates/executor-core/tests/stdlib.rs
Normal file
382
crates/executor-core/tests/stdlib.rs
Normal file
@@ -0,0 +1,382 @@
|
||||
//! Integration tests for the v1.1.0 stdlib utility modules.
|
||||
//!
|
||||
//! These exist alongside `sdk_contract.rs` rather than inside it
|
||||
//! because the stateless utilities aren't part of the same versioned
|
||||
//! SDK contract surface — `sdk_contract.rs` covers things that bump
|
||||
//! `SDK_VERSION` when they change; stdlib additions don't.
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
use picloud_executor_core::{Engine, ExecError, ExecRequest, InvocationType, Limits};
|
||||
use picloud_shared::{AppId, ExecutionId, RequestId, ScriptId, ScriptSandbox, Services};
|
||||
use serde_json::{json, Value};
|
||||
|
||||
// ----------------------------------------------------------------------------
|
||||
// Test harness — duplicated from sdk_contract.rs (each integration test
|
||||
// crate has its own; there is no tests/common/).
|
||||
// ----------------------------------------------------------------------------
|
||||
|
||||
fn engine() -> Engine {
|
||||
Engine::new(Limits::default(), Services::new())
|
||||
}
|
||||
|
||||
fn baseline_request() -> ExecRequest {
|
||||
let execution_id = ExecutionId::new();
|
||||
ExecRequest {
|
||||
execution_id,
|
||||
request_id: RequestId::new(),
|
||||
script_id: ScriptId::new(),
|
||||
script_name: "stdlib".into(),
|
||||
invocation_type: InvocationType::Http,
|
||||
path: "/stdlib-test".into(),
|
||||
headers: BTreeMap::new(),
|
||||
body: Value::Null,
|
||||
params: BTreeMap::new(),
|
||||
query: BTreeMap::new(),
|
||||
rest: String::new(),
|
||||
sandbox_overrides: ScriptSandbox::default(),
|
||||
app_id: AppId::new(),
|
||||
principal: None,
|
||||
trigger_depth: 0,
|
||||
root_execution_id: execution_id,
|
||||
}
|
||||
}
|
||||
|
||||
fn run(source: &str) -> Value {
|
||||
engine()
|
||||
.execute(source, baseline_request())
|
||||
.expect("stdlib test should execute cleanly")
|
||||
.body
|
||||
}
|
||||
|
||||
fn run_err(source: &str) -> ExecError {
|
||||
engine()
|
||||
.execute(source, baseline_request())
|
||||
.expect_err("stdlib test expected to throw")
|
||||
}
|
||||
|
||||
fn assert_runtime_err(err: ExecError, needle: &str) {
|
||||
match err {
|
||||
ExecError::Runtime(msg) => assert!(
|
||||
msg.contains(needle),
|
||||
"runtime error did not contain `{needle}`: {msg}"
|
||||
),
|
||||
other => panic!("expected Runtime error containing `{needle}`, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// regex
|
||||
// ============================================================================
|
||||
|
||||
#[test]
|
||||
fn regex_is_match_true_and_false() {
|
||||
assert_eq!(run(r#"regex::is_match("^h", "hello")"#), json!(true));
|
||||
assert_eq!(run(r#"regex::is_match("^x", "hello")"#), json!(false));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn regex_find_returns_first_match() {
|
||||
assert_eq!(run(r#"regex::find("\\d+", "abc 42 def 99")"#), json!("42"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn regex_find_returns_unit_when_no_match() {
|
||||
// () serializes to JSON null via dynamic_to_json.
|
||||
assert_eq!(run(r#"regex::find("\\d+", "abc")"#), Value::Null);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn regex_find_all_returns_array() {
|
||||
assert_eq!(
|
||||
run(r#"regex::find_all("\\d+", "a1 b22 c333")"#),
|
||||
json!(["1", "22", "333"])
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn regex_replace_first_only() {
|
||||
assert_eq!(
|
||||
run(r#"regex::replace("a", "banana", "X")"#),
|
||||
json!("bXnana")
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn regex_replace_all() {
|
||||
assert_eq!(
|
||||
run(r#"regex::replace_all("a", "banana", "X")"#),
|
||||
json!("bXnXnX")
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn regex_split() {
|
||||
assert_eq!(
|
||||
run(r#"regex::split(",\\s*", "a, b,c, d")"#),
|
||||
json!(["a", "b", "c", "d"])
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn regex_captures_extracts_groups() {
|
||||
assert_eq!(
|
||||
run(r#"regex::captures("(\\d+)-(\\w+)", "42-abc")"#),
|
||||
json!(["42-abc", "42", "abc"])
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn regex_captures_returns_unit_when_no_match() {
|
||||
assert_eq!(run(r#"regex::captures("(\\d+)", "abc")"#), Value::Null);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn regex_invalid_pattern_throws() {
|
||||
assert_runtime_err(run_err(r#"regex::is_match("(", "x")"#), "invalid regex");
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// random
|
||||
// ============================================================================
|
||||
|
||||
#[test]
|
||||
fn random_int_within_range() {
|
||||
// Run a few times to exercise the bounds — each call is independent.
|
||||
let body = run(r"
|
||||
let n = random::int(10, 20);
|
||||
n >= 10 && n <= 20
|
||||
");
|
||||
assert_eq!(body, json!(true));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn random_int_throws_when_min_greater_than_max() {
|
||||
assert_runtime_err(run_err("random::int(20, 10)"), "min");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn random_float_in_unit_interval() {
|
||||
let body = run(r"
|
||||
let f = random::float();
|
||||
f >= 0.0 && f < 1.0
|
||||
");
|
||||
assert_eq!(body, json!(true));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn random_bytes_returns_blob_of_correct_length() {
|
||||
assert_eq!(run("random::bytes(16).len()"), json!(16));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn random_bytes_rejects_negative() {
|
||||
assert_runtime_err(run_err("random::bytes(-1)"), "random::bytes");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn random_bytes_rejects_oversize() {
|
||||
assert_runtime_err(run_err("random::bytes(70000)"), "random::bytes");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn random_string_produces_alphanumeric_of_correct_length() {
|
||||
let body = run(r#"
|
||||
let s = random::string(32);
|
||||
s.len == 32 && regex::is_match("^[A-Za-z0-9]+$", s)
|
||||
"#);
|
||||
assert_eq!(body, json!(true));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn random_uuid_has_canonical_format() {
|
||||
let body = run(
|
||||
r#"regex::is_match("^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$", random::uuid())"#,
|
||||
);
|
||||
assert_eq!(body, json!(true));
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// time
|
||||
// ============================================================================
|
||||
|
||||
#[test]
|
||||
fn time_now_ms_is_positive() {
|
||||
let body = run("time::now_ms() > 0");
|
||||
assert_eq!(body, json!(true));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn time_now_string_looks_like_iso() {
|
||||
let body = run(r#"regex::is_match("^\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2}:\\d{2}", time::now())"#);
|
||||
assert_eq!(body, json!(true));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn time_parse_format_round_trip() {
|
||||
let body = run(r"
|
||||
let ms = 1700000000000;
|
||||
time::parse(time::format(ms)) == ms
|
||||
");
|
||||
assert_eq!(body, json!(true));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn time_add_seconds() {
|
||||
assert_eq!(run("time::add_seconds(0, 60)"), json!(60_000));
|
||||
assert_eq!(run("time::add_seconds(1000, -1)"), json!(0));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn time_diff_seconds_truncates() {
|
||||
assert_eq!(run("time::diff_seconds(0, 65_500)"), json!(65));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn time_parse_rejects_garbage() {
|
||||
assert_runtime_err(run_err(r#"time::parse("nonsense")"#), "time::parse");
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// json
|
||||
// ============================================================================
|
||||
|
||||
#[test]
|
||||
fn json_parse_then_stringify_round_trip() {
|
||||
let body = run(r#"
|
||||
let src = `{"a":1,"b":"x"}`;
|
||||
json::stringify(json::parse(src)) == src
|
||||
"#);
|
||||
assert_eq!(body, json!(true));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn json_stringify_compact() {
|
||||
assert_eq!(run(r"json::stringify(#{ a: 1 })"), json!(r#"{"a":1}"#));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn json_stringify_pretty_has_newlines() {
|
||||
let body = run(r#"json::stringify_pretty(#{ a: 1 }).contains("\n")"#);
|
||||
assert_eq!(body, json!(true));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn json_parse_invalid_throws() {
|
||||
assert_runtime_err(run_err(r#"json::parse("not json")"#), "json::parse");
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// base64
|
||||
// ============================================================================
|
||||
|
||||
#[test]
|
||||
fn base64_encode_string() {
|
||||
assert_eq!(run(r#"base64::encode("hi")"#), json!("aGk="));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn base64_decode_then_re_encode_round_trip() {
|
||||
assert_eq!(
|
||||
run(r#"base64::encode(base64::decode("aGVsbG8="))"#),
|
||||
json!("aGVsbG8=")
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn base64_encode_url_has_no_padding() {
|
||||
let body = run(r#"
|
||||
let s = base64::encode_url("hello world!?");
|
||||
!s.contains("=") && !s.contains("+") && !s.contains("/")
|
||||
"#);
|
||||
assert_eq!(body, json!(true));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn base64_decode_url_round_trip() {
|
||||
assert_eq!(
|
||||
run(r#"base64::encode_url(base64::decode_url("aGVsbG8"))"#),
|
||||
json!("aGVsbG8")
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn base64_decode_invalid_throws() {
|
||||
assert_runtime_err(run_err(r#"base64::decode("!!!")"#), "base64::decode");
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// hex
|
||||
// ============================================================================
|
||||
|
||||
#[test]
|
||||
fn hex_encode_produces_lowercase() {
|
||||
assert_eq!(run(r#"hex::encode("Z")"#), json!("5a"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn hex_decode_then_re_encode_round_trip() {
|
||||
// mixed-case input → lowercase output proves both case-insensitive
|
||||
// decode and lowercase encode.
|
||||
assert_eq!(
|
||||
run(r#"hex::encode(hex::decode("DeAdBeEf"))"#),
|
||||
json!("deadbeef")
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn hex_decode_returns_correct_length() {
|
||||
assert_eq!(run(r#"hex::decode("deadbeef").len()"#), json!(4));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn hex_decode_invalid_throws() {
|
||||
assert_runtime_err(run_err(r#"hex::decode("xyz")"#), "hex::decode");
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// url
|
||||
// ============================================================================
|
||||
|
||||
#[test]
|
||||
fn url_encode_basic() {
|
||||
assert_eq!(run(r#"url::encode("hello world")"#), json!("hello%20world"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn url_encode_preserves_unreserved() {
|
||||
assert_eq!(
|
||||
run(r#"url::encode("abcXYZ123-_.~")"#),
|
||||
json!("abcXYZ123-_.~")
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn url_decode_round_trip() {
|
||||
assert_eq!(
|
||||
run(r#"url::decode(url::encode("hello world!?"))"#),
|
||||
json!("hello world!?")
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn url_encode_query_basic() {
|
||||
// Map keys come out alphabetically (Rhai's Map is a BTreeMap).
|
||||
assert_eq!(
|
||||
run(r#"url::encode_query(#{ a: "1", b: "x y" })"#),
|
||||
json!("a=1&b=x%20y")
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn url_encode_query_coerces_non_strings() {
|
||||
// Numbers and bools shouldn't throw; they coerce via to_string().
|
||||
let body = run(r"url::encode_query(#{ n: 42, b: true })");
|
||||
// Order is alphabetical: b before n.
|
||||
assert_eq!(body, json!("b=true&n=42"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn url_decode_rejects_invalid_utf8() {
|
||||
assert_runtime_err(run_err(r#"url::decode("%FF%FE%80")"#), "url::decode");
|
||||
}
|
||||
215
docs/stdlib-reference.md
Normal file
215
docs/stdlib-reference.md
Normal file
@@ -0,0 +1,215 @@
|
||||
# Rhai stdlib reference
|
||||
|
||||
Everything in this document is callable from any user script without
|
||||
imports — Rhai's built-in standard library plus the seven PiCloud
|
||||
utility modules added in v1.1.0. Stateful service modules (KV, docs,
|
||||
HTTP, …) ship in subsequent v1.1.x releases and are documented
|
||||
separately.
|
||||
|
||||
For the architectural shape (why some modules are stateless and
|
||||
register at engine build, why others are per-call), see
|
||||
[sdk-shape.md](sdk-shape.md).
|
||||
|
||||
## Conventions
|
||||
|
||||
- **Throw on failure.** Every function throws a Rhai runtime error on
|
||||
bad input (invalid pattern, invalid encoding, out-of-range arg). Use
|
||||
`try { ... } catch (e) { ... }` if you want to handle it.
|
||||
- **`()` for absent.** Functions that semantically may have no result
|
||||
(e.g. `regex::find` when nothing matches) return `()`. Test with
|
||||
`if v == () { ... }`.
|
||||
- **`bool` for predicates.** Yes/no questions return `bool`.
|
||||
- **UTC, milliseconds, lowercase hex, RFC 3986.** Defaults chosen once,
|
||||
not per call.
|
||||
|
||||
---
|
||||
|
||||
## Rhai built-ins (free with every script)
|
||||
|
||||
These come with the Rhai engine itself. See the
|
||||
[Rhai book](https://rhai.rs/book/lib/index.html) for full signatures.
|
||||
|
||||
**Math:** `+ - * / %`, `min`, `max`, `abs`, `sqrt`, `pow`, `floor`,
|
||||
`ceil`, `round`, `to_int`, `to_float`, `sin`, `cos`, `tan`, `asin`,
|
||||
`acos`, `atan`, `exp`, `ln`, `log`, `PI()`, `E()`.
|
||||
|
||||
**String:** `len`, `is_empty`, `contains`, `starts_with`, `ends_with`,
|
||||
`index_of`, `split`, `trim`, `to_lower`, `to_upper`, `replace`, `chars`,
|
||||
`pad`, `sub_string`, `crop`, `+` (concatenation).
|
||||
|
||||
**Array:** `push`, `pop`, `shift`, `insert`, `remove`, `len`, `clear`,
|
||||
`truncate`, `extend`, `filter`, `map`, `reduce`, `reduce_rev`, `find`,
|
||||
`find_map`, `any`, `all`, `index_of`, `contains`, `sort`, `reverse`,
|
||||
`dedup`, `chunks`, `splice`, `[]` indexing.
|
||||
|
||||
**Map:** `len`, `is_empty`, `contains`, `keys`, `values`, `mixin`,
|
||||
`remove`, `clear`, `fill_with`, `+` (merge), `[]` and `.` access.
|
||||
|
||||
**Blob:** `len`, `push`, `pop`, `clear`, `as_string`, `parse_le_int`,
|
||||
`write_*`, `[]` indexing. Blobs are `Vec<u8>` at the Rust layer.
|
||||
|
||||
**Logging:** `log::trace`, `log::info`, `log::warn`, `log::error` —
|
||||
each takes a message and optionally a structured-data map. (Documented
|
||||
with the SDK contract; mentioned here for completeness.)
|
||||
|
||||
---
|
||||
|
||||
## `regex::` — regular expressions
|
||||
|
||||
Linear-time, no backtracking (powered by the Rust `regex` crate).
|
||||
Patterns compile per call.
|
||||
|
||||
| Function | Description |
|
||||
|---|---|
|
||||
| `regex::is_match(pattern, text) -> bool` | Whether `text` contains a match. |
|
||||
| `regex::find(pattern, text) -> String \| ()` | First match or `()` if none. |
|
||||
| `regex::find_all(pattern, text) -> Array` | All matches as `String` array. |
|
||||
| `regex::replace(pattern, text, replacement) -> String` | Replace first match only. |
|
||||
| `regex::replace_all(pattern, text, replacement) -> String` | Replace every match. |
|
||||
| `regex::split(pattern, text) -> Array` | Split `text` on matches. |
|
||||
| `regex::captures(pattern, text) -> Array \| ()` | `[full, group1, group2, ...]` from the first match; unmatched optional groups appear as `()`. |
|
||||
|
||||
Invalid patterns throw. Use `\\` to escape inside Rhai string literals
|
||||
(`"\\d+"`) or backtick strings to skip escaping (`` `\d+` ``).
|
||||
|
||||
```rhai
|
||||
if regex::is_match(`^/api/v\d+/`, ctx.request.path) {
|
||||
let cap = regex::captures(`/api/v(\d+)/(.+)`, ctx.request.path);
|
||||
let version = cap[1]; // "1"
|
||||
let rest = cap[2]; // "users"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## `random::` — cryptographically-secure randomness
|
||||
|
||||
All randomness comes from `OsRng`. There is deliberately no "fast
|
||||
non-crypto" variant — scripts shouldn't have to pick.
|
||||
|
||||
| Function | Description |
|
||||
|---|---|
|
||||
| `random::int(min, max) -> i64` | Uniform integer in `[min, max]` (inclusive). Throws if `min > max`. |
|
||||
| `random::float() -> f64` | Uniform float in `[0.0, 1.0)`. |
|
||||
| `random::bytes(n) -> Blob` | `n` random bytes. `n` in `0..=65536`. |
|
||||
| `random::string(n) -> String` | `n` random alphanumeric chars (`A-Za-z0-9`). `n` in `0..=4096`. |
|
||||
| `random::uuid() -> String` | UUID v4 in canonical 8-4-4-4-12 form. |
|
||||
|
||||
```rhai
|
||||
let token = random::uuid();
|
||||
let salt = random::bytes(16);
|
||||
let pin = random::int(100000, 999999);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## `time::` — UTC time
|
||||
|
||||
Canonical time value is **milliseconds since the Unix epoch** as `i64`.
|
||||
ISO 8601 / RFC 3339 strings are for I/O. UTC only — no timezone support.
|
||||
|
||||
| Function | Description |
|
||||
|---|---|
|
||||
| `time::now() -> String` | Current UTC time as ISO 8601 with ms (e.g. `"2026-05-30T20:15:00.123Z"`). |
|
||||
| `time::now_ms() -> i64` | Current ms since Unix epoch. |
|
||||
| `time::parse(iso) -> i64` | Parse RFC 3339 / ISO 8601 string to ms. Throws on bad input. |
|
||||
| `time::format(ms) -> String` | Format ms-since-epoch as ISO 8601 with ms precision. |
|
||||
| `time::add_seconds(ms, secs) -> i64` | `ms + secs*1000`, with overflow check. |
|
||||
| `time::diff_seconds(a_ms, b_ms) -> i64` | `(b_ms - a_ms) / 1000`, truncated. |
|
||||
|
||||
```rhai
|
||||
let started_at = time::now_ms();
|
||||
// ... do work ...
|
||||
let elapsed = time::diff_seconds(started_at, time::now_ms());
|
||||
|
||||
let deadline = time::format(time::add_seconds(time::now_ms(), 3600));
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## `json::` — JSON parse and stringify
|
||||
|
||||
| Function | Description |
|
||||
|---|---|
|
||||
| `json::parse(s) -> Dynamic` | Parse a JSON string. Returns Rhai maps, arrays, scalars, or `()` for null. Throws on invalid JSON. |
|
||||
| `json::stringify(v) -> String` | Compact JSON. |
|
||||
| `json::stringify_pretty(v) -> String` | Pretty-printed (2-space indent). |
|
||||
|
||||
```rhai
|
||||
let payload = json::parse(ctx.request.body); // if body came in as a string
|
||||
let body_str = json::stringify(#{ ok: true, items: [1, 2, 3] });
|
||||
```
|
||||
|
||||
Note: `ctx.request.body` is *already* parsed when the request body is
|
||||
`Content-Type: application/json` — only call `json::parse` on raw
|
||||
strings.
|
||||
|
||||
---
|
||||
|
||||
## `base64::` — standard and URL-safe Base64
|
||||
|
||||
Two alphabets: standard (with `=` padding) and URL-safe (no padding).
|
||||
Encoders accept both `String` and `Blob`; decoders always return `Blob`.
|
||||
|
||||
| Function | Description |
|
||||
|---|---|
|
||||
| `base64::encode(input) -> String` | Standard alphabet, padded. `input` is `String` or `Blob`. |
|
||||
| `base64::decode(s) -> Blob` | Decode standard alphabet. Throws on invalid. |
|
||||
| `base64::encode_url(input) -> String` | URL-safe alphabet, **no padding**. |
|
||||
| `base64::decode_url(s) -> Blob` | Decode URL-safe alphabet. Throws on invalid. |
|
||||
|
||||
```rhai
|
||||
let token = base64::encode_url(random::bytes(32)); // URL-safe session token
|
||||
let raw = base64::decode("aGVsbG8=");
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## `hex::` — hexadecimal
|
||||
|
||||
Encode produces lowercase. Decode accepts mixed case.
|
||||
|
||||
| Function | Description |
|
||||
|---|---|
|
||||
| `hex::encode(input) -> String` | Lowercase hex. `input` is `String` or `Blob`. |
|
||||
| `hex::decode(s) -> Blob` | Decode hex (case-insensitive). Throws on invalid. |
|
||||
|
||||
```rhai
|
||||
let fingerprint = hex::encode(random::bytes(20));
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## `url::` — percent-encoding
|
||||
|
||||
Unreserved set per RFC 3986 (`A-Z`, `a-z`, `0-9`, `-`, `_`, `.`, `~`)
|
||||
is preserved; everything else is percent-encoded.
|
||||
|
||||
| Function | Description |
|
||||
|---|---|
|
||||
| `url::encode(s) -> String` | Percent-encode a component value. |
|
||||
| `url::decode(s) -> String` | Percent-decode. Throws on invalid UTF-8 in the decoded output. |
|
||||
| `url::encode_query(map) -> String` | Build `k1=v1&k2=v2` from a Map. Both keys and values are percent-encoded. Non-string values are coerced via `to_string()`. |
|
||||
|
||||
`url::encode_query` emits keys in the Map's natural order, which is
|
||||
alphabetical (Rhai's `Map` is a `BTreeMap`). RFC 3986 leaves query
|
||||
parameter ordering unspecified, so this is fine for any conforming
|
||||
consumer; if you need a specific ordering, build the string by hand.
|
||||
|
||||
```rhai
|
||||
let qs = url::encode_query(#{ q: "rust regex", page: 2 });
|
||||
// → "page=2&q=rust%20regex"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## What's not here
|
||||
|
||||
- **Crypto** (sha256/hmac/argon2/encryption) — deferred to a focused
|
||||
later PR.
|
||||
- **Timezones** — UTC only in v1.1.0. Format with an offset upstream
|
||||
if you need local time.
|
||||
- **JWT, YAML, XML, CSV, Markdown** — not planned for v1.1.x.
|
||||
- **Stateful services** (KV, docs, HTTP, cron, files, pubsub, secrets,
|
||||
email, users, queue, invoke) — land per the v1.1.x roadmap in the
|
||||
[blueprint §12](../serverless_cloud_blueprint.md).
|
||||
502
docs/v1.1.x-design-notes.md
Normal file
502
docs/v1.1.x-design-notes.md
Normal file
@@ -0,0 +1,502 @@
|
||||
# v1.1.x design notes — in-flight decisions + revised roadmap
|
||||
|
||||
Planning document for the v1.1.x release series. Companion to:
|
||||
|
||||
- [`serverless_cloud_blueprint.md`](../serverless_cloud_blueprint.md) — authoritative design
|
||||
- [`docs/sdk-shape.md`](sdk-shape.md) — SDK conventions (settled in v1.1.0)
|
||||
- [`docs/stdlib-reference.md`](stdlib-reference.md) — stdlib API (settled in v1.1.0)
|
||||
- [`docs/versioning.md`](versioning.md) — versioning policy (post-1.0 carve-out settled with v1.1.0)
|
||||
|
||||
Items in this doc are either **tentatively decided but not yet shipped** or **open calls awaiting the maintainer's decision**. Once an item ships, its content moves into the blueprint and the corresponding section here gets pruned.
|
||||
|
||||
This document was created at the v1.1.0 → v1.1.1 boundary, capturing the architectural conversations that followed v1.1.0 but haven't yet landed in code or in the blueprint.
|
||||
|
||||
---
|
||||
|
||||
## 1. The three messaging primitives
|
||||
|
||||
PiCloud will expose three distinct messaging concepts. The right way to slice them is along **recipient model** and **delivery semantics**:
|
||||
|
||||
| | Recipients | Durability | Delivery | Retry on script failure | Mental model |
|
||||
|---|---|---|---|---|---|
|
||||
| **`invoke(script_id, args)`** | One **named** script | None (or fire-and-forget durable) | At-most-once sync, or at-least-once async | Caller-controlled via `retry::*` | Function call |
|
||||
| **`pubsub::publish(topic, msg)`** | **All** scripts subscribed via trigger | Through outbox | **At-least-once per subscriber** | Per-subscriber retry up to N, then dead-letter | Fan-out broadcast |
|
||||
| **`queue::enqueue(name, msg)`** | **Exactly one** consumer wins | Durable table | **At-least-once total** | Visibility timeout + nack-on-throw | Work distribution |
|
||||
|
||||
**Critical distinction:** pub/sub and queue both end up at-least-once, but the **subscriber model** differs. Queue: 1 message → 1 delivery record → consumers compete. Pub/sub: 1 message → N delivery records (one per subscriber) → no competition.
|
||||
|
||||
### Pub/sub reframe — drop ephemeral, use the outbox
|
||||
|
||||
The original blueprint plan was pub/sub via Postgres `LISTEN/NOTIFY` (ephemeral, sub-millisecond fan-out). Reframe to **reuse the triggers framework's outbox infrastructure**:
|
||||
|
||||
- `pubsub::publish(topic, msg)` writes to the outbox
|
||||
- Dispatcher fans out one delivery record per subscribed script trigger
|
||||
- Each delivery retried on failure with the same machinery as KV / doc / file triggers
|
||||
- After N retries → dead-letter (see §4)
|
||||
|
||||
**Wins:** one delivery model in the whole system, durable pub/sub for free, shared observability/retry/dead-letter tooling across every event-firing surface.
|
||||
|
||||
**Cost:** ~1ms Postgres write per publish (vs in-memory NOTIFY). For solo-dev / consumer hardware, the right tradeoff. If sub-ms ever matters, `pubsub::publish_ephemeral` is a future addition that bypasses the outbox.
|
||||
|
||||
### Queue stays separate
|
||||
|
||||
Pub/sub-through-outbox cannot model "work distribution with backpressure" cleanly. Queue keeps its own table:
|
||||
|
||||
- Producer: `queue::enqueue(name, msg)` → queue table
|
||||
- Consumer: `queue:receive` trigger fires when message available; runtime claims with `FOR UPDATE SKIP LOCKED` + visibility timeout
|
||||
- Script returns successfully → auto-ack (delete row)
|
||||
- Script throws → auto-nack (clear claim; message becomes visible again)
|
||||
- Visibility timeout exceeded → reclaim allowed (handles crashed consumers)
|
||||
- Max delivery attempts → dead-letter
|
||||
|
||||
The queue table IS the outbox for queue semantics — no double-buffering.
|
||||
|
||||
### Status
|
||||
|
||||
- **Pub/sub via trigger outbox**: leaning yes; needs final ack.
|
||||
- **Queue stays separate from pub/sub**: leaning yes.
|
||||
- **Drop `LISTEN/NOTIFY` plan**: leaning yes.
|
||||
|
||||
### Open calls
|
||||
|
||||
1. Pub/sub durability via trigger outbox (durable, at-least-once) — confirmed?
|
||||
2. Queue and pub/sub stay separate concepts (rather than unifying under a "messaging" abstraction with a subscription-mode flag) — confirmed?
|
||||
|
||||
---
|
||||
|
||||
## 2. Universal trigger outbox
|
||||
|
||||
The triggers framework's outbox should be the universal substrate for **async dispatch**. Every event source that fires scripts asynchronously writes to the same outbox table; one dispatcher reads from it and routes to the executor with shared load control, retry, dead-letter, and trigger-depth tracking.
|
||||
|
||||
### What runs through the outbox
|
||||
|
||||
| Ingress | Path | Reason |
|
||||
|---|---|---|
|
||||
| **HTTP request (sync)** | Direct: orchestrator → executor → response (with NATS-style indirection — see §3) | Caller is waiting; the inbox pattern makes this work via the outbox |
|
||||
| **HTTP request (async, opt-in)** | Orchestrator writes outbox → returns 202 → dispatcher → executor | Webhooks, fire-and-forget endpoints; explicit opt-in via route config |
|
||||
| **Cron tick** | Scheduler writes outbox → dispatcher → executor | No caller; naturally async |
|
||||
| **KV / doc / file change** | Service writes outbox → dispatcher → executor | No caller; the originating script already returned |
|
||||
| **Pub/sub publish** | Service writes outbox → dispatcher → executor (per subscriber) | Fan-out semantics |
|
||||
| **Queue message** | Queue table IS the outbox; dispatcher claims via `FOR UPDATE SKIP LOCKED` | Avoids double-buffering |
|
||||
| **Inbound email** | SMTP receiver writes outbox → dispatcher → executor | No caller |
|
||||
|
||||
### What this gives
|
||||
|
||||
1. **One dispatcher = one place** for load control (the existing `ExecutionGate`), retry, dead-letter, trigger-depth tracking, fan-out. New event source = "write to outbox in this shape", nothing else.
|
||||
2. **Routes become a trigger kind**, conceptually. A route is `(source=http, filter=method+path, script_id, dispatch_mode=sync|async)`. Schema-wise the `routes` table likely stays separate from the new `triggers` table (polymorphic JSON columns get ugly), but the mental model collapses to "everything that fires a script is a trigger".
|
||||
3. **`dispatch_mode = async` is a per-route opt-in**. Webhook handlers can return 202 immediately and process in the background — dispatcher handles retries, caller gets a snappy ack.
|
||||
4. **Replay and debugging.** Every async invocation has an outbox row; admin can re-fire a trigger by re-dispatching the row.
|
||||
5. **Decoupled lifecycle.** Dispatcher can be paused for maintenance without affecting HTTP ingress (it just queues); HTTP can degrade (overflow 503s) without affecting async work already in the outbox.
|
||||
|
||||
### What this doesn't change
|
||||
|
||||
- Sync HTTP still hits the `ExecutionGate` the same way (now via the dispatcher).
|
||||
- Async outbox dispatch also hits the gate when the dispatcher picks a row. Sync and async share the cap on actual blocking-thread-in-use.
|
||||
- Trigger CRUD likely stays in per-kind tables for schema sanity; the unification is conceptual + dispatch-layer, not schema-layer.
|
||||
|
||||
### Status
|
||||
|
||||
- **Universal outbox for async dispatch**: leaning yes.
|
||||
- **Routes-as-trigger conceptually**: leaning yes.
|
||||
- **Routes-as-trigger schema-wise**: leaning no (keep separate tables).
|
||||
- **Per-route `dispatch_mode: sync|async`**: leaning yes for v1.1.1 since the dispatcher is already being built.
|
||||
|
||||
### Open calls
|
||||
|
||||
1. Sync HTTP via outbox + per-request inbox pattern (NATS-style; see §3) — confirmed, or keep direct dispatch for sync?
|
||||
2. Ship `dispatch_mode: async` for HTTP routes in v1.1.1, or defer to a later release?
|
||||
3. Keep `routes` and `triggers` as separate tables (unified at the dispatcher only), or merge schemas?
|
||||
|
||||
---
|
||||
|
||||
## 3. NATS-style request/reply for sync HTTP
|
||||
|
||||
The constraint that makes "universal outbox" tricky: HTTP has a caller waiting. We can't write to outbox, return 202, and walk away — the user's browser expects `200 OK` with body. NATS's request/reply pattern resolves this elegantly.
|
||||
|
||||
### Pattern
|
||||
|
||||
```
|
||||
HTTP request → orchestrator generates inbox_id, registers a oneshot channel
|
||||
→ writes outbox row { source: http, payload, reply_to: inbox_id }
|
||||
→ awaits on the channel (with timeout = script's wall-clock + buffer)
|
||||
|
||||
Dispatcher → picks outbox row
|
||||
→ dispatches to executor (gate + spawn_blocking + Rhai)
|
||||
→ if reply_to.is_some(): resolves the channel with the result
|
||||
→ if reply_to.is_none(): records completion + retries on failure per trigger config
|
||||
|
||||
Orchestrator → channel resolves → returns response to HTTP caller
|
||||
→ on timeout: returns 504 or 500 → see status-code calls below
|
||||
```
|
||||
|
||||
The HTTP caller's experience is unchanged (synchronous request/response). Under the hood, dispatch is identical for every invocation source.
|
||||
|
||||
### Implementation by deployment mode
|
||||
|
||||
| Mode | Mechanism | Trade-off |
|
||||
|---|---|---|
|
||||
| **In-process (v1.1.1, MVP)** | Per-orchestrator `HashMap<InboxId, oneshot::Sender<Result>>`; dispatcher resolves the oneshot | Sub-ms wake-up; fails across process boundaries |
|
||||
| **Cross-process (cluster mode v1.3+)** | Postgres `LISTEN/NOTIFY` keyed on `inbox_id`, with a `responses` row as durable backup | Sub-10ms wake-up; survives across nodes; needs careful long-listener management |
|
||||
| **Polling fallback** | Orchestrator polls `responses` table for `inbox_id` every ~10ms | Simple; ~10ms minimum latency; only as fallback |
|
||||
|
||||
### Latency cost (honest numbers)
|
||||
|
||||
Per sync HTTP request, NATS-style adds: ~1-2ms Postgres write (outbox) + sub-ms dispatcher wake (in-process channel) + ~1ms response resolve = **~2-5ms overhead**. For most scripts (10-100ms execution), this is noise. PiCloud isn't optimizing for sub-ms; the architectural unification is worth a few ms.
|
||||
|
||||
### Retry policy — `reply_to` IS the signal
|
||||
|
||||
| Outbox row | Retry behavior |
|
||||
|---|---|
|
||||
| `reply_to.is_some()` | **Never retry.** Caller is waiting; retrying means the script might run twice and the caller gets one of two outcomes. Always: one attempt, surface result (success or failure) to inbox. |
|
||||
| `reply_to.is_none()` | Retry per trigger's configured policy. Default: 3 attempts, exponential backoff (1s, 2s, 4s), dead-letter after. |
|
||||
|
||||
Per-trigger config lives on the trigger row:
|
||||
|
||||
```
|
||||
trigger { source: cron, schedule: "0 */5 * * * *",
|
||||
retry: { max_attempts: 5, backoff: exponential, base_ms: 1000 } }
|
||||
|
||||
trigger { source: pubsub, topic: "user.created",
|
||||
retry: { max_attempts: 3, backoff: linear, base_ms: 500 } }
|
||||
|
||||
trigger { source: http, method: POST, path: "/api/foo",
|
||||
dispatch_mode: sync } // retry absent — sync HTTP is always 1-attempt
|
||||
```
|
||||
|
||||
### Failure / crash handling
|
||||
|
||||
With NATS-style indirection, there are new ways for a sync HTTP request to vanish. Every failure path must resolve the orchestrator's oneshot channel with something:
|
||||
|
||||
| Failure mode | Detection | Caller sees |
|
||||
|---|---|---|
|
||||
| Script throws / runtime error | Executor returns `ExecError::Runtime` → written to inbox | 502 (or 500 — see status-code discussion) |
|
||||
| Script exceeds wall-clock | `tokio::time::timeout` fires inside dispatcher → written to inbox | 504 (or 500) |
|
||||
| Operation budget exceeded | Executor returns `ExecError::OperationBudgetExceeded` → inbox | 507 (or 500) |
|
||||
| Executor process crashes mid-execution | `JoinError` → `ExecError::Runtime` → inbox | 500 |
|
||||
| Dispatcher process dies between claim and reply | Orchestrator's wait times out | 500 |
|
||||
| Outbox write fails (Postgres unavailable) | Orchestrator never publishes; immediate error | 500 |
|
||||
| Orchestrator's own wait times out unexpectedly | Channel timeout fires before inbox resolves | 504 (or 500) |
|
||||
|
||||
Every path resolves the channel with a result. The orchestrator's outer timeout is the backstop for "dispatcher just died completely".
|
||||
|
||||
### Status code strategy — open question
|
||||
|
||||
Today's orchestrator distinguishes 422 / 502 / 503 / 504 / 507 / 500. User raised "everything should be 500" framing. Two options:
|
||||
|
||||
- **Option A (recommended):** keep existing distinctions. Script crashes → 502, timeouts → 504, overloaded → 503, parse errors → 422, dispatcher vanished → 500. Clients get actionable info.
|
||||
- **Option B:** flatten to 500 for everything that's "platform couldn't return a useful response". Simpler surface; loses actionable distinctions.
|
||||
|
||||
### Status
|
||||
|
||||
- **NATS-style for sync HTTP**: leaning yes; resolves the outbox vs direct-dispatch tension.
|
||||
- **`reply_to` presence as the "don't retry" signal**: leaning yes.
|
||||
- **Default retry policy** (3 attempts, exp backoff 1s/2s/4s): proposed.
|
||||
|
||||
### Open calls
|
||||
|
||||
1. NATS-style request/reply for sync HTTP — confirmed?
|
||||
2. Status code strategy: keep existing distinctions (A, recommended) or flatten to 500 (B)?
|
||||
3. Default retry policy on triggers: 3 attempts with exp backoff (1s/2s/4s), or different defaults?
|
||||
4. Cancel-on-timeout semantics: if orchestrator's wait times out but executor finishes successfully later, do we (a) discard the late result, (b) write it to an "abandoned executions" table for debugging, or (c) attempt to ack the caller late? Leaning (b) — log + discard but keep the row for forensics.
|
||||
|
||||
---
|
||||
|
||||
## 4. Dead-letter handling
|
||||
|
||||
Events that exhaust their retry policy land in a **separate `dead_letters` table** (not a flag on the outbox — outbox should stay a queue with fast inserts and scans). Users handle dead letters by registering a script for the new `dead_letter` **trigger kind**.
|
||||
|
||||
### Schema sketch
|
||||
|
||||
```sql
|
||||
CREATE TABLE dead_letters (
|
||||
id UUID PRIMARY KEY,
|
||||
app_id UUID NOT NULL REFERENCES apps(id) ON DELETE CASCADE,
|
||||
original_event_id UUID NOT NULL, -- the outbox row id
|
||||
source TEXT NOT NULL, -- "kv", "cron", "pubsub", "queue", "email"
|
||||
op TEXT NOT NULL,
|
||||
trigger_id UUID, -- which trigger config fired (null for direct dispatches)
|
||||
script_id UUID, -- which script failed
|
||||
payload JSONB NOT NULL, -- the event payload, verbatim
|
||||
attempt_count INT NOT NULL,
|
||||
first_attempt_at TIMESTAMPTZ NOT NULL,
|
||||
last_attempt_at TIMESTAMPTZ NOT NULL,
|
||||
last_error TEXT NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
resolved_at TIMESTAMPTZ, -- null = unresolved
|
||||
resolution TEXT -- "replayed" | "ignored" | "handled_by_script" | "handler_failed"
|
||||
);
|
||||
|
||||
CREATE INDEX idx_dead_letters_app_unresolved
|
||||
ON dead_letters(app_id) WHERE resolved_at IS NULL;
|
||||
```
|
||||
|
||||
### Dead letter as trigger source
|
||||
|
||||
```
|
||||
trigger {
|
||||
source: dead_letter,
|
||||
filter: { source: "kv" }, -- optional; defaults to "any source"
|
||||
script_id: <your handler>,
|
||||
dispatch_mode: async,
|
||||
retry: { max_attempts: 1 } -- forced — see recursion stop rule below
|
||||
}
|
||||
```
|
||||
|
||||
Filterable on:
|
||||
- `source`: only dead letters from a particular event source (kv, cron, pubsub, …)
|
||||
- `trigger_id`: only dead letters from a particular trigger config
|
||||
- `script_id`: only dead letters from a particular script
|
||||
- No filter: every dead letter fires this handler
|
||||
|
||||
`ctx.event` for a dead-letter handler:
|
||||
|
||||
```rhai
|
||||
ctx.event.source // "dead_letter"
|
||||
ctx.event.dead_letter = #{
|
||||
original: #{
|
||||
source: "kv",
|
||||
op: "insert",
|
||||
collection: "widgets",
|
||||
key: "k1",
|
||||
payload: #{ ... }
|
||||
},
|
||||
attempts: 3,
|
||||
last_error: "script timeout after 30s",
|
||||
trigger_id: "...",
|
||||
script_id: "...",
|
||||
first_attempt_at: "2026-05-30T12:00:00.000Z",
|
||||
last_attempt_at: "2026-05-30T12:00:14.000Z"
|
||||
}
|
||||
```
|
||||
|
||||
The handler can `log::error`, send `email::send` to admins, write to `docs::collection("incidents").create(...)`, post to external alerting via `http::post`, or call `dead_letters::replay(id)` if it decides retry is favorable.
|
||||
|
||||
### Recursion stop rule
|
||||
|
||||
**Dead-letter handlers execute once, no retry, and CANNOT themselves be dead-lettered.**
|
||||
|
||||
When the dispatcher invokes a dead-letter trigger, the resulting execution is marked `is_dead_letter_handler = true`. If it fails:
|
||||
|
||||
- Failure is logged to the structured log (full payload + error)
|
||||
- A metric is bumped (`picloud_dead_letter_handler_failures`)
|
||||
- Original dead-letter row annotated with `resolution = "handler_failed"`
|
||||
- **Nothing else is fired.** Chain stops definitively.
|
||||
|
||||
This is the only safe stop rule. If your alerting script is broken, the platform shouldn't try to alert about that with the same broken script.
|
||||
|
||||
### Defaults
|
||||
|
||||
**No automatic handler.** Dead letters silently land in the table. Users opt into handling by registering a trigger. The dashboard surfaces an unresolved-count badge so users notice.
|
||||
|
||||
This avoids over-engineering — most apps will run for months without a dead-letter trigger; the table is the durable record either way.
|
||||
|
||||
### Sync HTTP failures don't dead-letter
|
||||
|
||||
Sync HTTP requests (`reply_to.is_some()`) failures don't land in `dead_letters`. Caller already got an error response; every failed HTTP request landing in `dead_letters` would flood the table; `execution_logs` already captures sync request failures. If a user wants alerts on HTTP endpoint failures, that's **monitoring** (v1.3+ territory), not dead-lettering.
|
||||
|
||||
### Pub/sub fan-out dead-letters independently
|
||||
|
||||
One `pubsub::publish` → N subscribers → each retries independently → each can independently dead-letter. So one publish can produce N dead-letter rows (one per subscriber that exhausted retries). Subscribers are independent failure domains.
|
||||
|
||||
### Manual replay
|
||||
|
||||
| Surface | Use case |
|
||||
|---|---|
|
||||
| `POST /api/v1/admin/apps/{id}/dead_letters/{dl_id}/replay` | Admin clicks "replay" in dashboard |
|
||||
| `dead_letters::replay(id)` Rhai SDK | A handler script decides to retry programmatically |
|
||||
| `dead_letters::resolve(id, reason)` Rhai SDK | A handler decides "this is fine, don't bother me" |
|
||||
|
||||
Replay re-inserts the original event into the outbox; dispatcher tries again from scratch.
|
||||
|
||||
### Retention
|
||||
|
||||
Time-based: delete dead letters older than 30 days by default. Configurable per-app via app settings, or globally via env var (`PICLOUD_DEAD_LETTER_RETENTION_DAYS`). A weekly GC job in the manager handles the deletion using `FOR UPDATE SKIP LOCKED`.
|
||||
|
||||
### Status
|
||||
|
||||
- **Separate `dead_letters` table**: leaning yes.
|
||||
- **`dead_letter` as trigger kind**: leaning yes.
|
||||
- **Recursion stop rule** (handlers can't be dead-lettered): leaning yes.
|
||||
- **No default handler** (rows just sit in table): leaning yes.
|
||||
- **Sync HTTP failures don't dead-letter**: leaning yes.
|
||||
|
||||
### Open calls
|
||||
|
||||
1. Dead-letter handlers unretryable + can't be dead-lettered themselves — confirmed?
|
||||
2. No default dead-letter handler (rows just sit in the table); user opts in — confirmed, or do you want a built-in "log to admin notifications channel" default?
|
||||
3. 30-day default retention sensible, or longer/shorter?
|
||||
4. Include Rhai SDK (`dead_letters::list/replay/resolve`) in v1.1.1 alongside admin endpoints, or defer the script-side surface to a later release?
|
||||
|
||||
---
|
||||
|
||||
## 5. Realtime updates for external clients
|
||||
|
||||
Apps built on PiCloud need a way for browser/mobile clients to receive live updates (chat messages, dashboard data, multiplayer state, notifications). Today's pub/sub is internal-only (script ↔ script via triggers).
|
||||
|
||||
### The chosen approach
|
||||
|
||||
**Option C (from prior debate): topics with opt-in external subscription.**
|
||||
|
||||
- One `pubsub::publish(topic, msg)` API for scripts — produces a single event
|
||||
- Topics are **internal-only by default** — script triggers can subscribe; external clients cannot
|
||||
- Apps explicitly mark topics as externally-subscribable (per-topic config in dashboard / API)
|
||||
- External clients connect to `GET /realtime/topics/{topic}` via SSE and receive only messages published to topics they're permitted to subscribe to
|
||||
|
||||
**Wins:** one publish API for scripts (DRY), topics don't leak by default (security), external visibility is an explicit opt-in per topic.
|
||||
|
||||
### Transport: SSE first
|
||||
|
||||
SSE (Server-Sent Events) for v1.x:
|
||||
|
||||
- Simpler than WebSocket; works through any HTTP proxy without protocol upgrade
|
||||
- Browsers auto-reconnect on disconnect
|
||||
- Sufficient for "server-pushed events to the browser" (the dominant use case)
|
||||
|
||||
WebSocket is added later if bidirectional comms (chat-style) warrant it.
|
||||
|
||||
### Auth model for external subscribers
|
||||
|
||||
Three flavors, ordered by complexity:
|
||||
|
||||
- **Public topics** — anyone with the URL connects. For marketing-style broadcasts, public stat boards.
|
||||
- **Token-gated topics** — client presents a token issued by a script. Pusher / Ably-style. Token can be a PiCloud API key (v1.1.6) or a users-SDK session token (v1.1.8+).
|
||||
- **Script-mediated** — a script handles each subscribe request and decides yes/no. Most flexible, defer to v1.2.
|
||||
|
||||
Ship public + token-gated in v1.1.6; defer script-mediated.
|
||||
|
||||
### Status
|
||||
|
||||
- **Approach C (opt-in external subscription)**: leaning yes.
|
||||
- **SSE first, WebSocket later**: leaning yes.
|
||||
- **Public + token-gated auth in v1.1.6**: leaning yes.
|
||||
|
||||
### Open calls
|
||||
|
||||
1. Approach C confirmed (vs A: pubsub IS realtime, B: separate `channels::` service)?
|
||||
2. SSE first, WebSocket deferred — confirmed, or ship both in v1.1.6?
|
||||
3. Auth: public + API-key gating in v1.1.6, defer users-SDK-based tokens to v1.1.8 follow-up — confirmed?
|
||||
|
||||
---
|
||||
|
||||
## 6. Frontend client library
|
||||
|
||||
Strategic positioning question: how much should PiCloud expose to frontend developers building apps on top of it?
|
||||
|
||||
### The two ends of the spectrum
|
||||
|
||||
| End | Frontend gets | Examples |
|
||||
|---|---|---|
|
||||
| **Minimalist** | HTTP to dev-defined script endpoints + SSE on dev-marked-public topics. Nothing else. | AWS Lambda + API Gateway, Cloudflare Workers, Deno Deploy |
|
||||
| **Maximalist** | Direct client-side access to KV/docs/users/files. Frontend writes `kv.get()`, `docs.find()`, no Rhai script for trivial reads. | Firebase, Supabase, AWS Amplify |
|
||||
|
||||
PiCloud today sits at the minimalist end (services exist for scripts to use, not for frontends). Crossing to maximalist would be a real product pivot, not a feature add.
|
||||
|
||||
### The chosen approach: hybrid
|
||||
|
||||
**Ship a client library that talks to scripts, not to services.** Specifically, three things:
|
||||
|
||||
1. **Typed HTTP client to dev-defined endpoints** — `picloud.endpoint('/api/users').post({ name: 'alice' })`. Fetch wrapper with auth header injection, retry logic, structured error handling.
|
||||
2. **SSE subscription** — `picloud.subscribe('chat-room-123', msg => …)`. Auto-reconnect, token refresh, backpressure.
|
||||
3. **Auth flow helpers** — `picloud.auth.login(email, password)`, `picloud.auth.logout()`, `picloud.auth.token`. These call **dev-defined** endpoints under the hood (`/api/auth/login` etc.); the lib just standardizes the dance + token storage.
|
||||
|
||||
Crucially: **no `picloud.kv.get()` or `picloud.docs.find()` from the frontend.** Those stay server-side, behind dev-written Rhai scripts.
|
||||
|
||||
### Why hybrid, not maximalist
|
||||
|
||||
Firebase trades security for DX; the security-rule misconfiguration footgun is the #1 cause of accidental data exposure in serverless apps. PiCloud's "solo dev / consumer hardware" audience does not have the operational capacity to defend a Firebase-style attack surface against misconfiguration. The script layer is also where PiCloud differentiates — if frontends bypass scripts to talk directly to services, we're competing with Supabase head-to-head (unwinnable, they're better-resourced and have a 5-year head start).
|
||||
|
||||
### Why hybrid, not pure minimalist
|
||||
|
||||
A frontend dev shouldn't have to hand-roll fetch wrappers, SSE reconnect logic, and token-refresh dances. That stuff is identical across every app. Shipping it as `@picloud/client` is genuinely valuable — it doesn't expand the security surface (scripts still gate everything), it just removes boilerplate.
|
||||
|
||||
### TypeScript first
|
||||
|
||||
Ship TypeScript first. Cross-language story (Python, Swift, Kotlin, Rust, …) deferred until demand emerges. TS covers the dominant "web app + mobile via React Native" segment.
|
||||
|
||||
### Status
|
||||
|
||||
- **Hybrid model (frontend through scripts only)**: leaning yes.
|
||||
- **TypeScript first, other languages deferred**: leaning yes.
|
||||
- **Co-ship with realtime as v1.1.6**: leaning yes (SSE wrapper is the killer feature of the lib).
|
||||
|
||||
### Open calls
|
||||
|
||||
1. Hybrid model — confirmed, or do you want to seriously evaluate Firebase-mode?
|
||||
2. TypeScript first, multi-language deferred — confirmed?
|
||||
3. Co-ship realtime + client lib as v1.1.6, or split (server in v1.1.6, client lib later)?
|
||||
4. Type safety: hand-written types only, or aim for codegen from script-declared schemas? Codegen is big — defer to v1.2+ if at all?
|
||||
|
||||
---
|
||||
|
||||
## 7. Revised v1.1.x roadmap
|
||||
|
||||
Net changes vs the [blueprint §12](../serverless_cloud_blueprint.md) roadmap:
|
||||
|
||||
- **v1.1.5 pub/sub**: now via trigger outbox (drops `LISTEN/NOTIFY` plan), tightening implementation scope
|
||||
- **NEW v1.1.6 Realtime Channels & Client Library**: realtime SSE + `@picloud/client` TS package; co-shipped
|
||||
- **v1.1.7+ items shifted by one** (was v1.1.6/7/8 → now v1.1.7/8/9)
|
||||
- **Dead letters and the unified outbox/dispatcher** are absorbed into v1.1.1's existing scope (triggers framework)
|
||||
|
||||
| Version | Capability |
|
||||
|---|---|
|
||||
| **v1.1.0** | **Foundation & Standard Library** — SDK shape, `Services` bundle, `SdkCallCx`, `ExecutionGate`, `ServiceEventEmitter` trait shape; stdlib utilities (regex, random, time, json, base64, hex, url). ✓ Shipped. |
|
||||
| **v1.1.1** | **Storage & Events** — KV store keyed `(app_id, collection, key)`; triggers framework (universal outbox + dispatcher + NATS-style sync HTTP via inbox + per-trigger retry config + dead-letter table & `dead_letter` trigger source + trigger CRUD + `ctx.event` + depth limit); KV trigger kinds. |
|
||||
| **v1.1.2** | **Documents** — `docs::collection(name).create/find/update/delete/list` with `docs:*` triggers. |
|
||||
| **v1.1.3** | **Modules** — `scripts.kind`, per-app resolver replaces `DummyModuleResolver`, AST cache + dep-graph invalidation. |
|
||||
| **v1.1.4** | **Outbound HTTP & Scheduled Tasks** — `http::*` with SSRF deny-list; cron triggers (small now that the framework exists). |
|
||||
| **v1.1.5** | **Files & Pub/Sub** — filesystem-backed blobs (`files/<app_id>/<id[0:2]>/<id>`) with `files:*` triggers; pub/sub via the universal outbox with `pubsub:*` triggers. |
|
||||
| **v1.1.6** | **Realtime Channels & Client Library** *(new)* — SSE-based external subscription to per-app pub/sub topics (public + API-key auth modes); `@picloud/client` TypeScript package (typed HTTP, SSE subscription, auth helpers). |
|
||||
| **v1.1.7** | **Configuration & Email** *(was v1.1.6)* — encrypted per-app secrets; outbound `email::send/send_html` + inbound `email:receive` trigger. |
|
||||
| **v1.1.8** | **User Management** *(was v1.1.7)* — `users::*` for in-script CRUD, auth, roles, invites, password reset. |
|
||||
| **v1.1.9** | **Durable Queues & Function Composition** *(was v1.1.8)* — `queue::*` with `queue:receive` trigger; `invoke()` + `retry::*` (closures-as-args, re-entrant Rhai). |
|
||||
| **v1.2** | **Workflows & Hierarchies** (per blueprint §Phase 5) — DAG execution, advanced docs query, interceptors, read triggers, audit log, script-mediated realtime auth. |
|
||||
| **v1.3+** | **Scale & Ops** (per blueprint §Phase 6) — cluster mode (NATS-style request/reply swaps to `LISTEN/NOTIFY`), cross-app data sharing, script versioning + rollback, rate limiting, richer auth, metrics, distributed tracing, webhooks, S3, monitoring/alerting on HTTP endpoint failures. |
|
||||
|
||||
The v1.1.9 release marks the end of the v1.1.x expansion cadence. v1.2 is the next minor product bump (phase milestone per [versioning policy](versioning.md)).
|
||||
|
||||
---
|
||||
|
||||
## Consolidated open calls
|
||||
|
||||
Numbered for easy reference in conversation. All currently un-answered.
|
||||
|
||||
### §1 — Messaging primitives
|
||||
1. Pub/sub durability via trigger outbox (durable, at-least-once) — confirmed?
|
||||
2. Queue and pub/sub stay separate concepts (rather than unifying under a "messaging" abstraction with a subscription-mode flag) — confirmed?
|
||||
|
||||
### §2 — Universal trigger outbox
|
||||
3. Sync HTTP via outbox + per-request inbox pattern (NATS-style; see §3) — confirmed, or keep direct dispatch for sync?
|
||||
4. Ship `dispatch_mode: async` for HTTP routes in v1.1.1, or defer to a later release?
|
||||
5. Keep `routes` and `triggers` as separate tables (unified at the dispatcher only), or merge schemas?
|
||||
|
||||
### §3 — NATS-style sync HTTP
|
||||
6. NATS-style request/reply for sync HTTP — confirmed?
|
||||
7. Status code strategy: keep existing distinctions (recommended) or flatten to 500?
|
||||
8. Default retry policy on triggers: 3 attempts with exp backoff (1s/2s/4s), or different defaults?
|
||||
9. Cancel-on-timeout semantics: discard late results (a), write to "abandoned executions" table for debugging (b — recommended), or attempt to ack the caller late (c)?
|
||||
|
||||
### §4 — Dead letters
|
||||
10. Dead-letter handlers unretryable + can't be dead-lettered themselves — confirmed?
|
||||
11. No default dead-letter handler (rows just sit in the table); user opts in — confirmed, or built-in "log to admin notifications channel" default?
|
||||
12. 30-day default retention sensible, or longer/shorter?
|
||||
13. Include Rhai SDK (`dead_letters::list/replay/resolve`) in v1.1.1 alongside admin endpoints, or defer the script-side surface to a later release?
|
||||
|
||||
### §5 — Realtime
|
||||
14. Approach C confirmed (opt-in external subscription on pub/sub topics) vs A: pubsub IS realtime, B: separate `channels::` service?
|
||||
15. SSE first, WebSocket deferred — confirmed, or ship both in v1.1.6?
|
||||
16. Auth: public + API-key gating in v1.1.6, defer users-SDK-based tokens to v1.1.8 follow-up — confirmed?
|
||||
|
||||
### §6 — Frontend client library
|
||||
17. Hybrid model (frontend through scripts only) — confirmed, or seriously evaluate Firebase-mode?
|
||||
18. TypeScript first, multi-language deferred — confirmed?
|
||||
19. Co-ship realtime + client lib as v1.1.6, or split (server in v1.1.6, client lib later)?
|
||||
20. Type safety: hand-written types only, or aim for codegen from script-declared schemas? Defer codegen to v1.2+ if at all?
|
||||
|
||||
---
|
||||
|
||||
## Lifecycle of this document
|
||||
|
||||
- **Created** at the v1.1.0 → v1.1.1 boundary (after the foundation PR series shipped).
|
||||
- **Each section gets pruned** once its decisions ship and land in the blueprint.
|
||||
- **Open calls are answered** in conversation, then folded into the corresponding section as "Decided: X" with the date.
|
||||
- **Document deleted** when v1.1.9 ships — everything by then is either in the blueprint, in code, or explicitly deferred to v1.2+.
|
||||
@@ -14,8 +14,8 @@ All of these carry the same version and are bumped together:
|
||||
|
||||
- Every crate in the Cargo workspace (via `version.workspace = true`)
|
||||
- The dashboard's `package.json`
|
||||
- Docker image tags (`picloud:0.2.0`)
|
||||
- Git tags (`v0.2.0`)
|
||||
- Docker image tags (`picloud:1.1.0`)
|
||||
- Git tags (`v1.1.0`)
|
||||
|
||||
Defined once in [`Cargo.toml`](../Cargo.toml) under `[workspace.package]`. There is no scenario where one crate is at a different version than another in the same build.
|
||||
|
||||
@@ -106,19 +106,15 @@ A versioning scheme without enforcement decays in months. Five cheap mechanical
|
||||
|
||||
## When to bump what
|
||||
|
||||
The product version follows SemVer applied pragmatically — we're pre-1.0, so the rules are looser:
|
||||
The product version uses SemVer with one carve-out for the platform's expansion cadence:
|
||||
|
||||
- **Patch** (`0.2.0 → 0.2.1`) — bug fixes, no surface change
|
||||
- **Minor** (`0.2 → 0.3`) — any surface bump, new features, or breaking changes (pre-1.0 license)
|
||||
- **Major** (`0 → 1`) — first stable release; SDK and API both committed to long-term compatibility
|
||||
- **Major** (`1.x → 2.0`) — surface major bump on a user-facing contract: removed/renamed/retyped SDK function, retired API version, breaking schema change that requires user action, breaking wire-protocol change.
|
||||
- **Minor** (`1.1 → 1.2`) — phase milestone or coherent capability cluster. Bumped when the maintainer marks a release as "the platform moved forward in a way that warrants a number". Typically aligned with blueprint Phase boundaries (Phase 5 → v1.2, Phase 6 → v1.3+).
|
||||
- **Patch** (`1.1.0 → 1.1.1`) — everything else: bug fixes AND **additive-only surface changes**. New SDK function, new admin endpoint, new schema migration that only adds tables/columns, new env var, new trigger kind — all patch.
|
||||
|
||||
After `1.0`, the product version follows strict SemVer based on the *worst* surface change:
|
||||
**Why the carve-out:** PiCloud ships in many small additive PRs (every v1.1.x release adds SDK surface). A strict "minor product bump per minor surface bump" rule would inflate the product version faster than the actual user-perceived "platform changed" milestones warrant. Patch-for-additions keeps the minor digit aligned with capability clusters, not individual feature drops.
|
||||
|
||||
- Any surface major bump → product major bump
|
||||
- Any surface minor bump → product minor bump (at minimum)
|
||||
- No surface changes → product patch
|
||||
|
||||
A surface can hit its own `1.0` independently of the product. The SDK in particular is likely to stabilize before the platform does, since scripts in production demand it.
|
||||
**Surface versions follow their own rules** (table above) and don't track the product version. A surface can independently hit its own `1.0` or `2.0`. The SDK in particular is likely to stabilize before the platform does, since scripts in production demand it.
|
||||
|
||||
---
|
||||
|
||||
@@ -126,7 +122,7 @@ A surface can hit its own `1.0` independently of the product. The SDK in particu
|
||||
|
||||
| | Version |
|
||||
|---|---|
|
||||
| Product | `0.6.0` |
|
||||
| Product | `1.1.0` |
|
||||
| SDK | `1.1` (adds `ctx.request.params`, `ctx.request.query`, `ctx.request.rest`) |
|
||||
| API | `1` (additive: `Script.app_id`, `Route.app_id`, `ExecutionLog.app_id`, new `/api/v1/admin/apps/*` and `/api/v1/admin/api-keys/*` endpoints, `?app=` filter on script list, `Authorization: Bearer pic_…` credential type, 403 responses on previously-401-only admin endpoints when the caller lacks the required capability) |
|
||||
| Schema | `6` (matches `migrations/0006_users_authz.sql`) |
|
||||
@@ -138,15 +134,19 @@ Read live from `GET /version` on any running instance.
|
||||
|
||||
## Examples
|
||||
|
||||
**Adding a `kv.*` SDK in v1.1+:**
|
||||
- Workspace bump: `0.2.0 → 0.3.0` (pre-1.0 minor)
|
||||
- SDK bump: `"1.0" → "1.1"` (added functions only)
|
||||
- API bump: none (no new endpoints affect existing API contract)
|
||||
- Schema bump: `1 → 2` (`0002_kv_store.sql` adds the `kv_store` table)
|
||||
**Adding a `kv.*` SDK in v1.1.1:**
|
||||
- Workspace bump: `1.1.0 → 1.1.1` (patch — additive SDK + schema, no breakage)
|
||||
- SDK bump: `"1.1" → "1.2"` (added functions only)
|
||||
- API bump: none (admin endpoints for trigger CRUD are additive)
|
||||
- Schema bump: `6 → 7` (`0007_kv_store.sql` adds the `kv_store` table)
|
||||
|
||||
**Cutting the v1.2 release (Phase 5: workflows, advanced query, interceptors):**
|
||||
- Workspace bump: `1.1.8 → 1.2.0` (minor — phase milestone)
|
||||
- Even if no individual change is breaking, the maintainer-marked phase transition warrants the minor digit.
|
||||
|
||||
**Renaming `ctx.execution_id` to `ctx.exec_id`:**
|
||||
- SDK bump: `"1.x" → "2.0"` (breaking)
|
||||
- Product: minor bump pre-1.0, major bump post-1.0
|
||||
- SDK bump: `"1.x" → "2.0"` (breaking — removed/retyped script-visible field)
|
||||
- Workspace bump: `1.x.y → 2.0.0` (product major — user-facing contract break)
|
||||
- Migration path: keep `ctx.execution_id` available in 1.x for a deprecation window, add `ctx.exec_id` alongside; flip to 2.0 only when both fields have shipped together for a release.
|
||||
|
||||
**Adding pagination to `GET /api/v1/admin/scripts`:**
|
||||
|
||||
Reference in New Issue
Block a user