test(cli): group-module lexical-resolution journeys + Phase 4b docs (C5)
Two journeys against a live server + Postgres: - `group_module_is_lexically_sealed_under_inheritance`: a group module + a group endpoint that imports it, inherited by an app — proves the §5.5 trust boundary (a leaf's same-named module does NOT shadow the inherited endpoint's import) and app-origin CoW (an app endpoint's import resolves the app's module). - `dangling_import_is_rejected_by_plan`: a manifest script importing a non-existent module is a `pic plan` error. Docs: mark §5.5 residual resolved + §11 Phase 4b ✅ in the design doc; update CLAUDE.md current-focus (extension points are the remaining §5.5 piece). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -10,7 +10,7 @@ Authoritative design: [serverless_cloud_blueprint.md](serverless_cloud_blueprint
|
|||||||
|
|
||||||
**v1.1.x — SDK foundation + services — is complete.** The SDK shape (handle pattern, `::` namespaces, `Services`/`SdkCallCx`; see [docs/sdk-shape.md](docs/sdk-shape.md), stdlib at [docs/stdlib-reference.md](docs/stdlib-reference.md)) fixed in v1.1.0, then KV, docs, modules, HTTP, cron, files, pub/sub, email, users, and durable queues + `invoke()` filled it in through **v1.1.9** — blueprint §12 has the table. Earlier groundwork: blueprint Phase 3 (admin auth, multi-app scoping, Phase 3.5 capability gating — `manager-core::authz::{can, require, Capability}`, migration `0006_users_authz.sql`).
|
**v1.1.x — SDK foundation + services — is complete.** The SDK shape (handle pattern, `::` namespaces, `Services`/`SdkCallCx`; see [docs/sdk-shape.md](docs/sdk-shape.md), stdlib at [docs/stdlib-reference.md](docs/stdlib-reference.md)) fixed in v1.1.0, then KV, docs, modules, HTTP, cron, files, pub/sub, email, users, and durable queues + `invoke()` filled it in through **v1.1.9** — blueprint §12 has the table. Earlier groundwork: blueprint Phase 3 (admin auth, multi-app scoping, Phase 3.5 capability gating — `manager-core::authz::{can, require, Capability}`, migration `0006_users_authz.sql`).
|
||||||
|
|
||||||
**Current focus: v1.2 _Hierarchies_ — groups + the declarative project tool** ([docs/design/groups-and-project-tool.md](docs/design/groups-and-project-tool.md)). That doc's §11 uses its own **Phase 1–6 numbering, distinct from the blueprint product-phase numbering above — do not conflate them** (its "Phase 3" = group-inherited config, not admin auth). Implemented on `feat/groups-*` branches: §11 Phase 1 (declarative `pic plan`/`apply`/`prune` + env overlays), Phase 2 (single-parent groups tree + hierarchy-aware RBAC), Phase 3 (group-inherited, env-scoped `vars` + secrets resolved **live** via a recursive CTE — no materialized cache), Phase 4-lite (group-owned **endpoint** scripts: `scripts` polymorphic owner in `0050_group_scripts.sql`, `get_by_name_inherited`/`is_invocable_by_app` chain resolution, inherited `invoke()` + declarative route/trigger binding — all **live**, no body materialization; group modules + the lexical import resolver are Phase 4b), Phase 5 (the **declarative project tool maps onto the group tree**: the reconcile engine generalized to `ApplyOwner{App|Group}`, a `[group]` manifest kind, and a single atomic **tree apply** — `pic plan/apply --dir` reconciles a whole directory tree of `picloud.toml` nodes in one Postgres transaction, groups-before-apps so an app route can bind a group script created in the same tx; the bound token folds in each group's `structure_version`. Multi-repo single-owner/attach-point and per-env approval gating are deferred; groups pre-exist). Next: Phase 4b (group modules + lexical import resolver) or §11.6 (group-level collections, v1.3).
|
**Current focus: v1.2 _Hierarchies_ — groups + the declarative project tool** ([docs/design/groups-and-project-tool.md](docs/design/groups-and-project-tool.md)). That doc's §11 uses its own **Phase 1–6 numbering, distinct from the blueprint product-phase numbering above — do not conflate them** (its "Phase 3" = group-inherited config, not admin auth). Implemented on `feat/groups-*` branches: §11 Phase 1 (declarative `pic plan`/`apply`/`prune` + env overlays), Phase 2 (single-parent groups tree + hierarchy-aware RBAC), Phase 3 (group-inherited, env-scoped `vars` + secrets resolved **live** via a recursive CTE — no materialized cache), Phase 4-lite (group-owned **endpoint** scripts: `scripts` polymorphic owner in `0050_group_scripts.sql`, `get_by_name_inherited`/`is_invocable_by_app` chain resolution, inherited `invoke()` + declarative route/trigger binding — all **live**, no body materialization), Phase 5 (the **declarative project tool maps onto the group tree**: the reconcile engine generalized to `ApplyOwner{App|Group}`, a `[group]` manifest kind, and a single atomic **tree apply** — `pic plan/apply --dir` reconciles a whole directory tree of `picloud.toml` nodes in one Postgres transaction, groups-before-apps so an app route can bind a group script created in the same tx; the bound token folds in each group's `structure_version`. Multi-repo single-owner/attach-point and per-env approval gating are deferred; groups pre-exist), Phase 4b (group **modules** + the **lexical (sealed-by-default) import resolver**, §5.5: owner-polymorphic `ModuleScript`, origin-rooted `ModuleSource::resolve` walking the importing node's chain, `ExecRequest.script_owner` threaded from every dispatch + `invoke()` site, `_source`-driven lexical chaining in `PicloudModuleResolver` with the compiled-module cache re-keyed by `ScriptId`, group modules/imports allowed, single-node dangling-import `plan` check — an inherited group script's imports **seal to the group**, a leaf can't shadow them). Next: §5.5 **opt-in extension points** (the one deferred §5.5 piece) or §11.6 (group-level collections, v1.3).
|
||||||
|
|
||||||
**Data-model invariant:** app-owned data-plane tables (KV, docs, files, …) start with `app_id UUID NOT NULL REFERENCES apps(id) ON DELETE CASCADE`; the group-inheritable tables — _config_ (`vars`, `secrets`) and now group-owned _code_ (`scripts`, `0050`) — instead carry a **polymorphic owner**: nullable `group_id` and `app_id` with an exactly-one CHECK and per-owner partial-unique indexes (config is `ON DELETE CASCADE`, scripts `RESTRICT` — code is not data). Inheritance resolves **live** down `apps.group_id → groups.parent_id` via `CHAIN_LEVELS_CTE` (no materialized view); nearest-owner-wins with an app's own row shadowing the inherited one (CoW). Every Rhai SDK call resolves its app from `cx.app_id`, never a script-passed arg, and a group script always runs under the *inheriting* app's `cx.app_id` (the cross-app isolation boundary).
|
**Data-model invariant:** app-owned data-plane tables (KV, docs, files, …) start with `app_id UUID NOT NULL REFERENCES apps(id) ON DELETE CASCADE`; the group-inheritable tables — _config_ (`vars`, `secrets`) and now group-owned _code_ (`scripts`, `0050`) — instead carry a **polymorphic owner**: nullable `group_id` and `app_id` with an exactly-one CHECK and per-owner partial-unique indexes (config is `ON DELETE CASCADE`, scripts `RESTRICT` — code is not data). Inheritance resolves **live** down `apps.group_id → groups.parent_id` via `CHAIN_LEVELS_CTE` (no materialized view); nearest-owner-wins with an app's own row shadowing the inherited one (CoW). Every Rhai SDK call resolves its app from `cx.app_id`, never a script-passed arg, and a group script always runs under the *inheriting* app's `cx.app_id` (the cross-app isolation boundary).
|
||||||
|
|
||||||
|
|||||||
@@ -23,6 +23,7 @@ mod dead_letters;
|
|||||||
mod email_queue;
|
mod email_queue;
|
||||||
mod enabled;
|
mod enabled;
|
||||||
mod env_overlay;
|
mod env_overlay;
|
||||||
|
mod group_modules;
|
||||||
mod group_scripts;
|
mod group_scripts;
|
||||||
mod group_secrets;
|
mod group_secrets;
|
||||||
mod groups;
|
mod groups;
|
||||||
|
|||||||
218
crates/picloud-cli/tests/group_modules.rs
Normal file
218
crates/picloud-cli/tests/group_modules.rs
Normal file
@@ -0,0 +1,218 @@
|
|||||||
|
//! Phase 4b group-owned modules + the lexical import resolver (§5.5), e2e via `pic`:
|
||||||
|
//! * a group owns a `module` and an endpoint that imports it; an app under the
|
||||||
|
//! group inherits the endpoint and resolves the module down the chain,
|
||||||
|
//! * **trust boundary** — the inheriting app defines a same-named module of
|
||||||
|
//! its own; the inherited group endpoint's import must STILL bind the
|
||||||
|
//! group's module (a leaf cannot shadow it),
|
||||||
|
//! * **CoW / app origin** — an app-owned endpoint importing that name resolves
|
||||||
|
//! the app's module,
|
||||||
|
//! * a manifest whose script imports a non-existent module is a `plan` error.
|
||||||
|
|
||||||
|
use std::fs;
|
||||||
|
|
||||||
|
use tempfile::TempDir;
|
||||||
|
|
||||||
|
use crate::common;
|
||||||
|
use crate::common::cleanup::{AppGuard, GroupGuard, ScriptGuard};
|
||||||
|
|
||||||
|
fn manifest_dir() -> TempDir {
|
||||||
|
let dir = TempDir::new().expect("tempdir");
|
||||||
|
fs::create_dir_all(dir.path().join("scripts")).expect("scripts dir");
|
||||||
|
dir
|
||||||
|
}
|
||||||
|
|
||||||
|
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||||
|
#[test]
|
||||||
|
fn group_module_is_lexically_sealed_under_inheritance() {
|
||||||
|
let Some(fx) = common::fixture_or_skip() else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
let env = common::admin_env(fx);
|
||||||
|
let group = common::unique_slug("gm-grp");
|
||||||
|
let child = common::unique_slug("gm-child");
|
||||||
|
|
||||||
|
// GroupGuard first so it drops LAST — after the app + group scripts.
|
||||||
|
let _g = GroupGuard::new(&env.url, &env.token, &group);
|
||||||
|
common::pic_as(&env)
|
||||||
|
.args(["groups", "create", &group])
|
||||||
|
.assert()
|
||||||
|
.success();
|
||||||
|
|
||||||
|
let dir = manifest_dir();
|
||||||
|
|
||||||
|
// Group module `util` and a group endpoint `welcome` that imports it.
|
||||||
|
fs::write(
|
||||||
|
dir.path().join("scripts/util.rhai"),
|
||||||
|
r#"fn greet(n) { "group:" + n }"#,
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
common::pic_as(&env)
|
||||||
|
.args(["scripts", "deploy"])
|
||||||
|
.arg(dir.path().join("scripts/util.rhai"))
|
||||||
|
.args(["--group", &group, "--name", "util", "--kind", "module"])
|
||||||
|
.assert()
|
||||||
|
.success();
|
||||||
|
|
||||||
|
fs::write(
|
||||||
|
dir.path().join("scripts/welcome.rhai"),
|
||||||
|
r#"import "util" as u; u::greet("x")"#,
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
common::pic_as(&env)
|
||||||
|
.args(["scripts", "deploy"])
|
||||||
|
.arg(dir.path().join("scripts/welcome.rhai"))
|
||||||
|
.args(["--group", &group, "--name", "welcome"])
|
||||||
|
.assert()
|
||||||
|
.success();
|
||||||
|
// Group scripts block group deletion (ON DELETE RESTRICT) — guard both.
|
||||||
|
let _gu = ScriptGuard::new(&env.url, &env.token, &group_script_id(&env, &group, "util"));
|
||||||
|
let _gw = ScriptGuard::new(
|
||||||
|
&env.url,
|
||||||
|
&env.token,
|
||||||
|
&group_script_id(&env, &group, "welcome"),
|
||||||
|
);
|
||||||
|
|
||||||
|
// App under the group; a `caller` invokes the inherited `welcome`.
|
||||||
|
let _child = AppGuard::new(&env.url, &env.token, &child);
|
||||||
|
common::pic_as(&env)
|
||||||
|
.args(["apps", "create", &child, "--group", &group])
|
||||||
|
.assert()
|
||||||
|
.success();
|
||||||
|
fs::write(
|
||||||
|
dir.path().join("scripts/caller.rhai"),
|
||||||
|
r#"invoke("welcome", #{})"#,
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
common::pic_as(&env)
|
||||||
|
.args(["scripts", "deploy"])
|
||||||
|
.arg(dir.path().join("scripts/caller.rhai"))
|
||||||
|
.args(["--app", &child, "--name", "caller"])
|
||||||
|
.assert()
|
||||||
|
.success();
|
||||||
|
let caller_id = app_script_id(&env, &child, "caller");
|
||||||
|
|
||||||
|
// Inheritance: welcome resolves `util` from the group → "group:x".
|
||||||
|
assert_eq!(
|
||||||
|
invoke_body(&env, &caller_id),
|
||||||
|
serde_json::json!("group:x"),
|
||||||
|
"inherited endpoint must resolve the group's module"
|
||||||
|
);
|
||||||
|
|
||||||
|
// TRUST BOUNDARY: the app defines its OWN `util` module. The group
|
||||||
|
// endpoint's import must STILL bind the GROUP's util (sealed from below).
|
||||||
|
fs::write(
|
||||||
|
dir.path().join("scripts/apputil.rhai"),
|
||||||
|
r#"fn greet(n) { "app:" + n }"#,
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
common::pic_as(&env)
|
||||||
|
.args(["scripts", "deploy"])
|
||||||
|
.arg(dir.path().join("scripts/apputil.rhai"))
|
||||||
|
.args(["--app", &child, "--name", "util", "--kind", "module"])
|
||||||
|
.assert()
|
||||||
|
.success();
|
||||||
|
assert_eq!(
|
||||||
|
invoke_body(&env, &caller_id),
|
||||||
|
serde_json::json!("group:x"),
|
||||||
|
"a leaf's same-named module must NOT shadow an inherited group endpoint's import"
|
||||||
|
);
|
||||||
|
|
||||||
|
// CoW / app origin: an app-owned endpoint importing `util` gets the APP's.
|
||||||
|
fs::write(
|
||||||
|
dir.path().join("scripts/appcaller.rhai"),
|
||||||
|
r#"import "util" as u; u::greet("y")"#,
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
common::pic_as(&env)
|
||||||
|
.args(["scripts", "deploy"])
|
||||||
|
.arg(dir.path().join("scripts/appcaller.rhai"))
|
||||||
|
.args(["--app", &child, "--name", "appcaller"])
|
||||||
|
.assert()
|
||||||
|
.success();
|
||||||
|
let appcaller_id = app_script_id(&env, &child, "appcaller");
|
||||||
|
assert_eq!(
|
||||||
|
invoke_body(&env, &appcaller_id),
|
||||||
|
serde_json::json!("app:y"),
|
||||||
|
"an app-owned script's import must resolve the app's own module"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||||
|
#[test]
|
||||||
|
fn dangling_import_is_rejected_by_plan() {
|
||||||
|
let Some(fx) = common::fixture_or_skip() else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
let env = common::admin_env(fx);
|
||||||
|
let app = common::unique_slug("gm-dangle");
|
||||||
|
|
||||||
|
let _app = AppGuard::new(&env.url, &env.token, &app);
|
||||||
|
common::pic_as(&env)
|
||||||
|
.args(["apps", "create", &app])
|
||||||
|
.assert()
|
||||||
|
.success();
|
||||||
|
|
||||||
|
// A manifest whose endpoint imports a module that exists nowhere on the
|
||||||
|
// chain. `pic plan` must refuse it (the §5.5 dangling-import check) rather
|
||||||
|
// than apply a script that 404s its import at runtime.
|
||||||
|
let dir = manifest_dir();
|
||||||
|
fs::write(
|
||||||
|
dir.path().join("scripts/broken.rhai"),
|
||||||
|
r#"import "ghost" as g; g::run()"#,
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
let manifest = format!(
|
||||||
|
"[app]\nslug = \"{app}\"\nname = \"Dangle\"\n\n\
|
||||||
|
[[scripts]]\nname = \"broken\"\nfile = \"scripts/broken.rhai\"\n"
|
||||||
|
);
|
||||||
|
let manifest_path = dir.path().join("picloud.toml");
|
||||||
|
fs::write(&manifest_path, &manifest).unwrap();
|
||||||
|
|
||||||
|
let out = common::pic_as(&env)
|
||||||
|
.args(["plan", "--file"])
|
||||||
|
.arg(&manifest_path)
|
||||||
|
.output()
|
||||||
|
.expect("plan");
|
||||||
|
assert!(!out.status.success(), "plan must reject a dangling import");
|
||||||
|
let stderr = String::from_utf8_lossy(&out.stderr).to_lowercase();
|
||||||
|
assert!(
|
||||||
|
stderr.contains("ghost") || stderr.contains("unknown module") || stderr.contains("import"),
|
||||||
|
"plan error should name the missing module:\n{stderr}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Id of the named script in a group (`pic scripts ls --group <g>`).
|
||||||
|
fn group_script_id(env: &common::TestEnv, group: &str, name: &str) -> String {
|
||||||
|
named_id(env, &["scripts", "ls", "--group", group], name)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Id of the named script in an app (`pic scripts ls --app <a>`).
|
||||||
|
fn app_script_id(env: &common::TestEnv, app: &str, name: &str) -> String {
|
||||||
|
named_id(env, &["scripts", "ls", "--app", app], name)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Run a `scripts ls` and pick the id of the row whose name column matches.
|
||||||
|
fn named_id(env: &common::TestEnv, args: &[&str], name: &str) -> String {
|
||||||
|
let ls = common::pic_as(env).args(args).output().expect("scripts ls");
|
||||||
|
let table = String::from_utf8(ls.stdout).unwrap();
|
||||||
|
// Rows are `id\t<owner_slug>\tname\tversion\tupdated_at`.
|
||||||
|
table
|
||||||
|
.lines()
|
||||||
|
.map(common::cells)
|
||||||
|
.find(|c| c.get(2) == Some(&name))
|
||||||
|
.and_then(|c| c.first().map(|s| (*s).to_string()))
|
||||||
|
.unwrap_or_else(|| panic!("script `{name}` not found:\n{table}"))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn invoke_body(env: &common::TestEnv, id: &str) -> serde_json::Value {
|
||||||
|
let out = common::pic_as(env)
|
||||||
|
.args(["scripts", "invoke", id])
|
||||||
|
.output()
|
||||||
|
.expect("scripts invoke");
|
||||||
|
assert!(
|
||||||
|
out.status.success(),
|
||||||
|
"invoke failed: {}",
|
||||||
|
String::from_utf8_lossy(&out.stderr)
|
||||||
|
);
|
||||||
|
serde_json::from_slice(&out.stdout).expect("invoke body is JSON")
|
||||||
|
}
|
||||||
@@ -545,11 +545,16 @@ trust inversion). The rule:
|
|||||||
extension point with no provider in a given app is an error for that app (a hard failure, joining
|
extension point with no provider in a given app is an error for that app (a hard failure, joining
|
||||||
§4.7).
|
§4.7).
|
||||||
|
|
||||||
> **Residual (verified):** executor-core's `PicloudModuleResolver` is app-scoped today and ignores the
|
> **Resolved (Phase 4b ✅).** The **lexical (sealed-by-default)** core shipped: `ModuleScript` carries
|
||||||
> importing script's origin (`module_resolver.rs` passes `_source` unused). Rhai *does* expose that
|
> a polymorphic owner; `ModuleSource::resolve(origin, name)` walks the chain rooted at the importing
|
||||||
> origin, so the lexical-vs-dynamic split is expressible — but it requires re-keying the resolver cache
|
> node (app-rooted `CHAIN_LEVELS_CTE` or the new group-rooted CTE); the resolved script's owner threads
|
||||||
> by owner identity and adding per-import policy (sealed vs. extension point), i.e. a real
|
> through `ExecRequest.script_owner` (the executor's `default_origin`) and every dispatch + `invoke()`
|
||||||
> resolver+cache redesign, not a parameter tweak. Lands with phasing step 4.
|
> site; the `PicloudModuleResolver` reads the importing node from Rhai's `_source` (set to each module's
|
||||||
|
> owner via `AST::set_source(encode(owner))` before `eval_ast_as_new` — the lexical chaining), and the
|
||||||
|
> cache is re-keyed by resolved `ScriptId`. Group modules + group-script imports are now allowed
|
||||||
|
> (`group_scripts_api`), and the single-node apply runs the §5.5 dangling-import `plan` check. **Opt-in
|
||||||
|
> extension points** (the dynamic-resolution branch + `[extension_points]` manifest declaration) remain
|
||||||
|
> the one deferred piece of §5.5 — a clean additive follow-up on top of the now-origin-aware resolver.
|
||||||
|
|
||||||
### 5.6 Tree lifecycle: delete, reparent, rename
|
### 5.6 Tree lifecycle: delete, reparent, rename
|
||||||
|
|
||||||
@@ -978,10 +983,16 @@ Resolved items now live inline next to their topic. What genuinely remains:
|
|||||||
> endpoint (`resolve_inherited_targets` → `name_to_id`), with the diff resolving the group-bound
|
> endpoint (`resolve_inherited_targets` → `name_to_id`), with the diff resolving the group-bound
|
||||||
> route's id → name so **re-apply is idempotent**.
|
> route's id → name so **re-apply is idempotent**.
|
||||||
>
|
>
|
||||||
> Deliberate Phase-4-lite limits (deferred to Phase 4b): group **modules** + the origin-aware
|
> **Phase 4b: ✅ shipped — group modules + the lexical (sealed-by-default) import resolver (§5.5).**
|
||||||
> (lexical) **import** resolver (§5.5) — group scripts must be self-contained; **invoke-by-id** stays
|
> Group `kind = module` scripts and group-script imports are now allowed; `import` resolves lexically
|
||||||
> app-scoped (inheritance is by-name only); CoW is **redefine-in-app + re-apply** (no live
|
> against the importing script's **own defining node** (the group for an inherited script — a leaf
|
||||||
> auto-rebinding). Sharp edge to track: `routes.script_id` is `ON DELETE CASCADE`, so deleting a
|
> can't shadow it; verified e2e). Mechanism: owner-polymorphic `ModuleScript`, origin-rooted
|
||||||
|
> `ModuleSource::resolve`, `ExecRequest.script_owner` threaded from every dispatch + `invoke()` site,
|
||||||
|
> `_source`-driven lexical chaining in the resolver (cache re-keyed by `ScriptId`), and the
|
||||||
|
> single-node dangling-import `plan` check. Still deferred: **opt-in extension points** (the only
|
||||||
|
> remaining §5.5 piece) and **invoke-by-id** staying app-scoped (inheritance is by-name only); CoW
|
||||||
|
> is **redefine-in-app + re-apply** (no live auto-rebinding). Sharp edge to track: `routes.script_id`
|
||||||
|
> is `ON DELETE CASCADE`, so deleting a
|
||||||
> group script removes descendant apps' bound routes (within group-editor authority; the *group*
|
> group script removes descendant apps' bound routes (within group-editor authority; the *group*
|
||||||
> delete itself stays `RESTRICT`).
|
> delete itself stays `RESTRICT`).
|
||||||
5. **Project tool maps onto groups.** Nested manifests, attach point, single-owner, server-computed
|
5. **Project tool maps onto groups.** Nested manifests, attach point, single-owner, server-computed
|
||||||
|
|||||||
Reference in New Issue
Block a user