test(cli): project-tree journeys + Phase 5 docs
Phase 5 C5. Three `pic ... --dir` tree journeys:
* a group + a nested app apply atomically as one tree (the app's route binds
the group's same-apply `shared` script); the group node's script is
created; re-plan is all-noop (idempotent),
* one invalid node aborts the whole tree — the valid group node's script is
NOT created (true all-or-nothing),
* a `pic groups reparent` between `pic plan --dir` and `pic apply --dir` trips
the bound-plan (StateMoved) check via the tree's structure version.
Docs: `groups-and-project-tool.md` §11.5 status (✅ shipped — single-repo
nested tree apply; multi-repo single-owner + per-env approval gating deferred
per §11.1) + CLAUDE.md current-focus.
Gates: fmt, clippy -D warnings, cargo test --workspace, check-versioning (50
migrations — Phase 5 added none), schema snapshot unchanged, full journey suite
108/108. (Pre-existing flake `queue_e2e::queue_receive_acks_on_success` — a
racy immediate `count==0` after the async ack; flaky at the pre-Phase-5
baseline too, unrelated to this work.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -10,7 +10,7 @@ Authoritative design: [serverless_cloud_blueprint.md](serverless_cloud_blueprint
|
||||
|
||||
**v1.1.x — SDK foundation + services — is complete.** The SDK shape (handle pattern, `::` namespaces, `Services`/`SdkCallCx`; see [docs/sdk-shape.md](docs/sdk-shape.md), stdlib at [docs/stdlib-reference.md](docs/stdlib-reference.md)) fixed in v1.1.0, then KV, docs, modules, HTTP, cron, files, pub/sub, email, users, and durable queues + `invoke()` filled it in through **v1.1.9** — blueprint §12 has the table. Earlier groundwork: blueprint Phase 3 (admin auth, multi-app scoping, Phase 3.5 capability gating — `manager-core::authz::{can, require, Capability}`, migration `0006_users_authz.sql`).
|
||||
|
||||
**Current focus: v1.2 _Hierarchies_ — groups + the declarative project tool** ([docs/design/groups-and-project-tool.md](docs/design/groups-and-project-tool.md)). That doc's §11 uses its own **Phase 1–6 numbering, distinct from the blueprint product-phase numbering above — do not conflate them** (its "Phase 3" = group-inherited config, not admin auth). Implemented on `feat/groups-*` branches: §11 Phase 1 (declarative `pic plan`/`apply`/`prune` + env overlays), Phase 2 (single-parent groups tree + hierarchy-aware RBAC), Phase 3 (group-inherited, env-scoped `vars` + secrets resolved **live** via a recursive CTE — no materialized cache), Phase 4-lite (group-owned **endpoint** scripts: `scripts` polymorphic owner in `0050_group_scripts.sql`, `get_by_name_inherited`/`is_invocable_by_app` chain resolution, inherited `invoke()` + declarative route/trigger binding — all **live**, no body materialization; group modules + the lexical import resolver are Phase 4b). Next: the project tool mapping onto groups (§11 Phase 5).
|
||||
**Current focus: v1.2 _Hierarchies_ — groups + the declarative project tool** ([docs/design/groups-and-project-tool.md](docs/design/groups-and-project-tool.md)). That doc's §11 uses its own **Phase 1–6 numbering, distinct from the blueprint product-phase numbering above — do not conflate them** (its "Phase 3" = group-inherited config, not admin auth). Implemented on `feat/groups-*` branches: §11 Phase 1 (declarative `pic plan`/`apply`/`prune` + env overlays), Phase 2 (single-parent groups tree + hierarchy-aware RBAC), Phase 3 (group-inherited, env-scoped `vars` + secrets resolved **live** via a recursive CTE — no materialized cache), Phase 4-lite (group-owned **endpoint** scripts: `scripts` polymorphic owner in `0050_group_scripts.sql`, `get_by_name_inherited`/`is_invocable_by_app` chain resolution, inherited `invoke()` + declarative route/trigger binding — all **live**, no body materialization; group modules + the lexical import resolver are Phase 4b), Phase 5 (the **declarative project tool maps onto the group tree**: the reconcile engine generalized to `ApplyOwner{App|Group}`, a `[group]` manifest kind, and a single atomic **tree apply** — `pic plan/apply --dir` reconciles a whole directory tree of `picloud.toml` nodes in one Postgres transaction, groups-before-apps so an app route can bind a group script created in the same tx; the bound token folds in each group's `structure_version`. Multi-repo single-owner/attach-point and per-env approval gating are deferred; groups pre-exist). Next: Phase 4b (group modules + lexical import resolver) or §11.6 (group-level collections, v1.3).
|
||||
|
||||
**Data-model invariant:** app-owned data-plane tables (KV, docs, files, …) start with `app_id UUID NOT NULL REFERENCES apps(id) ON DELETE CASCADE`; the group-inheritable tables — _config_ (`vars`, `secrets`) and now group-owned _code_ (`scripts`, `0050`) — instead carry a **polymorphic owner**: nullable `group_id` and `app_id` with an exactly-one CHECK and per-owner partial-unique indexes (config is `ON DELETE CASCADE`, scripts `RESTRICT` — code is not data). Inheritance resolves **live** down `apps.group_id → groups.parent_id` via `CHAIN_LEVELS_CTE` (no materialized view); nearest-owner-wins with an app's own row shadowing the inherited one (CoW). Every Rhai SDK call resolves its app from `cx.app_id`, never a script-passed arg, and a group script always runs under the *inheriting* app's `cx.app_id` (the cross-app isolation boundary).
|
||||
|
||||
|
||||
@@ -38,5 +38,6 @@ mod routes;
|
||||
mod scripts;
|
||||
mod secrets;
|
||||
mod staleness;
|
||||
mod tree;
|
||||
mod triggers;
|
||||
mod vars;
|
||||
|
||||
254
crates/picloud-cli/tests/tree.rs
Normal file
254
crates/picloud-cli/tests/tree.rs
Normal file
@@ -0,0 +1,254 @@
|
||||
//! Phase 5 project-tree apply, end to end via `pic plan/apply --dir`:
|
||||
//! * a group + a nested app under it apply atomically as one tree; the app
|
||||
//! binds a route to the group's inherited script (created in the SAME
|
||||
//! apply); re-plan is a no-op,
|
||||
//! * one invalid node aborts the whole tree (nothing written),
|
||||
//! * a `pic groups reparent` between plan and apply trips the bound-plan
|
||||
//! (StateMoved) check via the tree's structure version.
|
||||
|
||||
use std::fs;
|
||||
|
||||
use predicates::prelude::*;
|
||||
use tempfile::TempDir;
|
||||
|
||||
use crate::common;
|
||||
use crate::common::cleanup::{AppGuard, GroupGuard, ScriptGuard};
|
||||
|
||||
/// A project tree on disk: root group manifest + a nested app manifest that
|
||||
/// binds `GET /greet` to the group's (inherited) `shared` script.
|
||||
fn tree_dir(group: &str, app: &str, group_script_src: &str, app_extra: &str) -> TempDir {
|
||||
let dir = TempDir::new().expect("tempdir");
|
||||
fs::create_dir_all(dir.path().join("scripts")).unwrap();
|
||||
fs::create_dir_all(dir.path().join(app).join("scripts")).unwrap();
|
||||
fs::write(dir.path().join("scripts/shared.rhai"), group_script_src).unwrap();
|
||||
fs::write(
|
||||
dir.path().join("picloud.toml"),
|
||||
format!(
|
||||
"[group]\nslug = \"{group}\"\nname = \"Tree Group\"\n\n\
|
||||
[[scripts]]\nname = \"shared\"\nfile = \"scripts/shared.rhai\"\n\n\
|
||||
[vars]\nregion = \"eu\"\n"
|
||||
),
|
||||
)
|
||||
.unwrap();
|
||||
fs::write(
|
||||
dir.path().join(app).join("picloud.toml"),
|
||||
format!(
|
||||
"[app]\nslug = \"{app}\"\nname = \"Tree App\"\n\n\
|
||||
[[routes]]\nscript = \"shared\"\nmethod = \"GET\"\n\
|
||||
host_kind = \"any\"\npath_kind = \"exact\"\npath = \"/greet\"\n{app_extra}"
|
||||
),
|
||||
)
|
||||
.unwrap();
|
||||
dir
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn tree_applies_group_and_app_atomically_then_noop() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
let group = common::unique_slug("tr-grp");
|
||||
let app = common::unique_slug("tr-app");
|
||||
|
||||
// Group + app must pre-exist (the manifest owns content, not tree shape).
|
||||
let _g = GroupGuard::new(&env.url, &env.token, &group);
|
||||
common::pic_as(&env)
|
||||
.args(["groups", "create", &group])
|
||||
.assert()
|
||||
.success();
|
||||
let _a = AppGuard::new(&env.url, &env.token, &app);
|
||||
common::pic_as(&env)
|
||||
.args(["apps", "create", &app, "--group", &group])
|
||||
.assert()
|
||||
.success();
|
||||
|
||||
let dir = tree_dir(&group, &app, "\"hi from the group\"", "");
|
||||
|
||||
// Plan the whole tree: a group script+var create and an app route create.
|
||||
let plan = String::from_utf8(
|
||||
common::pic_as(&env)
|
||||
.args(["plan", "--dir"])
|
||||
.arg(dir.path())
|
||||
.output()
|
||||
.unwrap()
|
||||
.stdout,
|
||||
)
|
||||
.unwrap();
|
||||
assert!(
|
||||
plan.contains("shared") && plan.contains("region") && plan.contains("/greet"),
|
||||
"tree plan should cover both nodes:\n{plan}"
|
||||
);
|
||||
|
||||
// Apply the whole tree in one transaction.
|
||||
common::pic_as(&env)
|
||||
.args(["apply", "--dir"])
|
||||
.arg(dir.path())
|
||||
.assert()
|
||||
.success();
|
||||
// The group node created its `shared` script (it must drop before its
|
||||
// group at teardown — ON DELETE RESTRICT).
|
||||
let listed = String::from_utf8(
|
||||
common::pic_as(&env)
|
||||
.args(["scripts", "ls", "--group", &group])
|
||||
.output()
|
||||
.unwrap()
|
||||
.stdout,
|
||||
)
|
||||
.unwrap();
|
||||
assert!(
|
||||
listed.contains("shared"),
|
||||
"the group node's script should exist after the tree apply:\n{listed}"
|
||||
);
|
||||
let gscript_id = group_script_id(&env, &group);
|
||||
let _gs = ScriptGuard::new(&env.url, &env.token, &gscript_id);
|
||||
|
||||
// Re-plan is a clean no-op (idempotent across both nodes — incl. the app's
|
||||
// route bound to the inherited group script).
|
||||
let replan = String::from_utf8(
|
||||
common::pic_as(&env)
|
||||
.args(["plan", "--dir"])
|
||||
.arg(dir.path())
|
||||
.output()
|
||||
.unwrap()
|
||||
.stdout,
|
||||
)
|
||||
.unwrap();
|
||||
assert!(
|
||||
!replan.contains("create") && !replan.contains("update"),
|
||||
"re-plan of the tree must be a no-op:\n{replan}"
|
||||
);
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn tree_apply_is_atomic_on_an_invalid_node() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
let group = common::unique_slug("tra-grp");
|
||||
let app = common::unique_slug("tra-app");
|
||||
|
||||
let _g = GroupGuard::new(&env.url, &env.token, &group);
|
||||
common::pic_as(&env)
|
||||
.args(["groups", "create", &group])
|
||||
.assert()
|
||||
.success();
|
||||
let _a = AppGuard::new(&env.url, &env.token, &app);
|
||||
common::pic_as(&env)
|
||||
.args(["apps", "create", &app, "--group", &group])
|
||||
.assert()
|
||||
.success();
|
||||
|
||||
// The app node carries an INVALID Rhai script → the whole tree must abort.
|
||||
let dir = TempDir::new().unwrap();
|
||||
fs::create_dir_all(dir.path().join("scripts")).unwrap();
|
||||
fs::create_dir_all(dir.path().join(&app).join("scripts")).unwrap();
|
||||
fs::write(dir.path().join("scripts/shared.rhai"), "\"ok\"").unwrap();
|
||||
fs::write(dir.path().join(&app).join("scripts/bad.rhai"), "let x = ;").unwrap();
|
||||
fs::write(
|
||||
dir.path().join("picloud.toml"),
|
||||
format!(
|
||||
"[group]\nslug = \"{group}\"\nname = \"G\"\n\n\
|
||||
[[scripts]]\nname = \"shared\"\nfile = \"scripts/shared.rhai\"\n"
|
||||
),
|
||||
)
|
||||
.unwrap();
|
||||
fs::write(
|
||||
dir.path().join(&app).join("picloud.toml"),
|
||||
format!(
|
||||
"[app]\nslug = \"{app}\"\nname = \"A\"\n\n\
|
||||
[[scripts]]\nname = \"bad\"\nfile = \"scripts/bad.rhai\"\n"
|
||||
),
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
common::pic_as(&env)
|
||||
.args(["apply", "--dir"])
|
||||
.arg(dir.path())
|
||||
.assert()
|
||||
.failure();
|
||||
|
||||
// Atomic: the valid group node's `shared` must NOT have been created.
|
||||
let listed = String::from_utf8(
|
||||
common::pic_as(&env)
|
||||
.args(["scripts", "ls", "--group", &group])
|
||||
.output()
|
||||
.unwrap()
|
||||
.stdout,
|
||||
)
|
||||
.unwrap();
|
||||
assert!(
|
||||
!listed.contains("shared"),
|
||||
"a failed tree apply must leave nothing behind:\n{listed}"
|
||||
);
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn tree_reparent_between_plan_and_apply_trips_state_moved() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
let parent = common::unique_slug("trp-par");
|
||||
let group = common::unique_slug("trp-grp");
|
||||
let app = common::unique_slug("trp-app");
|
||||
|
||||
// parent → group → app.
|
||||
let _p = GroupGuard::new(&env.url, &env.token, &parent);
|
||||
common::pic_as(&env)
|
||||
.args(["groups", "create", &parent])
|
||||
.assert()
|
||||
.success();
|
||||
let _g = GroupGuard::new(&env.url, &env.token, &group);
|
||||
common::pic_as(&env)
|
||||
.args(["groups", "create", &group, "--parent", &parent])
|
||||
.assert()
|
||||
.success();
|
||||
let _a = AppGuard::new(&env.url, &env.token, &app);
|
||||
common::pic_as(&env)
|
||||
.args(["apps", "create", &app, "--group", &group])
|
||||
.assert()
|
||||
.success();
|
||||
|
||||
let dir = tree_dir(&group, &app, "\"x\"", "");
|
||||
|
||||
// Record a bound plan for the whole tree.
|
||||
common::pic_as(&env)
|
||||
.args(["plan", "--dir"])
|
||||
.arg(dir.path())
|
||||
.assert()
|
||||
.success();
|
||||
|
||||
// Reparent the group to the instance root — bumps its structure version.
|
||||
common::pic_as(&env)
|
||||
.args(["groups", "reparent", &group])
|
||||
.assert()
|
||||
.success();
|
||||
|
||||
// Apply the recorded plan: the structure moved underneath it → refuse.
|
||||
// (The apply aborts before any write, so no group script is created and the
|
||||
// GroupGuards tear down cleanly.)
|
||||
common::pic_as(&env)
|
||||
.args(["apply", "--dir"])
|
||||
.arg(dir.path())
|
||||
.assert()
|
||||
.failure()
|
||||
.stderr(
|
||||
predicate::str::contains("changed")
|
||||
.or(predicate::str::contains("plan"))
|
||||
.or(predicate::str::contains("409")),
|
||||
);
|
||||
}
|
||||
|
||||
fn group_script_id(env: &common::TestEnv, group: &str) -> String {
|
||||
let ls = common::pic_as(env)
|
||||
.args(["scripts", "ls", "--group", group])
|
||||
.output()
|
||||
.expect("scripts ls --group");
|
||||
common::parse_first_id(std::str::from_utf8(&ls.stdout).unwrap())
|
||||
.expect("group should have one script")
|
||||
}
|
||||
@@ -986,6 +986,37 @@ Resolved items now live inline next to their topic. What genuinely remains:
|
||||
> delete itself stays `RESTRICT`).
|
||||
5. **Project tool maps onto groups.** Nested manifests, attach point, single-owner, server-computed
|
||||
tree plan, per-env approval gating.
|
||||
|
||||
> **Status (Phase 5): ✅ shipped — single-repo nested tree apply, atomic.** A directory tree of
|
||||
> `picloud.toml` manifests (each declaring an `[app]` or `[group]` node) applies as ONE
|
||||
> server-computed plan in ONE Postgres transaction. Per the §11.1 review, the **multi-repo
|
||||
> single-owner / attach-point / takeover** layer (§7) is **deferred** for the solo-dev / single-repo
|
||||
> start, as is **per-env approval-policy gating** (`[project.environments]`); env overlays
|
||||
> (`picloud.<env>.toml`) already exist. Groups must **pre-exist** (`pic groups create`) — a manifest
|
||||
> owns each node's *content*, not the tree *shape* (declarative group create/reparent is a later
|
||||
> add).
|
||||
>
|
||||
> Shipped surface:
|
||||
> - **Engine:** the reconcile engine generalized from app-only to an `ApplyOwner { App | Group }`
|
||||
> (a group node has only scripts + vars; routes/triggers/secret-values stay app-only). The in-tx
|
||||
> reconcile is `reconcile_node_tx`, shared by single-node and tree apply. `apply_tree` locks every
|
||||
> node key (sorted), reconciles **groups first** (recording each group's `name → id`), then apps —
|
||||
> so an app route/trigger can bind a group script **created in the same transaction** (resolved
|
||||
> nearest-ancestor-wins across the in-tree index + pre-existing ancestors; the pool resolver can't
|
||||
> see uncommitted rows). One commit, one post-commit route refresh.
|
||||
> - **Bound plan covers structure:** the tree token folds every node's `state_token` **plus every
|
||||
> in-scope group's `structure_version`**, so a reparent (or content edit) between plan and apply
|
||||
> trips `StateMoved` — the §4.2 "content **and** tree-structure version" check.
|
||||
> - **API:** `POST /api/v1/admin/{groups/{id},tree}/{plan,apply}`; per-node capability gating (the
|
||||
> actor must hold the read/write caps for **every** node touched).
|
||||
> - **CLI:** a `[group]` manifest kind; `pic plan/apply` on one node (app or group); `pic
|
||||
> plan/apply --dir <root>` discovers + applies the whole tree, with a single `<tree>` bound token.
|
||||
>
|
||||
> Live- and journey-validated: a group + nested app apply atomically (app route binds the group's
|
||||
> same-apply script); re-plan is all-noop; one invalid node aborts the whole tree (nothing written);
|
||||
> a `pic groups reparent` between plan and apply is refused. The §5.1 materialized effective-view +
|
||||
> fan-out invalidation was **not** built — group config/scripts resolve **live** (Phase 3/4),
|
||||
> so the tree apply needs no cache-invalidation protocol.
|
||||
6. **(Much later) group-level collections/topics** — the v1.3 cross-app data-sharing problem, with a
|
||||
real shared-scope authz model. Optionally, trigger/route **templates** (§4.5) if cardinality
|
||||
demands.
|
||||
|
||||
Reference in New Issue
Block a user