From 3ef3038eb7611f52b4ee3dfb76e5ebaa52b6b3f8 Mon Sep 17 00:00:00 2001 From: MechaCat02 Date: Thu, 25 Jun 2026 22:43:20 +0200 Subject: [PATCH] test(cli): project-tree journeys + Phase 5 docs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Phase 5 C5. Three `pic ... --dir` tree journeys: * a group + a nested app apply atomically as one tree (the app's route binds the group's same-apply `shared` script); the group node's script is created; re-plan is all-noop (idempotent), * one invalid node aborts the whole tree — the valid group node's script is NOT created (true all-or-nothing), * a `pic groups reparent` between `pic plan --dir` and `pic apply --dir` trips the bound-plan (StateMoved) check via the tree's structure version. Docs: `groups-and-project-tool.md` §11.5 status (✅ shipped — single-repo nested tree apply; multi-repo single-owner + per-env approval gating deferred per §11.1) + CLAUDE.md current-focus. Gates: fmt, clippy -D warnings, cargo test --workspace, check-versioning (50 migrations — Phase 5 added none), schema snapshot unchanged, full journey suite 108/108. (Pre-existing flake `queue_e2e::queue_receive_acks_on_success` — a racy immediate `count==0` after the async ack; flaky at the pre-Phase-5 baseline too, unrelated to this work.) Co-Authored-By: Claude Opus 4.8 (1M context) --- CLAUDE.md | 2 +- crates/picloud-cli/tests/cli.rs | 1 + crates/picloud-cli/tests/tree.rs | 254 +++++++++++++++++++++++++ docs/design/groups-and-project-tool.md | 31 +++ 4 files changed, 287 insertions(+), 1 deletion(-) create mode 100644 crates/picloud-cli/tests/tree.rs diff --git a/CLAUDE.md b/CLAUDE.md index 7822b2f..389ab6d 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -10,7 +10,7 @@ Authoritative design: [serverless_cloud_blueprint.md](serverless_cloud_blueprint **v1.1.x — SDK foundation + services — is complete.** The SDK shape (handle pattern, `::` namespaces, `Services`/`SdkCallCx`; see [docs/sdk-shape.md](docs/sdk-shape.md), stdlib at [docs/stdlib-reference.md](docs/stdlib-reference.md)) fixed in v1.1.0, then KV, docs, modules, HTTP, cron, files, pub/sub, email, users, and durable queues + `invoke()` filled it in through **v1.1.9** — blueprint §12 has the table. Earlier groundwork: blueprint Phase 3 (admin auth, multi-app scoping, Phase 3.5 capability gating — `manager-core::authz::{can, require, Capability}`, migration `0006_users_authz.sql`). -**Current focus: v1.2 _Hierarchies_ — groups + the declarative project tool** ([docs/design/groups-and-project-tool.md](docs/design/groups-and-project-tool.md)). That doc's §11 uses its own **Phase 1–6 numbering, distinct from the blueprint product-phase numbering above — do not conflate them** (its "Phase 3" = group-inherited config, not admin auth). Implemented on `feat/groups-*` branches: §11 Phase 1 (declarative `pic plan`/`apply`/`prune` + env overlays), Phase 2 (single-parent groups tree + hierarchy-aware RBAC), Phase 3 (group-inherited, env-scoped `vars` + secrets resolved **live** via a recursive CTE — no materialized cache), Phase 4-lite (group-owned **endpoint** scripts: `scripts` polymorphic owner in `0050_group_scripts.sql`, `get_by_name_inherited`/`is_invocable_by_app` chain resolution, inherited `invoke()` + declarative route/trigger binding — all **live**, no body materialization; group modules + the lexical import resolver are Phase 4b). Next: the project tool mapping onto groups (§11 Phase 5). +**Current focus: v1.2 _Hierarchies_ — groups + the declarative project tool** ([docs/design/groups-and-project-tool.md](docs/design/groups-and-project-tool.md)). That doc's §11 uses its own **Phase 1–6 numbering, distinct from the blueprint product-phase numbering above — do not conflate them** (its "Phase 3" = group-inherited config, not admin auth). Implemented on `feat/groups-*` branches: §11 Phase 1 (declarative `pic plan`/`apply`/`prune` + env overlays), Phase 2 (single-parent groups tree + hierarchy-aware RBAC), Phase 3 (group-inherited, env-scoped `vars` + secrets resolved **live** via a recursive CTE — no materialized cache), Phase 4-lite (group-owned **endpoint** scripts: `scripts` polymorphic owner in `0050_group_scripts.sql`, `get_by_name_inherited`/`is_invocable_by_app` chain resolution, inherited `invoke()` + declarative route/trigger binding — all **live**, no body materialization; group modules + the lexical import resolver are Phase 4b), Phase 5 (the **declarative project tool maps onto the group tree**: the reconcile engine generalized to `ApplyOwner{App|Group}`, a `[group]` manifest kind, and a single atomic **tree apply** — `pic plan/apply --dir` reconciles a whole directory tree of `picloud.toml` nodes in one Postgres transaction, groups-before-apps so an app route can bind a group script created in the same tx; the bound token folds in each group's `structure_version`. Multi-repo single-owner/attach-point and per-env approval gating are deferred; groups pre-exist). Next: Phase 4b (group modules + lexical import resolver) or §11.6 (group-level collections, v1.3). **Data-model invariant:** app-owned data-plane tables (KV, docs, files, …) start with `app_id UUID NOT NULL REFERENCES apps(id) ON DELETE CASCADE`; the group-inheritable tables — _config_ (`vars`, `secrets`) and now group-owned _code_ (`scripts`, `0050`) — instead carry a **polymorphic owner**: nullable `group_id` and `app_id` with an exactly-one CHECK and per-owner partial-unique indexes (config is `ON DELETE CASCADE`, scripts `RESTRICT` — code is not data). Inheritance resolves **live** down `apps.group_id → groups.parent_id` via `CHAIN_LEVELS_CTE` (no materialized view); nearest-owner-wins with an app's own row shadowing the inherited one (CoW). Every Rhai SDK call resolves its app from `cx.app_id`, never a script-passed arg, and a group script always runs under the *inheriting* app's `cx.app_id` (the cross-app isolation boundary). diff --git a/crates/picloud-cli/tests/cli.rs b/crates/picloud-cli/tests/cli.rs index 2129c12..102f1ca 100644 --- a/crates/picloud-cli/tests/cli.rs +++ b/crates/picloud-cli/tests/cli.rs @@ -38,5 +38,6 @@ mod routes; mod scripts; mod secrets; mod staleness; +mod tree; mod triggers; mod vars; diff --git a/crates/picloud-cli/tests/tree.rs b/crates/picloud-cli/tests/tree.rs new file mode 100644 index 0000000..cb7ac49 --- /dev/null +++ b/crates/picloud-cli/tests/tree.rs @@ -0,0 +1,254 @@ +//! Phase 5 project-tree apply, end to end via `pic plan/apply --dir`: +//! * a group + a nested app under it apply atomically as one tree; the app +//! binds a route to the group's inherited script (created in the SAME +//! apply); re-plan is a no-op, +//! * one invalid node aborts the whole tree (nothing written), +//! * a `pic groups reparent` between plan and apply trips the bound-plan +//! (StateMoved) check via the tree's structure version. + +use std::fs; + +use predicates::prelude::*; +use tempfile::TempDir; + +use crate::common; +use crate::common::cleanup::{AppGuard, GroupGuard, ScriptGuard}; + +/// A project tree on disk: root group manifest + a nested app manifest that +/// binds `GET /greet` to the group's (inherited) `shared` script. +fn tree_dir(group: &str, app: &str, group_script_src: &str, app_extra: &str) -> TempDir { + let dir = TempDir::new().expect("tempdir"); + fs::create_dir_all(dir.path().join("scripts")).unwrap(); + fs::create_dir_all(dir.path().join(app).join("scripts")).unwrap(); + fs::write(dir.path().join("scripts/shared.rhai"), group_script_src).unwrap(); + fs::write( + dir.path().join("picloud.toml"), + format!( + "[group]\nslug = \"{group}\"\nname = \"Tree Group\"\n\n\ + [[scripts]]\nname = \"shared\"\nfile = \"scripts/shared.rhai\"\n\n\ + [vars]\nregion = \"eu\"\n" + ), + ) + .unwrap(); + fs::write( + dir.path().join(app).join("picloud.toml"), + format!( + "[app]\nslug = \"{app}\"\nname = \"Tree App\"\n\n\ + [[routes]]\nscript = \"shared\"\nmethod = \"GET\"\n\ + host_kind = \"any\"\npath_kind = \"exact\"\npath = \"/greet\"\n{app_extra}" + ), + ) + .unwrap(); + dir +} + +#[ignore = "needs DATABASE_URL pointing at a running Postgres"] +#[test] +fn tree_applies_group_and_app_atomically_then_noop() { + let Some(fx) = common::fixture_or_skip() else { + return; + }; + let env = common::admin_env(fx); + let group = common::unique_slug("tr-grp"); + let app = common::unique_slug("tr-app"); + + // Group + app must pre-exist (the manifest owns content, not tree shape). + let _g = GroupGuard::new(&env.url, &env.token, &group); + common::pic_as(&env) + .args(["groups", "create", &group]) + .assert() + .success(); + let _a = AppGuard::new(&env.url, &env.token, &app); + common::pic_as(&env) + .args(["apps", "create", &app, "--group", &group]) + .assert() + .success(); + + let dir = tree_dir(&group, &app, "\"hi from the group\"", ""); + + // Plan the whole tree: a group script+var create and an app route create. + let plan = String::from_utf8( + common::pic_as(&env) + .args(["plan", "--dir"]) + .arg(dir.path()) + .output() + .unwrap() + .stdout, + ) + .unwrap(); + assert!( + plan.contains("shared") && plan.contains("region") && plan.contains("/greet"), + "tree plan should cover both nodes:\n{plan}" + ); + + // Apply the whole tree in one transaction. + common::pic_as(&env) + .args(["apply", "--dir"]) + .arg(dir.path()) + .assert() + .success(); + // The group node created its `shared` script (it must drop before its + // group at teardown — ON DELETE RESTRICT). + let listed = String::from_utf8( + common::pic_as(&env) + .args(["scripts", "ls", "--group", &group]) + .output() + .unwrap() + .stdout, + ) + .unwrap(); + assert!( + listed.contains("shared"), + "the group node's script should exist after the tree apply:\n{listed}" + ); + let gscript_id = group_script_id(&env, &group); + let _gs = ScriptGuard::new(&env.url, &env.token, &gscript_id); + + // Re-plan is a clean no-op (idempotent across both nodes — incl. the app's + // route bound to the inherited group script). + let replan = String::from_utf8( + common::pic_as(&env) + .args(["plan", "--dir"]) + .arg(dir.path()) + .output() + .unwrap() + .stdout, + ) + .unwrap(); + assert!( + !replan.contains("create") && !replan.contains("update"), + "re-plan of the tree must be a no-op:\n{replan}" + ); +} + +#[ignore = "needs DATABASE_URL pointing at a running Postgres"] +#[test] +fn tree_apply_is_atomic_on_an_invalid_node() { + let Some(fx) = common::fixture_or_skip() else { + return; + }; + let env = common::admin_env(fx); + let group = common::unique_slug("tra-grp"); + let app = common::unique_slug("tra-app"); + + let _g = GroupGuard::new(&env.url, &env.token, &group); + common::pic_as(&env) + .args(["groups", "create", &group]) + .assert() + .success(); + let _a = AppGuard::new(&env.url, &env.token, &app); + common::pic_as(&env) + .args(["apps", "create", &app, "--group", &group]) + .assert() + .success(); + + // The app node carries an INVALID Rhai script → the whole tree must abort. + let dir = TempDir::new().unwrap(); + fs::create_dir_all(dir.path().join("scripts")).unwrap(); + fs::create_dir_all(dir.path().join(&app).join("scripts")).unwrap(); + fs::write(dir.path().join("scripts/shared.rhai"), "\"ok\"").unwrap(); + fs::write(dir.path().join(&app).join("scripts/bad.rhai"), "let x = ;").unwrap(); + fs::write( + dir.path().join("picloud.toml"), + format!( + "[group]\nslug = \"{group}\"\nname = \"G\"\n\n\ + [[scripts]]\nname = \"shared\"\nfile = \"scripts/shared.rhai\"\n" + ), + ) + .unwrap(); + fs::write( + dir.path().join(&app).join("picloud.toml"), + format!( + "[app]\nslug = \"{app}\"\nname = \"A\"\n\n\ + [[scripts]]\nname = \"bad\"\nfile = \"scripts/bad.rhai\"\n" + ), + ) + .unwrap(); + + common::pic_as(&env) + .args(["apply", "--dir"]) + .arg(dir.path()) + .assert() + .failure(); + + // Atomic: the valid group node's `shared` must NOT have been created. + let listed = String::from_utf8( + common::pic_as(&env) + .args(["scripts", "ls", "--group", &group]) + .output() + .unwrap() + .stdout, + ) + .unwrap(); + assert!( + !listed.contains("shared"), + "a failed tree apply must leave nothing behind:\n{listed}" + ); +} + +#[ignore = "needs DATABASE_URL pointing at a running Postgres"] +#[test] +fn tree_reparent_between_plan_and_apply_trips_state_moved() { + let Some(fx) = common::fixture_or_skip() else { + return; + }; + let env = common::admin_env(fx); + let parent = common::unique_slug("trp-par"); + let group = common::unique_slug("trp-grp"); + let app = common::unique_slug("trp-app"); + + // parent → group → app. + let _p = GroupGuard::new(&env.url, &env.token, &parent); + common::pic_as(&env) + .args(["groups", "create", &parent]) + .assert() + .success(); + let _g = GroupGuard::new(&env.url, &env.token, &group); + common::pic_as(&env) + .args(["groups", "create", &group, "--parent", &parent]) + .assert() + .success(); + let _a = AppGuard::new(&env.url, &env.token, &app); + common::pic_as(&env) + .args(["apps", "create", &app, "--group", &group]) + .assert() + .success(); + + let dir = tree_dir(&group, &app, "\"x\"", ""); + + // Record a bound plan for the whole tree. + common::pic_as(&env) + .args(["plan", "--dir"]) + .arg(dir.path()) + .assert() + .success(); + + // Reparent the group to the instance root — bumps its structure version. + common::pic_as(&env) + .args(["groups", "reparent", &group]) + .assert() + .success(); + + // Apply the recorded plan: the structure moved underneath it → refuse. + // (The apply aborts before any write, so no group script is created and the + // GroupGuards tear down cleanly.) + common::pic_as(&env) + .args(["apply", "--dir"]) + .arg(dir.path()) + .assert() + .failure() + .stderr( + predicate::str::contains("changed") + .or(predicate::str::contains("plan")) + .or(predicate::str::contains("409")), + ); +} + +fn group_script_id(env: &common::TestEnv, group: &str) -> String { + let ls = common::pic_as(env) + .args(["scripts", "ls", "--group", group]) + .output() + .expect("scripts ls --group"); + common::parse_first_id(std::str::from_utf8(&ls.stdout).unwrap()) + .expect("group should have one script") +} diff --git a/docs/design/groups-and-project-tool.md b/docs/design/groups-and-project-tool.md index f2f2cb5..6b75308 100644 --- a/docs/design/groups-and-project-tool.md +++ b/docs/design/groups-and-project-tool.md @@ -986,6 +986,37 @@ Resolved items now live inline next to their topic. What genuinely remains: > delete itself stays `RESTRICT`). 5. **Project tool maps onto groups.** Nested manifests, attach point, single-owner, server-computed tree plan, per-env approval gating. + + > **Status (Phase 5): ✅ shipped — single-repo nested tree apply, atomic.** A directory tree of + > `picloud.toml` manifests (each declaring an `[app]` or `[group]` node) applies as ONE + > server-computed plan in ONE Postgres transaction. Per the §11.1 review, the **multi-repo + > single-owner / attach-point / takeover** layer (§7) is **deferred** for the solo-dev / single-repo + > start, as is **per-env approval-policy gating** (`[project.environments]`); env overlays + > (`picloud..toml`) already exist. Groups must **pre-exist** (`pic groups create`) — a manifest + > owns each node's *content*, not the tree *shape* (declarative group create/reparent is a later + > add). + > + > Shipped surface: + > - **Engine:** the reconcile engine generalized from app-only to an `ApplyOwner { App | Group }` + > (a group node has only scripts + vars; routes/triggers/secret-values stay app-only). The in-tx + > reconcile is `reconcile_node_tx`, shared by single-node and tree apply. `apply_tree` locks every + > node key (sorted), reconciles **groups first** (recording each group's `name → id`), then apps — + > so an app route/trigger can bind a group script **created in the same transaction** (resolved + > nearest-ancestor-wins across the in-tree index + pre-existing ancestors; the pool resolver can't + > see uncommitted rows). One commit, one post-commit route refresh. + > - **Bound plan covers structure:** the tree token folds every node's `state_token` **plus every + > in-scope group's `structure_version`**, so a reparent (or content edit) between plan and apply + > trips `StateMoved` — the §4.2 "content **and** tree-structure version" check. + > - **API:** `POST /api/v1/admin/{groups/{id},tree}/{plan,apply}`; per-node capability gating (the + > actor must hold the read/write caps for **every** node touched). + > - **CLI:** a `[group]` manifest kind; `pic plan/apply` on one node (app or group); `pic + > plan/apply --dir ` discovers + applies the whole tree, with a single `` bound token. + > + > Live- and journey-validated: a group + nested app apply atomically (app route binds the group's + > same-apply script); re-plan is all-noop; one invalid node aborts the whole tree (nothing written); + > a `pic groups reparent` between plan and apply is refused. The §5.1 materialized effective-view + + > fan-out invalidation was **not** built — group config/scripts resolve **live** (Phase 3/4), + > so the tree apply needs no cache-invalidation protocol. 6. **(Much later) group-level collections/topics** — the v1.3 cross-app data-sharing problem, with a real shared-scope authz model. Optionally, trigger/route **templates** (§4.5) if cardinality demands.