test(cli): project-tree journeys + Phase 5 docs

Phase 5 C5. Three `pic ... --dir` tree journeys:
  * a group + a nested app apply atomically as one tree (the app's route binds
    the group's same-apply `shared` script); the group node's script is
    created; re-plan is all-noop (idempotent),
  * one invalid node aborts the whole tree — the valid group node's script is
    NOT created (true all-or-nothing),
  * a `pic groups reparent` between `pic plan --dir` and `pic apply --dir` trips
    the bound-plan (StateMoved) check via the tree's structure version.

Docs: `groups-and-project-tool.md` §11.5 status ( shipped — single-repo
nested tree apply; multi-repo single-owner + per-env approval gating deferred
per §11.1) + CLAUDE.md current-focus.

Gates: fmt, clippy -D warnings, cargo test --workspace, check-versioning (50
migrations — Phase 5 added none), schema snapshot unchanged, full journey suite
108/108. (Pre-existing flake `queue_e2e::queue_receive_acks_on_success` — a
racy immediate `count==0` after the async ack; flaky at the pre-Phase-5
baseline too, unrelated to this work.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
MechaCat02
2026-06-25 22:43:20 +02:00
parent 1662d7806a
commit 3ef3038eb7
4 changed files with 287 additions and 1 deletions

View File

@@ -10,7 +10,7 @@ Authoritative design: [serverless_cloud_blueprint.md](serverless_cloud_blueprint
**v1.1.x — SDK foundation + services — is complete.** The SDK shape (handle pattern, `::` namespaces, `Services`/`SdkCallCx`; see [docs/sdk-shape.md](docs/sdk-shape.md), stdlib at [docs/stdlib-reference.md](docs/stdlib-reference.md)) fixed in v1.1.0, then KV, docs, modules, HTTP, cron, files, pub/sub, email, users, and durable queues + `invoke()` filled it in through **v1.1.9** — blueprint §12 has the table. Earlier groundwork: blueprint Phase 3 (admin auth, multi-app scoping, Phase 3.5 capability gating — `manager-core::authz::{can, require, Capability}`, migration `0006_users_authz.sql`). **v1.1.x — SDK foundation + services — is complete.** The SDK shape (handle pattern, `::` namespaces, `Services`/`SdkCallCx`; see [docs/sdk-shape.md](docs/sdk-shape.md), stdlib at [docs/stdlib-reference.md](docs/stdlib-reference.md)) fixed in v1.1.0, then KV, docs, modules, HTTP, cron, files, pub/sub, email, users, and durable queues + `invoke()` filled it in through **v1.1.9** — blueprint §12 has the table. Earlier groundwork: blueprint Phase 3 (admin auth, multi-app scoping, Phase 3.5 capability gating — `manager-core::authz::{can, require, Capability}`, migration `0006_users_authz.sql`).
**Current focus: v1.2 _Hierarchies_ — groups + the declarative project tool** ([docs/design/groups-and-project-tool.md](docs/design/groups-and-project-tool.md)). That doc's §11 uses its own **Phase 16 numbering, distinct from the blueprint product-phase numbering above — do not conflate them** (its "Phase 3" = group-inherited config, not admin auth). Implemented on `feat/groups-*` branches: §11 Phase 1 (declarative `pic plan`/`apply`/`prune` + env overlays), Phase 2 (single-parent groups tree + hierarchy-aware RBAC), Phase 3 (group-inherited, env-scoped `vars` + secrets resolved **live** via a recursive CTE — no materialized cache), Phase 4-lite (group-owned **endpoint** scripts: `scripts` polymorphic owner in `0050_group_scripts.sql`, `get_by_name_inherited`/`is_invocable_by_app` chain resolution, inherited `invoke()` + declarative route/trigger binding — all **live**, no body materialization; group modules + the lexical import resolver are Phase 4b). Next: the project tool mapping onto groups (§11 Phase 5). **Current focus: v1.2 _Hierarchies_ — groups + the declarative project tool** ([docs/design/groups-and-project-tool.md](docs/design/groups-and-project-tool.md)). That doc's §11 uses its own **Phase 16 numbering, distinct from the blueprint product-phase numbering above — do not conflate them** (its "Phase 3" = group-inherited config, not admin auth). Implemented on `feat/groups-*` branches: §11 Phase 1 (declarative `pic plan`/`apply`/`prune` + env overlays), Phase 2 (single-parent groups tree + hierarchy-aware RBAC), Phase 3 (group-inherited, env-scoped `vars` + secrets resolved **live** via a recursive CTE — no materialized cache), Phase 4-lite (group-owned **endpoint** scripts: `scripts` polymorphic owner in `0050_group_scripts.sql`, `get_by_name_inherited`/`is_invocable_by_app` chain resolution, inherited `invoke()` + declarative route/trigger binding — all **live**, no body materialization; group modules + the lexical import resolver are Phase 4b), Phase 5 (the **declarative project tool maps onto the group tree**: the reconcile engine generalized to `ApplyOwner{App|Group}`, a `[group]` manifest kind, and a single atomic **tree apply**`pic plan/apply --dir` reconciles a whole directory tree of `picloud.toml` nodes in one Postgres transaction, groups-before-apps so an app route can bind a group script created in the same tx; the bound token folds in each group's `structure_version`. Multi-repo single-owner/attach-point and per-env approval gating are deferred; groups pre-exist). Next: Phase 4b (group modules + lexical import resolver) or §11.6 (group-level collections, v1.3).
**Data-model invariant:** app-owned data-plane tables (KV, docs, files, …) start with `app_id UUID NOT NULL REFERENCES apps(id) ON DELETE CASCADE`; the group-inheritable tables — _config_ (`vars`, `secrets`) and now group-owned _code_ (`scripts`, `0050`) — instead carry a **polymorphic owner**: nullable `group_id` and `app_id` with an exactly-one CHECK and per-owner partial-unique indexes (config is `ON DELETE CASCADE`, scripts `RESTRICT` — code is not data). Inheritance resolves **live** down `apps.group_id → groups.parent_id` via `CHAIN_LEVELS_CTE` (no materialized view); nearest-owner-wins with an app's own row shadowing the inherited one (CoW). Every Rhai SDK call resolves its app from `cx.app_id`, never a script-passed arg, and a group script always runs under the *inheriting* app's `cx.app_id` (the cross-app isolation boundary). **Data-model invariant:** app-owned data-plane tables (KV, docs, files, …) start with `app_id UUID NOT NULL REFERENCES apps(id) ON DELETE CASCADE`; the group-inheritable tables — _config_ (`vars`, `secrets`) and now group-owned _code_ (`scripts`, `0050`) — instead carry a **polymorphic owner**: nullable `group_id` and `app_id` with an exactly-one CHECK and per-owner partial-unique indexes (config is `ON DELETE CASCADE`, scripts `RESTRICT` — code is not data). Inheritance resolves **live** down `apps.group_id → groups.parent_id` via `CHAIN_LEVELS_CTE` (no materialized view); nearest-owner-wins with an app's own row shadowing the inherited one (CoW). Every Rhai SDK call resolves its app from `cx.app_id`, never a script-passed arg, and a group script always runs under the *inheriting* app's `cx.app_id` (the cross-app isolation boundary).

View File

@@ -38,5 +38,6 @@ mod routes;
mod scripts; mod scripts;
mod secrets; mod secrets;
mod staleness; mod staleness;
mod tree;
mod triggers; mod triggers;
mod vars; mod vars;

View File

@@ -0,0 +1,254 @@
//! Phase 5 project-tree apply, end to end via `pic plan/apply --dir`:
//! * a group + a nested app under it apply atomically as one tree; the app
//! binds a route to the group's inherited script (created in the SAME
//! apply); re-plan is a no-op,
//! * one invalid node aborts the whole tree (nothing written),
//! * a `pic groups reparent` between plan and apply trips the bound-plan
//! (StateMoved) check via the tree's structure version.
use std::fs;
use predicates::prelude::*;
use tempfile::TempDir;
use crate::common;
use crate::common::cleanup::{AppGuard, GroupGuard, ScriptGuard};
/// A project tree on disk: root group manifest + a nested app manifest that
/// binds `GET /greet` to the group's (inherited) `shared` script.
fn tree_dir(group: &str, app: &str, group_script_src: &str, app_extra: &str) -> TempDir {
let dir = TempDir::new().expect("tempdir");
fs::create_dir_all(dir.path().join("scripts")).unwrap();
fs::create_dir_all(dir.path().join(app).join("scripts")).unwrap();
fs::write(dir.path().join("scripts/shared.rhai"), group_script_src).unwrap();
fs::write(
dir.path().join("picloud.toml"),
format!(
"[group]\nslug = \"{group}\"\nname = \"Tree Group\"\n\n\
[[scripts]]\nname = \"shared\"\nfile = \"scripts/shared.rhai\"\n\n\
[vars]\nregion = \"eu\"\n"
),
)
.unwrap();
fs::write(
dir.path().join(app).join("picloud.toml"),
format!(
"[app]\nslug = \"{app}\"\nname = \"Tree App\"\n\n\
[[routes]]\nscript = \"shared\"\nmethod = \"GET\"\n\
host_kind = \"any\"\npath_kind = \"exact\"\npath = \"/greet\"\n{app_extra}"
),
)
.unwrap();
dir
}
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
#[test]
fn tree_applies_group_and_app_atomically_then_noop() {
let Some(fx) = common::fixture_or_skip() else {
return;
};
let env = common::admin_env(fx);
let group = common::unique_slug("tr-grp");
let app = common::unique_slug("tr-app");
// Group + app must pre-exist (the manifest owns content, not tree shape).
let _g = GroupGuard::new(&env.url, &env.token, &group);
common::pic_as(&env)
.args(["groups", "create", &group])
.assert()
.success();
let _a = AppGuard::new(&env.url, &env.token, &app);
common::pic_as(&env)
.args(["apps", "create", &app, "--group", &group])
.assert()
.success();
let dir = tree_dir(&group, &app, "\"hi from the group\"", "");
// Plan the whole tree: a group script+var create and an app route create.
let plan = String::from_utf8(
common::pic_as(&env)
.args(["plan", "--dir"])
.arg(dir.path())
.output()
.unwrap()
.stdout,
)
.unwrap();
assert!(
plan.contains("shared") && plan.contains("region") && plan.contains("/greet"),
"tree plan should cover both nodes:\n{plan}"
);
// Apply the whole tree in one transaction.
common::pic_as(&env)
.args(["apply", "--dir"])
.arg(dir.path())
.assert()
.success();
// The group node created its `shared` script (it must drop before its
// group at teardown — ON DELETE RESTRICT).
let listed = String::from_utf8(
common::pic_as(&env)
.args(["scripts", "ls", "--group", &group])
.output()
.unwrap()
.stdout,
)
.unwrap();
assert!(
listed.contains("shared"),
"the group node's script should exist after the tree apply:\n{listed}"
);
let gscript_id = group_script_id(&env, &group);
let _gs = ScriptGuard::new(&env.url, &env.token, &gscript_id);
// Re-plan is a clean no-op (idempotent across both nodes — incl. the app's
// route bound to the inherited group script).
let replan = String::from_utf8(
common::pic_as(&env)
.args(["plan", "--dir"])
.arg(dir.path())
.output()
.unwrap()
.stdout,
)
.unwrap();
assert!(
!replan.contains("create") && !replan.contains("update"),
"re-plan of the tree must be a no-op:\n{replan}"
);
}
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
#[test]
fn tree_apply_is_atomic_on_an_invalid_node() {
let Some(fx) = common::fixture_or_skip() else {
return;
};
let env = common::admin_env(fx);
let group = common::unique_slug("tra-grp");
let app = common::unique_slug("tra-app");
let _g = GroupGuard::new(&env.url, &env.token, &group);
common::pic_as(&env)
.args(["groups", "create", &group])
.assert()
.success();
let _a = AppGuard::new(&env.url, &env.token, &app);
common::pic_as(&env)
.args(["apps", "create", &app, "--group", &group])
.assert()
.success();
// The app node carries an INVALID Rhai script → the whole tree must abort.
let dir = TempDir::new().unwrap();
fs::create_dir_all(dir.path().join("scripts")).unwrap();
fs::create_dir_all(dir.path().join(&app).join("scripts")).unwrap();
fs::write(dir.path().join("scripts/shared.rhai"), "\"ok\"").unwrap();
fs::write(dir.path().join(&app).join("scripts/bad.rhai"), "let x = ;").unwrap();
fs::write(
dir.path().join("picloud.toml"),
format!(
"[group]\nslug = \"{group}\"\nname = \"G\"\n\n\
[[scripts]]\nname = \"shared\"\nfile = \"scripts/shared.rhai\"\n"
),
)
.unwrap();
fs::write(
dir.path().join(&app).join("picloud.toml"),
format!(
"[app]\nslug = \"{app}\"\nname = \"A\"\n\n\
[[scripts]]\nname = \"bad\"\nfile = \"scripts/bad.rhai\"\n"
),
)
.unwrap();
common::pic_as(&env)
.args(["apply", "--dir"])
.arg(dir.path())
.assert()
.failure();
// Atomic: the valid group node's `shared` must NOT have been created.
let listed = String::from_utf8(
common::pic_as(&env)
.args(["scripts", "ls", "--group", &group])
.output()
.unwrap()
.stdout,
)
.unwrap();
assert!(
!listed.contains("shared"),
"a failed tree apply must leave nothing behind:\n{listed}"
);
}
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
#[test]
fn tree_reparent_between_plan_and_apply_trips_state_moved() {
let Some(fx) = common::fixture_or_skip() else {
return;
};
let env = common::admin_env(fx);
let parent = common::unique_slug("trp-par");
let group = common::unique_slug("trp-grp");
let app = common::unique_slug("trp-app");
// parent → group → app.
let _p = GroupGuard::new(&env.url, &env.token, &parent);
common::pic_as(&env)
.args(["groups", "create", &parent])
.assert()
.success();
let _g = GroupGuard::new(&env.url, &env.token, &group);
common::pic_as(&env)
.args(["groups", "create", &group, "--parent", &parent])
.assert()
.success();
let _a = AppGuard::new(&env.url, &env.token, &app);
common::pic_as(&env)
.args(["apps", "create", &app, "--group", &group])
.assert()
.success();
let dir = tree_dir(&group, &app, "\"x\"", "");
// Record a bound plan for the whole tree.
common::pic_as(&env)
.args(["plan", "--dir"])
.arg(dir.path())
.assert()
.success();
// Reparent the group to the instance root — bumps its structure version.
common::pic_as(&env)
.args(["groups", "reparent", &group])
.assert()
.success();
// Apply the recorded plan: the structure moved underneath it → refuse.
// (The apply aborts before any write, so no group script is created and the
// GroupGuards tear down cleanly.)
common::pic_as(&env)
.args(["apply", "--dir"])
.arg(dir.path())
.assert()
.failure()
.stderr(
predicate::str::contains("changed")
.or(predicate::str::contains("plan"))
.or(predicate::str::contains("409")),
);
}
fn group_script_id(env: &common::TestEnv, group: &str) -> String {
let ls = common::pic_as(env)
.args(["scripts", "ls", "--group", group])
.output()
.expect("scripts ls --group");
common::parse_first_id(std::str::from_utf8(&ls.stdout).unwrap())
.expect("group should have one script")
}

View File

@@ -986,6 +986,37 @@ Resolved items now live inline next to their topic. What genuinely remains:
> delete itself stays `RESTRICT`). > delete itself stays `RESTRICT`).
5. **Project tool maps onto groups.** Nested manifests, attach point, single-owner, server-computed 5. **Project tool maps onto groups.** Nested manifests, attach point, single-owner, server-computed
tree plan, per-env approval gating. tree plan, per-env approval gating.
> **Status (Phase 5): ✅ shipped — single-repo nested tree apply, atomic.** A directory tree of
> `picloud.toml` manifests (each declaring an `[app]` or `[group]` node) applies as ONE
> server-computed plan in ONE Postgres transaction. Per the §11.1 review, the **multi-repo
> single-owner / attach-point / takeover** layer (§7) is **deferred** for the solo-dev / single-repo
> start, as is **per-env approval-policy gating** (`[project.environments]`); env overlays
> (`picloud.<env>.toml`) already exist. Groups must **pre-exist** (`pic groups create`) — a manifest
> owns each node's *content*, not the tree *shape* (declarative group create/reparent is a later
> add).
>
> Shipped surface:
> - **Engine:** the reconcile engine generalized from app-only to an `ApplyOwner { App | Group }`
> (a group node has only scripts + vars; routes/triggers/secret-values stay app-only). The in-tx
> reconcile is `reconcile_node_tx`, shared by single-node and tree apply. `apply_tree` locks every
> node key (sorted), reconciles **groups first** (recording each group's `name → id`), then apps —
> so an app route/trigger can bind a group script **created in the same transaction** (resolved
> nearest-ancestor-wins across the in-tree index + pre-existing ancestors; the pool resolver can't
> see uncommitted rows). One commit, one post-commit route refresh.
> - **Bound plan covers structure:** the tree token folds every node's `state_token` **plus every
> in-scope group's `structure_version`**, so a reparent (or content edit) between plan and apply
> trips `StateMoved` — the §4.2 "content **and** tree-structure version" check.
> - **API:** `POST /api/v1/admin/{groups/{id},tree}/{plan,apply}`; per-node capability gating (the
> actor must hold the read/write caps for **every** node touched).
> - **CLI:** a `[group]` manifest kind; `pic plan/apply` on one node (app or group); `pic
> plan/apply --dir <root>` discovers + applies the whole tree, with a single `<tree>` bound token.
>
> Live- and journey-validated: a group + nested app apply atomically (app route binds the group's
> same-apply script); re-plan is all-noop; one invalid node aborts the whole tree (nothing written);
> a `pic groups reparent` between plan and apply is refused. The §5.1 materialized effective-view +
> fan-out invalidation was **not** built — group config/scripts resolve **live** (Phase 3/4),
> so the tree apply needs no cache-invalidation protocol.
6. **(Much later) group-level collections/topics** — the v1.3 cross-app data-sharing problem, with a 6. **(Much later) group-level collections/topics** — the v1.3 cross-app data-sharing problem, with a
real shared-scope authz model. Optionally, trigger/route **templates** (§4.5) if cardinality real shared-scope authz model. Optionally, trigger/route **templates** (§4.5) if cardinality
demands. demands.