From a00454e5de89f43f415e9e170fd1a46c0f63b507 Mon Sep 17 00:00:00 2001 From: MechaCat02 Date: Tue, 30 Jun 2026 19:09:41 +0200 Subject: [PATCH] =?UTF-8?q?test(cli):=20shared-docs=20journey=20+=20docs?= =?UTF-8?q?=20(=C2=A711.6=20docs=20C5)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit End-to-end docs journey: a group declares a kv AND a docs collection via the string-or-table manifest; an authenticated app A docs::shared_collection( "articles").create(#{...}), app B finds it back across the subtree (exercising the shared find DSL); a sibling-subtree app gets CollectionNotShared; collections ls shows both kinds; re-apply NoOp. Full journey suite 116/116. Docs: groups-and-project-tool §11.6 (KV+docs slices shipped; files/topics/queue deferred), sdk-shape.md (docs::shared_collection + string-or-table), CLAUDE.md. Co-Authored-By: Claude Opus 4.8 (1M context) --- CLAUDE.md | 2 +- crates/picloud-cli/tests/collections.rs | 131 ++++++++++++++++++++++++ docs/design/groups-and-project-tool.md | 39 ++++--- docs/sdk-shape.md | 9 ++ 4 files changed, 164 insertions(+), 17 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index d9a22c8..3280440 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -10,7 +10,7 @@ Authoritative design: [serverless_cloud_blueprint.md](serverless_cloud_blueprint **v1.1.x — SDK foundation + services — is complete.** The SDK shape (handle pattern, `::` namespaces, `Services`/`SdkCallCx`; see [docs/sdk-shape.md](docs/sdk-shape.md), stdlib at [docs/stdlib-reference.md](docs/stdlib-reference.md)) fixed in v1.1.0, then KV, docs, modules, HTTP, cron, files, pub/sub, email, users, and durable queues + `invoke()` filled it in through **v1.1.9** — blueprint §12 has the table. Earlier groundwork: blueprint Phase 3 (admin auth, multi-app scoping, Phase 3.5 capability gating — `manager-core::authz::{can, require, Capability}`, migration `0006_users_authz.sql`). -**Current focus: v1.2 _Hierarchies_ — groups + the declarative project tool** ([docs/design/groups-and-project-tool.md](docs/design/groups-and-project-tool.md)). That doc's §11 uses its own **Phase 1–6 numbering, distinct from the blueprint product-phase numbering above — do not conflate them** (its "Phase 3" = group-inherited config, not admin auth). Implemented on `feat/groups-*` branches: §11 Phase 1 (declarative `pic plan`/`apply`/`prune` + env overlays), Phase 2 (single-parent groups tree + hierarchy-aware RBAC), Phase 3 (group-inherited, env-scoped `vars` + secrets resolved **live** via a recursive CTE — no materialized cache), Phase 4-lite (group-owned **endpoint** scripts: `scripts` polymorphic owner in `0050_group_scripts.sql`, `get_by_name_inherited`/`is_invocable_by_app` chain resolution, inherited `invoke()` + declarative route/trigger binding — all **live**, no body materialization), Phase 5 (the **declarative project tool maps onto the group tree**: the reconcile engine generalized to `ApplyOwner{App|Group}`, a `[group]` manifest kind, and a single atomic **tree apply** — `pic plan/apply --dir` reconciles a whole directory tree of `picloud.toml` nodes in one Postgres transaction, groups-before-apps so an app route can bind a group script created in the same tx; the bound token folds in each group's `structure_version`. Multi-repo single-owner/attach-point and per-env approval gating are deferred; groups pre-exist), Phase 4b (group **modules** + the **lexical (sealed-by-default) import resolver**, §5.5: owner-polymorphic `ModuleScript`, origin-rooted `ModuleSource::resolve` walking the importing node's chain, `ExecRequest.script_owner` threaded from every dispatch + `invoke()` site, `_source`-driven lexical chaining in `PicloudModuleResolver` with the compiled-module cache re-keyed by `ScriptId`, group modules/imports allowed, single-node dangling-import `plan` check — an inherited group script's imports **seal to the group**, a leaf can't shadow them), §5.5 **extension points** (opt-in polymorphism — **§5.5 now complete**: marker table `0051_extension_points.sql` (owner-polymorphic, CASCADE — structurally a `secrets` name; default body = a co-located `kind=module` script), `ModuleSource::resolve_policy` with **nearest-declaration-kind-wins** — a concrete module resolves lexically, an EP marker resolves **dynamically against the inheriting app** (its override else the default body up-chain), `NoProvider` is a hard error; declarative-only authoring via the `[app]`/`[group]` manifest key `extension_points = [...]`, reconcile mirrors `secrets`, single-node no-provider `plan` check, read-only `pic extension-points ls` + `pull` round-trip — the app can **override** a group default, the deliberate inverse of the Phase 4b sealed import), §11.6 **group-level collections — KV slice** (full cross-app shared read/write: a group declares a collection shared via the `[group]` manifest `collections = [...]` → owner-polymorphic marker `0052_group_collections.sql` (`kind='kv'`) + the group-keyed store `0053_group_kv_entries.sql` (no `app_id` — a shared row belongs to the group; CASCADE on group delete, an app delete leaves the data). Scripts use the **explicit** `kv::shared_collection("name")` handle (`shared` alone is a Rhai reserved word); `GroupKvServiceImpl` resolves the owning group from `cx.app_id`'s ancestor chain (nearest-wins) — **that walk is the isolation boundary**, a foreign app gets `CollectionNotShared`. **Reads open** to any subtree script (anonymous incl. — the declaration is the grant), **writes require an authenticated editor+** on the owning group (`GroupKvRead`/`GroupKvWrite`, `script_gate_require_principal` fails closed on anon). Declarative reconcile mirrors `extension_points`; read-only `pic collections ls --group`. Deferred: shared-collection triggers (the "group trigger has no app to watch" problem), docs/files/topics/queue shared collections, per-group quotas). Next: extend §11.6 to docs/files/topics, or multi-node cluster mode. +**Current focus: v1.2 _Hierarchies_ — groups + the declarative project tool** ([docs/design/groups-and-project-tool.md](docs/design/groups-and-project-tool.md)). That doc's §11 uses its own **Phase 1–6 numbering, distinct from the blueprint product-phase numbering above — do not conflate them** (its "Phase 3" = group-inherited config, not admin auth). Implemented on `feat/groups-*` branches: §11 Phase 1 (declarative `pic plan`/`apply`/`prune` + env overlays), Phase 2 (single-parent groups tree + hierarchy-aware RBAC), Phase 3 (group-inherited, env-scoped `vars` + secrets resolved **live** via a recursive CTE — no materialized cache), Phase 4-lite (group-owned **endpoint** scripts: `scripts` polymorphic owner in `0050_group_scripts.sql`, `get_by_name_inherited`/`is_invocable_by_app` chain resolution, inherited `invoke()` + declarative route/trigger binding — all **live**, no body materialization), Phase 5 (the **declarative project tool maps onto the group tree**: the reconcile engine generalized to `ApplyOwner{App|Group}`, a `[group]` manifest kind, and a single atomic **tree apply** — `pic plan/apply --dir` reconciles a whole directory tree of `picloud.toml` nodes in one Postgres transaction, groups-before-apps so an app route can bind a group script created in the same tx; the bound token folds in each group's `structure_version`. Multi-repo single-owner/attach-point and per-env approval gating are deferred; groups pre-exist), Phase 4b (group **modules** + the **lexical (sealed-by-default) import resolver**, §5.5: owner-polymorphic `ModuleScript`, origin-rooted `ModuleSource::resolve` walking the importing node's chain, `ExecRequest.script_owner` threaded from every dispatch + `invoke()` site, `_source`-driven lexical chaining in `PicloudModuleResolver` with the compiled-module cache re-keyed by `ScriptId`, group modules/imports allowed, single-node dangling-import `plan` check — an inherited group script's imports **seal to the group**, a leaf can't shadow them), §5.5 **extension points** (opt-in polymorphism — **§5.5 now complete**: marker table `0051_extension_points.sql` (owner-polymorphic, CASCADE — structurally a `secrets` name; default body = a co-located `kind=module` script), `ModuleSource::resolve_policy` with **nearest-declaration-kind-wins** — a concrete module resolves lexically, an EP marker resolves **dynamically against the inheriting app** (its override else the default body up-chain), `NoProvider` is a hard error; declarative-only authoring via the `[app]`/`[group]` manifest key `extension_points = [...]`, reconcile mirrors `secrets`, single-node no-provider `plan` check, read-only `pic extension-points ls` + `pull` round-trip — the app can **override** a group default, the deliberate inverse of the Phase 4b sealed import), §11.6 **group-level collections — KV + DOCS slices** (full cross-app shared read/write: a group declares a collection shared via the `[group]` manifest `collections = [...]` → owner-polymorphic marker `0052_group_collections.sql` with a `kind` discriminator + a per-kind group-keyed store: `0053_group_kv_entries.sql` (`kind='kv'`) and `0054_group_docs.sql` (`kind='docs'`, the queryable-JSON store) — no `app_id`, a shared row belongs to the group; CASCADE on group delete, an app delete leaves the data. Scripts use the **explicit** `kv::shared_collection("name")` / `docs::shared_collection("name")` handles (`shared` alone is a Rhai reserved word); `GroupKv`/`GroupDocsServiceImpl` resolve the owning group from `cx.app_id`'s ancestor chain **filtered by kind** (nearest-wins) — **that walk is the isolation boundary**, a foreign app gets `CollectionNotShared`; a `kv` and a `docs` collection of the same name are distinct stores. The docs slice reuses the `docs_filter` DSL — `build_find_query` generalized on its owner column (`docs`/`app_id` vs `group_docs`/`group_id`, both literals). **Reads open** to any subtree script (anonymous incl. — the declaration is the grant), **writes require an authenticated editor+** on the owning group (`GroupKvRead/Write`, `GroupDocsRead/Write`, `script_gate_require_principal` fails closed on anon). Declarative authoring is the **string-or-table** form `collections = ["catalog", { name = "articles", kind = "docs" }]` (bare string = kv); reconcile keys markers by `(name, kind)`; read-only `pic collections ls --group` shows a kind column. Deferred: shared-collection triggers (the "group trigger has no app to watch" problem), files/topics/queue shared collections (files needs group-keyed disk paths), per-group quotas). Next: extend §11.6 to files, or multi-node cluster mode. **Data-model invariant:** app-owned data-plane tables (KV, docs, files, …) start with `app_id UUID NOT NULL REFERENCES apps(id) ON DELETE CASCADE`; the group-inheritable tables — _config_ (`vars`, `secrets`) and now group-owned _code_ (`scripts`, `0050`) — instead carry a **polymorphic owner**: nullable `group_id` and `app_id` with an exactly-one CHECK and per-owner partial-unique indexes (config is `ON DELETE CASCADE`, scripts `RESTRICT` — code is not data). Inheritance resolves **live** down `apps.group_id → groups.parent_id` via `CHAIN_LEVELS_CTE` (no materialized view); nearest-owner-wins with an app's own row shadowing the inherited one (CoW). Every Rhai SDK call resolves its app from `cx.app_id`, never a script-passed arg, and a group script always runs under the *inheriting* app's `cx.app_id` (the cross-app isolation boundary). diff --git a/crates/picloud-cli/tests/collections.rs b/crates/picloud-cli/tests/collections.rs index 2cb228e..170012e 100644 --- a/crates/picloud-cli/tests/collections.rs +++ b/crates/picloud-cli/tests/collections.rs @@ -263,6 +263,137 @@ fn nearest_declaring_group_wins() { ); } +/// §11.6 docs slice: a group declares a `docs`-kind shared collection (via the +/// string-or-table manifest, mixed with a `kv` one); an authenticated app A +/// `create`s a document and app B `find`s it back across the subtree; a +/// sibling-subtree app gets `CollectionNotShared`; `collections ls` shows both +/// kinds. +#[ignore = "needs DATABASE_URL pointing at a running Postgres"] +#[test] +fn shared_docs_collection_is_read_write_across_the_subtree() { + let Some(fx) = common::fixture_or_skip() else { + return; + }; + let env = common::admin_env(fx); + let group = common::unique_slug("dcoll-grp"); + let sibling = common::unique_slug("dcoll-sib"); + let app_a = common::unique_slug("dcoll-a"); + let app_b = common::unique_slug("dcoll-b"); + let foreign = common::unique_slug("dcoll-f"); + + let _g = GroupGuard::new(&env.url, &env.token, &group); + let _gs = GroupGuard::new(&env.url, &env.token, &sibling); + common::pic_as(&env) + .args(["groups", "create", &group]) + .assert() + .success(); + common::pic_as(&env) + .args(["groups", "create", &sibling]) + .assert() + .success(); + + // The group declares a kv AND a docs collection (string-or-table form). + let dir = manifest_dir(); + let gmanifest = format!( + "[group]\nslug = \"{group}\"\nname = \"DColl\"\n\n\ + collections = [\"catalog\", {{ name = \"articles\", kind = \"docs\" }}]\n" + ); + let gpath = dir.path().join("group.toml"); + fs::write(&gpath, &gmanifest).unwrap(); + common::pic_as(&env) + .args(["apply", "--file"]) + .arg(&gpath) + .assert() + .success(); + + // `collections ls` shows both names with their kinds. + let ls = String::from_utf8( + common::pic_as(&env) + .args(["collections", "ls", "--group", &group]) + .output() + .unwrap() + .stdout, + ) + .unwrap(); + assert!( + ls.contains("articles") && ls.contains("docs") && ls.contains("catalog"), + "ls should list both kinds:\n{ls}" + ); + + // Re-apply is a no-op (both markers already exist). + common::pic_as(&env) + .args(["apply", "--file"]) + .arg(&gpath) + .assert() + .success(); + + let _a = AppGuard::new(&env.url, &env.token, &app_a); + let _b = AppGuard::new(&env.url, &env.token, &app_b); + let _f = AppGuard::new(&env.url, &env.token, &foreign); + for (app, grp) in [(&app_a, &group), (&app_b, &group), (&foreign, &sibling)] { + common::pic_as(&env) + .args(["apps", "create", app, "--group", grp]) + .assert() + .success(); + } + + // App A creates a document in the shared docs collection (authenticated). + fs::write( + dir.path().join("scripts/dwriter.rhai"), + r#"docs::shared_collection("articles").create(#{ t: "hi" })"#, + ) + .unwrap(); + common::pic_as(&env) + .args(["scripts", "deploy"]) + .arg(dir.path().join("scripts/dwriter.rhai")) + .args(["--app", &app_a, "--name", "dwriter"]) + .assert() + .success(); + common::pic_as(&env) + .args(["scripts", "invoke", &app_script_id(&env, &app_a, "dwriter")]) + .assert() + .success(); + + // App B finds the SAME doc back — cross-app docs sharing + the find DSL. + fs::write( + dir.path().join("scripts/dreader.rhai"), + r#"docs::shared_collection("articles").find(#{ t: "hi" })[0].data.t"#, + ) + .unwrap(); + common::pic_as(&env) + .args(["scripts", "deploy"]) + .arg(dir.path().join("scripts/dreader.rhai")) + .args(["--app", &app_b, "--name", "dreader"]) + .assert() + .success(); + assert_eq!( + invoke_body(&env, &app_script_id(&env, &app_b, "dreader")), + serde_json::json!("hi"), + "app B finds the document app A created in the shared docs collection" + ); + + // The foreign app (sibling subtree) → CollectionNotShared. + fs::write( + dir.path().join("scripts/dpeek.rhai"), + r#"docs::shared_collection("articles").find(#{})"#, + ) + .unwrap(); + common::pic_as(&env) + .args(["scripts", "deploy"]) + .arg(dir.path().join("scripts/dpeek.rhai")) + .args(["--app", &foreign, "--name", "dpeek"]) + .assert() + .success(); + let out = common::pic_as(&env) + .args(["scripts", "invoke", &app_script_id(&env, &foreign, "dpeek")]) + .output() + .expect("invoke"); + assert!( + !out.status.success(), + "a foreign app must not resolve another subtree's shared docs collection" + ); +} + /// Id of the named script in an app (`pic scripts ls --app `). fn app_script_id(env: &common::TestEnv, app: &str, name: &str) -> String { let ls = common::pic_as(env) diff --git a/docs/design/groups-and-project-tool.md b/docs/design/groups-and-project-tool.md index d5900e2..600a830 100644 --- a/docs/design/groups-and-project-tool.md +++ b/docs/design/groups-and-project-tool.md @@ -1044,24 +1044,31 @@ Resolved items now live inline next to their topic. What genuinely remains: real shared-scope authz model. Optionally, trigger/route **templates** (§4.5) if cardinality demands. - > **Shipped — §11.6 KV slice (full shared read/write).** A group declares a collection - > group-shared (`[group]` manifest `collections = ["catalog"]`, owner-polymorphic marker table - > `0052_group_collections`, `kind='kv'`); the data lives in `0053_group_kv_entries`, keyed by the - > owning `group_id` (NOT app — a shared row belongs to the group). Scripts read/write via the - > **explicit** `kv::shared_collection("catalog")` handle (a distinct `GroupKvHandle`; `shared` - > alone is a Rhai reserved word, hence `shared_collection`). The service resolves the owning group - > from `cx.app_id`'s ancestor chain (nearest-group-wins) — **that walk is the isolation boundary**: - > a foreign app's chain never contains the owning group, so the name returns `CollectionNotShared`. - > **Trust model:** reads are open to any subtree script (anonymous public HTTP included — the - > declaration *is* the grant); **writes require an authenticated editor+** on the owning group - > (`GroupKvWrite`, fails closed on an anonymous principal). CASCADE on group delete; an app delete - > leaves the shared data. Live- + journey-validated (app A writes, app B reads; a sibling-subtree - > app gets `CollectionNotShared`). + > **Shipped — §11.6 KV + DOCS slices (full shared read/write).** A group declares a collection + > group-shared (`[group]` manifest `collections = [...]`, owner-polymorphic marker table + > `0052_group_collections` with a `kind` discriminator); the data lives in a per-kind store keyed by + > the owning `group_id` (NOT app — a shared row belongs to the group): `0053_group_kv_entries` + > (`kind='kv'`) and `0054_group_docs` (`kind='docs'`, the queryable-JSON store). Scripts read/write + > via the **explicit** `kv::shared_collection("catalog")` / `docs::shared_collection("articles")` + > handles (distinct `GroupKvHandle`/`GroupDocsHandle`; `shared` alone is a Rhai reserved word, hence + > `shared_collection`). The service resolves the owning group from `cx.app_id`'s ancestor chain + > **filtered by kind**, nearest-group-wins — **that walk is the isolation boundary**: a foreign + > app's chain never contains the owning group, so the name returns `CollectionNotShared`; a `kv` and + > a `docs` collection of the same name are distinct stores. **Trust model:** reads are open to any + > subtree script (anonymous public HTTP included — the declaration *is* the grant); **writes require + > an authenticated editor+** on the owning group (`GroupKvWrite`/`GroupDocsWrite`, fail closed on an + > anonymous principal). The docs slice reuses the `docs_filter` DSL — `build_find_query` was + > generalized on its owner column (`docs`/`app_id` vs `group_docs`/`group_id`), both compile-time + > literals, so the find SQL has one source. Declarative authoring uses the **string-or-table** form: + > `collections = ["catalog", { name = "articles", kind = "docs" }]` (bare string = `kv`). CASCADE on + > group delete; an app delete leaves the shared data. Live- + journey-validated for both kinds (app + > A writes, app B reads/finds; a sibling-subtree app gets `CollectionNotShared`). > > **Deferred (documented gaps):** write-triggers/events on shared collections (the "group trigger - > has no app to watch" problem); docs/files/topics/queue shared collections (the `kind` column - > generalizes); per-group total-size quotas + write-rate limits; CAS/`set_if`; app-declared - > collections. Multi-node tree-apply leans on the runtime backstop for no-op edges, as elsewhere. + > has no app to watch" problem); **files/topics/queue** shared collections (the `kind` column + > generalizes — files needs group-keyed disk paths); per-group total-size quotas + write-rate + > limits; CAS/`set_if`; app-declared collections. Multi-node tree-apply leans on the runtime + > backstop for no-op edges, as elsewhere. ### 11.1 Re-sequencing review (post-Phase-3) diff --git a/docs/sdk-shape.md b/docs/sdk-shape.md index 50582b6..9ccd6d3 100644 --- a/docs/sdk-shape.md +++ b/docs/sdk-shape.md @@ -99,6 +99,15 @@ script. **Reads** are open to any subtree script (including anonymous public endpoints — declaring the collection *is* the grant); **writes** require an authenticated principal with editor+ on the owning group. +**Docs too.** The same handle exists for the queryable-JSON store: +`docs::shared_collection("articles")` returns a shared-docs handle with the +full `create`/`get`/`find`/`find_one`/`update`/`delete`/`list` surface (same +trust model + isolation boundary as the KV one). A `[group]` declares a +collection's store with a kind: `collections = ["catalog", { name = +"articles", kind = "docs" }]` — a bare string is `kv`, a `{ name, kind }` +table sets it. A `kv` and a `docs` collection of the same name are distinct +stores. (Files/topics/queue shared collections are not yet implemented.) + ## Error convention - **Throw on failure.** `widgets.set("k", "v")` throws a Rhai runtime