c04c684a2e3385c95c652a112afac37456b3fe04
11 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
MechaCat02
|
c04c684a2e |
feat(hierarchies): cross-repo template expansion (M7)
Route/trigger templates declared at a group now fan out to EVERY descendant
app in the DB subtree, not just the app nodes present in the current
`pic apply --dir`. A template change at group N reaches subtree(N) across
repos, and a removed/disabled template reaps its expansions on those
descendants too.
- group_repo: recursive `descendant_app_ids` (+ `_tx`) — inverse of the
ancestor chain CTE, app_id-ordered, depth-bounded.
- apply_service: Phase B2 expands templates into descendants of every in-tree
group not already handled in Phase B. All descendant locks are taken up
front in the single sorted batch (their union is invariant under reparenting
in-tree groups), so there is no out-of-order mid-tx locking / deadlock.
Per-recipient write caps (AppWriteRoute / AppManageTriggers /
AppSecretsRead-for-email) are required AUTHORITATIVELY IN-TX against the
post-reparent, already-locked app — checked set == written set, no TOCTOU.
expand_*_templates_tx are now self-contained (load hand-declared identities
from the tx), so collisions are caught on descendants too. Blast radius now
counts the true DB subtree. Descendant expansion errors are tagged with the
app slug (e.g. an {env} template on a NULL-environment descendant names it).
- apply_api: the pre-tx descendant authz pass is removed in favour of the
in-tx gate (sound via the hierarchy-aware effective_app_role).
- templates journey: out-of-tree descendant at depth 1 AND 2 receives the
expansion and is reaped on template removal.
- doc §4.5: strike the in-apply-only scope limitation.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
MechaCat02
|
4d2eed4e81 |
feat(hierarchies): trigger templates — per-app fan-out (M4b)
Completes §4.5 templates (M4a shipped routes). A group declares a trigger once;
the tree apply fans it out into one concrete per-app_id trigger on each
descendant app, reusing the M4a expansion engine over the trigger insert path.
- Migration 0054: `trigger_templates` table (group-owned; the whole
BundleTrigger wire object stored as `spec` JSONB, placeholders unresolved) +
`triggers.from_template` provenance column + index.
- `template_repo.rs`: trigger-template CRUD + chain-load.
- Manifest: `[group]` `[[trigger_templates]]` (flat: name, kind, script, + kind
params); build_bundle emits them; rejected on an app node.
- Apply: group nodes reconcile trigger-template rows; tree Phase B expands each
chain trigger template into a concrete trigger per descendant app —
placeholders resolved in every spec string leaf, then the typed trigger is
rebuilt and inserted through the shared `insert_bundle_trigger_tx` (email
secrets resolved + re-sealed per recipient app). Idempotent via from_template
(one trigger per template per app; semantic-identity compare → no-op /
delete+recreate / reap). Collision with a hand-declared trigger or between
templates is a hard error. Each resolved trigger is re-validated with the
per-kind shape check (cron/queue/etc.) so a {var:}-injected bad
schedule/timeout can't slip in.
- Authz: declaring → GroupScriptsWrite; receiving → AppManageTriggers, plus
AppSecretsRead for email-trigger recipients (parity with hand-declared email).
- Plan/token/report extended for trigger templates; blast radius covers both.
- Tests: tests/templates.rs trigger fan-out (kv + {app_slug}, stable-ids,
prune-reap); resolve_placeholders_in_json unit test; schema golden reblessed.
Reviewed (no CRITICAL; isolation + per-app email-secret sealing verified
sound). Closed a HIGH validation-parity gap (re-validate resolved triggers) and
a MEDIUM authz gap (AppSecretsRead for email expansions). v1.2 Hierarchies
template work (M4) complete.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
MechaCat02
|
0396866698 |
feat(hierarchies): route templates — group-declared, per-app fan-out (M4a)
§4.5 of the groups/project-tool design. A group declares a route ONCE; the
tree apply fans it out into one concrete per-app_id route on every descendant
app — instantiation, not inheritance (the cross-app isolation boundary is
unchanged; expanded rows stay app-owned).
- Migration 0053: `route_templates` table (group-owned) + `routes.from_template`
provenance column + index.
- `template_repo.rs`: tx-aware CRUD + chain-load (mirrors group_repo/route_repo).
- Manifest: `[group]` `[[route_templates]]`; build_bundle emits them; rejected
on an app node (422).
- Apply: group nodes reconcile template rows (create/update/delete via the
plan); tree Phase B expands each chain template into a concrete route per
descendant app node — placeholders {app_slug}/{env}/{var:NAME} resolved per
app (unknown var/placeholder = hard error), script bound nearest-owner-wins.
Idempotent via from_template (diffed create/update-as-replace/delete; re-apply
is no-churn, --prune reaps expansions). Collision with a hand-declared route,
or between two templates, is a hard error. Each RESOLVED expansion is
validated like a hand-declared route (reserved-path, path/host parse,
host-claim) so a {var:}-injected path or unclaimed strict host can't slip in.
- Plan: route-template diff at the group node + blast-radius (descendant app
count in this apply); state_token folds each template (edit trips StateMoved).
- Authz: declaring → GroupScriptsWrite; an app receiving expansions →
AppWriteRoute (gated even with no hand-declared routes).
- Tests: tests/templates.rs (fan-out + per-slug + idempotent-stable-ids +
prune-reap + collision-rejected); placeholder/validation unit tests; schema
golden reblessed.
Reviewed (no CRITICAL/HIGH; isolation core verified sound). Closed the two
validation-parity MEDIUMs (host-claim + reserved-path/pattern on expansions)
and added an out-of-apply-descendant prune warning. Scope: expansion targets
app nodes in the tree apply (not yet cross-repo descendants); {var:NAME} uses
committed vars. M4b (trigger templates) reuses this engine next.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
MechaCat02
|
193336a8a6 |
feat(hierarchies): multi-repo single-owner ownership + structural prune (M3)
§7 of the groups/project-tool design. A group node is now authoritatively managed by exactly one project-root; `pic apply --dir` claims, conflicts, takes over, and (with --prune) structurally reaps owned nodes. - Migration 0052: `projects(id, key UNIQUE)` + promote the inert `groups.owner_project` (0047) to a real FK (ON DELETE SET NULL) + index. - CLI mints a stable, gitignored project key in `.picloud/project.json` (`pic init`, or lazily on first tree plan/apply) and presents it on every tree request; `pic apply --dir --takeover` flag. - Server: prepare_tree resolves ownership read-only (plan surfaces conflicts + prune candidates; token folds each group's owner key). apply_tree upserts the project in-tx, claims created groups on insert, reconciles existing-group ownership under the per-node advisory lock (first-commit-wins), and prunes owned-but-undeclared groups leaf-first (delete=RESTRICT, never another repo's or a UI-owned node). - Authz (§7.4, ownership ⟂ RBAC): takeover requires GroupAdmin per contested node — enforced in authz_tree (pre-tx) AND re-verified in-tx at the ownership decision, so --force (which waives the staleness token) can't open a takeover-without-admin window. The attacker-supplied project key is length/charset-validated server-side. - Tests: tests/ownership.rs (claim → conflict → takeover → flip; non-admin takeover → 403; prune-owned-only); format_conflicts unit test; schema golden reblessed. tree_shape M2 journey updated to reparent in-place (a fresh dir is now a distinct project and would correctly conflict). Closes the M3 milestone; reviewed (4 findings: 1 authz-bypass via --force + key validation + 2 LOW, all fixed). M4 (trigger/route templates) remains. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
MechaCat02
|
c18ce7c2c4 |
feat(hierarchies): declarative group create/reparent — tree shape (M2)
M2 of the remaining-hierarchies work. `pic apply --dir` now owns the org-tree
SHAPE, not just node content: a `[group]` manifest for a group that doesn't
exist is CREATED under its declared parent, and an existing group whose declared
parent changed is REPARENTED — all inside the single tree-apply transaction
(create + reparent; structural prune is deferred to M3 with the ownership layer).
group_repo: extract transaction-aware structural mutations — `create_group_tx`,
`reparent_group_tx` (the ancestor-walk cycle guard, now reading through the tx so
it sees in-tx writes), `delete_group_tx`, and `acquire_structural_lock_tx`. The
trait `create`/`reparent`/`delete` delegate to them (one SQL definition,
behavior-preserving: the coarse structural advisory lock + structure_version
bump + delete=RESTRICT all preserved).
apply_service: `TreeNode` gains `parent_slug` + `name`. `prepare_tree` classifies
group nodes into existing (resolved id, maybe reparent) vs to-create (absent →
deferred), returning a `PreparedTree { prepared, creates, reparents, token }`.
`apply_tree` adds Phase 0 (`reconcile_tree_structure_tx`): create absent groups
parent-first (topo loop with cycle/unresolved-parent detection) and reparent
existing ones, then Phase A/B reconcile content as before. A to-create group
reconciles against an empty CurrentState (all-Create) — so it needs no DB read
and sidesteps the pool-vs-tx coupling. The bound-plan token folds a
declared-absent marker, so a group created out-of-band between plan and apply
trips StateMoved.
authz (apply_api `authz_tree`): a to-create group mirrors the interactive create
gate — root-level needs `InstanceCreateGroup`, a subgroup under an existing
parent needs only `GroupAdmin(parent)`; a reparent needs `GroupAdmin` at the
group, the SOURCE parent, and the DESTINATION parent (§5.6, parity with
`reparent_group`). No 404 on a to-create node.
CLI: `[group] parent` manifest key (else the parent is inferred from the nearest
ancestor directory's group); `build_tree` emits `parent_slug` + `name` per group
node; the apply report shows a `groups +N reparented M` line.
Reviewed by subagent (core mechanism verified sound: topo termination, empty-
CurrentState reconcile, in-tx cycle guard, atomicity, StateMoved, idempotency);
fixed the two authz-parity gaps it found (missing source-parent check on
reparent; over-gated subgroup create). Tests: a new `tree_shape` journey
(create + reparent + no-op re-plan); 393 manager-core lib + the Phase-5 tree/
group/ext journeys all green; clippy -D warnings clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
MechaCat02
|
4dcb8d1136 |
feat(hierarchies): extension points — opt-in dynamic module resolution (§5.5)
M1 of the remaining-hierarchies work. An extension point lets a PARENT/group
node opt a module name out of sealed lexical resolution: a parent script's
`import "x"` then resolves against the INHERITING app's module (with a declared
default body as fallback) instead of the parent's sealed chain — so each
descendant app supplies its own `x`. Only the declaring parent can open the
inversion; a descendant can never make a parent's sealed import dynamic or
hijack one (the "is this an extension point" decision keys on the importer's
trusted defining node, never a script-passed value).
Resolver (executor-core):
- `ModuleSource` gains `resolve_extension_point(origin, name)` and a
chain-constrained `resolve_by_id(origin, id)`. The resolve seam checks for an
ext point on the importer's chain first; on a hit it resolves against
`App(cx.app_id)`, else the declared default, else ModuleNotFound. The Postgres
query returns Some only when the NEAREST declaration of the name is an ext
point (a peer/nearer concrete module shadows it). A group origin can't reach
app-owned rows — the trust boundary holds. Cache stays id-keyed (no bleed).
Apply (manager-core, mirrors the `vars` pattern):
- migration 0051: owner-polymorphic `extension_points` table (group XOR app),
optional `default_script_id` FK ON DELETE SET NULL, per-owner LOWER(name)
unique indexes.
- Bundle/Plan/CurrentState gain extension_points; `diff_extension_points`
(name-based, default change = Update), reconcile (upsert after scripts so the
default resolves) + prune, `validate_bundle` (module-name shape; default must
be a local module), `check_imports_resolve` (a declared/on-chain ext point
satisfies an import), and the state_token fold. Writes gate on the
script-write capability (AppWriteScript / GroupScriptsWrite).
- GET /apps|groups/{id}/extension-points so `pic pull` round-trips them — a
re-applied pulled manifest is all-NoOp, so `--prune` can't silently drop one.
CLI: `[[extension_points]]` manifest table; plan/apply build + render; pull
exports them.
Tests: 5 resolver units (inversion, default fallback, no-provider error,
no-hijack of a sealed import, no cross-tenant cache bleed), 2 diff/state-token
units, 2 journeys (per-app resolution + default fallback via invoke; pull
round-trip). Schema golden re-blessed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
MechaCat02
|
8a559ea178 |
feat(apply): atomic project-tree apply with cross-node inherited binding (Phase 5 C3)
Phase 5 C3 — the headline. A whole project subtree (group + app nodes) is
reconciled in ONE Postgres transaction, all-or-nothing.
`POST /api/v1/admin/tree/{plan,apply}` take a `TreeBundle { nodes: [{kind,
slug, bundle}] }`. The actor must hold the relevant read/write caps for EVERY
node (per-kind, widened on prune) — `authz_tree` + reusable
`require_{app,group}_node_writes` (now shared with the single-node handlers).
Engine:
* The in-tx reconcile body of `apply_owner` is extracted to `reconcile_node_tx`
(returns the node's post-create `name → script_id`), so single-node and
tree apply share one implementation — behaviour-preserving (391 unit tests
+ app journeys green).
* `prepare_tree` resolves each slug→owner, validates (an app's route/trigger
target may bind to an in-tree ancestor group's script), computes each
node's plan, and folds a combined bound-plan token = every node's
`state_token` + every in-scope group's `structure_version` (so a reparent
or content edit between plan and apply trips StateMoved).
* `apply_tree`: lock every node key (sorted, deadlock-free); reconcile GROUPS
first, recording each group's `name → id` in an in-memory index; then APPS,
resolving inherited route/trigger targets nearest-ancestor-wins across the
in-tree index (incl. scripts created earlier in THIS tx, invisible to the
pool resolver) and pre-existing out-of-tree ancestors. One commit, one
post-commit route refresh.
Live-validated against the dev DB:
* Headline: a group node creates `shared`; an app node in the SAME apply
binds `GET /greet` to it — the route ends up bound to the group-owned
script, and re-plan is all-noop (idempotent).
* Atomicity: one invalid node → 422 and ZERO rows written (the valid group
node's script is not created).
CLI tree discovery + `pic plan/apply` tree mode is C4; journeys + docs are C5.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
MechaCat02
|
65049a6262 |
feat(apply): owner-generic apply engine — group-node plan/apply (Phase 5 C1)
Phase 5 C1. Generalize the reconcile engine from app-only to an `ApplyOwner
{ App | Group }`, so a GROUP's own config + scripts can be declaratively
reconciled — the foundation for the tree-spanning project tool.
A group node is a subset of an app node (scripts + vars only; no routes /
triggers / secrets-values), so the diff, script/route/trigger/var CRUD, prune,
idempotency, and the Phase-4 inherited-target resolution are all reused
unchanged. Only the owner-varying bits switch on `ApplyOwner`:
* `load_current` — `list_for_group` + `VarOwner::Group` for a group (empty
routes/triggers/secrets); `list_for_app` for an app, exactly as before.
* script create owner (`NewScript{app_id|group_id}`), var writes (new
`set_group_var_tx`/`delete_group_var_tx` mirroring the app variants at
scope `*`), and the advisory lock (owner-tagged).
* `validate_bundle_for` rejects routes/triggers on a group bundle (422);
route-host validation, inherited-target resolution, the post-commit route
refresh, and the unreachable-endpoint warning are app-only.
`apply`/`plan` are unchanged thin wrappers over the new `apply_owner`/
`plan_owner`. New endpoints `POST /groups/{id}/{plan,apply}` gated by
`GroupScripts{Read,Write}` / `GroupVarsWrite`. `ApplyService` gains a `groups`
repo (the tree apply in C3 needs it too).
Live-validated: group plan → apply (group-owned script + var) → idempotent
re-plan noop → routes rejected 422 → DB confirms group ownership. App apply
unchanged: 391 manager-core unit tests + the apply/prune/staleness/overlay
journeys all green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
MechaCat02
|
e517f6dc8c |
feat(apply): reconcile app-owned vars in the declarative engine
Extends the Phase-1 apply engine to manage app `vars` declaratively, alongside scripts/routes/triggers/secrets. The `Bundle` gains a `vars` map (key → JSON value; values are non-secret, so they live inline, unlike secret names). `diff_vars` is value-sensitive: a changed value is an Update, a live var the manifest dropped is a Delete (applied only under `--prune` — vars are prunable config, unlike never-pruned secrets). Writes go through `set_app_var_tx`/`delete_app_var_tx` inside the apply transaction (mirroring `PostgresVarsRepo::set` for `VarOwner::App`, scope `*`), so they commit atomically with the rest of the apply and roll back together on failure. `CurrentState` loads the app's OWN scope-`*`, non-tombstone vars (inherited group vars and tombstones are out of scope for the manifest); `state_token` folds them in so the bound-plan check catches an out-of-band `pic vars set` between plan and apply. Keys are validated against the same kebab rule as the vars admin API, and the apply handler now requires `AppVarsWrite` when vars are touched or `--prune` is set. Scope: app-owned vars at scope `*` (an app *is* an environment). Group vars via manifest and tombstone-via-manifest wait for the nested-manifest model. Unit tests cover the create/update/noop/delete diff and the state-token value sensitivity. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
MechaCat02
|
be5df06a48 |
feat(project-tool): bound-plan staleness check (content fingerprint)
`pic plan` now records a fingerprint of the live state it diffed against; `pic apply` replays it and the server refuses (HTTP 409) if the app changed underneath the reviewed plan — the §4.2 "apply exactly what you reviewed" guarantee, in its content-addressed form (no migration, no changes to interactive write paths). Server (manager-core): - `state_token(CurrentState)`: deterministic FNV-1a fingerprint over what the diff keys on — script name+version (version bumps on any edit), route identity+binding/attrs, trigger membership+enabled, secret names. Order-independent; a collision can only yield a false "unchanged", never a false refusal. - `plan` returns it (flattened onto the plan JSON, so the wire stays additive); `apply` takes an optional `expected_token` and, under the apply lock before any write, returns `StateMoved` (409) on mismatch. CLI: - `.picloud/` link state (`linkstate`): `pic plan` writes the token scoped to the app slug; `pic apply` replays it, then clears it on success (the token is single-use — the next apply re-plans). `--force` skips the check; apply with no recorded plan still works standalone (today's behavior). `.picloud/` is already gitignored by `pic init`. The tree-structure version (the other half of §4.2's counter) stays a deliberate no-op until groups exist — it only guards reparent/structural moves, which don't exist single-app. Tested: state_token unit test (stable/order-independent/sensitive) + a staleness journey (plan → out-of-band deploy → apply refuses → --force applies); manager-core lib 363 + cli bins 31 + 13 journeys green; clippy -D warnings clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
MechaCat02
|
3b650a2b14 |
feat: declarative project-tool foundation (pull/plan/apply/prune)
Add a server-side, atomic, declarative reconcile loop for a single app —
the foundation of the project-tool design. Developers describe an app's
scripts, routes, triggers, and secret-names in `picloud.toml`, then
`pic pull / plan / apply [--prune]` to converge live state to the manifest.
Server (manager-core):
- apply_service: a pure diff engine (compute_diff) shared by plan and
apply, plus an ApplyService that composes the existing per-repo writes
into ONE Postgres transaction. Identity keys mirror the DB UNIQUE
constraints (script=lower(name); route=(method,host_kind,host,
path_kind,path); trigger=per-kind semantic tuple; secret=name).
Apply takes a per-app advisory lock, recomputes the diff in-tx, applies
scripts -> routes -> triggers, prunes dependents-first, commits, then
refreshes the route table once post-commit.
- apply_api: POST /apps/{id}/plan (AppRead) and /apps/{id}/apply.
Apply requires the per-kind write caps the bundle exercises (all three
when --prune), plus AppSecretsRead when it binds an email trigger.
- tx-accepting repo siblings (insert/update/delete *_tx) so the existing
create/update/delete delegate to one SQL definition each.
- email triggers reference an inbound secret by NAME; the value is
resolved, decrypted (AAD-bound), and re-sealed server-side at apply —
it never travels in the manifest.
CLI (picloud-cli):
- manifest.rs (picloud.toml model), client plan/apply, and the pull/plan/
apply commands. pull rejects filesystem-unsafe script names up front.
Safety properties enforced and tested:
- idempotent: a freshly-pulled manifest re-applies as all-NoOp.
- atomic: a mid-bundle failure rolls back with nothing written.
- routes delete-before-insert so a freed binding is reusable in one apply.
- queue one-consumer invariant held inside the shared tx.
- email triggers are never pruned, and a script that still owns an
email/dead-letter trigger can't be pruned (the FK cascade would destroy
the sealed secret) — refused with a pointer to `pic triggers rm`.
- plan and apply agree on unset email-secret references.
No migration: the existing schema's UNIQUE constraints serve as identity
keys. Groups, env-scoping, and the `enabled` toggle are later milestones.
Tested: manager-core lib (360) + CLI bins (27) + 8 project-tool journeys
(pull/plan/apply/prune/email+queue), all green; clippy -D warnings clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|