Commit Graph

272 Commits

Author SHA1 Message Date
MechaCat02
41a21c0551 feat(workflows): M3 — conditional when + input templating at run time
Wire the two pure M1 modules into the orchestrator's execute path:

- Before running a step, build a `RunContext` from the run input + every prior
  succeeded step's output (`load_run_step_outputs`, read fresh at execution
  time so a step sees all upstream results).
- `workflow_expr::eval` the step's `when` condition (if any): false → the step
  is `skipped` — never executed — and counts as satisfied-but-empty for its
  dependents (a new `StepOutcome::Skipped`, written in `complete_step_and_advance`
  then advanced; `compute_advance` already treats skipped as satisfying deps).
- `workflow_template::resolve` the step's `input` against the context: an
  exact single `{{ ref }}` preserves the referenced JSON type, an embedded ref
  interpolates as text; a missing reference is a hard step failure (a definition
  bug surfaced, not hidden), never a silent null.

`when` and templates were already parse-validated at apply time (M1); this is
the runtime half.

Tests (DB-gated, end-to-end via a scripted fake executor):
- `when_false_skips_step` — b skipped, never executed, omitted from run output
- `step_output_flows_into_downstream_input` — a's `{n:7}` feeds b's input,
  type-preserved for a bare ref and interpolated in a larger string
- `missing_input_ref_fails_the_step` — an unresolved ref fails b → run fails

Verified: cargo fmt, clippy -D warnings clean, 448 manager-core lib tests,
11 workflow_orchestrator DB tests (3 new).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 17:18:01 +02:00
MechaCat02
a55c2f112e feat(workflows): M2 — durable orchestrator worker (claim/execute/advance)
The v1.2 Workflows durable DAG engine gains its runtime. A dedicated
background worker (`workflow_orchestrator.rs`, mirroring `cron_scheduler`,
NOT folded into the dispatcher) advances every in-flight run step-by-step,
durably, surviving restarts.

Per tick, two phases:
  A. Claim + execute — up to a small batch of `ready`, due steps are claimed
     with the same `FOR UPDATE SKIP LOCKED` competing-consumer lease the queue
     uses (`claim_ready_step`), one execution-gate permit per step acquired
     BEFORE the claim so the shared gate bounds real parallelism. Each step
     resolves its function by name in the run's app scope (never a
     script-passed arg — the isolation boundary), builds an `ExecRequest`, and
     runs through the injected `ExecutorClient`. Claimed steps run concurrently.
  B. Advance — the outcome is written and the DAG advanced in one token-gated
     transaction (`complete_step_and_advance`): pending steps whose deps are
     satisfied flip to `ready`, and the run's terminal status is recomputed.
     Fan-in falls out (a join waits until its last dep flips it); a stale worker
     matches zero rows and writes nothing.

The graph-advance decision is a pure, DB-free function (`compute_advance`) —
promotions + terminal run status, folding `on_error` fail-vs-continue and
skipped/failed dependency satisfaction — so it is unit-tested in isolation.

Retry uses the step's own policy via `compute_backoff`; a second, slower
cadence reclaims steps leased by a crashed worker (`reclaim_stale_steps`) — the
durability safety net. Steps run with no principal (like invoke_async), and log
under the new `ExecutionSource::Workflow` so `pic logs` surfaces them.

M2 executes function steps only; `when` + input templating land in M3, nested
sub-workflows in M4. Seams present (`StepTarget`, `run_input`, `workflow_depth`).

- migration 0072: widen the `execution_logs.source` CHECK with `workflow`
- shared: `ExecutionSource::Workflow`, `StepStatus::is_terminal`
- workflow_repo: run/step state — `start_run`, `claim_ready_step`,
  `complete_step_and_advance`, `reclaim_stale_steps`, `get_run`,
  `list_run_steps`, `compute_advance` (+ 7 pure unit tests)
- workflow_orchestrator: the worker + config (`PICLOUD_WORKFLOW_*` knobs),
  spawned in `picloud/src/lib.rs` beside the dispatcher/cron scheduler
- tests: 8 DB-gated integration tests (linear, parallel fan-out/fan-in, retry,
  on_error fail/continue, double-complete idempotency, stale-lease reclaim,
  end-to-end tick with a fake executor)

Verified: cargo fmt, clippy -D warnings clean, 448 manager-core lib tests,
8 workflow_orchestrator DB tests, schema snapshot reblessed, M1 workflow CLI
journeys still pass (binary boots with the orchestrator wired in).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 17:12:42 +02:00
MechaCat02
44f992cbb0 feat(workflows): M1 — schema + definition validation + declarative reconcile
First milestone of the v1.2 Workflows track (blueprint §9.1/§9.2): the durable
DAG engine's foundation — the definition model, its validation, and the
declarative `apply` reconcile path. No execution yet (the orchestrator is M2).

- Migration 0071_workflows: `workflows` (owner-polymorphic like scripts, app-
  owned in M1), `workflow_runs`, and `workflow_run_steps` (the per-step
  competing-consumer lease table the M2 orchestrator will claim). Schema golden
  reblessed.
- `shared::workflow`: the `WorkflowDefinition` / `WorkflowStepDef` DTOs (shared
  by the CLI, apply, and the future orchestrator) + run/step status enums.
- Pure, DB-free `workflow_template` (input `{{ input.x }}` / `{{ steps.a.output.y }}`
  resolution, type-preserving) and `workflow_expr` (a small safe JSON-predicate
  evaluator for `when` — keeps manager-core free of a scripting engine).
- `workflow_repo`: read trait + reconcile tx free-fns (insert/update/delete).
- apply_service: `BundleWorkflow` + `Plan.workflows` + `CurrentState.workflows`
  + `ApplyReport.workflows_*`; server-side `validate_workflow_definition`
  (unique/acyclic steps via topological sort, deps exist, function XOR workflow,
  `when`/template parse) rejecting on a group node; `diff_workflows` by
  lower(name) (Update on a definition change) + in-tx reconcile.
- CLI: `[[workflows]]` + `[[workflows.steps]]` manifest structs → wire bundle;
  plan/apply render + report counts.
- Tests: 16 manager-core lib tests (template, expr, definition validation) +
  the `workflows` CLI journey (apply → NoOp → prune; cyclic DAG rejected).

Deferred to later milestones: the orchestrator worker (M2), conditional/template
runtime wiring (M3), nested sub-workflows (M4), the SDK/API/CLI run surface
(M5), the dashboard (M6). The `group_id` column + run/step tables ship now so
those slot in without a migration churn.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 16:46:32 +02:00
MechaCat02
c8c4f012ff docs: reconcile blueprint/design-doc/CLAUDE.md to shipped v1.2 state
A five-agent roadmap-verification pass found the v1.2 Hierarchies code fully and
correctly implemented, but the three source-of-truth docs carried stale
"deferred/remaining/v1.3" notes — and one self-contradiction — for features that
have since shipped (§3 M3 server-side approval gate, §6 group create/reparent, the
Track A closeout, and the 2026-07-11 audit remediation). Reconcile all three to
reflect the true code; no behavior change.

CLAUDE.md: retract the inline "groups pre-exist" / "per-env approval deferred" /
"shared-topic SSE deferred" / "group dead-letter store deferred" / "email v0/no-AAD"
notes → mark §6, §3 M3, and Track A M1–M6 shipped; add per-group KV/docs byte
quotas + set_if; note migration 0063; refresh the "Out of MVP" section.

design doc: fix the header ("Phases 4–6 remain" → all shipped); resolve the §11.6
self-contradiction (the "Deferred" list still named shared-topic SSE / byte quotas /
set_if / operator admin API, all shipped — line 1317 already said SSE shipped); mark
the D3 dead-letter store shipped; retract the stale "groups pre-exist / §6 deferred"
forward references.

blueprint (comprehensive sweep): dashboard Alpine.js → SvelteKit + CodeMirror
(diagram, §3.3, tech table); Docker-per-execution → embedded in-process Rhai
(diagram + data flow); §12 Phase 4 "current focus" → shipped through v1.1.9; Phase 5
"in active development" + "Remaining" block → Hierarchies complete; per-app RBAC
"v1.3+" → shipped Phase 3.5; SDK reference → handle-pattern notation note, S3 tag
v1.1 → v1.3+; MVP schema + docker-compose flagged non-authoritative; §9 header noted
Hierarchies-shipped / Workflows-future; top status line refreshed.

Also fix two behavior-neutral stale in-code comments (sdk/kv.rs `kv::shared` →
`kv::shared_collection`; dispatcher.rs `q_terminal` "no group dead-letter store yet"
→ dead-letters to group_dead_letters, Track A M2).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 14:53:34 +02:00
MechaCat02
5e1bda1f32 fix(core): cron double-fire, materialize warnings, group quota fast path, SSE channel cap
Remediate the MEDIUM backend correctness/perf findings from the 2026-07-11 audit.

B1 — a group cron template toggled disabled→enabled no longer re-fires every
descendant's cron. Materialization now KEEPS a disabled cron template's copies
(syncing `enabled` in place) instead of delete+recreate, preserving
`last_fired_at`; the scheduler already skips disabled copies, so keeping them is
inert. Queue/email arms still delete-on-disable (freeing the one-consumer slot).

B2 — `rematerialize_stateful_templates` warnings are now logged at the
app-create/delete and group-reparent chokepoints (were silently dropped); only
the `Err` arm was handled before.

B3 — the group KV/docs byte-quota check skips the O(rows) `SUM(octet_length(..))`
scan when a cheap upper bound (`rows_after * max_value_bytes`) is already under
the cap, so the full aggregate only runs near-cap. Every row is validated
≤ max_value_bytes, so the bound is sound (never lets an over-quota write through).

B6 — the in-process SSE broadcaster caps live channels per map
(`PICLOUD_REALTIME_MAX_CHANNELS`, default 100k); a subscribe that would open a
NEW channel past the cap is refused with 503 + `Retry-After: 1` rather than
growing the (app,topic)/(group,topic) maps unboundedly. Check+insert is atomic
under the map mutex (no TOCTOU); an existing-channel subscribe is exempt.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 23:47:47 +02:00
MechaCat02
86448beb06 fix(security): server-authoritative approval gate + auth/session/file hardening
Remediate the HIGH and security-relevant findings from the 2026-07-11 audit.

H1 — the per-env approval gate is now server-authoritative. The governing
project is resolved from the target node's nearest-claimed ancestor
(`governing_env_policy`/`_tree` + `governing_project_id` +
`ProjectRepository::get_environments_by_id`), independent of the client-supplied
`[project]`. Omitting or spoofing the project block can no longer skip a gate the
owning project established; a to-create group resolves its declared parent's
chain so a fresh subtree node inherits the gate. Fails closed on any read error.

H2 — the API-key prefix slice (`&rest[..8]`) is now the boundary-safe
`rest.get(..8)`, so an attacker-supplied multibyte bearer can't panic the request
task (unauthenticated per-request DoS). Regression test added.

C1 — admin sessions gain an absolute lifetime cap (migration 0070,
`PICLOUD_SESSION_ABSOLUTE_TTL_HOURS`, default 30d): `lookup` filters it, `touch`
clamps the sliding bump to it, so a continuously-used or stolen-but-warm token
self-expires. Mirrors the data-plane app-user cap.

C2 — `Cache-Control: no-store` on the login and API-key-mint responses (the two
that return a raw credential), so a proxy/CDN/browser cache can't retain it.

B8 — file downloads are header-safe: `sanitize_stored_filename` guarantees a
valid `HeaderValue` (no panic on a control-char name) and BOTH the per-app and
group download paths now set attachment + `X-Content-Type-Options: nosniff` +
a restrictive CSP, closing a group-path stored-XSS gap.

Also folds in the server-side plan-warning plumbing (`plan_warnings`,
`PlanResult::warnings`) and the `app_only_reject` message helper that the CLI
plan-preview change builds on, plus operator security notes (reads-open shared-
topic SSE; the `--env` label is advisory, not a boundary).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 23:47:35 +02:00
MechaCat02
b8b047368e feat(realtime): external SSE subscription for group shared topics (§11.6 D2 / Track A M6)
Shared TOPICS fanned out only to in-cluster trigger handlers; external clients
could not subscribe (per-app topics already can). Add SSE for shared topics.

- RealtimeBroadcaster gains a parallel (group_id, topic) channel map:
  subscribe_group / publish_group / drop_group_topic (default no-ops so
  NoopRealtimeBroadcaster + test doubles are untouched). InProcessBroadcaster
  implements them with a second map; GC + channel_count span both.
- Route GET /realtime/shared/topics/{topic}: Host->app dispatch (as the per-app
  route), then RealtimeAuthority::authorize_subscribe_shared resolves the OWNING
  GROUP from the app's chain (kind=topic, root segment). Reads-open model — the
  resolution IS the authorization, consistent with in-script shared reads; a
  foreign-subtree app never resolves (404, the isolation boundary). No principal
  machinery needed.
- GroupPubsubServiceImpl::with_realtime bridges a shared-topic publish to the
  owning-group channel (best-effort) after the durable trigger fan-out.
- Host wires the broadcaster into the group pubsub service + the collection
  resolver into the authority.

Auth-model note: chose reads-open (subtree app's Host is the grant) over
"authenticated principal + GroupKvRead" — it's both simpler and faithful to how
shared-collection reads already work. Pinned by realtime broadcaster group-map
tests, realtime_api shared-route tests (404 + stream), and
group_pubsub_service::publish_bridges_to_the_group_broadcaster. No migration.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 15:33:39 +02:00
MechaCat02
0f05c270d1 feat(kv): set_if compare-and-swap for KV (per-app + shared + SDK) (Track A M5)
No atomic CAS existed, so concurrent writers to the same KV key raced. Add
set_if(key, expected, new): writes only when the current value equals expected,
or (expected absent) only when the key is missing — the primitive for lock-free
counters / optimistic concurrency.

- KvService + GroupKvService traits gain set_if with a defaulted "unsupported"
  impl (Noop + other impls untouched); the real services override it.
- kv_repo/group_kv_repo: atomic set_if — a single UPDATE ... WHERE value =
  $expected (JSONB semantic equality) or INSERT ... ON CONFLICT DO NOTHING.
  Atomic without a transaction; returns whether the write happened.
- Services enforce the same value-size cap + (group) writes-authed + row/byte
  quotas as set; the event fires only on a successful swap.
- Executor Rhai SDK: kv::collection(c).set_if(key, expected, new) and the
  shared_collection variant; Rhai () for expected means "only if absent".

Pinned by kv_service/group_kv_service set_if unit tests (CAS + fail-closed) +
sdk_kv::kv_set_if_compare_and_swap (through a real script). No migration.
docs/sdk-shape.md documents the primitive.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 15:19:03 +02:00
MechaCat02
636106e9ea feat(quota): per-group total-byte quotas for shared KV + docs (Track A M4)
M3 quotas capped a group's shared KV/docs by ROW count only; a group could still
blow past storage with few-but-huge values. Add the symmetric total-bytes ceiling
(files already had one).

- group_quota: PICLOUD_GROUP_{KV,DOCS}_MAX_TOTAL_BYTES env vars (default 256 MiB)
  + accessors.
- GroupKvRepo/GroupDocsRepo: total_bytes(group) = SUM(octet_length(value::text))
  (default Ok(0) so non-Postgres impls skip the check).
- GroupKv/DocsServiceImpl: check the PROJECTED total (old value's bytes
  subtracted, new added) on every set/create/update, so a same-or-smaller update
  near the cap is still allowed (unlike a naive used+new check). New
  TotalBytesQuotaExceeded errors; +with_max_total_bytes for tests.
- KV set switched has->get to obtain the old value's size for the delta.

Pinned by group_kv_service + group_docs_service per_group_byte_quota_uses_the_
projected_total unit tests (reject over-cap, allow same/smaller update near cap).
No migration. CLAUDE.md env-var table updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 15:09:01 +02:00
MechaCat02
1a69778c0c feat(secrets): AAD-bind email-trigger inbound secrets v0->v1 (Track A M3)
email_trigger_details.inbound_secret_encrypted was sealed v0 (no AAD, bound only
to the master key) because the table had no version column — a ciphertext could
in principle be relocated between rows (audit 2026-06-11 H-D1, Medium). Bring the
AAD versioning the `secrets` table gained in 0042 to email secrets.

- Migration 0069: `email_trigger_details.inbound_secret_version SMALLINT DEFAULT 0`.
- secrets_service: `seal_email`/`open_email` seal v1 with AAD bound to the
  SEALING OWNER (`email:{app}` / `email:group:{group}`) — deliberately NOT the
  per-row trigger_id, so a group template's sealed bytes stay valid when the
  materializer copies them verbatim to each descendant (all share the group AAD).
  v0 rows keep their exact legacy no-AAD read path.
- Both email-trigger create paths (declarative apply resolve_and_seal +
  triggers_api create_email_trigger) seal v1 under the trigger's owner; the
  version threads through CreateEmailTrigger + insert_email_trigger_tx.
- materialize copies inbound_secret_version verbatim with the bytes.
- email_inbound_target recovers the sealing owner (materialized_from ->
  template.group_id, else the app) so receive_inbound_email opens a v1 secret
  under the right AAD.

Cross-app/tenant relocation now fails the GCM tag; a same-owner swap is not
distinguished (accepted low-severity residual). Pinned by
email_secret_aad::materialized_email_copy_decrypts_under_the_group_aad (the
novel materialized-copy path) + the extended secret_round_trips_through_seal_open
lib test. Schema golden reblessed; materialization + email e2e regressions green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 14:58:35 +02:00
MechaCat02
fd4336e883 feat(queue): dead-letter store for group shared queues (§11.6 D3 / Track A M2)
An exhausted SHARED durable-queue message was `drop_exhausted()`-ed with a
warning — silent data loss. Per-app queues persist to `dead_letters` (0010);
add the symmetric group store so an exhausted shared-queue message is preserved
and operator-visible.

- Migration 0068: `group_dead_letters`, keyed by (group_id, collection),
  CASCADE on the group (an app delete leaves the data — it belongs to the
  group, not the consuming app).
- `GroupQueueRepo::dead_letter` (replaces `drop_exhausted`): one tx that INSERTs
  the dead-letter + DELETEs the live message, filtered by claim_token so a lost
  lease can't dead-letter a re-claimed message (mirrors queue_repo::dead_letter).
- Dispatcher `q_terminal` shared arm now dead-letters instead of dropping. It
  returns None (not the dl id) so the per-app `fan_out_dead_letter` is SKIPPED —
  firing the consuming app's per-app handlers on a shared message (competing
  consumers → nondeterministic app) would be wrong. Fan-out to a *shared*
  dead_letter trigger is deferred (needs a new trigger kind).
- Read side: `GroupDeadLetterRepo::list_for_group` backs a new read-only
  operator endpoint GET /api/v1/admin/groups/{id}/dead-letters (GroupKvRead,
  mirrors the M4 group-blobs surface). `pic dead-letters ls --group` deferred
  (optional; the HTTP endpoint is the operator surface).

Pinned by group_queue::dead_letter_moves_an_exhausted_message_to_the_group_store
(store move + claim-token-mismatch guard + operator read). Schema golden
reblessed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 14:41:04 +02:00
MechaCat02
c06a9e801e feat(apply): make the per-env approval gate hermetic (§3 M3 / Track A M1)
The per-env approval policy was applier-supplied — a hand-crafted request
that omitted `project.environments` was ungated, and flipping a gate to
`confirm = false` in the same request un-gated it. Persist the policy
server-side and enforce against `persisted ∪ declared`.

- Migration 0067: `project_environments (project_id, env_name, confirm)`,
  CASCADE on the project. Written declaratively (delete-then-insert) inside
  `upsert_project_tx`, same tx as the project row + node claim.
- `ProjectRepository::get_environments_by_slug` (read side) +
  `ApplyService::effective_env_policy` union the persisted policy with the
  request's declared one (confirm if EITHER says so — monotonic).
- `env_gate_check` now evaluates the effective policy; the three handlers load
  it before the gate. `plan.approvals_required` is the effective gated set, so
  CI sees a persisted gate even when the manifest omits it.
- The union rule closes both bypasses: an omitted policy still trips a
  persisted gate, and an ungating apply must itself pass the gate (the
  declarative replace then takes effect next time) — TOCTOU-safe.

Pinned by env_approval::{persisted_policy_gates_a_request_that_omits_it,
flipping_a_gate_to_false_in_the_same_request_still_gates} +
projects_repo::get_environments_by_slug_returns_persisted_policy. Schema golden
reblessed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 14:31:15 +02:00
MechaCat02
4b4b3148b7 feat(apply): server-side enforcement of per-env approval gating (§3 M3 / §4.2)
The per-env approval gate (`[project.environments]`, `confirm = true`) was
client-side only — the policy was `#[serde(skip_serializing)]`, so a patched or
older CLI applying to a confirm-required env bypassed it entirely. This makes
the gate server-enforced, admin-gated, and audited (the deferred §4.2 piece).

- Wire: `ProjectDecl` gains `environments` (the CLI stops skip_serializing it and
  sends the selected `env` + the `--approve` set on the apply request). The
  server RE-DERIVES the gate from the request (`env_gate_check`): a
  confirm-required env not in `approved_envs` → 409 `ApprovalRequired`.
- Admin gate: an approved override additionally requires `AppAdmin`/`GroupAdmin`
  on EVERY declared node — a step up from the editor write caps an ordinary
  apply needs (reusing the admin caps per §4.2) — and emits a `tracing::info!`
  audit line. Enforced on all three handlers (single app, single group, tree);
  the check runs before any tx, so a denial (403) precedes any mutation.
- Token: the env policy folds into the TREE bound-plan token (a policy edit
  between plan and apply trips StateMoved). The single-node token is left
  unchanged (a bare live-state hash), so plan/apply still match without threading
  a project into it.
- CLI: single-node `apply`/`plan` now resolve the GOVERNING project (own block
  else nearest ancestor's) so a leaf apply carries the root's policy; `plan`
  surfaces `approvals_required`. Client-side `require_env_approval` fast-fail
  kept as UX.

Adapts the earlier feat/ownership-and-approval M5 (654e387) to main's DTOs; main
addresses tree group nodes by slug only, so the admin loop mirrors authz_tree
(app: resolve_app_id fail-closed; group: get_by_slug, skip to-create).

Threat-model note (honest scope, in the doc + code comments): this closes the
patched/older-CLI bypass and admin-gates + audits the override, but is NOT a
hermetic boundary against a hand-crafted request that OMITS the policy — the
policy is applier-supplied, not persisted server state. Full closure (persist a
`project_environments` source of truth + a server-determined env) is deferred.

Review (adversarial pass): fixed a MEDIUM (single-node `plan` didn't resolve the
governing project, so a leaf plan under-reported the gate `apply` enforces) and a
LOW (duplicate clippy attr); corrected overstated "can't bypass" wording.

Tests: ProjectDecl gate unit test; env_approval journeys
`server_enforces_the_gate_against_a_non_stock_client` (raw request → 409 without
approval, admin-approved → 200) and `approving_a_gated_apply_requires_admin`
(editor + --approve → 403). 417 lib tests + 33 CLI journeys + workspace clippy
-D warnings + fmt clean. No migration.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 14:01:04 +02:00
MechaCat02
bbfdb1bf47 fix(apply): bound the blast-radius subtree down-walk CTE
`group_blast_radius`'s `app_chain` up-walk is depth-capped (`ac.depth < 64`,
added by 76926de to stop a should-be-impossible `groups.parent_id` cycle from
spinning the recursive CTE forever), but the sibling `subtree` down-walk in the
same query was left unbounded. A `parent_id` cycle would loop `subtree`
indefinitely and hang the plan — the exact exposure 76926de set out to close,
left half-done on the descendant side.

Add a `depth` column + `WHERE s.depth < 64` to `subtree`, mirroring `app_chain`.
Defense-in-depth: the reparent cycle-guard already prevents `parent_id` cycles,
so this hardens against an unreachable state — but that is precisely the bar
76926de applied. Surfaced by a post-adoption audit of the §7/§6 ownership code.

Verified: 416 manager-core lib tests; 12 ownership/create/divergence/approval
journeys (incl. plan_previews_ownership_and_blast_radius, which drives this
query); clippy -D warnings + fmt clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 13:25:06 +02:00
MechaCat02
87292fc4e9 refactor(apply): split reconcile_group_structure_tx create/reparent branches
Some checks failed
CI / Rust — fmt, clippy, test (push) Failing after 19m24s
CI / Dashboard — check (push) Successful in 9m46s
Review #4 (LOW, cleanup). `reconcile_group_structure_tx` carried a
`#[allow(clippy::too_many_lines)]` over a ~145-line body doing create +
reparent + attach/RBAC/lock in one loop. Extract the create branch
(`create_group_node_tx`) and the divergence-resolution branch
(`reparent_diverged_group_tx`) into helper methods, threading the shared
attach-ceiling / principal / mode context through a small `ReconcileCtx` — the
loop is now thin enough to drop the allow. Also improve the child-before-parent
error in `resolve_declared_parent` to hint that group nodes must be ordered
parents-first (only a hand-rolled bundle hits it; the CLI already sorts).

No behavior change. Documents the Tier-1 review follow-ups in the design doc.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 21:49:18 +02:00
MechaCat02
6ec034fb1d refactor(apply): unify divergence detection and drop the plan-path N+1
Review #3 (LOW). `divergence_preview` (plan path) compared parent slugs
case-sensitively from its own `get_by_slug`, while the apply path
(`reconcile_group_structure_tx`) compares resolved parent ids. On a mixed-case
parent slug the two could disagree — `pic plan` reporting `diverged` while
`pic apply` hard-errors "parent does not exist".

`resolve_existing_group_ids` becomes `load_existing_groups`, returning the full
`Group` rows it already loads; `plan_tree` derives the id map from them and
passes the map to `divergence_preview`, which now reads each node's own row
without a per-node `get_by_slug` (kills the N+1), resolves an in-tree parent's
slug from the same map, and compares parent slugs case-insensitively so the
preview agrees with the apply path's id-based check.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 21:49:05 +02:00
MechaCat02
2e3263002a fix(apply): gate group content writes in-tx to close a create-race authz hole
Review #1 (HIGH). M1's `authz_tree` skips the group content-write capability
check (`require_group_node_writes`) for a group node absent at API-request
time — a to-create group whose authorization is the service create-gate. But
that gate only fires when the group is STILL absent at reconcile, and the
content writer performs no authz. So a group created by a racer in the window
between the API check and the in-tx reconcile could have its scripts/vars
written by a principal with zero rights on it (the bound-plan token doesn't
save a hand-rolled request that omits `expected_token`).

Close it with an in-tx content-write re-check in `apply_tree` for every group
NOT structurally created/reparented by this apply: a group WE created is
covered by create-RBAC (`GroupAdmin(parent)` cascades) and its uncommitted row
is invisible to the pool-based authz cascade anyway, so it must not be
re-checked; a pre-existing (incl. racer-created) group is committed, so its
ancestry resolves and a missing cap denies. `require_group_node_writes` moves
from `apply_api` onto `ApplyService` (up the existing dependency edge) so the
API pre-check and the in-tx re-check share one implementation.

Pinned by `tests/group_create.rs::tree_apply_denies_group_content_write_without_the_cap`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 21:47:59 +02:00
MechaCat02
cb3d458cfd feat(apply): §6 M2 — structural-divergence detection + declarative reparent
With the declared parent on the wire (M1), `apply_tree` now compares each
EXISTING group node's server parent to the manifest's (directory-nesting)
parent and resolves a divergence per a request `StructureMode` (default
`Refuse` — a pre-M2 CLI never reshapes the tree):

- `Refuse` → `ApplyError::StructuralDivergence` (422), naming the flags.
- `ForceLocal` → reparent to the declared parent IN the apply tx via the
  extracted `group_repo::reparent_group_tx` (cycle guard + structure_version
  bump), §5.6-gated (`GroupAdmin` on node + source + destination) and
  attach-ceilinged.
- `AdoptServer` → keep the server shape, reconcile content in place.

M1's `create_missing_groups_tx` generalized to `reconcile_group_structure_tx`
(create + reparent share the coarse-lock / attach-ceiling / RBAC path; both are
recorded `structurally_changed` so the pool-based attach re-check skips their
uncommitted rows). `pic plan` previews the drift (`divergence_preview` → a
`structure` row naming server + manifest parents). CLI:
`--force-local-structure` / `--adopt-server-structure` (mutually exclusive) →
the `structure_mode` wire field; the 422 surfaces verbatim. A concurrent
`pic groups reparent` between plan and apply still trips `StateMoved` (the tree
token already folds structure_version).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 20:23:10 +02:00
MechaCat02
0078ae8b26 feat(apply): §6 M1 — declarative group create (directory-nesting parentage)
Lift "groups must pre-exist" for the tree apply: a `[group]` node now carries a
declared parent (inferred from directory nesting) and `pic apply --dir` creates
the ones the server lacks, in the same transaction.

Wire: `TreeNode` gains `parent`/`name`/`description` (`#[serde(default)]` — a
pre-§6 CLI still reconciles existing groups). `discover::build_tree` computes a
group's parent as the nearest ancestor directory holding a `[group]` (topmost →
the repo's `[project] parent_group` attach point, else instance root) and emits
nodes parents-first by directory depth.

Server: `apply_tree` runs a Phase 0 (`create_missing_groups_tx`) that
resolves-or-creates every group node IN the tx (a same-tree parent resolved
before its child via the new `group_repo::{create_group_tx,
read_group_id_by_slug_tx}`), under the coarse `GROUP_STRUCTURAL_LOCK_KEY` (taken
only when creating), enforcing the attach-point ceiling + RBAC
(`InstanceCreateGroup` / `GroupAdmin(parent)`) per create. A created group is
CLAIMED in Phase A alongside existing ones (`decide_group_claim` → `Claim`).
`prepare_tree` now takes a caller-supplied `resolved_ids` map and returns a
`to_create` list the plan path previews (empty current → full-create plan,
ownership `claim`); a to-create group's token part equals its post-create part
so plan/apply tokens match across a create. `validate_bundle_for` takes
`is_group: bool` (a to-create group has no id yet); `authz_tree` skips a
to-create group (its create authz is the service gate, not an existing-group
capability).

Reparent of a diverged existing group + the detect-and-refuse flags are M2.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 20:07:03 +02:00
MechaCat02
76926de369 fix(apply): bound the blast-radius up-walk; fix two stale comments
Second-review follow-ups on the ownership cleanups:

- The new `group_blast_radius` `app_chain` recursion walked app→root with no
  depth cap, unlike the `ancestors()` CTE it replaced (`WHERE c.depth < 64`).
  Add the same `ac.depth < 64` guard so a (should-be-impossible) cycle in
  `groups.parent_id` can't spin the recursive CTE unbounded and hang a plan.
- Drop the now-false "memoized per group" line from `group_blast_radius`'s doc
  (it is a single set-based query now).
- Correct the `PlanRequest` flatten comment: apply is deliberately NOT flattened
  (its pre-§7 wire was already `{bundle, …}`), so the two endpoints are
  asymmetric by history — the old comment wrongly claimed parity.

Verified: the blast-radius journey + ownership journeys still pass; the INNER
JOIN to `projects` is safe (owner_project FK is ON DELETE SET NULL, so a
non-NULL owner always references a live row — no dangling drop).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 19:21:03 +02:00
MechaCat02
86b51df50d refactor(apply): address ownership review follow-ups (#3–#6)
Four cleanups from the code review of the §7 ownership track, no behavior
change to any passing test (416 manager-core units + 133 CLI journeys green):

- #3 app-ownership read parity: `check_app_owner_single` now resolves the
  nearest-claimed ancestor WITHIN the apply tx (`nearest_claimed_in_tx`, the
  same mechanism the tree path uses) instead of on a separate pool connection,
  so the check and this apply's writes see one consistent snapshot. (Fully
  serializing against a concurrent claim on an out-of-tree ancestor stays part
  of the documented pool→tx follow-up.)

- #4 blast-radius N+1 → one query: `group_blast_radius` replaced the
  per-descendant-group `ancestors()` round-trips with a single recursive
  `subtree → app_chain → nearest` CTE that groups + counts server-side. Cost is
  now one query regardless of subtree size.

- #5 one ownership policy, two projections: `decide_group_claim` /
  `decide_app_owner` are generic over the identity key, and `preview_ownership`
  (plan side) now DERIVES its label from them (keyed on slugs, takeover-agnostic)
  rather than re-encoding the arms — plan and apply can no longer drift.

- #6 shared slug validator: `validate_project_slug` delegates to a new
  `groups_api::validate_slug_format`, which `groups_api::validate_slug` also now
  uses — the format rule lives once. Projects intentionally skip the
  reserved-word check (a project slug is never routed).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 07:49:52 +02:00
MechaCat02
ffa8b02506 fix(apply): restore bare-bundle wire compat on the plan endpoints
§7 M3 wrapped the three plan request bodies in a `{bundle, project}` struct,
silently breaking version-skew compatibility: a pre-§7 `pic plan` POSTs a BARE
bundle at the JSON top level and now fails deserialization (422 "missing field
bundle"), even though the sibling apply request deliberately kept its added
fields additive (`#[serde(default)]`) so an older CLI still works.

`#[serde(flatten)]` the bundle in `PlanRequest`/`TreePlanRequest` so the bundle
fields sit at the top level again: a bare bundle deserializes (`project`
defaults to `None`) and a newer client sends the same fields plus a top-level
`project` key. The CLI now builds that shape via a `plan_body` helper (bundle
object + optional `project`), which an older server also accepts (Bundle has no
`deny_unknown_fields`, so the extra key is ignored) — compatible in both skew
directions. Apply is left unwrapped (it was already `{bundle, ...}` pre-§7).

Pinned by `plan_request_accepts_a_bare_bundle` / `_a_flattened_project` /
`tree_plan_request_accepts_a_bare_bundle` unit tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 07:37:35 +02:00
MechaCat02
74f7a67be7 fix(apply): preserve project name on a name-less re-apply
`upsert_project_tx` defaulted a missing `[project].name` to the slug and then
`ON CONFLICT (slug) DO UPDATE SET name = EXCLUDED.name` wrote that fabricated
value unconditionally — so a re-apply from a clone whose manifest omits `name`
silently clobbered the stored display name back to the slug (visible in
`pic projects ls` / `pic groups ls`).

Bind the raw optional name once and guard both sides on it:
`VALUES ($1, COALESCE($2, $1), $3)` (first apply falls back to the slug for the
NOT NULL column) and `DO UPDATE SET name = COALESCE($2, projects.name)` (a
re-apply updates the name only when the manifest actually declares one). An
omitted optional field now preserves persisted data instead of mutating it.

Pinned by a new `reapply_without_name_preserves_project_name` journey.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 07:37:25 +02:00
MechaCat02
f673922d89 feat(cli): pic projects ls + §7 M3/track-complete docs
§7 M3 (part 3) — the ownership track becomes inspectable, completing M1–M3.

- server: GET /api/v1/admin/projects (projects_api) lists every project + its
  owned-group count (projects repo list_with_counts, one GROUP BY); behind the
  /admin middleware, like the groups list.
- CLI: pic projects ls (cmds/projects.rs + client projects_list); the
  apply_ownership plan-preview journey now also asserts projects ls (plat owns
  exactly 1 group; teamb listed).
- docs: design §7 + CLAUDE.md record M3 shipped and the whole §7 multi-repo
  ownership track (M1 claim · M2 attach ceiling · M3 preview + projects ls)
  COMPLETE; deferred = structural-divergence + declarative tree shape + per-env
  approval gating.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 21:26:10 +02:00
MechaCat02
30fe160f16 feat(apply): cross-repo blast-radius in the plan preview
§7 M3 (part 2): planning a change to a GROUP node now reports the descendant
apps owned by OTHER projects that the change fans out to — the cross-repo
signal an operator needs before touching shared config.

- OwnershipPreview gains `blast_radius: Vec<ProjectImpact>`; `group_blast_radius`
  walks the group's subtree (recursive CTE) and groups descendant apps by their
  nearest-claimed-ancestor project, excluding the planning project. Ancestor
  resolution is memoized per group (one walk per distinct descendant group, not
  per app).
- `pic plan` renders `blast_radius` rows (mode-agnostic).
- the apply_ownership journey gains a plan-preview case pinning both the
  ownership annotation (claim / conflict) and the blast radius (two other
  projects' apps surfaced) end to end.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 21:17:15 +02:00
MechaCat02
b4eb226d20 feat(apply): project on the plan wire + ownership preview annotation
§7 M3 (part 1): `pic plan` now surfaces the ownership outcome BEFORE apply.

- plan / plan_owner / plan_tree take the declaring `[project]`; each result
  carries an `ownership` preview (pure `preview_ownership`, unit-tested):
  claim (unclaimed + project), owned (already this project), conflict (owned by
  X — named), or unclaimed. A group node previews its OWN owner; an app node its
  nearest claimed ancestor (read-only, on the pool).
- the attach-point ceiling (M2) is now previewed at plan too, so `pic plan`
  outside the subtree fails early — consistent with apply.
- wire: PlanRequest / TreePlanRequest wrap { bundle, project } (project
  `#[serde(default)]` → a pre-M3 CLI still plans); the plan DTOs + `pic plan`
  gain an `ownership` row (mode-agnostic: shows in tsv + json).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 21:09:55 +02:00
MechaCat02
5bb1ccad5e feat(apply): [project] parent_group attach-point ceiling
§6/§7 M2: a project's attach point is its ceiling — applies are refused for
any node not strictly within the declared subtree, so a repo can't reach (or
claim) above its local root even with the RBAC to do so.

- `[project] parent_group = "<slug>"` (ManifestProject + ProjectDecl); absent
  = instance root = no ceiling (the default, backward-compatible).
- `check_within_attach`: the attach group must be a PROPER ancestor of a group
  node (you can't apply the attach point itself) or an ancestor (inclusive) of
  an app node's group; resolved via `groups.ancestors`. Enforced read-only in
  apply_owner (prologue) + apply_tree (per node), before the claim.
- `ApplyError::OutsideAttachPoint` → 422 (a scope error, like Invalid).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 20:57:03 +02:00
MechaCat02
b33c87e5c4 feat(cli): [project] manifest block, --takeover, groups ls owner column
Makes §7 ownership usable end to end from the CLI.

- manifest: a `[project]` block (slug + optional name), independent of the
  [app]/[group] XOR; threaded onto every apply from the repo. --dir reads it
  from the tree ROOT's picloud.toml (a [project] elsewhere is ignored with a
  note).
- client: apply_node/apply_tree carry project + takeover; the 409 branch now
  surfaces the server message verbatim (covers StateMoved AND OwnershipConflict,
  both self-contained). New groups_list_with_owner captures the owner slug.
- pic apply --takeover flag; pic groups ls gains an `owner` column (the owning
  project's slug, or — when unclaimed).
- server: /api/v1/admin/groups now returns each group flattened with its owner
  slug (via list_with_owner) — the shared Group deserialize ignores the extra
  field, so pic groups tree / the dashboard are unaffected.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 20:46:09 +02:00
MechaCat02
47072c481d feat(apply): §7 ownership claim state machine + project/takeover wire + 409
The heart of multi-repo ownership: an apply now claims/verifies node ownership
before it reconciles, so two repos can't clobber each other's subtree.

- Pure policy (DB-free, unit-tested): `decide_group_claim` (unclaimed→claim,
  owner→noop, foreign→conflict unless --takeover, no-project-into-claimed→
  conflict) and `decide_app_owner` (an app must match its nearest claimed
  ancestor; an unclaimed subtree stays open — backward-compatible).
- Execution under the per-node advisory lock: `upsert_project_tx` (first apply
  registers the project), `read/write_group_owner_tx`, `claim_group_owner`
  (takeover additionally requires `GroupAdmin` — ownership ⟂ RBAC),
  `check_app_owner_single` + tree `nearest_claimed_in_tx` (in-tree Phase-A
  claim overlays committed ancestors). No structure_version bump — a claim
  isn't a diff change, so it must not churn a pending bound plan.
- `apply`/`apply_owner`/`apply_tree` take an `OwnershipClaim` (project +
  takeover + principal); the claim slots after the lock, before load_current,
  so a conflict short-circuits before any diff work.
- New `ApplyError::OwnershipConflict` → HTTP 409 (actionable); wired through
  `ApplyRequest`/`TreeApplyRequest` (both `#[serde(default)]`, so the existing
  CLI stays compatible until it learns [project]).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 20:27:51 +02:00
MechaCat02
2cce051dc4 feat(projects): projects repo + group owner reads; wire into ApplyService
Read side of §7 ownership, ahead of the claim logic.

- project_repo: `ProjectRepository` trait + `PostgresProjectRepository`
  (list / get_by_id / get_by_slug). Writes are NOT here — a claim registers
  its project via an in-tx upsert (next commit).
- group_repo: `list_with_owner()` LEFT-JOINs projects for each group's owner
  slug (backs `pic groups ls`); columns qualified since id/slug/name exist on
  both tables.
- ApplyService gains a `projects` handle; constructed in the picloud binary.
- tests/projects_repo: round-trip pinning list_with_owner (claimed→slug,
  unclaimed→None) and that ancestors() surfaces owner_project nearest-first —
  the fold input the claim path walks.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 20:13:34 +02:00
MechaCat02
ba97c35aaf feat(projects): projects table + owner_project FK; ProjectId/Project types
First commit of §7 multi-repo ownership. A 'project' is a repo-root that
declaratively manages a slice of the shared group tree; each group node is
owned by exactly one project (single-owner-per-node). This lands the registry
+ shared types and wires the inert 0047 `owner_project` seam — no behavior
change yet (claim logic is the next commit).

- migration 0066: `projects` table (UUID pk, unique slug, name, created_by →
  admin_users ON DELETE SET NULL); `groups.owner_project` gains its FK →
  projects(id) ON DELETE SET NULL (un-claim, never cascade-destroy a tree) +
  an index.
- shared: `ProjectId` (id_type! macro) + `Project` struct; `Group` gains
  `owner_project: Option<ProjectId>`.
- group_repo: GROUP_COLS + the ancestors recursive-CTE term now carry
  `owner_project`, so `ancestors()` surfaces ownership for the nearest-claimed
  fold; GroupRow + From updated.
- schema snapshot re-blessed; /version schema assertion 65 → 66.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 20:01:57 +02:00
MechaCat02
50205e7b7e feat(shared-queues): materialized competing consumers + dispatcher branch (D3.2)
The consumption side of shared durable queues:
- Thread `shared` through BundleTrigger::Queue + QueueTriggerSpec + the
  trigger identity; validate_bundle_for requires a shared queue on a group
  to name a declared kind='queue' collection (a shared queue on an app is
  rejected by the existing app-owner shared guard).
- materialize: a shared queue template materializes a consumer per
  descendant (the M5 one-consumer-slot skip is bypassed for shared —
  competing consumers are intended; each descendant gets one copy).
- dispatcher: ActiveQueueConsumer gains shared_group (from the materialized
  copy's source template via LEFT JOIN); dispatch_one_queue +
  handle_queue_failure route claim/ack/nack/terminal to the group store
  when shared_group is Some, via q_claim/q_ack/q_nack/q_terminal helpers. A
  group claim is normalized to a ClaimedMessage under the consuming app so
  the handler path is unchanged; the reclaim task also drains the group
  store. Exhausted shared messages are dropped (no group dead-letter store
  yet — documented deferral).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 22:33:11 +02:00
MechaCat02
ccd3644aa4 feat(shared-queues): group-keyed queue store + enqueue service + SDK (D3.1)
The producer side of shared durable queues:
- migration 0065 group_queue_messages (mirrors 0034, keyed by (group_id,
  collection); CASCADE on group delete).
- GroupQueueRepo/PostgresGroupQueueRepo: enqueue + the competing-consumer
  claim (FOR UPDATE SKIP LOCKED) + ack/nack/drop_exhausted/reclaim/depth.
- GroupQueueService trait (shared) + GroupQueueServiceImpl: resolve owning
  group (kind='queue') from cx.app_id's chain, require editor+
  (GroupQueueEnqueue, fails closed on anon), size-cap, enqueue.
- SDK: `queue::shared_collection("name")` -> GroupQueueHandle with
  `.enqueue(msg[, opts])` / `.depth()` / `.depth_pending()`; wired through
  Services + the picloud binary.
- authz: Capability::GroupQueueEnqueue(GroupId), editor+ / script:write.

Deterministic test proves competing consumers claim each message exactly
once. Consumption wiring (materialized consumers + dispatcher branch) lands
in D3.2.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 22:08:21 +02:00
MechaCat02
19ba6dbfb4 test/docs(shared-topics): dispatch test + journey + docs (D2)
Deterministic manager-core test (tests/shared_topics.rs) proves the
namespace boundary both ways: a shared publish fans out only to the
group's shared pubsub trigger (not a per-app or non-shared group
template), and a per-app publish never hits the shared trigger. CLI
journey (shared_topics) covers authoring + `pic triggers ls --group`
shared column + the two validation rejections (undeclared topic; shared
on an app), mirroring the shared_triggers norm. Docs: CLAUDE.md + design
doc §11.6 record D1 + D2 as shipped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 21:54:32 +02:00
MechaCat02
9626d15863 feat(shared-topics): GroupPubsubService + shared publish fan-out + SDK handle (D2)
The runtime for shared topics:
- pubsub_repo: fan_out_publish gains `AND t.shared = FALSE` (a shared
  trigger never fires on a per-app publish); new fan_out_shared_publish
  matches `shared = true` pubsub triggers on the resolved owning group,
  each delivery stamped with the writer app_id (M2 model).
- GroupPubsubService trait (shared) + GroupPubsubServiceImpl: resolve the
  owning group (kind='topic') from cx.app_id's chain, require editor+
  (GroupPubsubPublish, fails closed for anon), size-cap, fan out.
- SDK: `pubsub::shared_topic("name")` -> GroupTopicHandle with
  `.publish(subtopic, msg)` / `.publish(msg)`; wired through Services +
  the picloud binary (reuses the one PubsubRepo).
- authz: Capability::GroupPubsubPublish(GroupId), editor+ / script:write.

The owning-group chain walk is the isolation boundary; a sibling-subtree
app never resolves the topic.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 21:48:59 +02:00
MechaCat02
1c7e8886d8 feat(shared-topics): thread the shared flag through pubsub triggers (D2)
Add `shared` to BundleTrigger::Pubsub + PubsubTriggerSpec + the trigger
identity (so toggling re-diffs) + current_trigger_identity. validate_bundle_for
now allows a `shared` pubsub trigger on a group and requires the topic
pattern's ROOT segment (events.* -> events) be a declared kind='topic'
collection; a wildcard root is a runtime match. Apply-side plumbing only —
the shared publish path + dispatch boundary land next.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 21:37:31 +02:00
MechaCat02
ae8f0be748 feat(shared-collections): admit 'topic' and 'queue' collection kinds (D2/D3)
Widen the group_collections kind allow-list (migration 0064) + the
manifest CollectionKind enum + apply_service COLLECTION_KINDS to include
'topic' (D2, storeless publish namespace) and 'queue' (D3, group-keyed
durable queue — store lands in 0065). Foundation shared by both
milestones; routes through the existing owner-generic reconcile +
resolve_owning_group path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 21:32:41 +02:00
MechaCat02
137103a429 feat(triggers): show a materialized column in pic triggers ls --app (D1)
A materialized copy of an M5 group stateful template (cron/queue/email) now
reads as `materialized = true` — inherited, read-only — distinct from a
hand-authored trigger. Threads a derived `materialized` bool (=
`materialized_from IS NOT NULL`) row → domain → API → CLI, mirroring the
`sealed`/`shared` columns; `list_for_app` SELECTs `materialized_from`. No
migration.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 21:30:32 +02:00
MechaCat02
849bd367b9 test(triggers): mirror the app_id-not-null inbound guard in the mock (review)
Adversarial review NIT: the #[cfg(test)] InMemoryTriggerRepo's
email_inbound_target `.expect()`s app_id, which would panic if a future
unit test fetched a group-owned email TEMPLATE. Filter `app_id.is_some()`
like the production query's `AND t.app_id IS NOT NULL` so the mock stays
faithful. Test-only; no production change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 21:05:23 +02:00
MechaCat02
da83222ea1 test(stateful-templates): group email template journey + load group secrets (M5.5)
CLI journey: set a group secret, apply a group [[triggers.email]] template
referencing it, create an app under the group → `pic triggers ls --app`
shows a materialized email trigger; re-apply is a NoOp.

Required loading a group's own set-secret names into CurrentState (was
hardcoded empty) so the apply plan-time email-secret check resolves the
ref against the GROUP store. Informational only — the secrets diff is
never executed on apply (secrets are set out-of-band via `pic secret
set`); the state token now folds in group secrets, matching apps.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 20:29:23 +02:00
MechaCat02
e2457efcbe test(stateful-templates): group email template materialization (M5.5)
Drive rematerialize directly with a group email template carrying a fake
sealed secret: a descendant app gets one copy whose inbound-secret
ciphertext + nonce byte-equal the template's (verbatim copy, no reseal),
the reconcile is idempotent, and reparenting the app out of the subtree
de-materializes the copy.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 20:25:22 +02:00
MechaCat02
9dc9191cb2 feat(stateful-templates): materialize group email templates (M5.5)
Close the last M5 gap: a [group] may now declare an email trigger
template, materialized into a per-descendant-app row like cron/queue.

Uses the shared-group-secret model: the template resolves its
inbound_secret_ref against the GROUP's own secret store once at apply and
seals it (resolve_and_seal + insert_email_trigger_tx generalized to a
SecretOwner / ScriptOwner). Email secrets are v0/no-AAD, so the sealed
blob is portable across rows — materialize copies the bytes verbatim into
each descendant (no master key, no reseal, no new schema column, no
threading into the CRUD hooks). Each copy has its own trigger_id / inbound
webhook URL.

- apply_service/manifest: drop the two group-email rejections; a group
  email template with an unset group secret still fails apply hard.
- materialize: add "email" to MATERIALIZED_KINDS + a byte-copy detail arm.
- trigger_repo: email_inbound_target gains AND t.app_id IS NOT NULL so a
  group TEMPLATE row is never directly invocable via its own webhook URL
  (mirrors the cron/queue app_id guards).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 20:24:10 +02:00
MechaCat02
05373814b2 fix(stateful-templates): serialize the materialization reconciler (review)
The reconciler read `should` and `existing` in separate statements under READ
COMMITTED, and its queue one-consumer check wasn't locked — so two concurrent
reconcilers (e.g. two parallel app-creates) could (a) spuriously DELETE a valid
materialized cron copy when one committed a copy between the other's two reads
(a lost copy self-healing only on the next mutation), or (b) both pass the
queue slot check and double-fill a consumer slot. Take a fixed xact-scoped
advisory lock (REMATERIALIZE_LOCK_KEY) at the top of the reconcile so each pass
is atomic w.r.t. the others. Reconciles are infrequent + fast, so serializing
is cheap; the ON CONFLICT guard stays as defense in depth.

Found by the M5 review. Pinned by the existing stateful_templates test + the
full journey suite (which previously flaked on the race).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 20:06:11 +02:00
MechaCat02
aa5bb3e8cc feat(stateful-templates): journey + docs + concurrency fix; email deferred (M5.6)
- 0063: partial unique index on (app_id, materialized_from) + ON CONFLICT DO
  NOTHING in the reconciler, so a materialized copy is idempotent under
  concurrent reconciles (two parallel app-creates no longer duplicate a copy).
- stateful_templates journey: a group cron template + a descendant app → a
  materialized cron copy in `pic triggers ls --app`; re-apply is a NoOp.
- docs (§4.5 + CLAUDE.md): cron+queue materialization implemented; group EMAIL
  templates deferred (per-descendant inbound-secret reseal needs the master key
  threaded through the hooks + a secret-ref schema) and cleanly rejected at
  apply for now, alongside a `materialized` ls column.

Completes the v1.2 near-term batch (M1-M5, cron+queue); group email is the one
scoped follow-up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 21:54:34 +02:00
MechaCat02
ddb3589c49 feat(stateful-templates): materialize cron+queue templates + hooks (M5.2-M5.4)
materialize::rematerialize_stateful_templates reconciles a group cron/queue
template into one app-owned copy per descendant (materialized_from = template),
via the all-apps app_chain CTE ⋈ group-owned stateful templates. A precise
create/delete diff preserves cron last_fired_at on unchanged copies; a queue
copy is skipped-with-warning when the app already fills that queue's consumer
slot (the one-consumer invariant). Called full-live at the route-rebuild
chokepoints: apply (single + tree), app create/delete (apps_api), group
reparent (groups_api) — AppsState/GroupsState gain a pool. Pinned by
tests/stateful_templates.rs (per-app copy, idempotent, new-app materializes,
reparent de-materializes).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 21:47:27 +02:00
MechaCat02
456e972336 feat(stateful-templates): schema + dispatch guards + allow cron/queue on group (M5.1)
0062 adds `materialized_from` on triggers (a managed app-owned copy links back
to its group template; CASCADE). The scheduler + queue-consumer queries gain
`AND t.app_id IS NOT NULL` so a group-owned stateful TEMPLATE is never
dispatched directly — only the per-descendant materialized app rows are.
validate_bundle_for + manifest parse now allow cron/queue on a group (email
stays rejected pending its per-app inbound-secret handling in M5.5). The
materialization reconcile that expands templates into app rows lands in M5.2.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 21:34:14 +02:00
MechaCat02
c21e82fb9f feat(group-blobs): read-only operator admin API for shared collections (M4.1+M4.2)
group_blobs_api mirrors the per-app kv_api/files_api for groups: GET
/groups/{id}/kv[/{c}/{k}], /docs[/{c}/{id}], /files[/{c}/{id}] — list +
fetch/download a group's §11.6 shared KV/docs/files. Authz GroupKvRead/
GroupDocsRead/GroupFilesRead (viewer tier, reads-open trust model). Read-only
by design (writes stay script-only). The host hoists the three group repos so
the SDK services and this router share one instance.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 21:22:07 +02:00
MechaCat02
2bc3fb677b feat(group-quota): per-group shared-collection quotas (M3)
Global env-var ceilings enforced in the group write path (mirrors the
per-value caps): PICLOUD_GROUP_KV_MAX_ROWS / _DOCS_MAX_ROWS (per-group row
count, checked only on a NEW key/doc — updates exempt) and
_FILES_MAX_TOTAL_BYTES (per-group total blob bytes). New group_quota env
helpers; count_rows/total_bytes repo methods (trait-default 0 for non-Postgres);
QuotaExceeded error variants on the three group error enums; a with_max_rows
test builder. Pinned by a group_kv_service unit test (3rd key rejected, update
of an existing key at the cap allowed).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 21:17:12 +02:00
MechaCat02
9a4b83dfcf feat(shared-triggers): visibility + tests + docs (M2.5)
- `shared` column in `pic triggers ls --group` (TriggerTemplateInfo + DTO +
  report + renderer).
- manager-core/tests/shared_triggers.rs: a shared write matches the group's
  shared trigger and NOT a same-named per-app trigger; a per-app write matches
  the app trigger and NOT the shared one (the `shared` flag is the boundary).
- shared_triggers journey: declarative authoring applies, ls --group shows
  shared, an undeclared-collection shared trigger + an app shared trigger are
  rejected.
- docs: §11.6 + CLAUDE.md move shared-collection triggers from Deferred to
  implemented.

Completes M2. (Pre-existing async-dispatcher timing flakes pass in isolation.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 21:05:30 +02:00
MechaCat02
f04eaed0b7 feat(shared-triggers): author + persist + validate shared templates (M2.4)
`shared = true` on a group [[triggers.kv|docs|files]] flows manifest → Bundle
→ insert_trigger_tx (new shared column), mirroring sealed: shared lives on the
Trigger DTO + is part of the apply diff identity (bundle + current sides) so a
toggle re-applies. validate_bundle_for rejects shared on an app owner, on a
non-collection kind (pubsub/cron/etc.), and on a concrete collection the group
does not declare as a shared collection of that kind. Emission (M2.3) now has
authored triggers to match.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 20:55:49 +02:00