Commit Graph

371 Commits

Author SHA1 Message Date
MechaCat02
bc2538b5b5 fix(cli): reject unknown fields in manifest entry specs (fail-loud on typos)
`ManifestScript`, `ManifestRoute`, and every trigger spec
(`KvTriggerSpec`…`QueueTriggerSpec`) lacked `#[serde(deny_unknown_fields)]`,
while their parent structs all had it. So a typo inside a `[[triggers.kv]]` /
`[[routes]]` / `[[scripts]]` entry (e.g. `collection` for `collection_glob`,
`methid` for `method`) was silently dropped instead of erroring — the manifest
applied with the mistyped directive missing. Add the attribute to the 9 entry
structs (all fields are exhaustive, so `pull`-emitted TOML still round-trips —
verified against the full journey suite, 151/151). Pinned by a new
`unknown_key_in_an_entry_spec_is_rejected` unit test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 19:50:02 +02:00
MechaCat02
486cc44a06 refactor(authz): route users_service through the shared authz-verdict mapper
`users_service::require` open-coded the `AuthzDenied → {Forbidden, Backend}`
match that `authz::script_gate` centralizes for every other stateful service —
the one straggler. Extract the mapping into `authz::require_mapped` (used now by
both `script_gate` variants and `users_service::require`), so the verdict→error
mapping lives in exactly one place. Also refresh the stale `to_app_user` comment
(it referenced an unshipped "commit 8" stub; roles are resolved via
`fetch_roles` and passed in). No behavior change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 19:46:22 +02:00
MechaCat02
5e6bb762f4 refactor(groups): dedup the group-service value-size check + shared-emit tail
Two clean, local dedups in the group shared-collection services:

- Extract `GroupKvServiceImpl::check_value_size` — `set` and `set_if` inlined
  the identical encode-and-cap block; now one method (mirrors the sibling
  `group_docs_service::check_data_size`).
- Add `group_collection_repo::best_effort_emit_shared` — the KV/docs/files
  services each copied the same emit-and-log-on-error tail for shared-collection
  triggers. Centralize the tail (logging the event's own `source`/`op`); each
  service still builds its own `ServiceEvent` (payload shapes differ).

Pure refactor, pinned by the existing group-service unit tests. (The
`owning_group` resolver was evaluated for dedup too but left as-is: the five
error enums diverge — `GroupFilesError::InvalidCollection` carries a `String`,
`GroupPubsubError` lacks the variant, and files/pubsub skip the empty-check kv/
docs/queue do — so a shared conversion would be a behavior change, not a
refactor.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 19:44:06 +02:00
MechaCat02
f1b480f2eb refactor(sdk): consolidate the SDK error/bridge helpers
`http.rs` hand-rolled its own `block_on` + `map_http_err`, and `secrets.rs` /
`email.rs` / `users.rs` each carried a byte-identical `runtime_err` — five
`#[allow(clippy::unnecessary_box_returns)]` across four files for one pattern.

Move `runtime_err` to `bridge.rs` beside `block_on` (single home, single allow)
and have the three bridges import it. `http.rs` now calls the shared
`bridge::block_on("http", …)` (same "http:"-prefixed error, so messages are
unchanged) and its `err` validation helper delegates to `bridge::runtime_err`.
Deletes 4 duplicate fns and 3 of the 5 allows; no behavior change (pinned by
the executor-core SDK contract tests).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 19:36:31 +02:00
MechaCat02
aa3f0531f0 fix(workflows): pull hard-errors on a definition-less workflow, not silent skip
Some checks failed
CI / Rust — fmt, clippy, test (push) Failing after 23m16s
CI / Dashboard — check (push) Successful in 9m54s
Re-review flagged that the previous warn-and-skip (aeebdd2) traded a loud
failure for a SILENT prune hazard: against a server too old to return a
workflow's `definition`, pull dropped it with only a stderr warning, exited 0,
and wrote a workflow-less manifest — which a later `apply --prune` uses to
DELETE the server-side workflow (and `pull --force` would clobber a
hand-authored `[[workflows]]` block first).

A live workflow always has ≥1 step (zero-step is rejected at apply), so an
empty `definition` can only mean an out-of-date server. Pull now bails BEFORE
writing anything — consistent with the existing clobber / unsafe-name fail-fast
gates — so nothing partial hits disk and the operator gets an actionable error.
wire_workflow_to_manifest reverts to infallible (the guard upholds its
precondition).

Verified: fmt + clippy -D warnings clean; 5 workflow CLI journeys pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 19:19:03 +02:00
MechaCat02
aeebdd2f3f fix(workflows): close re-review gaps in the pull round-trip + partial-failure UI
A re-review of eaf5ace surfaced three follow-ups, all in the fixes themselves:

- pull vs an old server: a returned empty `definition` can only mean a server
  predating the definition-in-list field (a real zero-step workflow is rejected
  at apply). Emitting a stepless `[[workflows]]` block would just fail the next
  apply, so `wire_workflow_to_manifest` now warns and skips it instead.
- base_ms/backoff defaults: the pull mapper keyed its default-omission off a
  hardcoded `500` literal. `default_base_ms` is now `pub` + re-exported, and the
  mapper omits whatever equals `default_base_ms()` / `WorkflowBackoff::default()`
  — no drift if a default ever changes.
- dashboard: a run that succeeds with an `on_error = continue` step failure now
  carries a partial-failure note in `run.error` (from the compute_advance
  change); the run-detail view rendered it as a red failure, contradicting the
  green "succeeded" badge. It now renders as a `.notice` (warning) for a
  succeeded run, `.error` only for a failed one.

Verified: fmt + clippy -D warnings clean; 450 lib tests; 5 workflow CLI
journeys (incl. the pull→plan round-trip); dashboard npm run check clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 19:08:21 +02:00
MechaCat02
eaf5ace30f fix(workflows): close review gaps — pull round-trip, dup-names, reclaim budget
Adversarial review of the v1.2 Workflows track surfaced two HIGH footguns and
several correctness/hardening gaps. Fixes:

HIGH
- pull round-trip: `pic pull` dropped `[[workflows]]`, so a later
  `apply --prune` silently deleted them. The list endpoint now returns the full
  `definition`; pull rebuilds the manifest block (inverse of workflow_to_wire).
- case-colliding names: two workflows differing only by case collided in the
  reconcile diff (keyed by lower(name)), silently dropping one. Rejected up
  front in validate_bundle_for.

MEDIUM
- reclaim retry budget: a crashed attempt (no outcome) consumed the retry
  budget. reclaim_stale_steps now decrements `attempt` (floored) and clears
  `next_attempt_at`, so a crash no longer counts as a failed try.
- on_error/backoff: typed the manifest fields against the shared enums so a
  bad value fails at TOML parse with a clear message, not an opaque 500.
- dedupe depends_on: a repeated dependency inflated the Kahn in-degree past the
  single decrement, reporting a spurious cycle for a valid DAG. Count distinct.
- canceled child: a canceled sub-workflow resolved the parent step with a
  message naming the cause instead of a generic "failed"; documented that a
  run-level cancel op is not yet supported.

LOW
- list_run_steps is now app-scoped at the query (JOIN workflow_runs) rather
  than relying on caller pre-verification.
- partial failure is surfaced: a run that succeeds with on_error=continue
  failures records them in the run's `error` field.
- admin-started runs log the root_execution_id against the principal.
- documented the deliberate when(missing→false) vs template(missing→fail)
  asymmetry; corrected the claim's atomicity comment.

Tests: unit (dedupe-deps), DB-gated reclaim-budget assertion, and two new CLI
journeys (duplicate-name rejection, pull→plan clean round-trip). fmt + clippy
-D warnings clean; 450 lib + 14 orchestrator DB + 5 workflow journeys pass;
schema snapshot unchanged (no migration).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 18:56:44 +02:00
MechaCat02
bd9c3fa4f0 feat(workflows): M6 — dashboard run-history + DAG view
A per-app "Workflows" tab: list definitions, browse run history, and inspect a
run's per-step progress with a static layered DAG.

- API: the run-detail endpoint now returns each step's `depends_on` (loaded
  from the definition via a new `get_workflow_by_id` reader) so the dashboard
  can draw the graph edges — the run's step rows don't carry them.
- dashboard `$lib/api`: a `workflows` namespace (list / runs / start / run) +
  the matching types.
- `apps/[slug]/workflows/+page.svelte`: workflow table → drill into runs →
  drill into a run. The run view renders a step table plus an SVG DAG (nodes =
  steps colored by status, laid out by longest-path level; edges = depends_on,
  arrowed) and a "Start run" button. Nested/parked steps show their child run.
- AppTabBar + the per-app layout gain the Workflows tab (admin-only).

Tests: `npm run check` clean (0 errors); two Playwright smoke tests
(navigation tabs — workflows loads cleanly + renders its empty state) pass
against the full stack.

With M6 the v1.2 Workflows track (M1 schema/validation → M2 durable
orchestrator → M3 when/templating → M4 nested → M5 SDK/API/CLI → M6 dashboard)
is COMPLETE.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 17:57:56 +02:00
MechaCat02
5a630e1d9f feat(workflows): M5 — SDK workflow::start + admin run API + pic CLI
The user-facing surface for the durable DAG engine. A workflow run can now be
started three ways, all resolving the workflow by name in the caller's app
(`cx.app_id` / the resolved app — never a passed-in arg, the isolation
boundary), seeding a run the orchestrator advances.

SDK seam (shared):
- `WorkflowService` trait + `NoopWorkflowService` + `WorkflowError` in
  `shared::workflow` (mirrors `InvokeService`); added to `Services` as a
  noop-defaulted `with_workflow` builder (all `Services::new` call sites
  unchanged).
- `WorkflowServiceImpl` (manager-core): resolves + seeds a run; authenticated
  callers gated on `AppInvoke` (anonymous skips, script-as-gate), the same gate
  `invoke()` uses.
- `executor-core::sdk::workflow` — the Rhai bridge `workflow::start(name, input)`
  / `workflow::start(name)`, registered in `register_all`. Returns the run id.

Admin API (manager-core `workflows_api`, mounted in the binary):
- `GET  /apps/{app}/workflows`              — definitions (AppRead)
- `POST /apps/{app}/workflows/{name}/runs`  — start a run (AppInvoke)
- `GET  /apps/{app}/workflows/{name}/runs`  — run history (AppRead)
- `GET  /apps/{app}/workflow-runs/{run_id}` — one run + its steps (AppRead)
  `app_id` scopes every query; a foreign run 404s.

CLI (`pic workflows`): `ls` · `run <name> [--input JSON]` · `runs <name>` ·
`run-status <run_id>` — all `--app`-scoped; run definitions stay declarative
(`[[workflows]]` → `pic apply`).

Also: `list_runs_for_workflow` repo reader.

Tests: `workflow_service_start_seeds_a_run` (DB-gated — the SDK/API-shared
entry) + `workflow_run_executes_end_to_end` CLI journey (apply → `pic workflows
run` → poll `run-status` → succeeded, through the real binary + orchestrator).
fmt + clippy -D warnings clean, 449 manager-core lib tests, 14 orchestrator DB
tests, 18 executor-core lib tests, 3 workflow CLI journeys green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 17:48:38 +02:00
MechaCat02
74d5874384 feat(workflows): M4 — nested sub-workflows (start_child, parent-park, depth ceiling)
A `workflow`-kind step now starts a nested sub-workflow run instead of a
function. The orchestrator's step processing is restructured into prepare →
dispatch (`Prep`): after the shared context/`when`/input resolution, a
function step runs through the executor as before, while a workflow step:

- checks the nesting depth ceiling (`PICLOUD_WORKFLOW_MAX_DEPTH`; the child
  would run at parent depth + 1) — over-deep → the step fails, not infinite;
- resolves the child workflow by name in the run's app scope;
- `start_child_and_park`: in one token-gated tx, seeds a child run (depth + 1,
  `parent_run_id`/`parent_step_id` linkage, correlated under the same
  `root_execution_id`) and PARKS the parent step (`running`, claim_token
  cleared, `child_run_id` set). A parked step is never re-claimed (only `ready`
  steps are) nor reclaimed (only *leased* running steps). The token-gated park
  runs before the child insert, so a stale claim writes nothing (no orphan).

Each tick first runs `resume_finished_children`: a parent step parked on a
now-terminal child is resolved (child output → parent step output, or child
error → parent step failed) and the parent run advanced — idempotent via a
`status='running'` gate. The child's own steps are claimed by the same global
scan, so nesting is just more runs. A failed sub-workflow honors the parent
step's `on_error`.

Shared plumbing extracted for reuse: `advance_run_tx` (out of
`complete_step_and_advance`) and `seed_run_tx` (out of `start_run`).

Apply-time soft-warn (`workflow_nesting_warnings`, wired into `plan_warnings`):
a workflow that nests into itself — directly or via a mutual cycle within the
bundle — is flagged at plan (bounded by the depth ceiling, but almost always a
bug). Not an error.

Tests: nested output-flows-to-parent + depth-ceiling-fails-the-run (DB-gated,
end-to-end), self/mutual-cycle warning (pure unit). fmt + clippy -D warnings
clean, 449 manager-core lib tests, 13 orchestrator DB tests, 26 CLI journeys
(workflows/apply/plan/prune) green, binary boots.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 17:30:16 +02:00
MechaCat02
41a21c0551 feat(workflows): M3 — conditional when + input templating at run time
Wire the two pure M1 modules into the orchestrator's execute path:

- Before running a step, build a `RunContext` from the run input + every prior
  succeeded step's output (`load_run_step_outputs`, read fresh at execution
  time so a step sees all upstream results).
- `workflow_expr::eval` the step's `when` condition (if any): false → the step
  is `skipped` — never executed — and counts as satisfied-but-empty for its
  dependents (a new `StepOutcome::Skipped`, written in `complete_step_and_advance`
  then advanced; `compute_advance` already treats skipped as satisfying deps).
- `workflow_template::resolve` the step's `input` against the context: an
  exact single `{{ ref }}` preserves the referenced JSON type, an embedded ref
  interpolates as text; a missing reference is a hard step failure (a definition
  bug surfaced, not hidden), never a silent null.

`when` and templates were already parse-validated at apply time (M1); this is
the runtime half.

Tests (DB-gated, end-to-end via a scripted fake executor):
- `when_false_skips_step` — b skipped, never executed, omitted from run output
- `step_output_flows_into_downstream_input` — a's `{n:7}` feeds b's input,
  type-preserved for a bare ref and interpolated in a larger string
- `missing_input_ref_fails_the_step` — an unresolved ref fails b → run fails

Verified: cargo fmt, clippy -D warnings clean, 448 manager-core lib tests,
11 workflow_orchestrator DB tests (3 new).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 17:18:01 +02:00
MechaCat02
a55c2f112e feat(workflows): M2 — durable orchestrator worker (claim/execute/advance)
The v1.2 Workflows durable DAG engine gains its runtime. A dedicated
background worker (`workflow_orchestrator.rs`, mirroring `cron_scheduler`,
NOT folded into the dispatcher) advances every in-flight run step-by-step,
durably, surviving restarts.

Per tick, two phases:
  A. Claim + execute — up to a small batch of `ready`, due steps are claimed
     with the same `FOR UPDATE SKIP LOCKED` competing-consumer lease the queue
     uses (`claim_ready_step`), one execution-gate permit per step acquired
     BEFORE the claim so the shared gate bounds real parallelism. Each step
     resolves its function by name in the run's app scope (never a
     script-passed arg — the isolation boundary), builds an `ExecRequest`, and
     runs through the injected `ExecutorClient`. Claimed steps run concurrently.
  B. Advance — the outcome is written and the DAG advanced in one token-gated
     transaction (`complete_step_and_advance`): pending steps whose deps are
     satisfied flip to `ready`, and the run's terminal status is recomputed.
     Fan-in falls out (a join waits until its last dep flips it); a stale worker
     matches zero rows and writes nothing.

The graph-advance decision is a pure, DB-free function (`compute_advance`) —
promotions + terminal run status, folding `on_error` fail-vs-continue and
skipped/failed dependency satisfaction — so it is unit-tested in isolation.

Retry uses the step's own policy via `compute_backoff`; a second, slower
cadence reclaims steps leased by a crashed worker (`reclaim_stale_steps`) — the
durability safety net. Steps run with no principal (like invoke_async), and log
under the new `ExecutionSource::Workflow` so `pic logs` surfaces them.

M2 executes function steps only; `when` + input templating land in M3, nested
sub-workflows in M4. Seams present (`StepTarget`, `run_input`, `workflow_depth`).

- migration 0072: widen the `execution_logs.source` CHECK with `workflow`
- shared: `ExecutionSource::Workflow`, `StepStatus::is_terminal`
- workflow_repo: run/step state — `start_run`, `claim_ready_step`,
  `complete_step_and_advance`, `reclaim_stale_steps`, `get_run`,
  `list_run_steps`, `compute_advance` (+ 7 pure unit tests)
- workflow_orchestrator: the worker + config (`PICLOUD_WORKFLOW_*` knobs),
  spawned in `picloud/src/lib.rs` beside the dispatcher/cron scheduler
- tests: 8 DB-gated integration tests (linear, parallel fan-out/fan-in, retry,
  on_error fail/continue, double-complete idempotency, stale-lease reclaim,
  end-to-end tick with a fake executor)

Verified: cargo fmt, clippy -D warnings clean, 448 manager-core lib tests,
8 workflow_orchestrator DB tests, schema snapshot reblessed, M1 workflow CLI
journeys still pass (binary boots with the orchestrator wired in).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 17:12:42 +02:00
MechaCat02
44f992cbb0 feat(workflows): M1 — schema + definition validation + declarative reconcile
First milestone of the v1.2 Workflows track (blueprint §9.1/§9.2): the durable
DAG engine's foundation — the definition model, its validation, and the
declarative `apply` reconcile path. No execution yet (the orchestrator is M2).

- Migration 0071_workflows: `workflows` (owner-polymorphic like scripts, app-
  owned in M1), `workflow_runs`, and `workflow_run_steps` (the per-step
  competing-consumer lease table the M2 orchestrator will claim). Schema golden
  reblessed.
- `shared::workflow`: the `WorkflowDefinition` / `WorkflowStepDef` DTOs (shared
  by the CLI, apply, and the future orchestrator) + run/step status enums.
- Pure, DB-free `workflow_template` (input `{{ input.x }}` / `{{ steps.a.output.y }}`
  resolution, type-preserving) and `workflow_expr` (a small safe JSON-predicate
  evaluator for `when` — keeps manager-core free of a scripting engine).
- `workflow_repo`: read trait + reconcile tx free-fns (insert/update/delete).
- apply_service: `BundleWorkflow` + `Plan.workflows` + `CurrentState.workflows`
  + `ApplyReport.workflows_*`; server-side `validate_workflow_definition`
  (unique/acyclic steps via topological sort, deps exist, function XOR workflow,
  `when`/template parse) rejecting on a group node; `diff_workflows` by
  lower(name) (Update on a definition change) + in-tx reconcile.
- CLI: `[[workflows]]` + `[[workflows.steps]]` manifest structs → wire bundle;
  plan/apply render + report counts.
- Tests: 16 manager-core lib tests (template, expr, definition validation) +
  the `workflows` CLI journey (apply → NoOp → prune; cyclic DAG rejected).

Deferred to later milestones: the orchestrator worker (M2), conditional/template
runtime wiring (M3), nested sub-workflows (M4), the SDK/API/CLI run surface
(M5), the dashboard (M6). The `group_id` column + run/step tables ship now so
those slot in without a migration churn.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 16:46:32 +02:00
MechaCat02
c8c4f012ff docs: reconcile blueprint/design-doc/CLAUDE.md to shipped v1.2 state
A five-agent roadmap-verification pass found the v1.2 Hierarchies code fully and
correctly implemented, but the three source-of-truth docs carried stale
"deferred/remaining/v1.3" notes — and one self-contradiction — for features that
have since shipped (§3 M3 server-side approval gate, §6 group create/reparent, the
Track A closeout, and the 2026-07-11 audit remediation). Reconcile all three to
reflect the true code; no behavior change.

CLAUDE.md: retract the inline "groups pre-exist" / "per-env approval deferred" /
"shared-topic SSE deferred" / "group dead-letter store deferred" / "email v0/no-AAD"
notes → mark §6, §3 M3, and Track A M1–M6 shipped; add per-group KV/docs byte
quotas + set_if; note migration 0063; refresh the "Out of MVP" section.

design doc: fix the header ("Phases 4–6 remain" → all shipped); resolve the §11.6
self-contradiction (the "Deferred" list still named shared-topic SSE / byte quotas /
set_if / operator admin API, all shipped — line 1317 already said SSE shipped); mark
the D3 dead-letter store shipped; retract the stale "groups pre-exist / §6 deferred"
forward references.

blueprint (comprehensive sweep): dashboard Alpine.js → SvelteKit + CodeMirror
(diagram, §3.3, tech table); Docker-per-execution → embedded in-process Rhai
(diagram + data flow); §12 Phase 4 "current focus" → shipped through v1.1.9; Phase 5
"in active development" + "Remaining" block → Hierarchies complete; per-app RBAC
"v1.3+" → shipped Phase 3.5; SDK reference → handle-pattern notation note, S3 tag
v1.1 → v1.3+; MVP schema + docker-compose flagged non-authoritative; §9 header noted
Hierarchies-shipped / Workflows-future; top status line refreshed.

Also fix two behavior-neutral stale in-code comments (sdk/kv.rs `kv::shared` →
`kv::shared_collection`; dispatcher.rs `q_terminal` "no group dead-letter store yet"
→ dead-letters to group_dead_letters, Track A M2).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 14:53:34 +02:00
MechaCat02
a3c4267f6c feat(cli): plan warnings, apply confirm + blast-radius gate, files --group, unified owner-XOR
Remediate the CLI/DX findings from the 2026-07-11 audit.

B4 — `pic plan` now previews the apply-time desired-state warnings (disabled
binding, unreachable endpoint, dangling `[suppress]`), so plan and apply agree
for CI review. Wire fields are `#[serde(default)]` (older-server tolerant).

B5 — `pic apply` no longer mutates on a first run (no recorded plan) without a
preview + confirm, and a large blast radius now triggers an extra confirmation.
Both gates check `is_terminal()` before any read — a non-TTY never hangs on
stdin and fails closed without `--yes`; `--yes`/`--force` bypass headlessly.
`--force` help now notes it also skips these prompts.

C3 — `pic files ls`/`get` gain `--group`, mirroring `pic kv --group`, so the
shipped group shared-files admin surface is reachable from the CLI.

C4 — a shared `require_one_owner(app, group)` helper (cmds/mod.rs) gives one
canonical message for the `--app`/`--group` XOR, wired into every such site
(kv, files, vars, secrets, suppress, extension-points, triggers ls, scripts
deploy). routes ls (positional script_id) and scripts ls (lists all) keep their
own shapes deliberately.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 23:47:59 +02:00
MechaCat02
5e1bda1f32 fix(core): cron double-fire, materialize warnings, group quota fast path, SSE channel cap
Remediate the MEDIUM backend correctness/perf findings from the 2026-07-11 audit.

B1 — a group cron template toggled disabled→enabled no longer re-fires every
descendant's cron. Materialization now KEEPS a disabled cron template's copies
(syncing `enabled` in place) instead of delete+recreate, preserving
`last_fired_at`; the scheduler already skips disabled copies, so keeping them is
inert. Queue/email arms still delete-on-disable (freeing the one-consumer slot).

B2 — `rematerialize_stateful_templates` warnings are now logged at the
app-create/delete and group-reparent chokepoints (were silently dropped); only
the `Err` arm was handled before.

B3 — the group KV/docs byte-quota check skips the O(rows) `SUM(octet_length(..))`
scan when a cheap upper bound (`rows_after * max_value_bytes`) is already under
the cap, so the full aggregate only runs near-cap. Every row is validated
≤ max_value_bytes, so the bound is sound (never lets an over-quota write through).

B6 — the in-process SSE broadcaster caps live channels per map
(`PICLOUD_REALTIME_MAX_CHANNELS`, default 100k); a subscribe that would open a
NEW channel past the cap is refused with 503 + `Retry-After: 1` rather than
growing the (app,topic)/(group,topic) maps unboundedly. Check+insert is atomic
under the map mutex (no TOCTOU); an existing-channel subscribe is exempt.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 23:47:47 +02:00
MechaCat02
86448beb06 fix(security): server-authoritative approval gate + auth/session/file hardening
Remediate the HIGH and security-relevant findings from the 2026-07-11 audit.

H1 — the per-env approval gate is now server-authoritative. The governing
project is resolved from the target node's nearest-claimed ancestor
(`governing_env_policy`/`_tree` + `governing_project_id` +
`ProjectRepository::get_environments_by_id`), independent of the client-supplied
`[project]`. Omitting or spoofing the project block can no longer skip a gate the
owning project established; a to-create group resolves its declared parent's
chain so a fresh subtree node inherits the gate. Fails closed on any read error.

H2 — the API-key prefix slice (`&rest[..8]`) is now the boundary-safe
`rest.get(..8)`, so an attacker-supplied multibyte bearer can't panic the request
task (unauthenticated per-request DoS). Regression test added.

C1 — admin sessions gain an absolute lifetime cap (migration 0070,
`PICLOUD_SESSION_ABSOLUTE_TTL_HOURS`, default 30d): `lookup` filters it, `touch`
clamps the sliding bump to it, so a continuously-used or stolen-but-warm token
self-expires. Mirrors the data-plane app-user cap.

C2 — `Cache-Control: no-store` on the login and API-key-mint responses (the two
that return a raw credential), so a proxy/CDN/browser cache can't retain it.

B8 — file downloads are header-safe: `sanitize_stored_filename` guarantees a
valid `HeaderValue` (no panic on a control-char name) and BOTH the per-app and
group download paths now set attachment + `X-Content-Type-Options: nosniff` +
a restrictive CSP, closing a group-path stored-XSS gap.

Also folds in the server-side plan-warning plumbing (`plan_warnings`,
`PlanResult::warnings`) and the `app_only_reject` message helper that the CLI
plan-preview change builds on, plus operator security notes (reads-open shared-
topic SSE; the `--env` label is advisory, not a boundary).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 23:47:35 +02:00
MechaCat02
b8b047368e feat(realtime): external SSE subscription for group shared topics (§11.6 D2 / Track A M6)
Shared TOPICS fanned out only to in-cluster trigger handlers; external clients
could not subscribe (per-app topics already can). Add SSE for shared topics.

- RealtimeBroadcaster gains a parallel (group_id, topic) channel map:
  subscribe_group / publish_group / drop_group_topic (default no-ops so
  NoopRealtimeBroadcaster + test doubles are untouched). InProcessBroadcaster
  implements them with a second map; GC + channel_count span both.
- Route GET /realtime/shared/topics/{topic}: Host->app dispatch (as the per-app
  route), then RealtimeAuthority::authorize_subscribe_shared resolves the OWNING
  GROUP from the app's chain (kind=topic, root segment). Reads-open model — the
  resolution IS the authorization, consistent with in-script shared reads; a
  foreign-subtree app never resolves (404, the isolation boundary). No principal
  machinery needed.
- GroupPubsubServiceImpl::with_realtime bridges a shared-topic publish to the
  owning-group channel (best-effort) after the durable trigger fan-out.
- Host wires the broadcaster into the group pubsub service + the collection
  resolver into the authority.

Auth-model note: chose reads-open (subtree app's Host is the grant) over
"authenticated principal + GroupKvRead" — it's both simpler and faithful to how
shared-collection reads already work. Pinned by realtime broadcaster group-map
tests, realtime_api shared-route tests (404 + stream), and
group_pubsub_service::publish_bridges_to_the_group_broadcaster. No migration.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 15:33:39 +02:00
MechaCat02
0f05c270d1 feat(kv): set_if compare-and-swap for KV (per-app + shared + SDK) (Track A M5)
No atomic CAS existed, so concurrent writers to the same KV key raced. Add
set_if(key, expected, new): writes only when the current value equals expected,
or (expected absent) only when the key is missing — the primitive for lock-free
counters / optimistic concurrency.

- KvService + GroupKvService traits gain set_if with a defaulted "unsupported"
  impl (Noop + other impls untouched); the real services override it.
- kv_repo/group_kv_repo: atomic set_if — a single UPDATE ... WHERE value =
  $expected (JSONB semantic equality) or INSERT ... ON CONFLICT DO NOTHING.
  Atomic without a transaction; returns whether the write happened.
- Services enforce the same value-size cap + (group) writes-authed + row/byte
  quotas as set; the event fires only on a successful swap.
- Executor Rhai SDK: kv::collection(c).set_if(key, expected, new) and the
  shared_collection variant; Rhai () for expected means "only if absent".

Pinned by kv_service/group_kv_service set_if unit tests (CAS + fail-closed) +
sdk_kv::kv_set_if_compare_and_swap (through a real script). No migration.
docs/sdk-shape.md documents the primitive.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 15:19:03 +02:00
MechaCat02
636106e9ea feat(quota): per-group total-byte quotas for shared KV + docs (Track A M4)
M3 quotas capped a group's shared KV/docs by ROW count only; a group could still
blow past storage with few-but-huge values. Add the symmetric total-bytes ceiling
(files already had one).

- group_quota: PICLOUD_GROUP_{KV,DOCS}_MAX_TOTAL_BYTES env vars (default 256 MiB)
  + accessors.
- GroupKvRepo/GroupDocsRepo: total_bytes(group) = SUM(octet_length(value::text))
  (default Ok(0) so non-Postgres impls skip the check).
- GroupKv/DocsServiceImpl: check the PROJECTED total (old value's bytes
  subtracted, new added) on every set/create/update, so a same-or-smaller update
  near the cap is still allowed (unlike a naive used+new check). New
  TotalBytesQuotaExceeded errors; +with_max_total_bytes for tests.
- KV set switched has->get to obtain the old value's size for the delta.

Pinned by group_kv_service + group_docs_service per_group_byte_quota_uses_the_
projected_total unit tests (reject over-cap, allow same/smaller update near cap).
No migration. CLAUDE.md env-var table updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 15:09:01 +02:00
MechaCat02
1a69778c0c feat(secrets): AAD-bind email-trigger inbound secrets v0->v1 (Track A M3)
email_trigger_details.inbound_secret_encrypted was sealed v0 (no AAD, bound only
to the master key) because the table had no version column — a ciphertext could
in principle be relocated between rows (audit 2026-06-11 H-D1, Medium). Bring the
AAD versioning the `secrets` table gained in 0042 to email secrets.

- Migration 0069: `email_trigger_details.inbound_secret_version SMALLINT DEFAULT 0`.
- secrets_service: `seal_email`/`open_email` seal v1 with AAD bound to the
  SEALING OWNER (`email:{app}` / `email:group:{group}`) — deliberately NOT the
  per-row trigger_id, so a group template's sealed bytes stay valid when the
  materializer copies them verbatim to each descendant (all share the group AAD).
  v0 rows keep their exact legacy no-AAD read path.
- Both email-trigger create paths (declarative apply resolve_and_seal +
  triggers_api create_email_trigger) seal v1 under the trigger's owner; the
  version threads through CreateEmailTrigger + insert_email_trigger_tx.
- materialize copies inbound_secret_version verbatim with the bytes.
- email_inbound_target recovers the sealing owner (materialized_from ->
  template.group_id, else the app) so receive_inbound_email opens a v1 secret
  under the right AAD.

Cross-app/tenant relocation now fails the GCM tag; a same-owner swap is not
distinguished (accepted low-severity residual). Pinned by
email_secret_aad::materialized_email_copy_decrypts_under_the_group_aad (the
novel materialized-copy path) + the extended secret_round_trips_through_seal_open
lib test. Schema golden reblessed; materialization + email e2e regressions green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 14:58:35 +02:00
MechaCat02
fd4336e883 feat(queue): dead-letter store for group shared queues (§11.6 D3 / Track A M2)
An exhausted SHARED durable-queue message was `drop_exhausted()`-ed with a
warning — silent data loss. Per-app queues persist to `dead_letters` (0010);
add the symmetric group store so an exhausted shared-queue message is preserved
and operator-visible.

- Migration 0068: `group_dead_letters`, keyed by (group_id, collection),
  CASCADE on the group (an app delete leaves the data — it belongs to the
  group, not the consuming app).
- `GroupQueueRepo::dead_letter` (replaces `drop_exhausted`): one tx that INSERTs
  the dead-letter + DELETEs the live message, filtered by claim_token so a lost
  lease can't dead-letter a re-claimed message (mirrors queue_repo::dead_letter).
- Dispatcher `q_terminal` shared arm now dead-letters instead of dropping. It
  returns None (not the dl id) so the per-app `fan_out_dead_letter` is SKIPPED —
  firing the consuming app's per-app handlers on a shared message (competing
  consumers → nondeterministic app) would be wrong. Fan-out to a *shared*
  dead_letter trigger is deferred (needs a new trigger kind).
- Read side: `GroupDeadLetterRepo::list_for_group` backs a new read-only
  operator endpoint GET /api/v1/admin/groups/{id}/dead-letters (GroupKvRead,
  mirrors the M4 group-blobs surface). `pic dead-letters ls --group` deferred
  (optional; the HTTP endpoint is the operator surface).

Pinned by group_queue::dead_letter_moves_an_exhausted_message_to_the_group_store
(store move + claim-token-mismatch guard + operator read). Schema golden
reblessed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 14:41:04 +02:00
MechaCat02
c06a9e801e feat(apply): make the per-env approval gate hermetic (§3 M3 / Track A M1)
The per-env approval policy was applier-supplied — a hand-crafted request
that omitted `project.environments` was ungated, and flipping a gate to
`confirm = false` in the same request un-gated it. Persist the policy
server-side and enforce against `persisted ∪ declared`.

- Migration 0067: `project_environments (project_id, env_name, confirm)`,
  CASCADE on the project. Written declaratively (delete-then-insert) inside
  `upsert_project_tx`, same tx as the project row + node claim.
- `ProjectRepository::get_environments_by_slug` (read side) +
  `ApplyService::effective_env_policy` union the persisted policy with the
  request's declared one (confirm if EITHER says so — monotonic).
- `env_gate_check` now evaluates the effective policy; the three handlers load
  it before the gate. `plan.approvals_required` is the effective gated set, so
  CI sees a persisted gate even when the manifest omits it.
- The union rule closes both bypasses: an omitted policy still trips a
  persisted gate, and an ungating apply must itself pass the gate (the
  declarative replace then takes effect next time) — TOCTOU-safe.

Pinned by env_approval::{persisted_policy_gates_a_request_that_omits_it,
flipping_a_gate_to_false_in_the_same_request_still_gates} +
projects_repo::get_environments_by_slug_returns_persisted_policy. Schema golden
reblessed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 14:31:15 +02:00
MechaCat02
4b4b3148b7 feat(apply): server-side enforcement of per-env approval gating (§3 M3 / §4.2)
The per-env approval gate (`[project.environments]`, `confirm = true`) was
client-side only — the policy was `#[serde(skip_serializing)]`, so a patched or
older CLI applying to a confirm-required env bypassed it entirely. This makes
the gate server-enforced, admin-gated, and audited (the deferred §4.2 piece).

- Wire: `ProjectDecl` gains `environments` (the CLI stops skip_serializing it and
  sends the selected `env` + the `--approve` set on the apply request). The
  server RE-DERIVES the gate from the request (`env_gate_check`): a
  confirm-required env not in `approved_envs` → 409 `ApprovalRequired`.
- Admin gate: an approved override additionally requires `AppAdmin`/`GroupAdmin`
  on EVERY declared node — a step up from the editor write caps an ordinary
  apply needs (reusing the admin caps per §4.2) — and emits a `tracing::info!`
  audit line. Enforced on all three handlers (single app, single group, tree);
  the check runs before any tx, so a denial (403) precedes any mutation.
- Token: the env policy folds into the TREE bound-plan token (a policy edit
  between plan and apply trips StateMoved). The single-node token is left
  unchanged (a bare live-state hash), so plan/apply still match without threading
  a project into it.
- CLI: single-node `apply`/`plan` now resolve the GOVERNING project (own block
  else nearest ancestor's) so a leaf apply carries the root's policy; `plan`
  surfaces `approvals_required`. Client-side `require_env_approval` fast-fail
  kept as UX.

Adapts the earlier feat/ownership-and-approval M5 (654e387) to main's DTOs; main
addresses tree group nodes by slug only, so the admin loop mirrors authz_tree
(app: resolve_app_id fail-closed; group: get_by_slug, skip to-create).

Threat-model note (honest scope, in the doc + code comments): this closes the
patched/older-CLI bypass and admin-gates + audits the override, but is NOT a
hermetic boundary against a hand-crafted request that OMITS the policy — the
policy is applier-supplied, not persisted server state. Full closure (persist a
`project_environments` source of truth + a server-determined env) is deferred.

Review (adversarial pass): fixed a MEDIUM (single-node `plan` didn't resolve the
governing project, so a leaf plan under-reported the gate `apply` enforces) and a
LOW (duplicate clippy attr); corrected overstated "can't bypass" wording.

Tests: ProjectDecl gate unit test; env_approval journeys
`server_enforces_the_gate_against_a_non_stock_client` (raw request → 409 without
approval, admin-approved → 200) and `approving_a_gated_apply_requires_admin`
(editor + --approve → 403). 417 lib tests + 33 CLI journeys + workspace clippy
-D warnings + fmt clean. No migration.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 14:01:04 +02:00
MechaCat02
bbfdb1bf47 fix(apply): bound the blast-radius subtree down-walk CTE
`group_blast_radius`'s `app_chain` up-walk is depth-capped (`ac.depth < 64`,
added by 76926de to stop a should-be-impossible `groups.parent_id` cycle from
spinning the recursive CTE forever), but the sibling `subtree` down-walk in the
same query was left unbounded. A `parent_id` cycle would loop `subtree`
indefinitely and hang the plan — the exact exposure 76926de set out to close,
left half-done on the descendant side.

Add a `depth` column + `WHERE s.depth < 64` to `subtree`, mirroring `app_chain`.
Defense-in-depth: the reparent cycle-guard already prevents `parent_id` cycles,
so this hardens against an unreachable state — but that is precisely the bar
76926de applied. Surfaced by a post-adoption audit of the §7/§6 ownership code.

Verified: 416 manager-core lib tests; 12 ownership/create/divergence/approval
journeys (incl. plan_previews_ownership_and_blast_radius, which drives this
query); clippy -D warnings + fmt clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 13:25:06 +02:00
MechaCat02
87292fc4e9 refactor(apply): split reconcile_group_structure_tx create/reparent branches
Some checks failed
CI / Rust — fmt, clippy, test (push) Failing after 19m24s
CI / Dashboard — check (push) Successful in 9m46s
Review #4 (LOW, cleanup). `reconcile_group_structure_tx` carried a
`#[allow(clippy::too_many_lines)]` over a ~145-line body doing create +
reparent + attach/RBAC/lock in one loop. Extract the create branch
(`create_group_node_tx`) and the divergence-resolution branch
(`reparent_diverged_group_tx`) into helper methods, threading the shared
attach-ceiling / principal / mode context through a small `ReconcileCtx` — the
loop is now thin enough to drop the allow. Also improve the child-before-parent
error in `resolve_declared_parent` to hint that group nodes must be ordered
parents-first (only a hand-rolled bundle hits it; the CLI already sorts).

No behavior change. Documents the Tier-1 review follow-ups in the design doc.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 21:49:18 +02:00
MechaCat02
6ec034fb1d refactor(apply): unify divergence detection and drop the plan-path N+1
Review #3 (LOW). `divergence_preview` (plan path) compared parent slugs
case-sensitively from its own `get_by_slug`, while the apply path
(`reconcile_group_structure_tx`) compares resolved parent ids. On a mixed-case
parent slug the two could disagree — `pic plan` reporting `diverged` while
`pic apply` hard-errors "parent does not exist".

`resolve_existing_group_ids` becomes `load_existing_groups`, returning the full
`Group` rows it already loads; `plan_tree` derives the id map from them and
passes the map to `divergence_preview`, which now reads each node's own row
without a per-node `get_by_slug` (kills the N+1), resolves an in-tree parent's
slug from the same map, and compares parent slugs case-insensitively so the
preview agrees with the apply path's id-based check.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 21:49:05 +02:00
MechaCat02
3f272daf04 fix(cli): make the per-env approval gate fail closed
Review #2 (MEDIUM). The M3 `[project.environments]` approval gate silently
failed open in three shapes:

- `EnvPolicy.confirm` was `#[serde(default)]`, so `production = {}` parsed as
  un-gated. `confirm` is now a REQUIRED field — an env listed with an empty
  policy is a load error (`missing field 'confirm'`), not a silent no-gate.
- `pic apply --file <leaf>` where the leaf carried no `[project]` skipped the
  gate even when the repo root gated the env. `run` now discovers the governing
  `[project]` by walking up from the manifest to the nearest ancestor
  `picloud.toml` that declares one (`find_governing_project`, loaded without the
  env overlay — only the block matters).
- A `[project].environments` in a non-root manifest under `--dir` was dropped
  with a generic "ignored" note; `build_tree` now warns explicitly that its
  approval gating is not enforced.

Pinned by `tests/env_approval.rs` (empty-policy load error; leaf `--file` apply
honoring the root gate).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 21:48:09 +02:00
MechaCat02
2e3263002a fix(apply): gate group content writes in-tx to close a create-race authz hole
Review #1 (HIGH). M1's `authz_tree` skips the group content-write capability
check (`require_group_node_writes`) for a group node absent at API-request
time — a to-create group whose authorization is the service create-gate. But
that gate only fires when the group is STILL absent at reconcile, and the
content writer performs no authz. So a group created by a racer in the window
between the API check and the in-tx reconcile could have its scripts/vars
written by a principal with zero rights on it (the bound-plan token doesn't
save a hand-rolled request that omits `expected_token`).

Close it with an in-tx content-write re-check in `apply_tree` for every group
NOT structurally created/reparented by this apply: a group WE created is
covered by create-RBAC (`GroupAdmin(parent)` cascades) and its uncommitted row
is invisible to the pool-based authz cascade anyway, so it must not be
re-checked; a pre-existing (incl. racer-created) group is committed, so its
ancestry resolves and a missing cap denies. `require_group_node_writes` moves
from `apply_api` onto `ApplyService` (up the existing dependency edge) so the
API pre-check and the in-tx re-check share one implementation.

Pinned by `tests/group_create.rs::tree_apply_denies_group_content_write_without_the_cap`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 21:47:59 +02:00
MechaCat02
8d71620e11 style(test): rustfmt the env-approval journey
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 20:28:59 +02:00
MechaCat02
f0e18d263b style(test): rustfmt the structural-divergence journey
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 20:28:37 +02:00
MechaCat02
e2bfe633a7 test/docs(cli): §3 M3 per-env approval journey; Tier 1 complete
`tests/env_approval.rs`: a `[project.environments]` gating `production` →
`pic apply --env production` refused (names --approve); `--yes` alone does not
bypass; `--approve production` applies; an un-gated `staging` applies freely.
Design doc §3 records M3 shipped and the §6/§3 group-tree Tier 1 (M1 create ·
M2 divergence/reparent · M3 per-env approval) COMPLETE.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 20:28:28 +02:00
MechaCat02
62710b5d81 feat(cli): §3 M3 — per-env approval gating ([project.environments])
A `[project.environments]` block maps an env name to `{ confirm = bool }`
(`ManifestProject.environments`, client-side only — `skip_serializing`).
`pic apply --env <e>` is refused before any request when `<e>` is
confirm-required unless it is explicitly `--approve <e>`d; a blanket `--yes`
does NOT cover a gated env (§4.2 "CI must opt in per environment"); an unlisted
or `confirm = false` env applies freely. `require_env_approval` gates both the
single (`run`) and tree (`run_tree`) apply paths; `--approve <env>` is a
repeatable flag. Per-env value overlays (`picloud.<env>.toml`) already merge
client-side, so this is the confirm-policy gate only.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 20:28:28 +02:00
MechaCat02
b8eced9d91 test/docs(apply): §6 M2 structural-divergence journey + design note
`tests/structural_divergence.rs`: a repo that nests a group under a different
parent than the server → `pic plan` shows it `diverged`; a bare apply is
refused (422); `--force-local-structure` reparents to the manifest shape;
`--adopt-server-structure` keeps the server placement. Design doc §6 records M2
shipped, leaving only per-env approval gating deferred.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 20:23:10 +02:00
MechaCat02
cb3d458cfd feat(apply): §6 M2 — structural-divergence detection + declarative reparent
With the declared parent on the wire (M1), `apply_tree` now compares each
EXISTING group node's server parent to the manifest's (directory-nesting)
parent and resolves a divergence per a request `StructureMode` (default
`Refuse` — a pre-M2 CLI never reshapes the tree):

- `Refuse` → `ApplyError::StructuralDivergence` (422), naming the flags.
- `ForceLocal` → reparent to the declared parent IN the apply tx via the
  extracted `group_repo::reparent_group_tx` (cycle guard + structure_version
  bump), §5.6-gated (`GroupAdmin` on node + source + destination) and
  attach-ceilinged.
- `AdoptServer` → keep the server shape, reconcile content in place.

M1's `create_missing_groups_tx` generalized to `reconcile_group_structure_tx`
(create + reparent share the coarse-lock / attach-ceiling / RBAC path; both are
recorded `structurally_changed` so the pool-based attach re-check skips their
uncommitted rows). `pic plan` previews the drift (`divergence_preview` → a
`structure` row naming server + manifest parents). CLI:
`--force-local-structure` / `--adopt-server-structure` (mutually exclusive) →
the `structure_mode` wire field; the 422 surfaces verbatim. A concurrent
`pic groups reparent` between plan and apply still trips `StateMoved` (the tree
token already folds structure_version).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 20:23:10 +02:00
MechaCat02
dcc387c345 test/docs(apply): §6 M1 group-create journey + design note
`tests/group_create.rs`: a nested `[group]` tree with no server groups yet →
`pic apply --dir` creates both (parent from directory nesting), claims them for
the `[project]`, and re-applies as a no-op; a member with only editor (no
group-admin) is refused and leaves nothing behind. Design doc §6 records M1
shipped (parent-by-nesting, Phase-0 create-in-tx, claim, RBAC + attach ceiling)
with reparent/divergence still deferred to M2.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 20:07:12 +02:00
MechaCat02
0078ae8b26 feat(apply): §6 M1 — declarative group create (directory-nesting parentage)
Lift "groups must pre-exist" for the tree apply: a `[group]` node now carries a
declared parent (inferred from directory nesting) and `pic apply --dir` creates
the ones the server lacks, in the same transaction.

Wire: `TreeNode` gains `parent`/`name`/`description` (`#[serde(default)]` — a
pre-§6 CLI still reconciles existing groups). `discover::build_tree` computes a
group's parent as the nearest ancestor directory holding a `[group]` (topmost →
the repo's `[project] parent_group` attach point, else instance root) and emits
nodes parents-first by directory depth.

Server: `apply_tree` runs a Phase 0 (`create_missing_groups_tx`) that
resolves-or-creates every group node IN the tx (a same-tree parent resolved
before its child via the new `group_repo::{create_group_tx,
read_group_id_by_slug_tx}`), under the coarse `GROUP_STRUCTURAL_LOCK_KEY` (taken
only when creating), enforcing the attach-point ceiling + RBAC
(`InstanceCreateGroup` / `GroupAdmin(parent)`) per create. A created group is
CLAIMED in Phase A alongside existing ones (`decide_group_claim` → `Claim`).
`prepare_tree` now takes a caller-supplied `resolved_ids` map and returns a
`to_create` list the plan path previews (empty current → full-create plan,
ownership `claim`); a to-create group's token part equals its post-create part
so plan/apply tokens match across a create. `validate_bundle_for` takes
`is_group: bool` (a to-create group has no id yet); `authz_tree` skips a
to-create group (its create authz is the service gate, not an existing-group
capability).

Reparent of a diverged existing group + the detect-and-refuse flags are M2.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 20:07:03 +02:00
MechaCat02
76926de369 fix(apply): bound the blast-radius up-walk; fix two stale comments
Second-review follow-ups on the ownership cleanups:

- The new `group_blast_radius` `app_chain` recursion walked app→root with no
  depth cap, unlike the `ancestors()` CTE it replaced (`WHERE c.depth < 64`).
  Add the same `ac.depth < 64` guard so a (should-be-impossible) cycle in
  `groups.parent_id` can't spin the recursive CTE unbounded and hang a plan.
- Drop the now-false "memoized per group" line from `group_blast_radius`'s doc
  (it is a single set-based query now).
- Correct the `PlanRequest` flatten comment: apply is deliberately NOT flattened
  (its pre-§7 wire was already `{bundle, …}`), so the two endpoints are
  asymmetric by history — the old comment wrongly claimed parity.

Verified: the blast-radius journey + ownership journeys still pass; the INNER
JOIN to `projects` is safe (owner_project FK is ON DELETE SET NULL, so a
non-NULL owner always references a live row — no dangling drop).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 19:21:03 +02:00
MechaCat02
86b51df50d refactor(apply): address ownership review follow-ups (#3–#6)
Four cleanups from the code review of the §7 ownership track, no behavior
change to any passing test (416 manager-core units + 133 CLI journeys green):

- #3 app-ownership read parity: `check_app_owner_single` now resolves the
  nearest-claimed ancestor WITHIN the apply tx (`nearest_claimed_in_tx`, the
  same mechanism the tree path uses) instead of on a separate pool connection,
  so the check and this apply's writes see one consistent snapshot. (Fully
  serializing against a concurrent claim on an out-of-tree ancestor stays part
  of the documented pool→tx follow-up.)

- #4 blast-radius N+1 → one query: `group_blast_radius` replaced the
  per-descendant-group `ancestors()` round-trips with a single recursive
  `subtree → app_chain → nearest` CTE that groups + counts server-side. Cost is
  now one query regardless of subtree size.

- #5 one ownership policy, two projections: `decide_group_claim` /
  `decide_app_owner` are generic over the identity key, and `preview_ownership`
  (plan side) now DERIVES its label from them (keyed on slugs, takeover-agnostic)
  rather than re-encoding the arms — plan and apply can no longer drift.

- #6 shared slug validator: `validate_project_slug` delegates to a new
  `groups_api::validate_slug_format`, which `groups_api::validate_slug` also now
  uses — the format rule lives once. Projects intentionally skip the
  reserved-word check (a project slug is never routed).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 07:49:52 +02:00
MechaCat02
ffa8b02506 fix(apply): restore bare-bundle wire compat on the plan endpoints
§7 M3 wrapped the three plan request bodies in a `{bundle, project}` struct,
silently breaking version-skew compatibility: a pre-§7 `pic plan` POSTs a BARE
bundle at the JSON top level and now fails deserialization (422 "missing field
bundle"), even though the sibling apply request deliberately kept its added
fields additive (`#[serde(default)]`) so an older CLI still works.

`#[serde(flatten)]` the bundle in `PlanRequest`/`TreePlanRequest` so the bundle
fields sit at the top level again: a bare bundle deserializes (`project`
defaults to `None`) and a newer client sends the same fields plus a top-level
`project` key. The CLI now builds that shape via a `plan_body` helper (bundle
object + optional `project`), which an older server also accepts (Bundle has no
`deny_unknown_fields`, so the extra key is ignored) — compatible in both skew
directions. Apply is left unwrapped (it was already `{bundle, ...}` pre-§7).

Pinned by `plan_request_accepts_a_bare_bundle` / `_a_flattened_project` /
`tree_plan_request_accepts_a_bare_bundle` unit tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 07:37:35 +02:00
MechaCat02
74f7a67be7 fix(apply): preserve project name on a name-less re-apply
`upsert_project_tx` defaulted a missing `[project].name` to the slug and then
`ON CONFLICT (slug) DO UPDATE SET name = EXCLUDED.name` wrote that fabricated
value unconditionally — so a re-apply from a clone whose manifest omits `name`
silently clobbered the stored display name back to the slug (visible in
`pic projects ls` / `pic groups ls`).

Bind the raw optional name once and guard both sides on it:
`VALUES ($1, COALESCE($2, $1), $3)` (first apply falls back to the slug for the
NOT NULL column) and `DO UPDATE SET name = COALESCE($2, projects.name)` (a
re-apply updates the name only when the manifest actually declares one). An
omitted optional field now preserves persisted data instead of mutating it.

Pinned by a new `reapply_without_name_preserves_project_name` journey.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 07:37:25 +02:00
MechaCat02
f673922d89 feat(cli): pic projects ls + §7 M3/track-complete docs
§7 M3 (part 3) — the ownership track becomes inspectable, completing M1–M3.

- server: GET /api/v1/admin/projects (projects_api) lists every project + its
  owned-group count (projects repo list_with_counts, one GROUP BY); behind the
  /admin middleware, like the groups list.
- CLI: pic projects ls (cmds/projects.rs + client projects_list); the
  apply_ownership plan-preview journey now also asserts projects ls (plat owns
  exactly 1 group; teamb listed).
- docs: design §7 + CLAUDE.md record M3 shipped and the whole §7 multi-repo
  ownership track (M1 claim · M2 attach ceiling · M3 preview + projects ls)
  COMPLETE; deferred = structural-divergence + declarative tree shape + per-env
  approval gating.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 21:26:10 +02:00
MechaCat02
30fe160f16 feat(apply): cross-repo blast-radius in the plan preview
§7 M3 (part 2): planning a change to a GROUP node now reports the descendant
apps owned by OTHER projects that the change fans out to — the cross-repo
signal an operator needs before touching shared config.

- OwnershipPreview gains `blast_radius: Vec<ProjectImpact>`; `group_blast_radius`
  walks the group's subtree (recursive CTE) and groups descendant apps by their
  nearest-claimed-ancestor project, excluding the planning project. Ancestor
  resolution is memoized per group (one walk per distinct descendant group, not
  per app).
- `pic plan` renders `blast_radius` rows (mode-agnostic).
- the apply_ownership journey gains a plan-preview case pinning both the
  ownership annotation (claim / conflict) and the blast radius (two other
  projects' apps surfaced) end to end.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 21:17:15 +02:00
MechaCat02
b4eb226d20 feat(apply): project on the plan wire + ownership preview annotation
§7 M3 (part 1): `pic plan` now surfaces the ownership outcome BEFORE apply.

- plan / plan_owner / plan_tree take the declaring `[project]`; each result
  carries an `ownership` preview (pure `preview_ownership`, unit-tested):
  claim (unclaimed + project), owned (already this project), conflict (owned by
  X — named), or unclaimed. A group node previews its OWN owner; an app node its
  nearest claimed ancestor (read-only, on the pool).
- the attach-point ceiling (M2) is now previewed at plan too, so `pic plan`
  outside the subtree fails early — consistent with apply.
- wire: PlanRequest / TreePlanRequest wrap { bundle, project } (project
  `#[serde(default)]` → a pre-M3 CLI still plans); the plan DTOs + `pic plan`
  gain an `ownership` row (mode-agnostic: shows in tsv + json).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 21:09:55 +02:00
MechaCat02
5a820d7262 test/docs(ownership): attach-ceiling journey + M2 status
The apply_ownership journey gains an attach-ceiling case: a group node below
the attach point applies; the attach point itself and a sibling subtree are
both refused (422, message names the attach point). Design doc §7 + CLAUDE.md
record M2 shipped and re-point 'Next' at M3.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 20:57:57 +02:00
MechaCat02
5bb1ccad5e feat(apply): [project] parent_group attach-point ceiling
§6/§7 M2: a project's attach point is its ceiling — applies are refused for
any node not strictly within the declared subtree, so a repo can't reach (or
claim) above its local root even with the RBAC to do so.

- `[project] parent_group = "<slug>"` (ManifestProject + ProjectDecl); absent
  = instance root = no ceiling (the default, backward-compatible).
- `check_within_attach`: the attach group must be a PROPER ancestor of a group
  node (you can't apply the attach point itself) or an ancestor (inclusive) of
  an app node's group; resolved via `groups.ancestors`. Enforced read-only in
  apply_owner (prologue) + apply_tree (per node), before the claim.
- `ApplyError::OutsideAttachPoint` → 422 (a scope error, like Invalid).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 20:57:03 +02:00
MechaCat02
5bd72956b1 test/docs(ownership): §7 apply_ownership journey + design/CLAUDE updates
End-to-end coverage of the M1 ownership claim through the real CLI + server:
claim-on-first-apply + owner column, idempotent re-apply by the owner, a
second project refused (409, naming the owner), --takeover by an admin,
app-inheritance from the nearest claimed ancestor (owning project ok; foreign
or absent project refused), and the capability gate — a member with only an
editor GROUP role can reconcile but is refused --takeover (needs group-admin),
with ownership unchanged after the failed takeover.

- tests/apply_ownership.rs (registered in cli.rs); a grant_group_membership
  helper in tests/common/member.rs (group-level, mirroring the app one).
- design doc §7 gains an M1-shipped status note; CLAUDE.md records §7 M1 and
  re-points 'Next' at M2 (attach ceiling) + M3 (blast-radius / pic projects ls).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 20:49:31 +02:00
MechaCat02
b33c87e5c4 feat(cli): [project] manifest block, --takeover, groups ls owner column
Makes §7 ownership usable end to end from the CLI.

- manifest: a `[project]` block (slug + optional name), independent of the
  [app]/[group] XOR; threaded onto every apply from the repo. --dir reads it
  from the tree ROOT's picloud.toml (a [project] elsewhere is ignored with a
  note).
- client: apply_node/apply_tree carry project + takeover; the 409 branch now
  surfaces the server message verbatim (covers StateMoved AND OwnershipConflict,
  both self-contained). New groups_list_with_owner captures the owner slug.
- pic apply --takeover flag; pic groups ls gains an `owner` column (the owning
  project's slug, or — when unclaimed).
- server: /api/v1/admin/groups now returns each group flattened with its owner
  slug (via list_with_owner) — the shared Group deserialize ignores the extra
  field, so pic groups tree / the dashboard are unaffected.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 20:46:09 +02:00
MechaCat02
47072c481d feat(apply): §7 ownership claim state machine + project/takeover wire + 409
The heart of multi-repo ownership: an apply now claims/verifies node ownership
before it reconciles, so two repos can't clobber each other's subtree.

- Pure policy (DB-free, unit-tested): `decide_group_claim` (unclaimed→claim,
  owner→noop, foreign→conflict unless --takeover, no-project-into-claimed→
  conflict) and `decide_app_owner` (an app must match its nearest claimed
  ancestor; an unclaimed subtree stays open — backward-compatible).
- Execution under the per-node advisory lock: `upsert_project_tx` (first apply
  registers the project), `read/write_group_owner_tx`, `claim_group_owner`
  (takeover additionally requires `GroupAdmin` — ownership ⟂ RBAC),
  `check_app_owner_single` + tree `nearest_claimed_in_tx` (in-tree Phase-A
  claim overlays committed ancestors). No structure_version bump — a claim
  isn't a diff change, so it must not churn a pending bound plan.
- `apply`/`apply_owner`/`apply_tree` take an `OwnershipClaim` (project +
  takeover + principal); the claim slots after the lock, before load_current,
  so a conflict short-circuits before any diff work.
- New `ApplyError::OwnershipConflict` → HTTP 409 (actionable); wired through
  `ApplyRequest`/`TreeApplyRequest` (both `#[serde(default)]`, so the existing
  CLI stays compatible until it learns [project]).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 20:27:51 +02:00
MechaCat02
2cce051dc4 feat(projects): projects repo + group owner reads; wire into ApplyService
Read side of §7 ownership, ahead of the claim logic.

- project_repo: `ProjectRepository` trait + `PostgresProjectRepository`
  (list / get_by_id / get_by_slug). Writes are NOT here — a claim registers
  its project via an in-tx upsert (next commit).
- group_repo: `list_with_owner()` LEFT-JOINs projects for each group's owner
  slug (backs `pic groups ls`); columns qualified since id/slug/name exist on
  both tables.
- ApplyService gains a `projects` handle; constructed in the picloud binary.
- tests/projects_repo: round-trip pinning list_with_owner (claimed→slug,
  unclaimed→None) and that ancestors() surfaces owner_project nearest-first —
  the fold input the claim path walks.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 20:13:34 +02:00