Phase 3 foundation (docs §3): env-filtered, proximity-first config
inheritance down the group tree.
- Migration 0048: `vars` (polymorphic group|app owner via two nullable FKs
+ CHECK exactly-one, env scope, JSONB value, explicit tombstone) +
`apps.environment` (the env marker the resolver filters on — 'an
environment is an app').
- config_resolver: a shared chain-walk CTE (mirrors effective_app_role) that
walks app → ancestor groups, env-filters in SQL, plus a pure Rust
resolution pass implementing the §3 semantics SQL can't — per-key
proximity-first, @E-over-* within a level, map deep-merge, tombstone
suppression, and provenance for --explain.
Verified: 8 unit tests (incl. the §3.2 proximity-beats-env-specificity call,
deep-merge, tombstone, boundary) + the candidate-fetch CTE against live
Postgres (env-filter drops a non-matching @production value; nearer level
shadows farther).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds the GitLab-style single-parent group tree above apps: migration
0047 (groups + group_members + apps.group_id NOT NULL/RESTRICT with
single-root backfill), tree repos with an ancestor-walk cycle guard
under a coarse structural advisory lock, slug-freeze and structure_version,
hierarchy-aware RBAC (effective role = MAX across app membership + every
ancestor group, resolved live so revocation is immediate), new
Group{Read,Write,Admin}/InstanceCreateGroup caps that bound API keys
cannot obtain, plus `pic groups` and a dashboard group tree/detail/members.
Verified before merge: fmt/clippy(-D warnings)/check clean; manager-core
378 unit tests pass (incl. hierarchy authz + group repos); dashboard
npm run check 0 errors; check-versioning OK (47 migrations sequential).
New CLI groups journeys pass against a live DB — cycle guard
(reparent-into-descendant rejected), delete=RESTRICT, and
inherited-group-admin deploy + live revoke. The 6 failing logs/scripts/roles
journeys are the same pre-existing format drift on main, not regressions.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
SvelteKit UI for Phase-2 groups, mirroring the apps/users patterns:
- api.ts: Group/GroupDetail/GroupMember types + an api.groups client
(list/get/create/update/reparent/delete + nested members CRUD); optional
group selector wired into app create.
- routes/groups: a collapsible tree overview (assembled from parent_id)
with a New-group form, and a detail page with breadcrumb, subgroups/apps
lists, a rename form (no slug field — slug is frozen), a reparent
control, a delete button that surfaces the 409 RESTRICT message, and a
Members tab reusing RoleChip/ConfirmModal/ActionMenu.
- +layout.svelte: a Groups nav entry beside Apps.
npm run check: 0 errors; npm run build: success.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds the `pic groups` command family over /api/v1/admin/groups:
ls/tree, create (--parent), show (path + subgroups + apps), rename
(name/description; slug frozen), reparent (--to), rm (--recursive,
leaf-first; never auto-deletes apps), and members add/set/rm/ls. Also
`pic apps create --group <slug>` to place an app under a group.
End-to-end journey tests (tests/groups.rs, DB-gated) cover tree CRUD,
delete=RESTRICT (409 on a non-empty group), reparent cycle rejection, and
the headline invariant: a group_admin on an ancestor group can deploy to
an app it is NOT a direct member of (inherited membership), and loses that
access immediately on revoke — all green against a live server.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Server-side foundation for Phase-2 groups (no group-owned resources yet):
Shared types:
- GroupId, Group; App gains group_id; AppRole::{precedence,max} for
folding the highest effective role across the membership chain.
Repos:
- group_repo: tree CRUD with reparent (ancestor-walk cycle guard under a
coarse instance-wide structural advisory lock; slug frozen; bumps
structure_version) and delete=RESTRICT (refuses non-empty groups).
- group_members_repo: per-(user, group) role grants, mirroring app_members.
Hierarchy-aware authz (§5.3):
- AuthzRepo gains effective_app_role / effective_group_role (default to
direct membership / none, so the ~18 existing test stubs are untouched);
the Postgres impl resolves each via one depth-bounded recursive CTE that
MAXes the app's own row with every ancestor group_members row.
- can(): the Member path now folds inherited group roles, so a group_admin
on any ancestor is implicitly app_admin beneath it. New Capability
variants InstanceCreateGroup / Group{Read,Write,Admin}; group caps carry
no app_id (bound API keys can't manage groups). 8 new unit tests.
Admin API:
- groups_api: group CRUD + reparent (admin at both source and destination
parent, §5.6) + per-group members, all capability-gated.
- apps: POST /apps takes an optional parent group (default root); app
responses carry group_id; my_role now reflects the effective role.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase 2 (blueprint §5): groups form a single-parent org tree ABOVE apps.
This migration adds the tree + membership tables and gives every app a
parent (§9 adoption):
- groups: id, parent_id (self-FK ON DELETE RESTRICT), slug (instance-
global UNIQUE, frozen at creation), name, description, structure_version
(bumped on structural mutation), owner_project (inert §7 seam).
- group_members: (group_id, user_id, role) with the SAME three role
literals as app_members so AppRole round-trips and the authz rank table
covers both.
- apps.group_id: nullable add → backfill every app under a seeded 'root'
group → promote to NOT NULL + FK RESTRICT.
No group-owned resources yet (scripts/secrets stay app-owned — Phase 3).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds the declarative `pic` project tool (init/pull/plan/apply/prune,
env-scoped overlays, config --effective), the three-state `enabled`
lifecycle on scripts/routes with dispatcher fire-time re-checks, the
trigger `name` column + backfill, and cross-app/disabled-execution
backstops on every async dispatch path. Design captured in
docs/design/groups-and-project-tool.md.
Verified before merge: cargo fmt/clippy(-D warnings)/check clean;
manager-core 370+ unit tests pass (incl. dispatcher enabled-gates and
cross-app isolation); all new CLI journey tests pass against a live DB.
The 6 failing logs/scripts/roles journey tests are pre-existing format
drift on main, not regressions from this branch.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Update §4.3: the fire-time `enabled` re-check is shipped and test-backed on
BOTH async arms (the unified outbox `!resolved.active` gate and the queue
arm's fresh-script nack), with the specific regression tests cited, plus the
same-app backstop now present on every async build path.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Incidental `cargo fmt --all` normalization of two pre-existing over-width
`.lock()` chains in DevEmailSink; no behavior change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Four review fixes on the `pic` surface:
- `config --effective` gains `--env`: it now resolves through the same
overlay as `plan`/`apply`, so the masked report targets the app those
commands act on instead of silently reading the base slug's secrets.
- `pull` gains `--force` and refuses to overwrite an existing
`picloud.toml` (and colliding `scripts/*.rhai`) without it — fail-fast
before any network call or write, mirroring `init`.
- Overlays (`ManifestOverlay`/`OverlayApp`) get `deny_unknown_fields`: a
`[[scripts]]`/`[[routes]]`/`[[triggers]]` table or a typo'd key in an
overlay now errors loudly instead of being silently dropped. +unit test.
- `apply` 409 (StateMoved — the only 409 the endpoint emits) surfaces an
actionable hint (re-run `pic plan`, or `--force`) instead of a bare
`HTTP 409`.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The interactive create path rejects a module whose name shadows a built-in
SDK namespace (`kv`, `secrets`, …) so `import "kv"` can't resolve to a user
module, but `pic apply`'s `validate_bundle` skipped the check — a parity gap
that let a manifest create what the dashboard forbids. Gate it on
`ScriptKind::Module` and share the same `RESERVED_MODULE_NAMES` const (now
`pub(crate)`); endpoints remain unrestricted.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The `enabled` lifecycle's "a disabled script is not executable via ANY
path" guarantee leaked on the queue arm, and the outbox trigger arm lacked
the cross-app backstop its siblings carry.
- Queue arm: `dispatch_one_queue` bypasses the unified `dispatch_one`
fire-time gate, so its only `enabled` guard was the per-tick
`list_active_queue_consumers` snapshot — a TOCTOU window for a message
claimed in a tick where the script was still enabled. Re-check the
freshly-read `script.enabled` before executing and release the claim
(`nack`) when disabled; the message stays queued and resumes on
re-enable. The nack is load-bearing: `reclaim_visibility_timeouts` only
reclaims claims for enabled triggers, so without it the message would
sit claimed indefinitely.
- Trigger arm: `resolve_trigger` built the ExecRequest with
`app_id = row.app_id` while sourcing the body from `trigger.script_id`
with no same-app check — the guard the queue/http/invoke arms already
have. Add it so a hand-edited/restored row can't run one app's script
under another's SdkCallCx.
Both regression-locked by new unit tests (full mock harness, verified to
fail if the gate is removed):
- queue_enabled_gate::disabled_queue_consumer_releases_claim_without_executing
- outbox_enabled_gate::disabled_http_outbox_row_is_dropped_without_executing
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Holistic Phase-1 review found the "a disabled script can't execute via
any path" guarantee (§4.3) held only for the sync user-route, the
execute-by-id bypass, and the trigger outbox arm — three paths still ran
disabled scripts:
- Queue arm: `list_active_queue_consumers` filtered the trigger's
`enabled` but not the bound script's. Add `JOIN scripts … AND
s.enabled = TRUE` so a disabled script's queue trigger stops consuming.
- Async-HTTP (202) + queued invoke() arms: `build_http_request` /
`build_invoke_request` hardcoded `active: true`. Set it to
`script.enabled`, and MOVE the dispatcher's fire-time `active` drop to
after the source-kind match so it covers all three arms uniformly
(previously trigger-only).
- `invoke()` (script-to-script): `resolve_id`/`resolve_name` checked
cross-app isolation but not `enabled`. A disabled target now resolves
to NotFound (indistinguishable from absent), matching the data plane.
Also (review LOW/§4.7):
- The route-on/script-off 404 returned `NotFound(script_id)`, leaking the
internal id and distinguishable from absent. Return the same flat "no
route matches" 404 as the unmatched case.
- Add the §4.7 "enabled endpoint with no route and no trigger" reachability
warning (was unimplemented; only disabled-target shipped).
Tested: manager-core lib 368 + orchestrator 75 + 22 project-tool journeys
(incl. a new enabled-route→disabled-script flat-404 e2e) green; clippy
-D warnings clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fold the implementation outcome back into the project-tool spec (§11) as
the design asks. Records what landed (init/pull/config/plan/apply/prune,
env overlays, enabled lifecycle, trigger name+backfill, content-fingerprint
bound-plan, apply lock) and the deliberate deferrals: tree-structure
version → groups; vars/--explain → Phase 3; trigger upsert-by-name + the
dashboard enabled toggle → tracked follow-ups beyond the §11 bullets.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Read the new `name` column through the data model: `Trigger`/`TriggerRow`
carry it, every trigger SELECT/RETURNING includes it, and the row→Trigger
assembly + the interactive create paths populate it (the create INSERTs
still omit it, so they take the migration's default). Behavior unchanged
— the apply diff still keys on the semantic tuple; B-2 switches it to
name-keying (enabling Update) and wires the manifest.
Tested: manager-core lib 367 + trigger journeys green; clippy clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Schema groundwork for the trigger `name` identity (the manifest merge key
that will let `apply` Update a trigger in place rather than only
Create/Delete). A `gen_random_uuid()` default keeps the existing write
paths valid and unique without code changes; existing rows are backfilled
to the readable `{kind}-{n}` form; `UNIQUE(app_id, name)` is enforced.
No behavior change yet — the manager-core write paths and the apply diff
start using the name in the follow-up. Verified the migration applies and
the trigger journeys (which create triggers via the unnamed path → default
name) stay green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The §4.1 base+overlay model for the single-app world: a base `picloud.toml`
holds the shared scripts/routes/triggers; a sparse `picloud.<env>.toml`
carries the per-environment slug (+ extra secret names). `pic plan/apply
--env <E>` merges the overlay onto the base before diffing/applying, so an
environment deploys to its own app ("an environment is an app", §2)
without duplicating the shared definitions.
Merge is sparse: overlay `app.slug`/`name` replace the base's; secret
names union. Scripts/routes/triggers always come from the base. Rich
per-key `vars` resolution stays Phase 3.
Tested: unit tests for the merge + overlay-path derivation; a journey
proving `apply --env staging` deploys to the staging app and leaves the
base app untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The §11 Phase-1 `config --effective` surface. Single-app today, "config"
= the app's secrets, so it cross-references the manifest's declared
secret names against what's set on the server and renders each masked:
`<set>` (managed / on-server-not-in-manifest) or `<unset>` (declared but
not pushed). Values are never fetched or shown (§4.6). Shaped to grow an
`--explain` mode and inherited `vars` when groups/vars arrive (Phase 3).
Tested: a journey verifying declared-but-unset → `<unset>`, post-`secret
set` → `<set>`/managed, and the value never appears in output.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Close the coverage asymmetry flagged in review: script-disable had a full
e2e journey but route-disable was only unit-tested (compile_routes) plus
the apply-refresh path. Add an HTTP-level journey — claim a Host for the
app (two-phase dispatch needs it), then serve `/hello` (200), flip the
route's `enabled = false` and re-apply (404, indistinguishable from
absent), and re-enable (200).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- §4.7 warning: `apply` now reports when an enabled route or trigger is
bound to a disabled script — deployed but unreachable (route 404s,
trigger won't fire). A warning, not an error: it's valid desired state,
the operator just shouldn't be surprised by the silent 404.
- E2E journey (`tests/enabled.rs`): apply an active script and confirm
`/api/v1/execute/{id}` 200s; flip `enabled = false` in the manifest and
re-apply → 404; re-enable → 200. Exercises the whole path: manifest →
diff → apply → runtime honoring.
Tested: manager-core lib 367 + cli bins 31 + 15 project-tool journeys
(incl. the new enabled e2e) green; clippy -D warnings clean; versioning
gate passes (migration 0045).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Close the §4.3 outbox gap: the match-time `enabled` check can't see a
trigger (or its target script) disabled *after* an outbox row was
enqueued, so a pending row would still fire. `resolve_trigger` now folds
`trigger.enabled && script.enabled` into the resolved row, and
`dispatch_one` drops the row at fire time when inactive instead of firing
a stale event.
The queue arm needs no change here: `list_active_queue_consumers` already
re-lists only enabled queue triggers each tick, so a disable is honored
promptly without a stale-row window.
Tested: manager-core lib 366 green; clippy -D warnings clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make the `enabled` flag bite at the data plane (§4.3).
- Routes: `compile_routes` drops disabled rows from the match table, so a
request to a disabled route 404s indistinguishably from an absent one
(no info leak). Rebuilt on every route write + at startup, as today.
- Scripts: the orchestrator's two execute paths — `/execute/{id}` bypass
and the user-route handler — 404 when the resolved script is disabled.
The route-handler check also covers an enabled-route-bound-to-disabled-
script (§4.7): the binding is unreachable rather than 500/executing.
Still pending: the dispatcher fire-time re-check for already-enqueued
outbox rows (next commit).
Tested: route_admin disabled-route-dropped unit test + orchestrator suite
(75) green; clippy -D warnings clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase-1 `enabled` three-state lifecycle (§4.3), data-model half. Triggers
already carried `enabled`; this adds it to scripts and routes and threads
it through the declarative project tool. Runtime honoring (disabled route
404 / script non-invocable / dispatcher fire-time re-check) is the next
commit — this change only stores and reconciles the flag.
- Migration 0045: `enabled BOOLEAN NOT NULL DEFAULT TRUE` on scripts +
routes. Default true ⇒ no behavior change on migrate.
- `Script`/`Route` (shared) gain `enabled` (serde default true via the new
`picloud_shared::default_true`); repos' SELECT/INSERT/UPDATE SQL, row
structs, `NewScript`/`NewRoute`/`ScriptPatch` all carry it.
- Apply diff treats `enabled` as a declarative field (omitted ⇒ active):
`script_update_reason` + `diff_routes` detect a toggle as an Update, and
the create/update/insert paths persist it. The bound-plan `state_token`
re-includes script/route `enabled` (removed in the earlier review fix
precisely because the diff didn't key on it yet — now it does).
- CLI manifest model + `build_bundle` + `pull` round-trip `enabled`
(serialized only when false; omitted ⇒ active). `pic init` scaffold and
the interactive script/route create paths default active.
Tested: manager-core lib 365 (incl. enabled diff + token sensitivity) +
cli bins 31 (incl. manifest skip-serialize) green; clippy -D warnings
clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Remediation after an independent review of the two feature commits.
pic init:
- Fix the headline `pic init --dir <new-dir>` flow: slug derivation used
`canonicalize()`, which fails on a not-yet-created directory → empty
slug → bail. Derive from the path's own final component first, falling
back to canonicalize only for `.`. Add a test that exercised the gap
(the existing tests all passed an explicit slug).
- `ensure_gitignored` no longer overwrites an existing-but-unreadable
`.gitignore` (only a genuinely-absent file is treated as empty).
Bound-plan staleness:
- Token trigger coverage now mirrors the diff exactly via
`current_trigger_identity`: dead-letter triggers (which the diff
ignores and the manifest can't represent) and the currently-inert
`enabled` flag are no longer hashed, so an out-of-band dead-letter
change can't force a spurious `--force`. Add a test.
- Correct two overclaiming comments: the check shares the diff's
pool-read apply-vs-interactive-write window (it is not tx-scoped), and
the token deliberately mirrors the diff's inputs.
- `.picloud/` self-ignores via a `*` `.gitignore` written alongside the
plan token, so projects created with `pic pull`/`plan` (not just
`init`) keep it out of git.
- `clear_plan` is now app-scoped: it won't discard a token recorded for a
different app sharing the directory.
Tested: manager-core lib 364 + cli bins 31 + 14 project-tool journeys
green; clippy -D warnings clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`pic plan` now records a fingerprint of the live state it diffed against;
`pic apply` replays it and the server refuses (HTTP 409) if the app
changed underneath the reviewed plan — the §4.2 "apply exactly what you
reviewed" guarantee, in its content-addressed form (no migration, no
changes to interactive write paths).
Server (manager-core):
- `state_token(CurrentState)`: deterministic FNV-1a fingerprint over what
the diff keys on — script name+version (version bumps on any edit),
route identity+binding/attrs, trigger membership+enabled, secret names.
Order-independent; a collision can only yield a false "unchanged", never
a false refusal.
- `plan` returns it (flattened onto the plan JSON, so the wire stays
additive); `apply` takes an optional `expected_token` and, under the
apply lock before any write, returns `StateMoved` (409) on mismatch.
CLI:
- `.picloud/` link state (`linkstate`): `pic plan` writes the token scoped
to the app slug; `pic apply` replays it, then clears it on success (the
token is single-use — the next apply re-plans). `--force` skips the
check; apply with no recorded plan still works standalone (today's
behavior). `.picloud/` is already gitignored by `pic init`.
The tree-structure version (the other half of §4.2's counter) stays a
deliberate no-op until groups exist — it only guards reparent/structural
moves, which don't exist single-app.
Tested: state_token unit test (stable/order-independent/sensitive) + a
staleness journey (plan → out-of-band deploy → apply refuses → --force
applies); manager-core lib 363 + cli bins 31 + 13 journeys green; clippy
-D warnings clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Offline project scaffolding — the on-ramp to pull/plan/apply. Writes a
`picloud.toml` describing a minimal *working* app (a `hello` endpoint
bound to GET /hello) plus commented examples for the other blocks,
`scripts/hello.rhai`, and a `.gitignore` that pre-ignores the project
tool's `.picloud/` link-state directory (the home for the bound-plan
state token, landing next). Refuses to overwrite an existing manifest
without `--force`; derives the app slug from the directory name when not
given (validated against the canonical slug rule).
The active manifest is rendered through the same `to_toml` model the rest
of the tool uses, so a freshly-init'd project is guaranteed valid and
immediately `pic plan`-able. Offline — no server contact, so the journey
tests need no DATABASE_URL.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Remediation of the single-app reconcile foundation after two independent
review passes. No new feature surface — closes correctness, parity, and
safety gaps in pull/plan/apply/prune.
apply engine (manager-core):
- validate_bundle reached parity with the interactive trigger API: reject
an empty kv/docs/files collection_glob and a malformed pubsub
topic_pattern (previously written and silently never matched), and lift
the queue visibility floor to MIN_QUEUE_VISIBILITY_TIMEOUT_SECS (30) —
apply had accepted a [5,29] value the dashboard refuses. Per-kind checks
extracted into a pure, unit-tested validate_trigger_shape.
- An omitted script `description` now means "leave as-is" (matching the
other optional fields and the function's own documented contract)
instead of clearing the stored value.
- Doc fixes: drop stale "next milestone" notes; record the one deliberate
plan/apply divergence (set-but-empty secret); document delete_route_tx
as intentionally idempotent for reconcile.
CLI:
- Gate destructive `apply --prune` behind confirmation: interactive y/N,
or `--yes` for CI; a non-interactive prune without `--yes` refuses
rather than silently deleting. Journey tests pass `--yes`; a new test
proves the gate refuses and deletes nothing.
- Harden pull's filename safety: reject all control characters and the
Unicode bidi-override / zero-width chars used for terminal/filename
spoofing, and cap length at 200 bytes (NAME_MAX headroom).
Tested: manager-core lib (incl. queue-floor + sparse-description parity)
+ CLI bins + 9 project-tool journeys green; clippy -D warnings clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a server-side, atomic, declarative reconcile loop for a single app —
the foundation of the project-tool design. Developers describe an app's
scripts, routes, triggers, and secret-names in `picloud.toml`, then
`pic pull / plan / apply [--prune]` to converge live state to the manifest.
Server (manager-core):
- apply_service: a pure diff engine (compute_diff) shared by plan and
apply, plus an ApplyService that composes the existing per-repo writes
into ONE Postgres transaction. Identity keys mirror the DB UNIQUE
constraints (script=lower(name); route=(method,host_kind,host,
path_kind,path); trigger=per-kind semantic tuple; secret=name).
Apply takes a per-app advisory lock, recomputes the diff in-tx, applies
scripts -> routes -> triggers, prunes dependents-first, commits, then
refreshes the route table once post-commit.
- apply_api: POST /apps/{id}/plan (AppRead) and /apps/{id}/apply.
Apply requires the per-kind write caps the bundle exercises (all three
when --prune), plus AppSecretsRead when it binds an email trigger.
- tx-accepting repo siblings (insert/update/delete *_tx) so the existing
create/update/delete delegate to one SQL definition each.
- email triggers reference an inbound secret by NAME; the value is
resolved, decrypted (AAD-bound), and re-sealed server-side at apply —
it never travels in the manifest.
CLI (picloud-cli):
- manifest.rs (picloud.toml model), client plan/apply, and the pull/plan/
apply commands. pull rejects filesystem-unsafe script names up front.
Safety properties enforced and tested:
- idempotent: a freshly-pulled manifest re-applies as all-NoOp.
- atomic: a mid-bundle failure rolls back with nothing written.
- routes delete-before-insert so a freed binding is reusable in one apply.
- queue one-consumer invariant held inside the shared tx.
- email triggers are never pruned, and a script that still owns an
email/dead-letter trigger can't be pruned (the FK cascade would destroy
the sealed secret) — refused with a pointer to `pic triggers rm`.
- plan and apply agree on unset email-secret references.
No migration: the existing schema's UNIQUE constraints serve as identity
keys. Groups, env-scoping, and the `enabled` toggle are later milestones.
Tested: manager-core lib (360) + CLI bins (27) + 8 project-tool journeys
(pull/plan/apply/prune/email+queue), all green; clippy -D warnings clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The architecture spec for the full vision — a declarative project tool
(picloud.toml + pull/plan/apply/prune) and a server-side nested-groups
hierarchy with live-resolve inheritance. Reviewed across several passes
for consistency, gaps, and contradictions; resolutions are folded in
beside the topics they settle.
This commit lands the spec only. The first slice of implementation (the
single-app reconcile foundation) follows; groups, env-scoping, and the
`enabled` toggle are later milestones called out in the doc.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Closes the gaps and the one security finding from the second end-to-end
CLI test (E2E_STASH_REPORT.md), plus the H1 boot-regression found while
re-reviewing those fixes.
Security
- S6: reserved-path validation (`check_reserved`) now case-folds before
comparing, so `/API/v2/x`, `/HEALTHZ`, `/Admin/x` are rejected like
their lowercase forms. Request-time matching stays case-sensitive.
- S10: "public route != public data" callout in sdk-shape.md (script_gate
skips authz when the principal is anonymous).
Observability / features
- G1: trigger executions now write `execution_logs`. Migration 0043 adds
a `source` column (CHECK mirrors ExecutionSource/OutboxSourceKind,
DEFAULT 'http' backfills history); a shared `build_execution_log` helper
in executor-core; dispatcher logging for outbox triggers + queue
consumers (skips sync-HTTP rows the orchestrator already logs). `pic
logs` gains a source column + `--source` filter.
- G5: dev-only in-memory email capture under PICLOUD_DEV_MODE with no SMTP
(email::send succeeds locally), readable at GET /api/v1/admin/dev/emails
(Owner/Admin only; route mounted only in capture mode).
- G6: generalized the Rhai in-place-mutation footgun note (trim/replace/
make_upper/make_lower/crop/truncate/pad return ()).
- G2/G3/G4 (CLI): `pic members`, `pic files`, `pic queues`, read-only
`pic kv` (+ new kv_api.rs); `pic deploy --timeout/--memory/--kind/
--sandbox`; first-class `pic triggers create-{docs,files,pubsub,queue,
email}` wrappers. All new client path segments percent-encoded via seg().
H1 regression fix (found in re-review)
- The S6 change also runs in `compile_routes`, which compiles every stored
route at boot and on each route CRUD. A single stored route the new
validation rejects (creatable while the S6 gap existed) made the whole
compile Err and aborted startup. `compile_routes` is now lenient: it
skips an un-compilable row with a warning instead of bricking boot
(route creation still validates separately). Migration 0044 sweeps
pre-existing reserved-path routes on upgrade (WHERE mirrors
check_reserved exactly). Added regression tests for both.
Verified: cargo fmt, clippy --all-targets --all-features -D warnings, the
schema_snapshot test, and the new S6/lenient-compile unit tests all pass;
boot-resilience and G1/G5 confirmed live.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Addresses findings from an independent review of the gap-closing commits:
- R1 (regression): add #[serde(default)] to HttpDispatchPayload.method so
async-HTTP outbox rows enqueued before the field existed still decode
after upgrade instead of dead-lettering on "missing field `method`".
Adds a regression test for the missing-key path.
- method case: uppercase ctx.request.method at the orchestrator boundary
so it honors its documented "uppercased" contract for extension verbs.
Route matching is already case-insensitive, so matching is unaffected.
- CLI hardening: percent-encode free-form path segments (app slug, topic
name, ids, secret name) in the reqwest client so a value containing
`/ ? #` can't break out of its URL segment. Adds a `seg()` helper +
unit test and applies it uniformly across all path-interpolating
methods (new and pre-existing).
- S1 (docs): correct the users::email_available enumeration framing — it
adds no new vector vs. create's uniqueness error, but is cheaper and
unthrottled; there is no built-in throttle/CAPTCHA primitive, so the
honest mitigation is a kv-based counter. Updated SDK doc-comments,
trait docs, and stdlib-reference.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Rebuilt the rich To-Do app end-to-end through `pic` alone (no raw admin
curl) after the gap fixes landed, and appended a per-finding verification
table + transcript to the report. Confirms B1/B2/F1–F4/S1–S3/O1 closed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- S2: document that the response envelope is statusCode-gated — a returned
map is only unwrapped into {statusCode, headers, body} when it contains
statusCode, else the whole map silently becomes the 200 body.
- S3: note that Rhai String.replace() mutates in place and returns (),
with the sub_string bearer-parsing idiom.
- Document ctx.request fields including the new `method`.
- O1: document PICLOUD_DEV_INSECURE_KEY next to PICLOUD_DEV_MODE in
CLAUDE.md (the acknowledgement var was previously only in the startup
error).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`users::find_by_email` requires an authenticated principal (F-S-003,
anti-enumeration), so the natural check-then-create register pattern 502s
for anonymous public scripts. Add `users::email_available(email) -> bool`,
gated like `create` (the registration write path) but without the
anonymous rejection: it leaks only a single boolean — no more than
`create`'s own uniqueness error already does — so self-serve register
scripts can pre-check. Also document find_by_email's principal
requirement in the SDK doc-comment.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The request map exposed path/headers/body/params/query/rest but not the
HTTP method, forcing one script per verb (the E2E app needed 7 scripts
where ~3 would do). Add a `method` field to ExecRequest (populated from
the HTTP method on the sync-execute bypass and the HTTP outbox payload;
empty for non-HTTP triggers) and surface it as `ctx.request.method`,
uppercased. A single script can now branch GET/POST on one route.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A CLI-only walkthrough (E2E_TODO_REPORT.md) found two control-plane
operations with no `pic` command — every CLI-created app 404'd until it
claimed a domain, and external SSE feeds needed a raw topic-registration
call — plus several friction points. All admin APIs already existed; this
adds thin wrappers mirroring `routes`/`triggers`:
- B1: `pic apps domains {ls,add,rm}` over apps/{id}/domains.
- B2: `pic topics {ls,create,update,rm}` over apps/{id}/topics, and a
triggers help note distinguishing a pubsub trigger from topic
registration.
- F2: `pic users {ls,show,reset-password,revoke-sessions}` over the app
end-user admin surface (read + the two admin actions; create/invitations
deferred).
- F1: `apps create` and `deploy` now honor `--output json`, emitting the
created object so scripts can capture the id.
- F3: non-interactive login via `--username` + `--password-stdin` (inline
passwords still rejected, mirroring the `--token` rule).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Review nit — the field is a non-optional Arc; the old '`None` for tests'
note was stale. Clarify that it's one shared Arc so revocation-side
evictions are visible to the middleware's resolve-side lookup.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
extract_remote_ip took the FIRST X-Forwarded-For entry, but Caddy (the
single trusted proxy) appends the real peer as the LAST hop — earlier
entries are client-supplied. An attacker could prepend a rotating
X-Forwarded-For value to get a fresh per-IP bucket every request and
evade the per-IP login limiter (the per-username bucket + global Argon2
semaphore still bounded it, but the per-IP layer was defeated). Take the
last hop so the key reflects the real client. Added unit tests covering
the spoof, the single-hop case, and the missing-header fallback.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
pic admins create/set --password and pic login --token accepted secrets
on argv, where they leak into shell history, ps aux, and
/proc/<pid>/cmdline. Now only --password -/--token - (stdin) is allowed;
an inline value is rejected with guidance. PICLOUD_TOKEN env still covers
CI for login.
Also (M2): the interactive admins password prompt used read_line, which
echoes despite a 'no echo' claim — switched to rpassword::prompt_password
(already a dep via login). Removed the now-dead ReadLine trait + Write
import.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Caddy request_body { max_size 12MB } in both Caddyfiles — a hard
ceiling replacing Caddy's multi-GB default; 12MB clears the
orchestrator's 10 MiB user-route read. Validated with caddy validate.
- email-inbound router gets an explicit DefaultBodyLimit::max(1MB):
the public unauthenticated webhook previously rode Axum's 2MB
extractor default; tightened so a flood can't force large
allocation + JSON parse per request.
Admin / execute handlers keep Axum's 2MB extractor default (already
bounded); the user-route path keeps its explicit 10 MiB manual read.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The admin files endpoints hit FsFilesRepo directly (not via
FilesServiceImpl, which validates), so only create/update guarded the
collection — head/get/list/delete built FS paths from an unvalidated
value. Today nothing can store a traversal-shaped collection, but one
future bad migration / restore tool inserting collection='../../etc'
would give the read/delete paths arbitrary host-file reach.
- FsFilesRepo::{head,get,list,delete} now call guard_collection (create
and update already did).
- The three admin endpoints (list/get/delete) call
validate_files_collection up front for a clean 422 instead of the
repo guard surfacing as an opaque 500.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ExecError::Runtime strings (filesystem paths, "blocked by SSRF policy:
link-local" cloud-metadata reconnaissance, pool-exhaustion timing,
panic fragments, and attacker-thrown strings) were returned verbatim in
the public /api/v1/execute + user-route 502 bodies. This handler only
serves anonymous data-plane callers (the authenticated script-test path
is in manager-core and keeps verbose errors), so log the full text
server-side under a correlation id and return a stable generic message
plus the id for operator grep.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
C-2 follow-up: sanitize_stored_content_type previously let a CRLF pass
through an allowlisted prefix (e.g. "image/png\r\nX-Injected: 1" matched
the image/ branch and returned the original), which would inject a
response header / panic HeaderValue on download. Now any control byte
(<0x20 or 0x7f) coerces to application/octet-stream up front.
F-SE-H-03: disable_symbol("debug") to match "print" (the comment
already claimed both were disabled) so scripts can't write
attacker-controlled bytes to the operator's stderr.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The PrincipalCache (token-hash -> resolved Principal, 60s TTL) had no
eviction hook, so deactivation / password change / API-key revocation
flipped the DB rows but the cache kept serving the stale principal for
up to the TTL window. Closes the audit's PrincipalCache revocation-lag
Medium and greens authz::deactivating_user_revokes_their_api_keys.
- PrincipalCache::evict_user(user_id) + evict_token(hash).
- One shared cache Arc threaded into AuthState / AdminsState /
ApiKeysState (a per-state cache would let the middleware's own copy
keep authenticating a revoked principal).
- Evict on: deactivation, password change (both evict_user), logout
(evict_token, precise), single API-key delete (evict_user).
- H-E1 note: DB-write invalidation stays best-effort by design (a blip
must not undo the rotation); the cache evict is what makes it take
effect on the next request.
- Renamed bearer_and_cookie_produce_same_principal ->
session_token_and_api_key_produce_same_principal (cookie auth was
removed in C-1; the test never tested cookies).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Postgres-backed integration tests surfaced two assertion fixes while
verifying the audit branch:
* email_inbound::create_without_inbound_secret_is_rejected (added in the
H-B2 commit) asserted 400; TriggersApiError::Invalid maps to 422
(UNPROCESSABLE_ENTITY). Corrected.
* api::version_includes_public_base_url had two rotted assertions —
`schema` pinned at 6 (it's migrations::latest_version(), which had
climbed to 41 and is now 42 after 0042_secrets_envelope_version.sql)
and `sdk` pinned at "1.1" (the crate is at "1.10"). The test is
#[ignore]-gated so the rot went unnoticed in normal runs. Both pinned
to current reality. These were pre-existing failures, not caused by
the security changes.
Verified against a throwaway Postgres: schema_snapshot, api (53),
authz (28 of 29 — see below), dispatcher_e2e, email_inbound, invoke_e2e,
queue_e2e all green.
Known pre-existing failure (NOT fixed here, out of Tier 1+2 scope):
authz::deactivating_user_revokes_their_api_keys fails identically on the
pre-branch merge-base. It's the PrincipalCache revocation-lag Medium the
audit flagged as unfixed (resolve_principal caches by token-hash with no
eviction on deactivation; lag bounded by the 60s TTL). Deferred to a
Tier 3/4 pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
At-rest secrets were AES-256-GCM sealed with no Associated
Authentication Data, so anyone with Postgres write access could
ciphertext-swap rows across apps (or rename via row edit) and the
decrypt would silently succeed under the wrong identity, returning
attacker-chosen plaintext. This breaks the cross-app isolation boundary
the moment DB write access is achieved.
Adds an AAD-bound envelope (v1) alongside the legacy no-AAD layout (v0),
discriminated by a per-row `version` column:
* shared::crypto — new encrypt_with_aad / decrypt_with_aad using
aes_gcm::aead::Payload { msg, aad }. Originals retained for v0 reads.
Tests: AAD round-trip, AAD-mismatch fails, empty-AAD round-trip.
* migration 0042 — adds `secrets.version SMALLINT NOT NULL DEFAULT 0`
and `app_secrets.realtime_signing_key_version SMALLINT NOT NULL
DEFAULT 0`. Existing rows stay v0; new writes are v1.
* secrets (SDK + admin API) — seal() now binds AAD =
"secret:{app_id}:{name}" and emits v1; open() dispatches on version.
StoredSecret gains a `version` field; SecretsRepo::set takes it.
Both secrets_service::set and secrets_api::set_secret go through the
v1 path. Tests prove a cross-app swap and a cross-name swap both
surface Corrupted, and that a hand-built v0 row still decrypts.
* app_secrets (realtime signing key) — get_or_create_signing_key writes
v1 with AAD = "app_secret:{app_id}:realtime_signing_key"; decode
dispatches on version. Tests cover v0 decode, v1 round-trip, and v1
decode-under-wrong-app failing.
* email-trigger inbound secret — kept on v0 (seal_legacy/open_legacy)
and explicitly deferred: email_trigger_details has no version column
and the trigger_id isn't known at seal time. The audit classes the
email-trigger AAD gap as Medium; folded into v1.2's key-versioning
pass per SECURITY_AUDIT.md.
* expected_schema.txt re-blessed by hand (no local Postgres) for the two
new columns + migration 0042.
Also folds in a let-else clippy fix in auth_api.rs (login Argon2
semaphore acquire, from the H-B1 commit) and two cargo-fmt reflows.
No re-encryption sweep — v0 rows decrypt as-is; the sweep is deferred to
v1.2's key-versioning pass (audit "Notes on remediation methodology").
Audit ref: security_audit/03_crypto_secrets.md (H-D1).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Rhai SDK email::send path went straight to the SMTP relay with no
admission control. EmailRateLimiter lived in users_service.rs (gating
verification + password-reset emails) but was unreachable from the SDK
surface. Any anonymous-callable HTTP-route script could loop email::send
to burn the operator's SMTP quota or BCC-bomb thousands of recipients
per request.
Two new caps on EmailServiceImpl, both fire BEFORE message assembly +
SMTP connect:
1. Per-message recipient cap (to+cc+bcc combined). Default 20, override
with PICLOUD_EMAIL_MAX_RECIPIENTS. New EmailError::TooManyRecipients
variant. Closes the BCC-bomb amplification path.
2. EmailRateLimiter inlined into EmailServiceImpl:
* Per-(app, recipient): RECIPIENT_BURST=5 / RECIPIENT_WINDOW=60min.
* Per-app daily: APP_DAILY_CAP=200 / 24h.
The structure mirrors users_service's private limiter; defense-in-
depth redundancy is fine since the gates are identical and never
conflict. New EmailError::RateLimited(&'static str) variant; the
string names which bucket tripped so the operator can act.
users_service::map_email_error grows two cases (mapped to
EmailTransport for the script-facing error shape).
Tests:
* too_many_recipients_rejected — 30 recipients, default cap 20, rejected
with TooManyRecipients.
* per_recipient_burst_caps_repeated_send_to_same_address — 6 sends to
the same address; 6th returns RateLimited("per-recipient burst").
Audit ref: security_audit/09_external_integrations.md#f-ext-h-001.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
