Commit Graph

86 Commits

Author SHA1 Message Date
MechaCat02
fd4336e883 feat(queue): dead-letter store for group shared queues (§11.6 D3 / Track A M2)
An exhausted SHARED durable-queue message was `drop_exhausted()`-ed with a
warning — silent data loss. Per-app queues persist to `dead_letters` (0010);
add the symmetric group store so an exhausted shared-queue message is preserved
and operator-visible.

- Migration 0068: `group_dead_letters`, keyed by (group_id, collection),
  CASCADE on the group (an app delete leaves the data — it belongs to the
  group, not the consuming app).
- `GroupQueueRepo::dead_letter` (replaces `drop_exhausted`): one tx that INSERTs
  the dead-letter + DELETEs the live message, filtered by claim_token so a lost
  lease can't dead-letter a re-claimed message (mirrors queue_repo::dead_letter).
- Dispatcher `q_terminal` shared arm now dead-letters instead of dropping. It
  returns None (not the dl id) so the per-app `fan_out_dead_letter` is SKIPPED —
  firing the consuming app's per-app handlers on a shared message (competing
  consumers → nondeterministic app) would be wrong. Fan-out to a *shared*
  dead_letter trigger is deferred (needs a new trigger kind).
- Read side: `GroupDeadLetterRepo::list_for_group` backs a new read-only
  operator endpoint GET /api/v1/admin/groups/{id}/dead-letters (GroupKvRead,
  mirrors the M4 group-blobs surface). `pic dead-letters ls --group` deferred
  (optional; the HTTP endpoint is the operator surface).

Pinned by group_queue::dead_letter_moves_an_exhausted_message_to_the_group_store
(store move + claim-token-mismatch guard + operator read). Schema golden
reblessed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 14:41:04 +02:00
MechaCat02
f673922d89 feat(cli): pic projects ls + §7 M3/track-complete docs
§7 M3 (part 3) — the ownership track becomes inspectable, completing M1–M3.

- server: GET /api/v1/admin/projects (projects_api) lists every project + its
  owned-group count (projects repo list_with_counts, one GROUP BY); behind the
  /admin middleware, like the groups list.
- CLI: pic projects ls (cmds/projects.rs + client projects_list); the
  apply_ownership plan-preview journey now also asserts projects ls (plat owns
  exactly 1 group; teamb listed).
- docs: design §7 + CLAUDE.md record M3 shipped and the whole §7 multi-repo
  ownership track (M1 claim · M2 attach ceiling · M3 preview + projects ls)
  COMPLETE; deferred = structural-divergence + declarative tree shape + per-env
  approval gating.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 21:26:10 +02:00
MechaCat02
2cce051dc4 feat(projects): projects repo + group owner reads; wire into ApplyService
Read side of §7 ownership, ahead of the claim logic.

- project_repo: `ProjectRepository` trait + `PostgresProjectRepository`
  (list / get_by_id / get_by_slug). Writes are NOT here — a claim registers
  its project via an in-tx upsert (next commit).
- group_repo: `list_with_owner()` LEFT-JOINs projects for each group's owner
  slug (backs `pic groups ls`); columns qualified since id/slug/name exist on
  both tables.
- ApplyService gains a `projects` handle; constructed in the picloud binary.
- tests/projects_repo: round-trip pinning list_with_owner (claimed→slug,
  unclaimed→None) and that ancestors() surfaces owner_project nearest-first —
  the fold input the claim path walks.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 20:13:34 +02:00
MechaCat02
50205e7b7e feat(shared-queues): materialized competing consumers + dispatcher branch (D3.2)
The consumption side of shared durable queues:
- Thread `shared` through BundleTrigger::Queue + QueueTriggerSpec + the
  trigger identity; validate_bundle_for requires a shared queue on a group
  to name a declared kind='queue' collection (a shared queue on an app is
  rejected by the existing app-owner shared guard).
- materialize: a shared queue template materializes a consumer per
  descendant (the M5 one-consumer-slot skip is bypassed for shared —
  competing consumers are intended; each descendant gets one copy).
- dispatcher: ActiveQueueConsumer gains shared_group (from the materialized
  copy's source template via LEFT JOIN); dispatch_one_queue +
  handle_queue_failure route claim/ack/nack/terminal to the group store
  when shared_group is Some, via q_claim/q_ack/q_nack/q_terminal helpers. A
  group claim is normalized to a ClaimedMessage under the consuming app so
  the handler path is unchanged; the reclaim task also drains the group
  store. Exhausted shared messages are dropped (no group dead-letter store
  yet — documented deferral).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 22:33:11 +02:00
MechaCat02
ccd3644aa4 feat(shared-queues): group-keyed queue store + enqueue service + SDK (D3.1)
The producer side of shared durable queues:
- migration 0065 group_queue_messages (mirrors 0034, keyed by (group_id,
  collection); CASCADE on group delete).
- GroupQueueRepo/PostgresGroupQueueRepo: enqueue + the competing-consumer
  claim (FOR UPDATE SKIP LOCKED) + ack/nack/drop_exhausted/reclaim/depth.
- GroupQueueService trait (shared) + GroupQueueServiceImpl: resolve owning
  group (kind='queue') from cx.app_id's chain, require editor+
  (GroupQueueEnqueue, fails closed on anon), size-cap, enqueue.
- SDK: `queue::shared_collection("name")` -> GroupQueueHandle with
  `.enqueue(msg[, opts])` / `.depth()` / `.depth_pending()`; wired through
  Services + the picloud binary.
- authz: Capability::GroupQueueEnqueue(GroupId), editor+ / script:write.

Deterministic test proves competing consumers claim each message exactly
once. Consumption wiring (materialized consumers + dispatcher branch) lands
in D3.2.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 22:08:21 +02:00
MechaCat02
9626d15863 feat(shared-topics): GroupPubsubService + shared publish fan-out + SDK handle (D2)
The runtime for shared topics:
- pubsub_repo: fan_out_publish gains `AND t.shared = FALSE` (a shared
  trigger never fires on a per-app publish); new fan_out_shared_publish
  matches `shared = true` pubsub triggers on the resolved owning group,
  each delivery stamped with the writer app_id (M2 model).
- GroupPubsubService trait (shared) + GroupPubsubServiceImpl: resolve the
  owning group (kind='topic') from cx.app_id's chain, require editor+
  (GroupPubsubPublish, fails closed for anon), size-cap, fan out.
- SDK: `pubsub::shared_topic("name")` -> GroupTopicHandle with
  `.publish(subtopic, msg)` / `.publish(msg)`; wired through Services +
  the picloud binary (reuses the one PubsubRepo).
- authz: Capability::GroupPubsubPublish(GroupId), editor+ / script:write.

The owning-group chain walk is the isolation boundary; a sibling-subtree
app never resolves the topic.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 21:48:59 +02:00
MechaCat02
ddb3589c49 feat(stateful-templates): materialize cron+queue templates + hooks (M5.2-M5.4)
materialize::rematerialize_stateful_templates reconciles a group cron/queue
template into one app-owned copy per descendant (materialized_from = template),
via the all-apps app_chain CTE ⋈ group-owned stateful templates. A precise
create/delete diff preserves cron last_fired_at on unchanged copies; a queue
copy is skipped-with-warning when the app already fills that queue's consumer
slot (the one-consumer invariant). Called full-live at the route-rebuild
chokepoints: apply (single + tree), app create/delete (apps_api), group
reparent (groups_api) — AppsState/GroupsState gain a pool. Pinned by
tests/stateful_templates.rs (per-app copy, idempotent, new-app materializes,
reparent de-materializes).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 21:47:27 +02:00
MechaCat02
c21e82fb9f feat(group-blobs): read-only operator admin API for shared collections (M4.1+M4.2)
group_blobs_api mirrors the per-app kv_api/files_api for groups: GET
/groups/{id}/kv[/{c}/{k}], /docs[/{c}/{id}], /files[/{c}/{id}] — list +
fetch/download a group's §11.6 shared KV/docs/files. Authz GroupKvRead/
GroupDocsRead/GroupFilesRead (viewer tier, reads-open trust model). Read-only
by design (writes stay script-only). The host hoists the three group repos so
the SDK services and this router share one instance.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 21:22:07 +02:00
MechaCat02
0210ace406 feat(shared-triggers): emit shared events from group services (M2.3)
ServiceEventEmitter gains emit_shared(cx, owning_group, event) (default no-op);
OutboxEventEmitter implements it — matches shared triggers on the owning group
and writes outbox rows stamped with the WRITER app_id (dispatch runs under the
writer). GroupKv/Docs/FilesServiceImpl gain an events field (Noop default +
with_events builder) and emit on set/create/update/delete; the host wires the
outbox emitter via with_events. Closes the "group trigger has no app to watch"
gap. Authoring + validation land in M2.4.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 20:33:09 +02:00
MechaCat02
96277e1e7c feat(routes): live group-route expansion + full-live invalidation (§11 tail R3)
The runtime half. Group route TEMPLATES now expand into every descendant
app's in-memory match slice, live, with no per-app materialization.

- `compile_effective_routes` consumes the `list_effective` expansion and
  applies NEAREST-OWNER-WINS shadowing: for one app, identical binding
  tuples (method+host+path) owned at multiple chain levels collapse to the
  nearest (an app's own route shadows an ancestor-group template); a nearer
  group beats a farther one. Non-identical bindings coexist and the
  existing matcher precedence resolves each request. `compile_route` now
  takes the effective app_id, so the matcher + dispatch are unchanged.
- `rebuild_route_table` is the single refresh chokepoint (list_effective →
  compile_effective_routes → replace_all). Route CRUD, the apply reconcile,
  and startup all route through it; the apply guards drop the "a group node
  touches no routes" assumption (a group apply may now create templates).
- FULL-LIVE INVALIDATION: app create/delete (apps_api) and group reparent
  (groups_api) rebuild the snapshot, so inherited routes take effect the
  instant the tree changes — matching the live model of triggers/vars.
  Best-effort, mirroring apply: a failed rebuild self-heals on the next
  write or restart, never failing the committed mutation.

The isolation boundary is the chain expansion: a template lands only in the
slices of apps whose ancestor chain contains the owning group. Pinned by a
deterministic live-DB integration test (descendant inherits, sibling
subtree does not, app shadows by identical binding, non-identical coexists).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 21:49:58 +02:00
MechaCat02
e7c0485dbf feat(modules): GroupFilesService + files::shared_collection handle (§11.6 files C3)
- shared/group_files: GroupFilesService trait (mirror FilesService, no
  group id arg — owner resolved from cx.app_id), GroupFilesError (clone of
  FilesError + CollectionNotShared) with From<FilesError>, Noop.
- services: group_files field + with_group_files setter (default Noop).
- group_files_service: GroupFilesServiceImpl — owning_group via the
  registry resolver (kind=files) → CollectionNotShared; reads open
  (script_gate), writes fail closed (script_gate_require_principal);
  reuses validate_files_collection + the NewFile/FileUpdate validators +
  sanitize_stored_content_type + the per-file size cap; no events.
- sdk/files: GroupFilesHandle + files::shared_collection(name) + the
  create/head/get/update/delete/list group methods (Blob in/out).
- picloud: construct FsGroupFilesRepo (sharing the files root + size cap)
  + GroupFilesServiceImpl, .with_group_files(...).

Unit tests: off-chain → CollectionNotShared; anon read OK / anon write
Forbidden; oversize → TooLarge.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 19:46:39 +02:00
MechaCat02
78a468de83 feat(modules): GroupDocsService + docs::shared_collection handle (§11.6 docs C3)
The runtime for shared group docs collections, mirroring the group-KV vertical
with the docs surface (filter parsing, JSON-object validation, value-size cap).

- shared: GroupDocsService trait + GroupDocsError (+ CollectionNotShared) +
  NoopGroupDocsService; group_docs field on Services + with_group_docs().
- manager-core: GroupDocsServiceImpl — owning_group resolves kind='docs' from
  cx.app_id's chain (CollectionNotShared off-chain); reads-open (script_gate
  GroupDocsRead) / writes-authed (script_gate_require_principal GroupDocsWrite);
  reuses parse_filter + docs value-size cap; no events. Unit tests cover
  off-chain CollectionNotShared, anon-read-OK/anon-write-Forbidden, non-object
  data rejected.
- executor-core: GroupDocsHandle + docs::shared_collection constructor + the
  create/get/find/find_one/update/delete/list methods (dispatch by receiver
  type), reusing doc_to_map/parse_doc_id.
- picloud: wire PostgresGroupDocsRepo + GroupDocsServiceImpl.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 08:04:30 +02:00
MechaCat02
d9766bcbc7 feat(modules): GroupKvService + kv::shared SDK handle (§11.6 C3)
The runtime for shared group collections. A script reaches a shared store via
the explicit kv::shared("name") handle (distinct GroupKvHandle Rhai type, so
private-vs-shared scope is a deliberate choice). The service resolves the
owning group from cx.app_id's ancestor chain — the script never names a group,
and that walk is the isolation boundary (a foreign app's chain never contains
the owning group → CollectionNotShared).

- shared: GroupKvService trait + GroupKvError + NoopGroupKvService; threaded
  into the Services bundle as a field defaulted to noop, opted into via a new
  with_group_kv() (keeps the ~14 Services::new call sites unchanged).
- manager-core: GroupKvServiceImpl with the reads-open/writes-authed split —
  reads use script_gate (anonymous public scripts skip), writes use the new
  script_gate_require_principal (anonymous fails closed). Owner resolution
  behind a GroupCollectionResolver trait (Postgres impl + injectable fake);
  unit tests cover off-chain CollectionNotShared, anonymous-read-OK/
  anonymous-write-Forbidden, authed-no-role-Forbidden.
- executor-core: GroupKvHandle + kv::shared constructor + get/set/has/delete/
  list (Rhai dispatches by receiver type, reusing the block_on bridge).
- picloud: wire PostgresGroupKvRepo + resolver + GroupKvServiceImpl.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 21:44:53 +02:00
MechaCat02
c8acac1d20 feat(modules): enable group modules + dangling-import plan check (Phase 4b C4)
Lift the Phase-4-lite restrictions now that import resolution is lexical:

- `create_group_script` (group_scripts_api): allow `kind = module` and
  group scripts with `import`s; record the import edges. Module-shape and
  sandbox-ceiling validation stay.
- The declarative apply path already reconciles group modules generically
  (`BundleScript.kind` + `owner.script_owner()` flow through
  `reconcile_node_tx`); no change needed beyond stale-comment fixes.

Adds the §5.5 dangling-import plan check (single-node plan/apply):
`check_imports_resolve` rejects an `import "<m>"` that resolves to neither a
bundle-local module nor a module reachable lexically up the node's chain —
caught at plan time instead of as a runtime 404. Roots resolution at the
node's owner (an app node reaches ancestor group modules; a group node walks
its own ancestry). ApplyService gains an injected `ModuleSource`. The tree
path opts out (a node may import a module created by another node in the same
uncommitted tx); the runtime check is the backstop there.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 07:29:18 +02:00
MechaCat02
65049a6262 feat(apply): owner-generic apply engine — group-node plan/apply (Phase 5 C1)
Phase 5 C1. Generalize the reconcile engine from app-only to an `ApplyOwner
{ App | Group }`, so a GROUP's own config + scripts can be declaratively
reconciled — the foundation for the tree-spanning project tool.

A group node is a subset of an app node (scripts + vars only; no routes /
triggers / secrets-values), so the diff, script/route/trigger/var CRUD, prune,
idempotency, and the Phase-4 inherited-target resolution are all reused
unchanged. Only the owner-varying bits switch on `ApplyOwner`:
  * `load_current` — `list_for_group` + `VarOwner::Group` for a group (empty
    routes/triggers/secrets); `list_for_app` for an app, exactly as before.
  * script create owner (`NewScript{app_id|group_id}`), var writes (new
    `set_group_var_tx`/`delete_group_var_tx` mirroring the app variants at
    scope `*`), and the advisory lock (owner-tagged).
  * `validate_bundle_for` rejects routes/triggers on a group bundle (422);
    route-host validation, inherited-target resolution, the post-commit route
    refresh, and the unreachable-endpoint warning are app-only.

`apply`/`plan` are unchanged thin wrappers over the new `apply_owner`/
`plan_owner`. New endpoints `POST /groups/{id}/{plan,apply}` gated by
`GroupScripts{Read,Write}` / `GroupVarsWrite`. `ApplyService` gains a `groups`
repo (the tree apply in C3 needs it too).

Live-validated: group plan → apply (group-owned script + var) → idempotent
re-plan noop → routes rejected 422 → DB confirms group ownership. App apply
unchanged: 391 manager-core unit tests + the apply/prune/staleness/overlay
journeys all green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-25 21:46:52 +02:00
MechaCat02
43ed4e8e7f feat(scripts): group-owned script admin API + pic scripts … --group
Phase 4-lite C2. Lets a group own scripts (templates inherited by descendant
apps), with the create/list/manage surface — but not yet the inherited
resolution (binding + runtime), which is C3.

Capabilities: add `GroupScriptsRead` (viewer+ on the group → script:read) and
`GroupScriptsWrite` (editor+ → script:write), mirroring the group-vars tier and
resolved through the same group-ancestor walk.

API:
  * New `group_scripts_api`: `POST/GET /groups/{id}/scripts` — create a
    group-owned endpoint script and list a group's own (non-inherited) rows.
    Phase 4-lite is endpoint-only and self-contained: `kind=module` and any
    `import` are rejected (group modules + the lexical resolver are Phase 4b).
    Owner resolved first (slug-or-uuid); capability bound to the resolved id.
  * The by-id `/scripts/{id}` get/update/delete/logs handlers are now
    owner-polymorphic via a `script_cap` helper: app-owned scripts gate on
    `App*` exactly as before; group-owned on `GroupScripts*`. This is what
    makes `deploy --group` idempotent (update reuses the by-id PUT) and lets a
    group script be deleted by id. (C1 had these fail closed for groups.)

Repo: `list_for_group(group_id)` (the group's own rows).

CLI: `pic scripts ls --group <g>`, `pic scripts deploy --group <g>` (create or
update by name; `--app`/`--group` are mutually exclusive, exactly one
required). `pic scripts delete <id>` already works for group scripts via the
owner-polymorphic by-id route.

Live-validated against the dev DB: create → update (v2 via the by-id PUT) →
list → delete, plus module rejection and the polymorphic row shape
(`app_id NULL`, `group_id` set). Group scripts still can't be routed/triggered
or invoked — that lands in C3.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-25 20:10:20 +02:00
MechaCat02
e517f6dc8c feat(apply): reconcile app-owned vars in the declarative engine
Extends the Phase-1 apply engine to manage app `vars` declaratively,
alongside scripts/routes/triggers/secrets. The `Bundle` gains a `vars`
map (key → JSON value; values are non-secret, so they live inline, unlike
secret names). `diff_vars` is value-sensitive: a changed value is an
Update, a live var the manifest dropped is a Delete (applied only under
`--prune` — vars are prunable config, unlike never-pruned secrets).

Writes go through `set_app_var_tx`/`delete_app_var_tx` inside the apply
transaction (mirroring `PostgresVarsRepo::set` for `VarOwner::App`, scope
`*`), so they commit atomically with the rest of the apply and roll back
together on failure. `CurrentState` loads the app's OWN scope-`*`,
non-tombstone vars (inherited group vars and tombstones are out of scope
for the manifest); `state_token` folds them in so the bound-plan check
catches an out-of-band `pic vars set` between plan and apply. Keys are
validated against the same kebab rule as the vars admin API, and the
apply handler now requires `AppVarsWrite` when vars are touched or
`--prune` is set.

Scope: app-owned vars at scope `*` (an app *is* an environment). Group
vars via manifest and tombstone-via-manifest wait for the nested-manifest
model. Unit tests cover the create/update/noop/delete diff and the
state-token value sensitivity.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-25 07:35:12 +02:00
MechaCat02
c914758c09 feat(secrets): group-secrets admin API, masked read, config/effective
Adds the Phase-3 admin surface on top of the group-secrets storage:

* `secrets_api` gains group routes under `/groups/{id}/secrets`
  (set/list/delete, env-scoped) gated `GroupSecretsWrite` (editor+), plus
  the ONE plaintext endpoint `GET /groups/{id}/secrets/{name}/value` gated
  `GroupSecretsRead` (group_admin only). That is the masked-secret
  boundary: a descendant app's dev sees a group secret EXISTS and consumes
  it at runtime via `secrets::get`, but only a reader at the OWNING group
  gets the value. App secrets stay env-agnostic (a stray `env` is rejected).
  The owner is resolved first, then the capability binds to the resolved
  id — never a path param.

* `config_api`: `GET /apps/{id}/config/effective` (gated `AppVarsRead`)
  returns the resolved view a dev would get — every inherited var with its
  value + provenance, and every inherited secret MASKED (name/owner/scope,
  never the value). Backed by a new `fetch_effective_secret_meta`
  (DISTINCT-ON nearest-wins, same ordering as the per-name resolver).

* authz: `GroupSecretsWrite` moves from `app:admin` to `script:write`
  scope so its API-key scope matches its editor role tier (closing the
  latent scope/role mismatch the checkpoint review flagged); the value
  read `GroupSecretsRead` stays at `app:admin`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 22:00:32 +02:00
MechaCat02
9ee85993d8 feat(vars): admin CRUD API + pic vars CLI
Completes the vars half of Phase 3 end-to-end:
- vars_repo: VarOwner{Group|App} upsert/delete/list (owner-kind-specific
  SQL; ON CONFLICT restates the partial-index predicate).
- vars_api: GET/PUT/DELETE under /apps/{id}/vars and /groups/{id}/vars,
  resolve-then-require gated on App/GroupVars{Read,Write}, secrets-style
  error mapping + key/env-scope validation.
- pic vars ls/set/rm (--group|--app, --env, --json, --tombstone).
- journey test: a group var is inherited by a descendant app's
  vars::get(), and an app-level value overrides it (proximity) — green.

386 manager-core lib tests + the vars journey pass; clippy clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 21:11:46 +02:00
MechaCat02
343f6d3b4d feat(vars): VarsService + vars:: SDK + config capabilities
Scripts can now read group-inherited config via vars::get(key) / vars::all().

- shared: VarsService trait (read-only get/all) + NoopVarsService; added as
  the 14th field on the Services bundle.
- manager-core: VarsServiceImpl resolves the calling app's config via
  config_resolver (derives app_id from cx.app_id; AppVarsRead-gated for
  authed principals, script-as-gate for anon).
- executor-core: vars:: Rhai bridge (get/all), registered in the SDK.
- authz: AppVarsRead/Write, GroupVarsRead/Write, GroupSecretsRead/Write
  capabilities (group caps resolve via the Phase-2 ancestor walk; the
  GroupSecretsRead human-read gate is group_admin-only, distinct from an
  app's runtime injection).
- wired VarsServiceImpl into the picloud binary; updated the executor-core
  test Services::new call sites.

386 manager-core lib tests green; clippy clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 20:59:23 +02:00
MechaCat02
a432091191 feat(groups): tree repos, hierarchy-aware RBAC, admin API
Server-side foundation for Phase-2 groups (no group-owned resources yet):

Shared types:
- GroupId, Group; App gains group_id; AppRole::{precedence,max} for
  folding the highest effective role across the membership chain.

Repos:
- group_repo: tree CRUD with reparent (ancestor-walk cycle guard under a
  coarse instance-wide structural advisory lock; slug frozen; bumps
  structure_version) and delete=RESTRICT (refuses non-empty groups).
- group_members_repo: per-(user, group) role grants, mirroring app_members.

Hierarchy-aware authz (§5.3):
- AuthzRepo gains effective_app_role / effective_group_role (default to
  direct membership / none, so the ~18 existing test stubs are untouched);
  the Postgres impl resolves each via one depth-bounded recursive CTE that
  MAXes the app's own row with every ancestor group_members row.
- can(): the Member path now folds inherited group roles, so a group_admin
  on any ancestor is implicitly app_admin beneath it. New Capability
  variants InstanceCreateGroup / Group{Read,Write,Admin}; group caps carry
  no app_id (bound API keys can't manage groups). 8 new unit tests.

Admin API:
- groups_api: group CRUD + reparent (admin at both source and destination
  parent, §5.6) + per-group members, all capability-gated.
- apps: POST /apps takes an optional parent group (default root); app
  responses carry group_id; my_role now reflects the effective role.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 19:59:00 +02:00
MechaCat02
3b650a2b14 feat: declarative project-tool foundation (pull/plan/apply/prune)
Add a server-side, atomic, declarative reconcile loop for a single app —
the foundation of the project-tool design. Developers describe an app's
scripts, routes, triggers, and secret-names in `picloud.toml`, then
`pic pull / plan / apply [--prune]` to converge live state to the manifest.

Server (manager-core):
- apply_service: a pure diff engine (compute_diff) shared by plan and
  apply, plus an ApplyService that composes the existing per-repo writes
  into ONE Postgres transaction. Identity keys mirror the DB UNIQUE
  constraints (script=lower(name); route=(method,host_kind,host,
  path_kind,path); trigger=per-kind semantic tuple; secret=name).
  Apply takes a per-app advisory lock, recomputes the diff in-tx, applies
  scripts -> routes -> triggers, prunes dependents-first, commits, then
  refreshes the route table once post-commit.
- apply_api: POST /apps/{id}/plan (AppRead) and /apps/{id}/apply.
  Apply requires the per-kind write caps the bundle exercises (all three
  when --prune), plus AppSecretsRead when it binds an email trigger.
- tx-accepting repo siblings (insert/update/delete *_tx) so the existing
  create/update/delete delegate to one SQL definition each.
- email triggers reference an inbound secret by NAME; the value is
  resolved, decrypted (AAD-bound), and re-sealed server-side at apply —
  it never travels in the manifest.

CLI (picloud-cli):
- manifest.rs (picloud.toml model), client plan/apply, and the pull/plan/
  apply commands. pull rejects filesystem-unsafe script names up front.

Safety properties enforced and tested:
- idempotent: a freshly-pulled manifest re-applies as all-NoOp.
- atomic: a mid-bundle failure rolls back with nothing written.
- routes delete-before-insert so a freed binding is reusable in one apply.
- queue one-consumer invariant held inside the shared tx.
- email triggers are never pruned, and a script that still owns an
  email/dead-letter trigger can't be pruned (the FK cascade would destroy
  the sealed secret) — refused with a pointer to `pic triggers rm`.
- plan and apply agree on unset email-secret references.

No migration: the existing schema's UNIQUE constraints serve as identity
keys. Groups, env-scoping, and the `enabled` toggle are later milestones.

Tested: manager-core lib (360) + CLI bins (27) + 8 project-tool journeys
(pull/plan/apply/prune/email+queue), all green; clippy -D warnings clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 21:52:21 +02:00
MechaCat02
51f14fa2b1 feat: E2E #2 (Stash) gap remediation + S6 hardening
Some checks failed
CI / Rust — fmt, clippy, test (push) Failing after 6m19s
CI / Dashboard — check (push) Successful in 9m48s
Closes the gaps and the one security finding from the second end-to-end
CLI test (E2E_STASH_REPORT.md), plus the H1 boot-regression found while
re-reviewing those fixes.

Security
- S6: reserved-path validation (`check_reserved`) now case-folds before
  comparing, so `/API/v2/x`, `/HEALTHZ`, `/Admin/x` are rejected like
  their lowercase forms. Request-time matching stays case-sensitive.
- S10: "public route != public data" callout in sdk-shape.md (script_gate
  skips authz when the principal is anonymous).

Observability / features
- G1: trigger executions now write `execution_logs`. Migration 0043 adds
  a `source` column (CHECK mirrors ExecutionSource/OutboxSourceKind,
  DEFAULT 'http' backfills history); a shared `build_execution_log` helper
  in executor-core; dispatcher logging for outbox triggers + queue
  consumers (skips sync-HTTP rows the orchestrator already logs). `pic
  logs` gains a source column + `--source` filter.
- G5: dev-only in-memory email capture under PICLOUD_DEV_MODE with no SMTP
  (email::send succeeds locally), readable at GET /api/v1/admin/dev/emails
  (Owner/Admin only; route mounted only in capture mode).
- G6: generalized the Rhai in-place-mutation footgun note (trim/replace/
  make_upper/make_lower/crop/truncate/pad return ()).
- G2/G3/G4 (CLI): `pic members`, `pic files`, `pic queues`, read-only
  `pic kv` (+ new kv_api.rs); `pic deploy --timeout/--memory/--kind/
  --sandbox`; first-class `pic triggers create-{docs,files,pubsub,queue,
  email}` wrappers. All new client path segments percent-encoded via seg().

H1 regression fix (found in re-review)
- The S6 change also runs in `compile_routes`, which compiles every stored
  route at boot and on each route CRUD. A single stored route the new
  validation rejects (creatable while the S6 gap existed) made the whole
  compile Err and aborted startup. `compile_routes` is now lenient: it
  skips an un-compilable row with a warning instead of bricking boot
  (route creation still validates separately). Migration 0044 sweeps
  pre-existing reserved-path routes on upgrade (WHERE mirrors
  check_reserved exactly). Added regression tests for both.

Verified: cargo fmt, clippy --all-targets --all-features -D warnings, the
schema_snapshot test, and the new S6/lenient-compile unit tests all pass;
boot-resilience and G1/G5 confirmed live.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-13 15:01:04 +02:00
MechaCat02
ec4a2aa24a style: cargo fmt across audit-2026-06-11 tier-3 changes
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-12 18:38:28 +02:00
MechaCat02
31ecfae86e fix(audit-2026-06-11/revocation-lag): evict PrincipalCache on credential change
The PrincipalCache (token-hash -> resolved Principal, 60s TTL) had no
eviction hook, so deactivation / password change / API-key revocation
flipped the DB rows but the cache kept serving the stale principal for
up to the TTL window. Closes the audit's PrincipalCache revocation-lag
Medium and greens authz::deactivating_user_revokes_their_api_keys.

- PrincipalCache::evict_user(user_id) + evict_token(hash).
- One shared cache Arc threaded into AuthState / AdminsState /
  ApiKeysState (a per-state cache would let the middleware's own copy
  keep authenticating a revoked principal).
- Evict on: deactivation, password change (both evict_user), logout
  (evict_token, precise), single API-key delete (evict_user).
- H-E1 note: DB-write invalidation stays best-effort by design (a blip
  must not undo the rotation); the cache evict is what makes it take
  effect on the next request.
- Renamed bearer_and_cookie_produce_same_principal ->
  session_token_and_api_key_produce_same_principal (cookie auth was
  removed in C-1; the test never tested cookies).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-12 18:12:54 +02:00
MechaCat02
8b5a02b3ef fix(audit-2026-06-11/H-B2): inbound_secret mandatory + bad-signature lockout on email triggers
Two gaps:

1. /api/v1/admin/apps/{id}/triggers/email accepted `inbound_secret: null`
   and treated it as "this trigger is unsigned". The runtime then
   skipped HMAC verification entirely. URL discovery (logs, ops
   dashboards, bruteforce on the 122-bit TriggerId space) then fan-out-
   amplified into the outbox for free — anonymous DoS via the
   dispatcher.

2. Even signed triggers had no per-trigger rate limit on signature-
   verify failures, so an attacker who knew the URL could pump unsigned
   POSTs to force Argon2-equivalent work (HMAC verify + nonce-dedup +
   stored-secret decrypt) at line rate.

Fix:

* triggers_api::create_email_trigger now requires a non-empty
  inbound_secret. 400 on missing / empty / whitespace-only.
* email_inbound_api::receive_inbound_email returns 401 immediately when
  the resolved target has no inbound_secret_encrypted column (pre-fix
  rows). The previous `if let Some(ct), Some(nonce)` branch is gone.
* New `BadSignatureLimiter`: per-`(app_id, trigger_id)` sliding-window
  bucket (10 fails / 60 s). On lockout the receiver returns 429 with
  Retry-After instead of 401. record_failure runs on both the
  "no-secret" 401 path and any verify_signature failure.

Test fallout:
* email_inbound integration tests that relied on None secret: removed
  the now-impossible `unsigned_trigger_accepts_without_signature` test,
  replaced with `create_without_inbound_secret_is_rejected` covering
  null / "" / "   "; updated `malformed_body_is_422` and
  `cross_app_path_is_404` to pass + sign with a secret.
* Two new unit tests pin the limiter behavior (burst lockout + per-
  trigger independence).

Audit refs: security_audit/08_dos_resource.md#h-3,
security_audit/09_external_integrations.md#f-ext-m-001.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-11 20:49:31 +02:00
MechaCat02
07ffc0b568 fix(audit-2026-06-11/H-B1): login rate limit + Argon2 concurrency cap
The login handler had no admission control before
`spawn_blocking(verify_password)`. On Pi-class hardware Argon2id is ~50-
100 ms per attempt, so a few dozen concurrent anonymous POSTs to
/auth/login park every blocking worker, wedging the entire admin API
and any other path that uses `spawn_blocking`.

Adds two cheap, in-process guards modeled on EmailRateLimiter:

* `LoginRateLimiter` — two sliding-window token buckets, one per
  `(remote_ip, username)` (burst 5/60s) and one per `username`
  (burst 10/15min). Per-(ip, user) defeats a single attacker pounding
  one account; per-user defeats credential-stuffing distributed across
  many IPs. Username is case-folded so case variants share a bucket.
  Maps GC lazily when they cross a soft size cap.

* Per-process `tokio::sync::Semaphore` — caps concurrent Argon2 verifies
  during login. Default 2 permits (Pi-class), overridable via
  `PICLOUD_LOGIN_ARGON2_PARALLELISM`. Acquired AFTER the bucket check so
  attackers can't queue.

Limit check fires before the DB credentials lookup, so even an unknown
username doesn't cost a query. Denied attempts return 429 + Retry-After.
Real client IP comes from the first X-Forwarded-For entry (Caddy is the
trusted single hop); falls back to "unknown" so the per-user bucket
still gates.

4 unit tests cover bucket burst exhaustion, per-user crossing IPs, case
folding, and per-user independence.

Audit ref: security_audit/08_dos_resource.md (H-2 / H-B1).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-11 20:42:51 +02:00
MechaCat02
3c9816daf3 fix(stage-1): audit High-severity backend correctness + CI unblocker
Closes 4 High-severity audit findings:

- describe_event broken for HTTP-async payloads: was reading source/op
  JSON fields that HttpDispatchPayload doesn't carry, producing empty
  source on DL rows. from_wire("") then fell back to Kv on replay, the
  dispatcher tried to resolve a non-existent trigger, and replay no-op'd
  silently. Now branches on OutboxSourceKind: HTTP rows emit
  "<method> <path>" + source="http"; Invoke rows emit "invoke_async" +
  source="invoke"; TriggerEvent payloads keep top-level field extraction.

- HTTP outbound Authorization leaked across cross-origin redirects.
  Manual redirect loop (Policy::none) reused header_map on every hop
  and only scrubbed Content-Type for POST->GET. Now compares
  url::Url::origin() across hops and strips Authorization,
  Proxy-Authorization, Cookie when origin changes — matching reqwest's
  default policy.

- F-S-010 inbound email HMAC had no replay protection. Signature was
  computed over body only with no timestamp or nonce, so captured POSTs
  were replayable indefinitely. Adds X-Picloud-Timestamp header bound
  into the HMAC input (signed string is ts || "." || body), 300s
  tolerance window, and a process-local LRU keyed on
  (ts, sha256(body)) with 600s TTL to catch within-window replays.
  Breaking change: webhook senders must include the timestamp header.

- expected_schema.txt re-blessed for migrations 0036-0039 (the previous
  snapshot stopped at 0035 so CI would have gone red on first run).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-10 21:03:10 +02:00
MechaCat02
adc719975f style: fmt + clippy cleanup after Phase C
cargo fmt regroups; three clippy fixes:
- F-S-006: hoist MAX_API_KEY_CANDIDATES to module scope to satisfy
  clippy::items_after_statements.
- F-S-009: use map_or instead of map().unwrap_or() per
  clippy::map_unwrap_or.
- F-P-012: replace `|c| c.encode()` closure with the method itself
  per clippy::redundant_closure_for_method_calls.

No behavior change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 21:19:00 +02:00
MechaCat02
f2b16da2b5 fix(manager-core): F-S-012 add AppInvoke capability + gate invoke() / invoke_async()
invoke_service::resolve and enqueue_async performed no authz check —
no AppInvoke capability existed. Same-app isolation was preserved
(cross-app guards work), but within one app an anonymous public-HTTP
script could trigger any other script (e.g. an admin-only worker that
hits secrets/files/external HTTP). Worse: invoke_async runs the
callee with principal: None, so the callee could hold capabilities
the original public caller shouldn't.

- Add Capability::AppInvoke(AppId). app_id() / scope_for_capability
  (script:write) / role_satisfies (editor+) are all updated.
- InvokeServiceImpl gains an optional `authz: Option<Arc<dyn AuthzRepo>>`
  + a `with_authz` builder. When set, resolve() runs script_gate on
  AppInvoke before doing the cross-app id check.
- picloud/src/lib.rs wires it: `InvokeServiceImpl::new(...).with_authz(...)`.
- Anonymous callers (cx.principal == None) continue to skip the check
  via script_gate, preserving the public-HTTP convention.

Existing 5 invoke_service unit tests still pass (the tests use the
authz-less constructor, so the gate is a no-op there).

AUDIT.md anchor: F-S-012.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 21:13:45 +02:00
MechaCat02
1911691420 fix(manager-core): F-Q-011 blanket Arc<T: ScriptRepository> impl; delete PostgresScriptRepoHandle
A 70-line newtype lived in picloud/src/lib.rs wrapping
Arc<PostgresScriptRepository> and re-implementing every ScriptRepository
method as `self.0.method(...).await`. Hand-delegation invited silent
skew when the trait gained a method, and forced 8 call sites to know
about the wrapper.

Add a blanket
  impl<T: ScriptRepository + ?Sized> ScriptRepository for Arc<T>
in manager-core::repo. The implementations forward via (**self).method(...)
so any owner of an Arc<dyn ScriptRepository> (or Arc<ConcreteImpl>) can
be passed where the trait is expected.

Migrate the 7 picloud-binary call sites:
  Arc::new(PostgresScriptRepoHandle(script_repo.clone())) → script_repo.clone()

Delete the newtype + its hand-delegated impl. Leaves a 6-line comment
documenting why the boilerplate is gone.

AUDIT.md anchor: F-Q-011.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 21:10:46 +02:00
MechaCat02
617c216429 fix(manager-core): F-P-009 cache resolved Principals (60s TTL) in auth middleware
attach_principal_if_present runs on every request that carries a
Bearer header, including data-plane paths that may not even need
authz. Each call paid three DB round-trips on the session path
(lookup + admin_users.get + touch) plus an Argon2 verify per
prefix-colliding candidate on the API-key path. A hot user with
multiple keys serialized every request behind N×Argon2.

Add a process-shared PrincipalCache keyed on hash_token(bearer) →
(Instant, Principal), TTL 60s. resolve_principal checks the cache
first; on miss falls through to the verify_api_key / verify_session
path and writes back on success.

- Lazy GC: when the cache exceeds 1024 entries, sweep expired before
  inserting (kept simple — production sees few unique hot tokens).
- Cache wired through AuthState; constructed once in picloud/src/lib.rs.
- Skip-when-route-doesn't-need-auth scaffolding is deferred to a
  follow-up — it requires touching the router shape, which is more
  invasive than the 60s cache alone warrants.

AUDIT.md anchor: F-P-009 (cache; skip-path deferred).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 20:23:17 +02:00
MechaCat02
47ea239eb6 fix(picloud): F-P-003 PICLOUD_DB_MAX_CONNECTIONS env knob, default 32 (matches gate)
init_db hard-coded max_connections(10). The execution gate
(ExecutionGate, PICLOUD_MAX_CONCURRENT_EXECUTIONS default 32) lets 32
script executions run concurrently, each doing multiple sequential
DB calls — plus the dispatcher tick every 100ms, the cron tick, three
GC sweeps, and the auth middleware running per request all draw from
the same pool. With max=10 vs gate=32, pool starvation surfaces as
acquire_timeout errors and tail-latency spikes under load.

- DEFAULT_DB_MAX_CONNECTIONS = 32 (matches gate default).
- Env knob: PICLOUD_DB_MAX_CONNECTIONS overrides; invalid/zero → default.
- Documented in CLAUDE.md runtime config table.

AUDIT.md anchor: F-P-003.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 20:03:56 +02:00
MechaCat02
e29ac1c03d fix(manager-core): F-S-001 cap kv/docs/pubsub/queue payload sizes (default 256 KB)
Files (per-file cap), secrets (64 KB default), and email (25 MB default)
already enforce limits; kv::set, docs::create/update, pubsub::publish_durable
and queue::enqueue accepted any JSON value straight to a JSONB column with
no size validation. An anonymous public-HTTP script could fill disk via
queue::enqueue or amplify a single publish into N outbox rows × payload bytes.

Adds four new error variants:
- KvError::ValueTooLarge { limit, actual }
- DocsError::ValueTooLarge { limit, actual }
- PubsubError::MessageTooLarge { limit, actual }
- QueueError::PayloadTooLarge { limit, actual }

Each stateful service grows a `max_value_bytes` field with:
- Conservative 256 KB default (DEFAULT_KV_MAX_VALUE_BYTES etc.).
- New `with_max_*` constructor preserving the old `new()` signature.
- Env-knob reader (`*_max_*_from_env()`) — mirrors SecretsConfig::from_env.

Validation runs at the entry point BEFORE authz so an anonymous DoS doesn't
pay a membership lookup per attempt.

Wired via env knobs:
- PICLOUD_KV_MAX_VALUE_BYTES
- PICLOUD_DOCS_MAX_VALUE_BYTES
- PICLOUD_PUBSUB_MAX_MESSAGE_BYTES
- PICLOUD_QUEUE_MAX_PAYLOAD_BYTES

Documented in CLAUDE.md runtime config table.

New unit test in queue_service verifying the cap fires before authz.

AUDIT.md anchor: F-S-001. Depends on F-Q-004 (Backend variant).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 20:00:36 +02:00
MechaCat02
c3baa87415 chore(v1.1.9): clippy clean — workspace -D warnings
Surfaced + fixed during the F3 attestation:

- executor-core/sdk/queue.rs: drop redundant .map(|()| ()) call sites;
  enqueue_blocking discards QueueMessageId via .map(|_id| ()) (intentional)
- executor-core/sdk/retry.rs: hoist use std::collections::hash_map::DefaultHasher
  + use std::hash::Hasher to the top of the file (clippy::items_after_statements);
  replace `as i64` casts with i64::try_from + clear comment about jitter
  bound (clippy::cast_possible_wrap)
- executor-core/sdk/invoke.rs: move _LIMITS_IS_COPY const before #[cfg(test)]
  mod tests (clippy::items_after_test_module)
- manager-core/dispatcher.rs: dispatch_one_queue gains #[allow(too_many_lines)]
  (it's the queue tick's whole logic — split makes it less readable than
  the lint); .map(...).unwrap_or(default) → .map_or(default, ...)
- picloud/lib.rs: Limits { trigger_depth_max, ..Limits::default() } via
  struct-update instead of let-mut-assign (clippy::field_reassign_with_default)
- tests: r#"..."# → r"..." where there are no `"` inside (clippy::needless_raw_string_hashes);
  combine InvokeTarget::Name | InvokeTarget::Path arms with identical bodies
  in sdk_invoke.rs (clippy::match_same_arms)

cargo fmt --all -- --check 2>&1 | tail -3: no output (exit 0)
cargo clippy --workspace --all-targets --all-features -- -D warnings:
  Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.41s
Workspace lib smoke: 432 passing (306 manager-core + 74 executor-core + 34 shared + 18 orchestrator-core).
DB-gated E2E (against docker compose postgres on localhost:15432):
  queue_e2e: 4 ok / invoke_e2e: 4 ok / retry_e2e: 3 ok / migration_queue_messages: 4 ok / schema_snapshot: 1 ok

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 11:02:22 +02:00
MechaCat02
714f6dae18 feat(v1.1.9): admin HTTP endpoints — queue triggers + read-only queues
triggers_api.rs adds:
  POST /api/v1/admin/apps/{id}/triggers/queue
    body: { script_id, queue_name, visibility_timeout_secs?,
            dispatch_mode?, retry_max_attempts?, retry_backoff?,
            retry_base_ms? }
    cap: AppManageTriggers
    409-equiv via the repo's "queue 'X' already has a consumer" error
    (TriggerRepoError::Invalid → 422 Invalid in the existing mapping)

queues_api.rs (new) adds read-only inspection endpoints:
  GET /api/v1/admin/apps/{id}/queues
    -> [{queue_name, total, pending, claimed}]

  GET /api/v1/admin/apps/{id}/queues/{queue_name}
    -> {queue_name, total, pending, claimed,
        consumer: Some(QueueConsumer) | None}

  QueueConsumer = { trigger_id, script_id, script_name,
                    visibility_timeout_secs, last_fired_at }

Capability: AppLogRead (same trust tier as execution-log read — these
are read-only operational views, not config changes).

No mutating queue endpoints — enqueue lives on the SDK side; purge /
requeue / delete-message stays at v1.2.

picloud/lib.rs wires the queues router into the guarded /admin chain.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-06 22:36:50 +02:00
MechaCat02
302c1df577 feat(v1.1.9): invoke:: Rhai SDK module — sync re-entry + invoke_async
executor-core/sdk/invoke.rs adds `invoke()` and `invoke_async()` as
top-level Rhai helpers (no `::` namespace, mirroring the brief):

  invoke(target, args)        -> Dynamic  (sync, returns callee's value)
  invoke_async(target, args)  -> String   (fire-and-forget; returns execution_id)

Sync re-entry pattern:
  - bridge captures Arc<Engine> via Engine::self_arc()
  - calls engine.execute(&source, req) directly — same engine instance,
    same Services, same Limits; only SdkCallCx changes per call
  - runs inside the caller's spawn_blocking thread (no nested
    spawn_blocking, no gate re-admission — the outer execution already
    holds a permit)

Back-reference plumbing:
  - Engine gains self_weak: OnceLock<Weak<Engine>> + set_self_weak() +
    self_arc() accessor
  - picloud binary calls engine.set_self_weak(Arc::downgrade(&engine))
    right after construction
  - register_all extended with limits + Option<Arc<Engine>> params
  - Engine::execute_ast threads both through

Limits.trigger_depth_max added — mirrors TriggerConfig::max_trigger_depth.
Default 8; the picloud binary syncs from TriggerConfig::from_env() so
PICLOUD_MAX_TRIGGER_DEPTH governs both the dispatcher's fan-out cap and
the invoke depth bound.

Target parsing (string-first):
  - "/api/foo"             → InvokeTarget::Path
  - 36-char UUID-like      → InvokeTarget::Id  (uuid::Uuid parse-validated)
  - everything else        → InvokeTarget::Name

Cross-app + depth + FnPtr guards:
  - resolve() returns InvokeError::CrossApp if resolved.app_id != cx.app_id
  - bridge throws "invoke: depth limit exceeded (max N)" when
    cx.trigger_depth + 1 > limits.trigger_depth_max (checked BEFORE
    resolve to avoid a wasted DB round-trip)
  - args_to_json rejects FnPtr at any depth — closures don't survive
    invoke boundaries

invoke_async path:
  - calls services.invoke.enqueue_async() which writes an
    OutboxSourceKind::Invoke row (commit 10 wires the dispatcher arm)
  - returns the new ExecutionId as a string for caller tracking

Integration tests (sdk_invoke.rs, 6 binaries):
  - sync invoke returns callee's value (script returns body.x + 1)
  - cross-app invoke rejected (target in other app)
  - depth limit exceeded (recursive script that calls itself; throws
    before stack overflow)
  - callee receives incremented depth in cx (smoke — strict assertion
    needs ctx.trigger_depth surface, deferred to v1.2)
  - args_to_json rejects FnPtr in payload
  - invoke_async returns valid UUID string + queues args payload

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-06 19:56:05 +02:00
MechaCat02
563c44ad95 feat(v1.1.9): InvokeServiceImpl + RouteTable wiring in picloud binary
manager-core/invoke_service.rs implements InvokeService over:
  - ScriptRepository (for Id + Name resolution)
  - RouteTable (for Path resolution via the orchestrator's matcher)
  - OutboxRepo (for invoke_async)

Same-app guard runs at the service entry point: resolve_id() always
verifies resolved.app_id == cx.app_id and returns InvokeError::CrossApp
otherwise. resolve_name() reads cx.app_id when querying (so a wrong
app_id from somewhere else couldn't slip through), but defense-in-depth
won't hurt — future hardening if it surfaces.

resolve_path() runs the matcher against this app's routes only
(RouteTable::snapshot_for_app). Method assumed POST→GET fallback for
v1.1.9 — surfacing method via the SDK is a future addition.

enqueue_async() resolves the target, then writes one outbox row with
source_kind = 'invoke'. The payload carries script_id, script_name,
args, fresh execution_id, root_execution_id (inherited from caller),
trigger_depth + 1. caller principal recorded as origin_principal
(forensic only — the executing script runs with no principal). One
shot — no retry policy on the row; users wrap in retry::with() if
they want retries.

picloud/lib.rs:
  - route_table construction moved BEFORE Services::new so the invoke
    service can hold it. populates from route_repo as before.
  - InvokeServiceImpl wired into Services::new in place of the
    NoopInvokeService placeholder.

Unit tests cover: resolve by Id same-app, cross-app rejected, resolve
by Name finds + not-found, enqueue_async writes Invoke outbox row with
trigger_depth=1.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-06 19:48:53 +02:00
MechaCat02
36bf046791 feat(v1.1.9): InvokeService trait + InvokeTarget + ScriptRepository::get_by_name
shared/invoke.rs introduces the InvokeService trait + InvokeTarget enum:

  - InvokeTarget::Path(String)  — resolves via the route trie
  - InvokeTarget::Name(String)  — (cx.app_id, name) -> script_id via get_by_name
  - InvokeTarget::Id(ScriptId)  — direct lookup

InvokeService::resolve enforces same-app at the service layer
(InvokeError::CrossApp). InvokeService::enqueue_async writes a v1.1.9
OutboxSourceKind::Invoke row for fire-and-forget composition.

Errors: NotFound, CrossApp, DepthExceeded(u32), Forbidden, Rejected,
Unavailable. NoopInvokeService surfaces all calls as Unavailable for
harnesses that don't wire the service.

ScriptRepository gains get_by_name(app_id, name) backed by the existing
(app_id, name) uniqueness constraint. Postgres impl + in-memory mock +
the picloud crate's PostgresScriptRepoHandle delegate added.

Services::new gains invoke: Arc<dyn InvokeService> positionally after
queue. with_noop_services wires NoopInvokeService. 12 test sites
threaded through. picloud/lib.rs binds NoopInvokeService at this commit
boundary — the real InvokeResolver lands in commit 8.

Deviation from plan (flagged for HANDBACK §7): the plan called for
moving ExecutorClient + ScriptIdentity from orchestrator-core to shared
so the invoke bridge could call a re-entrant variant. Re-evaluated:
the bridge lives in executor-core which can hold Arc<Engine> directly,
making the trait move unnecessary. Engine::execute is sync, returns
ExecResponse, and shares the engine instance + per-call SdkCallCx
exactly as required. AST cache reuse across invokes is a v1.2 optimization
(invokes recompile each call — ms-scale cost, acceptable).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-06 19:44:43 +02:00
MechaCat02
6891eda66c feat(v1.1.9): dispatcher queue arm + visibility-timeout reclaim task
Extends Dispatcher with the v1.1.9 queue path. The queue table IS the
outbox for queue semantics, so the dispatcher polls queue_messages
directly via FOR UPDATE SKIP LOCKED — no outbox indirection.

Per tick (every 100ms):
  1) outbox arm — unchanged
  2) queue arm: list_active_queue_consumers() → per (app_id, queue_name)
     attempt one claim. Bounded by registered-consumer count so one busy
     queue can't starve others.

Per claimed message:
  - Build TriggerEvent::Queue + ExecRequest (executes as the trigger's
    registering principal, matching design notes §4)
  - dispatch through executor.execute_with_identity (reuses AST cache)
  - success → queue.ack(id, claim_token) — DELETE WHERE id AND token
  - throw + attempt < max_attempts → queue.nack(...) — clear claim, set
    deliver_after = NOW() + compute_backoff(attempt, backoff, base_ms, jitter)
  - throw + exhausted → queue.dead_letter(...) atomic move to dead_letters
    + DELETE in one transaction. fan_out_dead_letter on the outbox arm
    fires registered dead_letter handlers off the new row without changes.

Visibility-timeout reclaim task: separate tokio::spawn ticking on
queue_reclaim_interval_ms (default 30000). UPDATE clears claim_token /
claimed_at on rows whose claimed_at exceeds the per-queue
visibility_timeout_secs (joined from queue_trigger_details). A crashed
consumer thus loses its lease and the message becomes claimable again.

Dispatcher gains queue: Arc<dyn QueueRepo>; picloud/lib.rs threads
queue_repo.clone() into the construction.

ActiveQueueConsumer extended with app_id so the queue claim can be
performed without a follow-up trigger lookup; list_active_queue_consumers
SQL extended to SELECT t.app_id.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-06 19:37:36 +02:00
MechaCat02
73e19ca626 feat(v1.1.9): QueueService trait + Postgres impl + Services bundle wiring
- shared/services.rs: Services::new gains queue: Arc<dyn QueueService>
  positionally after users (mirrors v1.1.8's append of users).
  with_noop_services adds NoopQueueService.
- manager-core/queue_service.rs: QueueServiceImpl wraps QueueRepo with
  script-as-gate authz on AppQueueEnqueue. enqueue clamps max_attempts
  to [1,20] and delay_ms to [0, 86_400_000ms]. depth/depth_pending are
  read-only — no authz check (scripts in the app can see their own
  queue depths). cx.principal threads through as enqueued_by_principal
  (forensic only).
- manager-core/authz.rs: AppQueueEnqueue(AppId) capability — script:write
  scope, granted to editor+ (same trust shape as AppPubsubPublish).
- picloud/lib.rs: wires PostgresQueueRepo + QueueServiceImpl into
  Services::new alongside the existing v1.1.7+ services.
- 11 sdk test binaries + manager-core/realtime_authority.rs updated to
  pass NoopQueueService to Services::new.

Unit tests cover empty queue_name reject, max_attempts clamping
(0/21 → invalid), delay_ms negative-reject, anonymous principal skips
authz, depth/depth_pending pass-through.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-06 19:30:34 +02:00
MechaCat02
7027e0dfb8 chore(v1.1.8): cargo fmt --all (reviewer)
8 files needed re-wrapping after the F2 clippy-fix pass landed
un-fmt'd. Pure cosmetic — no behavioral change. Re-runs of
`cargo fmt --all -- --check` now exit clean.

Reviewer-supplied per REVIEW.md §7; folded onto the branch
to avoid a NEEDS-CHANGES round-trip on a purely mechanical fix.
2026-06-06 18:44:52 +02:00
MechaCat02
5eb596611c chore(v1.1.8): GC sweep for app-user sessions + tokens
Extend the existing weekly retention sweeper with a new
spawn_app_user_token_gc function that prunes four tables in one
tick:

  * app_user_sessions — expired (sliding window or absolute cap) or
    explicitly revoked
  * app_user_email_verifications — consumed or expired
  * app_user_password_resets — consumed or expired
  * app_user_invitations — accepted or expired

Each underlying repo's gc(batch_size) uses FOR UPDATE SKIP LOCKED so
concurrent sweepers don't fight (cluster mode v1.3+ ready). No
configurable retention — rows die when their TTL or terminal state
hits, not on an arbitrary clock.

Wired into picloud's build_app alongside spawn_dead_letter_gc and
spawn_abandoned_gc.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-06 15:02:21 +02:00
MechaCat02
ff4f443531 feat(v1.1.8): F3 realtime auth_mode = 'session' (migration 0033)
Migration 0033 widens topics.auth_mode CHECK to include 'session'
alongside 'public' and 'token'.

TopicAuthMode enum gains a Session variant (as_str + from_db
extended uniformly).

RealtimeAuthorityImpl now takes Arc<dyn UsersService> as a third
constructor arg. The Session branch of authorize_subscribe
delegates to UsersService::verify_session_for_realtime(app_id,
token):
  * Returns Some(user) → allow. The service bumps the sliding TTL
    on success.
  * Returns None → Unauthorized.
  * Defense-in-depth: even though verify_session_for_realtime
    already enforces cross-app isolation, the branch re-checks
    user.app_id == app_id.

Tests added (4 new cases): valid session token allows; missing
token is Unauthorized; wrong token is Unauthorized; cross-app
session token is Unauthorized. All 12 realtime_authority tests
pass.

Dashboard: TopicAuthMode TypeScript union widened to include
'session'; the topic create + edit forms gain a third radio option
labeled "session — requires a per-app user session minted by
users::login (v1.1.8)".

picloud binary: construction order reshuffled so users is built
before realtime_authority. app_secrets_repo is now .clone()'d into
the pubsub realtime wiring so the original Arc can be re-used by
realtime_authority.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-06 15:00:47 +02:00
MechaCat02
3c2c4a3767 feat(v1.1.8): F1 drop plaintext realtime_signing_key (migration 0032)
v1.1.7 added at-rest encryption for app_secrets.realtime_signing_key
plus a startup task that backfilled encryption over the plaintext
column. v1.1.7's CHANGELOG committed v1.1.8 to dropping the
plaintext column; this commit follows through.

Migration 0032:
  * Guard query: refuses to apply if any row still has
    realtime_signing_key IS NOT NULL but realtime_signing_key_encrypted
    IS NULL. Forces operators who skipped v1.1.7 to apply it first.
  * ALTER TABLE app_secrets DROP COLUMN IF EXISTS
    realtime_signing_key.

app_secrets_repo:
  * decode_signing_key now reads encrypted+nonce only; the plaintext
    fallback is gone. (The schema still allows it via DROP IF EXISTS
    semantics on replay; once dropped, the column doesn't exist —
    the SELECT no longer requests it.)
  * Removed migrate_plaintext_keys (the v1.1.7 startup sweep).
  * Tests for the falls-back-to-plaintext path are gone with it; the
    remaining tests cover the encrypted-only happy path, the
    missing-columns None case, and the wrong-master-key Crypto error.

picloud/lib.rs: removed the migrate_plaintext_keys startup call
+ replaced with a comment explaining the upgrade-path requirement.

LOAD-BEARING: v1.1.8 requires v1.1.7 to have been applied first.
Operators upgrading directly from v1.1.6 or earlier must apply
v1.1.7 (which performs the encryption pass) before applying v1.1.8.
This is enforced both by the migration guard and by the CHANGELOG
(in a later commit).

Brief mentioned dropping a "realtime_signing_key_nonce_LEGACY_IF_EXISTS"
column — recon confirmed migration 0025 only added the plaintext
column + the encrypted/nonce pair, so no legacy nonce column exists
to drop. Documented in HANDBACK §7.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-06 14:56:27 +02:00
MechaCat02
aa2631ff61 feat(v1.1.8): admin HTTP — /apps/{id}/users + /invitations
users_admin_api router merged into the guarded admin tree at
/api/v1/admin/apps/{id_or_slug}/users/* + /invitations/*.

Endpoints (capability gates applied via the service layer):

  GET    /users                                       AppUsersRead
  GET    /users/{user_id}                             AppUsersRead
  POST   /users                                       AppUsersWrite
  PATCH  /users/{user_id}                             AppUsersWrite
  DELETE /users/{user_id}                             AppUsersWrite (admins satisfy implicitly)
  POST   /users/{user_id}/reset-password              AppUsersAdmin -> one-shot token
  POST   /users/{user_id}/revoke-sessions             AppUsersAdmin
  GET    /invitations                                 AppUsersAdmin
  POST   /invitations                                 AppUsersAdmin
  DELETE /invitations/{invite_id}                     AppUsersAdmin

DTOs never include password_hash, session tokens, or unrotated
reset tokens. The reset-password endpoint returns the raw one-shot
token exactly once in the response body so an admin can paste it
into a manual reset link (TTL 1h by default).

Synth_cx() factors the admin-side cx construction into one place
(marked) so the SDK and admin code paths share the service's authz
fan-out. Admin-mediated trait methods (admin_create_invitation,
admin_reset_password_token, admin_revoke_all_sessions,
list_invitations, revoke_invitation) take &Principal directly,
not the synthesized cx.

UsersServiceImpl: removed the NOT_YET_IMPL constant + unused
auth alias (every method has a real implementation now). Added
Serialize+Deserialize on the AppUser / Invitation shared DTOs so
the admin HTTP layer doesn't need a parallel set of types.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-06 12:18:29 +02:00
MechaCat02
3af99873c3 feat(v1.1.8): per-app roles + roles in user record (migration 0031)
migration 0031: app_user_roles table — composite PK (app_id, user_id,
role) so add is idempotent (ON CONFLICT DO NOTHING). v1.1.8 stores
strings only; permission matrices / hierarchies / role registry are
explicitly v1.2 work per the brief.

UsersServiceImpl wires roles: Arc<dyn AppUserRoleRepo>:

  * fetch_roles() now actually queries the repo (replacing the empty
    Vec stub from commit 4). Every AppUser returned from get /
    find_by_email / list / update / verify / login now carries its
    role list.
  * users::add_role gated on AppUsersAdmin; first checks the user
    exists in this app so a FK violation can't leak "no such user".
  * users::remove_role gated on AppUsersAdmin; idempotent.
  * users::has_role gated on AppUsersRead.
  * accept_invite now applies pre-staged roles atomically with the
    user creation; malformed role strings are skipped with a warn
    rather than aborting the whole accept (the invitation was an
    admin's promise — we honor as much of it as we can).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-06 12:13:35 +02:00
MechaCat02
b07382e64b feat(v1.1.8): invitations flow (migration 0030 + accept_invite returns session)
migration 0030: app_user_invitations table — surrogate id PK + unique
token_hash, app_id FK cascading, pre-stages email + display_name +
roles for a user that doesn't exist yet. One-shot via atomic UPDATE
SET accepted_at = NOW() WHERE accepted_at IS NULL.

UsersServiceImpl gains invitations: Arc<dyn AppUserInvitationRepo>
plus a mint_session() helper factored from login() and reused by
accept_invite().

users::invite(email, opts) is gated on AppUsersAdmin (per brief —
the most senior of the three new capabilities). Optional
EmailTemplateOpts inside InviteOpts: omitting the template skips the
email send so an admin can stamp invitations for out-of-band
delivery (mailers, printed onboarding letters, etc.). If the template
is present and the email service isn't configured, surfaces as
NotConfigured; non-NotConfigured failures are logged but kept silent
so the invitation row remains valid for retry.

users::accept_invite(token, password, display_name?) atomically
consumes the invitation, validates the new password, creates the
user (returning () on DuplicateEmail — sign-up beat acceptance,
they'll log in normally), and mints a fresh session via mint_session
so the caller can return both the user and a working session token
in one round trip.

Pre-staged roles are stored on the invitation row but not yet
applied — the app_user_roles table arrives in commit 8 (migration
0031). For commit 7 the staged-but-not-applied case logs an info
record so an operator can audit the gap.

list_invitations + revoke_invitation (admin-mediated, gated on
AppUsersAdmin) ship in this commit and become reachable from the
HTTP surface later in the series.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-06 12:10:32 +02:00
MechaCat02
45242e2d92 feat(v1.1.8): password reset flow (migration 0029 + revokes sessions)
migration 0029: app_user_password_resets table — same shape as
verification (token_hash PK, app_id + user_id FKs, expires_at,
consumed_at). One-shot via atomic UPDATE WHERE consumed_at IS NULL.
Default TTL 1h (shorter than verification's 48h — reset tokens are
higher-risk).

UsersServiceImpl gains password_resets: Arc<dyn AppUserPasswordResetRepo>.

users::request_password_reset(email, opts):
  * Returns Ok(()) regardless of whether the email matched — no
    existence-leak signal in script-land (per brief).
  * Email-not-configured surfaces as NotConfigured so scripts can
    fall back to a synchronous reset path. Other email errors are
    silently swallowed and logged server-side; surfacing them would
    leak which addresses produced a "real send attempted" signal vs
    a no-op.

users::complete_password_reset(token, new_password):
  * Atomically consumes the token, updates the Argon2id hash, and
    revokes EVERY active session for that user (anyone with a stale
    token shouldn't be able to ride out the reset). Emits
    users::password_changed.
  * Returns the user on success, () on bad/expired/already-used.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-06 12:06:47 +02:00
MechaCat02
c855739559 feat(v1.1.8): email verification flow (migration 0028 + SDK)
migration 0028: app_user_email_verifications table — token_hash PK,
app_id + user_id FKs cascading, expires_at, consumed_at. Single-use
via atomic UPDATE WHERE consumed_at IS NULL.

UsersServiceImpl gains:
  * verifications: Arc<dyn AppUserVerificationRepo>
  * email: Arc<dyn EmailService>

users::send_verification_email(user_id, opts) mints a 32-byte token,
stores SHA-256(token), and calls EmailService::send with the body
template's {link} placeholder substituted by link_base + ?token=raw.
EmailError::NotConfigured propagates as UsersError::EmailNotConfigured
so scripts already handling email-disabled mode (v1.1.7 email::send)
don't need new branches.

users::verify_email(token) atomically consumes the one-shot token via
the verifications repo, marks the user's email_verified_at = NOW(),
and emits a "users::email_verified" event for future triggers.

Internal email send uses a synthesized SdkCallCx with principal=None
so the email-service AppEmailSend authz check is skipped (the
users::* surface has already gated on AppUsersWrite — the internal
hop isn't the script's direct call). Documented in HANDBACK §7.

EmailTemplateOpts now requires `from` (the v1.1.7 email service needs
an envelope sender). The brief example omitted it; deviation logged
in HANDBACK §7.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-06 12:04:09 +02:00