Commit Graph

69 Commits

Author SHA1 Message Date
MechaCat02
bdcd3dc7f4 fix(executor): bound Rhai→JSON materialization to prevent anonymous OOM
The per-element Rhai sandbox caps (max_string_size 64 KiB, max_array_size /
max_map_size 10 000) do NOT bound the *materialized* size: Rhai shares strings
and arrays by Rc, so a 10 000-element array of one aliased 64 KiB string is
cheap to build (~10 000 ops, far under the 1 M op budget) yet dealiases to
~640 MiB of distinct JSON. Every Dynamic→JSON conversion deep-copies the
aliases; the HTTP response body path had NO size cap at all, and the KV/docs
value caps run only AFTER full materialization — so a handful of anonymous
requests could OOM the node (32 concurrent × ~640 MiB ≈ 20 GiB) on the stated
consumer-hardware target.

Add a byte-budgeted materializer that bails the moment the budget is exceeded,
so the transient allocation is bounded regardless of aliasing:
- bridge.rs: `dynamic_to_json_capped(value, max) -> Result<Json, JsonSizeError>`
  (charges a running budget) + `MAX_JSON_MATERIALIZE_BYTES` (16 MiB hard rail,
  far above the 256 KiB business caps). `dynamic_to_json` stays infallible for
  best-effort paths (logging) but is now internally bounded — it collapses an
  over-limit value to a marker string instead of OOMing.
- Route every user-value boundary through the fail-closed capped form: KV
  set/set_if (+ the CAS `expected`), docs create/update/find, secrets set,
  http request body, workflow input, `json::stringify`, and the HTTP RESPONSE
  body (previously the one fully-uncapped exit → now a 500). `invoke` args and
  the pubsub/queue message materializers get the same byte budget in place.
- `impl From<JsonSizeError> for Box<EvalAltResult>` so SDK sites protect
  themselves with a bare `?`.

Pinned by bridge unit tests: an aliased 10 000×64 KiB array errors on the
budget rather than materializing, and the infallible wrapper yields a marker
instead of OOMing. Workspace 914 + journeys 157/157 green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 21:50:16 +02:00
MechaCat02
d08df88df5 fix(interceptors): harden the §9.4 slice — seal, fail-closed, re-entrancy, shared coverage
An end-to-end audit of the (unmerged) interceptor slice surfaced seven ways a
KV allow/deny guard could be silently defeated or misbehave. Close all of them:

1. Shared-collection bypass — `kv::shared_collection(...).set/delete` skipped
   the hook entirely, so any `(kv, set/delete)` guard was circumvented by
   choosing the shared handle. `GroupKvHandle` now runs the same before-op hook.

2. Seal to the declaring owner — the marker resolved nearest-owner-wins but its
   script name was then re-resolved on the CALLER's chain, so a descendant app
   could shadow a group's mandatory guard with a same-named local script.
   `resolve_before` now returns the script sealed to the owner that declared the
   marker (one LEFT JOIN in manager-core), so the executor never re-resolves by
   name. A descendant can only override with its OWN explicit marker.

3. Re-entrancy — an "allow + audit-log" guard that itself wrote KV re-triggered
   itself to the depth cap and then DENIED the original write (plus ~8x
   resolve/compile/execute amplification). A thread-local guard makes a write
   performed by an interceptor bypass interception.

4. Fail-closed verdict — only `#{ allowed: false }` denied; a bare bool, a
   typo'd key, a non-bool, or a bare unit all ALLOWED. Now allow ONLY on an
   explicit `#{ allowed: true }`; every other shape denies.

5/6. Fail-closed edges — a dangling (missing/disabled) interceptor script and a
   missing engine back-reference now DENY instead of allowing.

7. AppInvoke coupling — resolution no longer routes through `invoke.resolve`, so
   installing a guard no longer denies writes to principals who hold KV-write
   but not AppInvoke.

The seam (`InterceptorService::resolve_before`) now returns an
`InterceptorResolution { None | Dangling | Run(ResolvedScript) }`; the executor
runs the sealed script straight through the shared `run_resolved_blocking` core
and drops its `InvokeService` dependency. executor-core stays Postgres-free.
No migration change (0073 unchanged).

Pinned by three new journeys in tests/interceptors.rs: the seal (a same-named
app script does NOT shadow the group's guard), shared-collection coverage, and
the fail-closed verdict. Full journey suite 157/157.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 21:20:20 +02:00
MechaCat02
2fc9476f9e feat(interceptors): §9.4 service interceptors — thin KV allow/deny slice
Smallest honest vertical slice of §9.4: a `[[interceptors]]` block (app OR
group) binds a script to run BEFORE `kv::set`/`delete`; it reads the operation
context (`ctx.request.body`: service, action, collection, key, value, caller
ids) and returns `#{ allowed, reason }` — `allowed == false` denies the op (the
write never runs, the caller gets a runtime error).

Reuses two existing mechanisms rather than inventing new ones:
- Registration mirrors extension points (§5.5): a marker table
  `0073_interceptors.sql` (owner-polymorphic app_id/group_id XOR, keyed
  (service, op) → script), `interceptor_repo` (insert/delete/list + the
  nearest-owner-wins `resolve_before` chain walk), reconciled through the
  declarative apply exactly like `vars` (create/update/delete, prunable).
- Execution reuses the `invoke()` re-entry path: the new `InterceptorService`
  (shared trait + Postgres-backed impl) only RESOLVES the script name (keeping
  executor-core Postgres-free); the executor's `sdk::interceptor::run_before`
  resolves that name and runs it via `run_resolved_blocking` (extracted from
  `invoke_blocking` — shared depth bound + AST cache). An un-hooked write pays
  one indexed `Ok(None)` resolve; no interceptor ⇒ zero overhead.

Nearest-owner-wins so an app overrides a group's interceptor, and a group
interceptor is inherited by every descendant app — the chain walk is the
isolation boundary (a sibling subtree never matches). `validate_bundle_for`
restricts the MVP to `service = "kv"`, `op ∈ {set, delete}`, one marker per
(service, op).

Deferred (documented in §9.4): the `data` transform return, services other
than kv, `after_*` hooks, chaining + circular-dependency guard, the timeout
policy, and a `pic interceptors ls` read surface (needs a server route).

Pinned by `tests/interceptors.rs` (deny blocks the write; allow passes;
group→app inheritance), schema snapshot re-blessed. 154/154 journeys pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 20:44:50 +02:00
MechaCat02
f1b480f2eb refactor(sdk): consolidate the SDK error/bridge helpers
`http.rs` hand-rolled its own `block_on` + `map_http_err`, and `secrets.rs` /
`email.rs` / `users.rs` each carried a byte-identical `runtime_err` — five
`#[allow(clippy::unnecessary_box_returns)]` across four files for one pattern.

Move `runtime_err` to `bridge.rs` beside `block_on` (single home, single allow)
and have the three bridges import it. `http.rs` now calls the shared
`bridge::block_on("http", …)` (same "http:"-prefixed error, so messages are
unchanged) and its `err` validation helper delegates to `bridge::runtime_err`.
Deletes 4 duplicate fns and 3 of the 5 allows; no behavior change (pinned by
the executor-core SDK contract tests).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 19:36:31 +02:00
MechaCat02
5a630e1d9f feat(workflows): M5 — SDK workflow::start + admin run API + pic CLI
The user-facing surface for the durable DAG engine. A workflow run can now be
started three ways, all resolving the workflow by name in the caller's app
(`cx.app_id` / the resolved app — never a passed-in arg, the isolation
boundary), seeding a run the orchestrator advances.

SDK seam (shared):
- `WorkflowService` trait + `NoopWorkflowService` + `WorkflowError` in
  `shared::workflow` (mirrors `InvokeService`); added to `Services` as a
  noop-defaulted `with_workflow` builder (all `Services::new` call sites
  unchanged).
- `WorkflowServiceImpl` (manager-core): resolves + seeds a run; authenticated
  callers gated on `AppInvoke` (anonymous skips, script-as-gate), the same gate
  `invoke()` uses.
- `executor-core::sdk::workflow` — the Rhai bridge `workflow::start(name, input)`
  / `workflow::start(name)`, registered in `register_all`. Returns the run id.

Admin API (manager-core `workflows_api`, mounted in the binary):
- `GET  /apps/{app}/workflows`              — definitions (AppRead)
- `POST /apps/{app}/workflows/{name}/runs`  — start a run (AppInvoke)
- `GET  /apps/{app}/workflows/{name}/runs`  — run history (AppRead)
- `GET  /apps/{app}/workflow-runs/{run_id}` — one run + its steps (AppRead)
  `app_id` scopes every query; a foreign run 404s.

CLI (`pic workflows`): `ls` · `run <name> [--input JSON]` · `runs <name>` ·
`run-status <run_id>` — all `--app`-scoped; run definitions stay declarative
(`[[workflows]]` → `pic apply`).

Also: `list_runs_for_workflow` repo reader.

Tests: `workflow_service_start_seeds_a_run` (DB-gated — the SDK/API-shared
entry) + `workflow_run_executes_end_to_end` CLI journey (apply → `pic workflows
run` → poll `run-status` → succeeded, through the real binary + orchestrator).
fmt + clippy -D warnings clean, 449 manager-core lib tests, 14 orchestrator DB
tests, 18 executor-core lib tests, 3 workflow CLI journeys green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 17:48:38 +02:00
MechaCat02
c8c4f012ff docs: reconcile blueprint/design-doc/CLAUDE.md to shipped v1.2 state
A five-agent roadmap-verification pass found the v1.2 Hierarchies code fully and
correctly implemented, but the three source-of-truth docs carried stale
"deferred/remaining/v1.3" notes — and one self-contradiction — for features that
have since shipped (§3 M3 server-side approval gate, §6 group create/reparent, the
Track A closeout, and the 2026-07-11 audit remediation). Reconcile all three to
reflect the true code; no behavior change.

CLAUDE.md: retract the inline "groups pre-exist" / "per-env approval deferred" /
"shared-topic SSE deferred" / "group dead-letter store deferred" / "email v0/no-AAD"
notes → mark §6, §3 M3, and Track A M1–M6 shipped; add per-group KV/docs byte
quotas + set_if; note migration 0063; refresh the "Out of MVP" section.

design doc: fix the header ("Phases 4–6 remain" → all shipped); resolve the §11.6
self-contradiction (the "Deferred" list still named shared-topic SSE / byte quotas /
set_if / operator admin API, all shipped — line 1317 already said SSE shipped); mark
the D3 dead-letter store shipped; retract the stale "groups pre-exist / §6 deferred"
forward references.

blueprint (comprehensive sweep): dashboard Alpine.js → SvelteKit + CodeMirror
(diagram, §3.3, tech table); Docker-per-execution → embedded in-process Rhai
(diagram + data flow); §12 Phase 4 "current focus" → shipped through v1.1.9; Phase 5
"in active development" + "Remaining" block → Hierarchies complete; per-app RBAC
"v1.3+" → shipped Phase 3.5; SDK reference → handle-pattern notation note, S3 tag
v1.1 → v1.3+; MVP schema + docker-compose flagged non-authoritative; §9 header noted
Hierarchies-shipped / Workflows-future; top status line refreshed.

Also fix two behavior-neutral stale in-code comments (sdk/kv.rs `kv::shared` →
`kv::shared_collection`; dispatcher.rs `q_terminal` "no group dead-letter store yet"
→ dead-letters to group_dead_letters, Track A M2).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 14:53:34 +02:00
MechaCat02
0f05c270d1 feat(kv): set_if compare-and-swap for KV (per-app + shared + SDK) (Track A M5)
No atomic CAS existed, so concurrent writers to the same KV key raced. Add
set_if(key, expected, new): writes only when the current value equals expected,
or (expected absent) only when the key is missing — the primitive for lock-free
counters / optimistic concurrency.

- KvService + GroupKvService traits gain set_if with a defaulted "unsupported"
  impl (Noop + other impls untouched); the real services override it.
- kv_repo/group_kv_repo: atomic set_if — a single UPDATE ... WHERE value =
  $expected (JSONB semantic equality) or INSERT ... ON CONFLICT DO NOTHING.
  Atomic without a transaction; returns whether the write happened.
- Services enforce the same value-size cap + (group) writes-authed + row/byte
  quotas as set; the event fires only on a successful swap.
- Executor Rhai SDK: kv::collection(c).set_if(key, expected, new) and the
  shared_collection variant; Rhai () for expected means "only if absent".

Pinned by kv_service/group_kv_service set_if unit tests (CAS + fail-closed) +
sdk_kv::kv_set_if_compare_and_swap (through a real script). No migration.
docs/sdk-shape.md documents the primitive.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 15:19:03 +02:00
MechaCat02
ccd3644aa4 feat(shared-queues): group-keyed queue store + enqueue service + SDK (D3.1)
The producer side of shared durable queues:
- migration 0065 group_queue_messages (mirrors 0034, keyed by (group_id,
  collection); CASCADE on group delete).
- GroupQueueRepo/PostgresGroupQueueRepo: enqueue + the competing-consumer
  claim (FOR UPDATE SKIP LOCKED) + ack/nack/drop_exhausted/reclaim/depth.
- GroupQueueService trait (shared) + GroupQueueServiceImpl: resolve owning
  group (kind='queue') from cx.app_id's chain, require editor+
  (GroupQueueEnqueue, fails closed on anon), size-cap, enqueue.
- SDK: `queue::shared_collection("name")` -> GroupQueueHandle with
  `.enqueue(msg[, opts])` / `.depth()` / `.depth_pending()`; wired through
  Services + the picloud binary.
- authz: Capability::GroupQueueEnqueue(GroupId), editor+ / script:write.

Deterministic test proves competing consumers claim each message exactly
once. Consumption wiring (materialized consumers + dispatcher branch) lands
in D3.2.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 22:08:21 +02:00
MechaCat02
9626d15863 feat(shared-topics): GroupPubsubService + shared publish fan-out + SDK handle (D2)
The runtime for shared topics:
- pubsub_repo: fan_out_publish gains `AND t.shared = FALSE` (a shared
  trigger never fires on a per-app publish); new fan_out_shared_publish
  matches `shared = true` pubsub triggers on the resolved owning group,
  each delivery stamped with the writer app_id (M2 model).
- GroupPubsubService trait (shared) + GroupPubsubServiceImpl: resolve the
  owning group (kind='topic') from cx.app_id's chain, require editor+
  (GroupPubsubPublish, fails closed for anon), size-cap, fan out.
- SDK: `pubsub::shared_topic("name")` -> GroupTopicHandle with
  `.publish(subtopic, msg)` / `.publish(msg)`; wired through Services +
  the picloud binary (reuses the one PubsubRepo).
- authz: Capability::GroupPubsubPublish(GroupId), editor+ / script:write.

The owning-group chain walk is the isolation boundary; a sibling-subtree
app never resolves the topic.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 21:48:59 +02:00
MechaCat02
e7c0485dbf feat(modules): GroupFilesService + files::shared_collection handle (§11.6 files C3)
- shared/group_files: GroupFilesService trait (mirror FilesService, no
  group id arg — owner resolved from cx.app_id), GroupFilesError (clone of
  FilesError + CollectionNotShared) with From<FilesError>, Noop.
- services: group_files field + with_group_files setter (default Noop).
- group_files_service: GroupFilesServiceImpl — owning_group via the
  registry resolver (kind=files) → CollectionNotShared; reads open
  (script_gate), writes fail closed (script_gate_require_principal);
  reuses validate_files_collection + the NewFile/FileUpdate validators +
  sanitize_stored_content_type + the per-file size cap; no events.
- sdk/files: GroupFilesHandle + files::shared_collection(name) + the
  create/head/get/update/delete/list group methods (Blob in/out).
- picloud: construct FsGroupFilesRepo (sharing the files root + size cap)
  + GroupFilesServiceImpl, .with_group_files(...).

Unit tests: off-chain → CollectionNotShared; anon read OK / anon write
Forbidden; oversize → TooLarge.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 19:46:39 +02:00
MechaCat02
78a468de83 feat(modules): GroupDocsService + docs::shared_collection handle (§11.6 docs C3)
The runtime for shared group docs collections, mirroring the group-KV vertical
with the docs surface (filter parsing, JSON-object validation, value-size cap).

- shared: GroupDocsService trait + GroupDocsError (+ CollectionNotShared) +
  NoopGroupDocsService; group_docs field on Services + with_group_docs().
- manager-core: GroupDocsServiceImpl — owning_group resolves kind='docs' from
  cx.app_id's chain (CollectionNotShared off-chain); reads-open (script_gate
  GroupDocsRead) / writes-authed (script_gate_require_principal GroupDocsWrite);
  reuses parse_filter + docs value-size cap; no events. Unit tests cover
  off-chain CollectionNotShared, anon-read-OK/anon-write-Forbidden, non-object
  data rejected.
- executor-core: GroupDocsHandle + docs::shared_collection constructor + the
  create/get/find/find_one/update/delete/list methods (dispatch by receiver
  type), reusing doc_to_map/parse_doc_id.
- picloud: wire PostgresGroupDocsRepo + GroupDocsServiceImpl.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 08:04:30 +02:00
MechaCat02
0973344515 feat(cli): collections manifest + ls + journeys; rename to kv::shared_collection (§11.6 C5)
Finish the §11.6 KV slice: declarative authoring, read-only inspection, the
runtime journey, and docs — plus a forced SDK-name fix.

- SDK name: `shared` is a Rhai RESERVED keyword, so `kv::shared(...)` is a parse
  error. Renamed the handle constructor to `kv::shared_collection(...)`
  (keeps the user's "shared" intent; unambiguous). Updated everywhere.
- CLI manifest: `collections = [...]` on `[group]` only (ManifestApp has no such
  field + deny_unknown_fields → an app manifest carrying it is a hard error);
  Manifest::collections() accessor; build_bundle emits it.
- plan/apply: a `collection` row group in `pic plan`; created/deleted counts in
  the apply summary; collections threaded through PlanDto/NodePlanDto/
  ApplyReportDto.
- read-only `pic collections ls --group` → GET /admin/groups/{id}/collections
  (viewer+), backed by ApplyService::collection_report + CollectionInfo.
- journey: a group declares `catalog`; app A writes, app B (same subtree) reads
  it back; a sibling-subtree app gets CollectionNotShared; `collections ls` +
  re-apply NoOp. Full suite 114/114.
- docs: groups-and-project-tool §11.6 (KV-only MVP shipped + deferrals),
  sdk-shape.md (the shared-collection handle + trust model), CLAUDE.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 22:26:43 +02:00
MechaCat02
d9766bcbc7 feat(modules): GroupKvService + kv::shared SDK handle (§11.6 C3)
The runtime for shared group collections. A script reaches a shared store via
the explicit kv::shared("name") handle (distinct GroupKvHandle Rhai type, so
private-vs-shared scope is a deliberate choice). The service resolves the
owning group from cx.app_id's ancestor chain — the script never names a group,
and that walk is the isolation boundary (a foreign app's chain never contains
the owning group → CollectionNotShared).

- shared: GroupKvService trait + GroupKvError + NoopGroupKvService; threaded
  into the Services bundle as a field defaulted to noop, opted into via a new
  with_group_kv() (keeps the ~14 Services::new call sites unchanged).
- manager-core: GroupKvServiceImpl with the reads-open/writes-authed split —
  reads use script_gate (anonymous public scripts skip), writes use the new
  script_gate_require_principal (anonymous fails closed). Owner resolution
  behind a GroupCollectionResolver trait (Postgres impl + injectable fake);
  unit tests cover off-chain CollectionNotShared, anonymous-read-OK/
  anonymous-write-Forbidden, authed-no-role-Forbidden.
- executor-core: GroupKvHandle + kv::shared constructor + get/set/has/delete/
  list (Rhai dispatches by receiver type, reusing the block_on bridge).
- picloud: wire PostgresGroupKvRepo + resolver + GroupKvServiceImpl.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 21:44:53 +02:00
MechaCat02
8f966783fe feat(modules): extension-point-aware resolver policy (§5.5 C2)
Add the runtime heart of extension points: `ModuleSource::resolve_policy`
decides lexical vs dynamic resolution by the NEAREST declaration of a name
walking up the importing origin's chain — a concrete module → lexical (seal
to origin, Phase 4b); an extension-point marker → dynamic, resolved against
the inheriting app (`cx.app_id`) so the app can override, falling back to the
default body up-chain; the EP wins a depth tie.

- shared: `ModuleResolution { Module | NoProvider | NotFound }` + a
  `resolve_policy(origin, inheriting_app, name)` trait method with a
  lexical-only default impl (existing fakes inherit it unchanged).
- PostgresModuleSource: one-round-trip nearest-declaration query (min EP depth
  vs min module depth over the chain CTEs), then the lexical/dynamic branch.
- module_resolver: calls `resolve_policy(origin, cx.app_id, path)`; maps
  NoProvider → a clear "extension point has no provider" error, NotFound →
  module-not-found. Cache + AST-source sealing unchanged.
- executor-core tests: a policy fake + 3 cases (app override binds dynamically,
  no-provider errors, non-EP stays lexical).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 20:07:59 +02:00
MechaCat02
b53eabb786 feat(modules): lexical import resolver — seal imports to defining node (Phase 4b C3)
Make `import` resolution lexical (§5.5): an inherited group script's
imports resolve against the module set at its OWN defining node, walking
up from there — a leaf app can't shadow them, and a group script behaves
identically under every inheriting app.

Mechanism — Rhai's `_source` carries the importing AST's origin tag:
- The entry script has no source tag → resolve from `default_origin` (its
  defining node, threaded in via C2).
- Each resolved module's AST source is set to `encode(module.owner())`
  before `eval_ast_as_new`, so its nested `import`s carry the module's own
  node as `_source` and resolve from there — lexical chaining.
- `encode_origin`/`decode_origin` map a `ScriptOwner` to/from `app:<uuid>`
  / `group:<uuid>`.

Cache rekeyed from `(app_id, name)` to the resolved `ScriptId`: a compiled
module is lexically self-contained, so its body is a pure function of
`(script_id, updated_at)` — this also dedupes a shared group module across
inheriting apps and keeps same-named cross-app modules distinct.

Adds two executor-core unit tests (origin-keyed fake) proving the trust
boundary: a group module's import binds the group's module even when an
app-owned module of the same name exists; `ScriptOwner` gains `Hash`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 07:22:25 +02:00
MechaCat02
1844bef132 feat(executor): thread the defining node into ExecRequest (Phase 4b C2)
Carry the resolved script's owner (its defining node) from dispatch into
the executor so `import` resolution can be lexical (§5.5). The executing
app (`app_id`) stays the SDK isolation boundary; `script_owner` is a
separate axis — the lexical origin for imports.

- `ExecRequest.script_owner: Option<ScriptOwner>` (serde default; `None`
  falls back to `App(app_id)` in the engine for old payloads / cluster wire).
- `ScriptOwner` gains `Serialize`/`Deserialize` for the wire.
- The engine computes `default_origin` and hands it to the resolver
  (used fully in C3; the resolve() lookup already uses it).
- Every dispatch site sets it from the resolved `Script.owner()`:
  the 4 dispatcher arms (queue/trigger/http/invoke_async, via a new
  `ResolvedTrigger.script_owner`), the orchestrator id-bypass, and the
  `invoke()` SDK path (new `ResolvedScript.owner`, so a group script
  invoked by an app resolves imports from the group).

Behaviour-preserving: app scripts still resolve `App(app_id)`; group
scripts can't yet carry imports (lifted in C4).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 07:17:33 +02:00
MechaCat02
7fef098b48 feat(modules): owner-aware module lookup — lexical chain walk (Phase 4b C1)
Make the module-loading contract origin-aware so group-owned modules
become resolvable and imports can resolve lexically (§5.5).

- `ModuleScript` gains a polymorphic owner (`app_id`/`group_id` + `owner()`),
  mirroring `Script`.
- `ModuleSource::lookup(&cx, name)` -> `resolve(origin: ScriptOwner, name)`:
  resolution walks the group chain rooted at the importing script's
  defining node (nearest-owner-wins), not the inheriting app's view.
- `PostgresModuleSource::resolve` branches on origin: app-rooted
  (`CHAIN_LEVELS_CTE`) or a new group-rooted `GROUP_CHAIN_LEVELS_CTE`,
  joining `kind='module'` rows.

The executor resolver passes a temporary `App(cx.app_id)` origin (behaviour-
preserving for app scripts; true lexical origin lands in C3). Group scripts
still can't carry imports until C4, so no trust-inversion window opens here.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 07:07:04 +02:00
MechaCat02
343f6d3b4d feat(vars): VarsService + vars:: SDK + config capabilities
Scripts can now read group-inherited config via vars::get(key) / vars::all().

- shared: VarsService trait (read-only get/all) + NoopVarsService; added as
  the 14th field on the Services bundle.
- manager-core: VarsServiceImpl resolves the calling app's config via
  config_resolver (derives app_id from cx.app_id; AppVarsRead-gated for
  authed principals, script-as-gate for anon).
- executor-core: vars:: Rhai bridge (get/all), registered in the SDK.
- authz: AppVarsRead/Write, GroupVarsRead/Write, GroupSecretsRead/Write
  capabilities (group caps resolve via the Phase-2 ancestor walk; the
  GroupSecretsRead human-read gate is group_admin-only, distinct from an
  app's runtime injection).
- wired VarsServiceImpl into the picloud binary; updated the executor-core
  test Services::new call sites.

386 manager-core lib tests green; clippy clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 20:59:23 +02:00
MechaCat02
51f14fa2b1 feat: E2E #2 (Stash) gap remediation + S6 hardening
Some checks failed
CI / Rust — fmt, clippy, test (push) Failing after 6m19s
CI / Dashboard — check (push) Successful in 9m48s
Closes the gaps and the one security finding from the second end-to-end
CLI test (E2E_STASH_REPORT.md), plus the H1 boot-regression found while
re-reviewing those fixes.

Security
- S6: reserved-path validation (`check_reserved`) now case-folds before
  comparing, so `/API/v2/x`, `/HEALTHZ`, `/Admin/x` are rejected like
  their lowercase forms. Request-time matching stays case-sensitive.
- S10: "public route != public data" callout in sdk-shape.md (script_gate
  skips authz when the principal is anonymous).

Observability / features
- G1: trigger executions now write `execution_logs`. Migration 0043 adds
  a `source` column (CHECK mirrors ExecutionSource/OutboxSourceKind,
  DEFAULT 'http' backfills history); a shared `build_execution_log` helper
  in executor-core; dispatcher logging for outbox triggers + queue
  consumers (skips sync-HTTP rows the orchestrator already logs). `pic
  logs` gains a source column + `--source` filter.
- G5: dev-only in-memory email capture under PICLOUD_DEV_MODE with no SMTP
  (email::send succeeds locally), readable at GET /api/v1/admin/dev/emails
  (Owner/Admin only; route mounted only in capture mode).
- G6: generalized the Rhai in-place-mutation footgun note (trim/replace/
  make_upper/make_lower/crop/truncate/pad return ()).
- G2/G3/G4 (CLI): `pic members`, `pic files`, `pic queues`, read-only
  `pic kv` (+ new kv_api.rs); `pic deploy --timeout/--memory/--kind/
  --sandbox`; first-class `pic triggers create-{docs,files,pubsub,queue,
  email}` wrappers. All new client path segments percent-encoded via seg().

H1 regression fix (found in re-review)
- The S6 change also runs in `compile_routes`, which compiles every stored
  route at boot and on each route CRUD. A single stored route the new
  validation rejects (creatable while the S6 gap existed) made the whole
  compile Err and aborted startup. `compile_routes` is now lenient: it
  skips an un-compilable row with a warning instead of bricking boot
  (route creation still validates separately). Migration 0044 sweeps
  pre-existing reserved-path routes on upgrade (WHERE mirrors
  check_reserved exactly). Added regression tests for both.

Verified: cargo fmt, clippy --all-targets --all-features -D warnings, the
schema_snapshot test, and the new S6/lenient-compile unit tests all pass;
boot-resilience and G1/G5 confirmed live.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-13 15:01:04 +02:00
MechaCat02
a91b134285 fix(review): harden the E2E-gap fixes after security/regression review
Addresses findings from an independent review of the gap-closing commits:

- R1 (regression): add #[serde(default)] to HttpDispatchPayload.method so
  async-HTTP outbox rows enqueued before the field existed still decode
  after upgrade instead of dead-lettering on "missing field `method`".
  Adds a regression test for the missing-key path.
- method case: uppercase ctx.request.method at the orchestrator boundary
  so it honors its documented "uppercased" contract for extension verbs.
  Route matching is already case-insensitive, so matching is unaffected.
- CLI hardening: percent-encode free-form path segments (app slug, topic
  name, ids, secret name) in the reqwest client so a value containing
  `/ ? #` can't break out of its URL segment. Adds a `seg()` helper +
  unit test and applies it uniformly across all path-interpolating
  methods (new and pre-existing).
- S1 (docs): correct the users::email_available enumeration framing — it
  adds no new vector vs. create's uniqueness error, but is cheaper and
  unthrottled; there is no built-in throttle/CAPTCHA primitive, so the
  honest mitigation is a kv-based counter. Updated SDK doc-comments,
  trait docs, and stdlib-reference.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-13 09:59:24 +02:00
MechaCat02
7ffbb8de87 feat(sdk): users::email_available for anonymous registration pre-check (S1)
`users::find_by_email` requires an authenticated principal (F-S-003,
anti-enumeration), so the natural check-then-create register pattern 502s
for anonymous public scripts. Add `users::email_available(email) -> bool`,
gated like `create` (the registration write path) but without the
anonymous rejection: it leaks only a single boolean — no more than
`create`'s own uniqueness error already does — so self-serve register
scripts can pre-check. Also document find_by_email's principal
requirement in the SDK doc-comment.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-12 20:43:18 +02:00
MechaCat02
ae2134e62d feat(executor): expose ctx.request.method to scripts (F4)
The request map exposed path/headers/body/params/query/rest but not the
HTTP method, forcing one script per verb (the E2E app needed 7 scripts
where ~3 would do). Add a `method` field to ExecRequest (populated from
the HTTP method on the sync-execute bypass and the HTTP outbox payload;
empty for non-HTTP triggers) and surface it as `ctx.request.method`,
uppercased. A single script can now branch GET/POST on one route.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-12 20:43:10 +02:00
MechaCat02
95fddf31bc fix(audit-2026-06-11/C-2+F-SE-H-03): reject control chars in content-type; disable Rhai debug
C-2 follow-up: sanitize_stored_content_type previously let a CRLF pass
through an allowlisted prefix (e.g. "image/png\r\nX-Injected: 1" matched
the image/ branch and returned the original), which would inject a
response header / panic HeaderValue on download. Now any control byte
(<0x20 or 0x7f) coerces to application/octet-stream up front.

F-SE-H-03: disable_symbol("debug") to match "print" (the comment
already claimed both were disabled) so scripts can't write
attacker-controlled bytes to the operator's stderr.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-12 18:14:36 +02:00
MechaCat02
513c4a2d3c fix(audit-2026-06-11/H-D1): bind AES-GCM AAD on secrets + realtime signing key
At-rest secrets were AES-256-GCM sealed with no Associated
Authentication Data, so anyone with Postgres write access could
ciphertext-swap rows across apps (or rename via row edit) and the
decrypt would silently succeed under the wrong identity, returning
attacker-chosen plaintext. This breaks the cross-app isolation boundary
the moment DB write access is achieved.

Adds an AAD-bound envelope (v1) alongside the legacy no-AAD layout (v0),
discriminated by a per-row `version` column:

* shared::crypto — new encrypt_with_aad / decrypt_with_aad using
  aes_gcm::aead::Payload { msg, aad }. Originals retained for v0 reads.
  Tests: AAD round-trip, AAD-mismatch fails, empty-AAD round-trip.

* migration 0042 — adds `secrets.version SMALLINT NOT NULL DEFAULT 0`
  and `app_secrets.realtime_signing_key_version SMALLINT NOT NULL
  DEFAULT 0`. Existing rows stay v0; new writes are v1.

* secrets (SDK + admin API) — seal() now binds AAD =
  "secret:{app_id}:{name}" and emits v1; open() dispatches on version.
  StoredSecret gains a `version` field; SecretsRepo::set takes it.
  Both secrets_service::set and secrets_api::set_secret go through the
  v1 path. Tests prove a cross-app swap and a cross-name swap both
  surface Corrupted, and that a hand-built v0 row still decrypts.

* app_secrets (realtime signing key) — get_or_create_signing_key writes
  v1 with AAD = "app_secret:{app_id}:realtime_signing_key"; decode
  dispatches on version. Tests cover v0 decode, v1 round-trip, and v1
  decode-under-wrong-app failing.

* email-trigger inbound secret — kept on v0 (seal_legacy/open_legacy)
  and explicitly deferred: email_trigger_details has no version column
  and the trigger_id isn't known at seal time. The audit classes the
  email-trigger AAD gap as Medium; folded into v1.2's key-versioning
  pass per SECURITY_AUDIT.md.

* expected_schema.txt re-blessed by hand (no local Postgres) for the two
  new columns + migration 0042.

Also folds in a let-else clippy fix in auth_api.rs (login Argon2
semaphore acquire, from the H-B1 commit) and two cargo-fmt reflows.

No re-encryption sweep — v0 rows decrypt as-is; the sweep is deferred to
v1.2's key-versioning pass (audit "Notes on remediation methodology").

Audit ref: security_audit/03_crypto_secrets.md (H-D1).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-12 17:15:09 +02:00
MechaCat02
99eb0025fa fix(audit-2026-06-11/H-C1): engine.on_progress deadline check terminates runaway scripts
tokio::time::timeout over JoinHandle only drops the future — the
spawn_blocking OS thread keeps running until the Rhai script self-
completes or hits the per-op budget. A `loop {}` body with a generous
max_operations could pin a blocking worker for tens of seconds; on Pi-
class hardware one anonymous-callable script call could permanently
subtract a worker from the pool.

Installs an `engine.on_progress` hook that consults a thread-local
deadline; when `Instant::now() >= deadline` the hook returns `Some(_)`,
triggering `ErrorTerminated` inside the Rhai loop. The deadline is set
by `Engine::execute_with_deadline` / `Engine::execute_ast_with_deadline`
via an RAII `DeadlineGuard`, called from the orchestrator client; the
old `execute` / `execute_ast` paths are unchanged so tests, validation,
and the parse-only path keep working.

Invoke re-entry (`sdk/invoke.rs`) inherits the parent's deadline
transparently — the thread-local is set for the entire spawn_blocking
lifetime, and Rhai is single-threaded so the same thread services the
parent + every sub-script.

New tests:
* `deadline_terminates_a_runaway_loop` — `loop {}` body with
  `max_operations = u64::MAX` and a 100 ms deadline aborts within 2 s
  (typically <200 ms). Maps to `ExecError::Runtime`.
* `no_deadline_set_does_not_abort` — `None` deadline keeps the pre-
  audit behavior; the op-budget still bites.

Audit ref: security_audit/05_sandbox_exec.md#f-se-h-01.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-11 20:33:01 +02:00
MechaCat02
05ed9b00bb fix(stage-6): dashboard hardening + audit Lows cherry-pick
Closes 4 dashboard hardening findings and 5 of the Lows from the audit.

Dashboard hardening:
- Subtabs no longer re-fetch the app via api.apps.get on every page
  load. users/files/dead-letters drop the fetch outright (the variable
  was set but never read); queues + queues/[name] now consume the
  layout's AppContext via getContext for the page title. The layout's
  reloadApp() owns the historical-slug redirect — subtab-local redirect
  blocks are removed so there's no race.
- The global :global(details > summary::before) chevron is now scoped
  to details.chevron. The script editor's "Advanced sandbox" details
  and the inbound-email-shape help-text both opt in; the script
  exec-list logs no longer inherit a spurious chevron.
- deriveTab now matches the path segment by anchored ===, so a future
  /apps/<slug>/queues-archived route wouldn't activate the queues tab.

Lows cherry-pick:
- ExecError gains Serialize/Deserialize derives + a snake_case tag so
  RemoteExecutorClient (cluster mode v1.3+) can round-trip the variant.
- triggers_api rejects queue triggers whose visibility_timeout_secs is
  below the dispatcher's per-message executor budget; with no minimum
  the reclaim task races the handler and the queue silently
  double-delivers. Existing test using 5s updated to 30s.
- New migration 0040: execution_logs.script_id cascade switched from
  ON DELETE CASCADE to ON DELETE SET NULL so deleting a script no
  longer wipes the forensic history that motivated the delete.
- New migration 0041: dead_letters composite index on
  (app_id, created_at DESC) so the "list all" dashboard view stops
  falling back to seqscan + sort when unresolved=false.
- Schema snapshot re-blessed.

Deferred to v1.2: the ExecRequest principal serde(skip) marker
(documented in-place; the cluster-mode PR will introduce the wire-safe
snapshot at that point) and the `pic --help` mention of
`picloud admin reset-password` (one-line follow-up).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-10 21:50:58 +02:00
MechaCat02
547c9f9950 fix(executor-core): F-P-014 thread-local LRU regex cache (128 entries)
regex::is_match, find, find_all, replace, replace_all, split, captures
all called Regex::new(pattern) per invocation. A script doing
`regex::is_match("\\d+", x)` in a tight loop paid the compile every
iteration. Regex compile dominates is_match on short strings.

Add a thread-local LruCache<String, Arc<Regex>> (cap 128). The
compile helper now memoises by pattern string, returning Arc<Regex>
on hit. Cap is well above what any sensible script uses but bounded
enough to keep memory predictable on long-running threads.

AUDIT.md anchor: F-P-014.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 20:51:20 +02:00
MechaCat02
fea95bd63b fix(manager-core): F-P-012 keyset cursor on app_user_repo::list (created_at, id) tiebreaker
ORDER BY created_at DESC, id DESC but cursor was `WHERE created_at <
$2` — when two users were created at the same instant, pagination
could skip the second row at a page boundary or return it twice.

- Add ListCursor { created_at, id } with `<rfc3339>_<uuid>` encode /
  decode helpers.
- Change WHERE predicate to `(created_at, id) < ($2, $3)` matching the
  ORDER BY.
- Surface the opaque cursor string through UsersListOpts /
  UsersListPage / users_admin_api ListUsersResponse instead of the raw
  DateTime — keeps the wire format stable while the id half stops the
  boundary bug.

Same shape as F-P-005 (execution_logs).

AUDIT.md anchor: F-P-012.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 20:50:25 +02:00
MechaCat02
dc40bc7926 style: fmt + clippy cleanup after Phase B
cargo fmt regroups, plus three Phase-B clippy fixes:
- F-P-004: #[allow(clippy::type_complexity)] on the invoke_ast_cache Mutex.
- F-P-008: hoist `futures::stream::{self, TryStreamExt}` to the top-of-
  file imports to satisfy clippy::items_after_statements.
- F-P-005: rename `_legacy_offset` to `legacy_offset` + #[allow(dead_code)]
  to satisfy clippy's "field is pub but `_`-prefixed" check.

No behavior change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 20:38:11 +02:00
MechaCat02
aae107a5ff fix(executor-core): F-P-004 cache AST across invoke() re-entry (per-Engine cache)
Two paths bypassed the AST cache: LocalExecutorClient::execute (tests +
fallback) and the synchronous invoke() re-entry in the executor SDK.
The latter is the hot one — composed workflows multiplied parse cost
by depth, so a 4-deep invoke chain on a 200-line script paid the parse
budget × 4 per call.

Add a per-Engine HashMap<ScriptId, (updated_at, Arc<AST>)> + a
`compile_for_identity(script_id, updated_at, source)` helper that
behaves like LocalExecutorClient::get_or_compile but lives on the
Engine. Update the SDK invoke synchronous re-entry to:
  resolved → compile_for_identity → execute_ast

The orchestrator-core LocalExecutorClient cache (HTTP-path dispatch)
is left untouched — it caches a different access pattern at a
different boundary.

AUDIT.md anchor: F-P-004.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 20:13:28 +02:00
MechaCat02
6f2bd3e949 style: cargo fmt after Phase A
Whitespace + import-grouping fixups in the 9 SDK modules touched by
F-Q-002. No behavior change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 20:02:19 +02:00
MechaCat02
5c50ce2e11 fix(executor-core): F-Q-002 promote sdk::bridge::block_on, migrate 9 SDK modules
Replace ten near-identical `block_on` helpers (one per SDK module)
with a single shared `sdk::bridge::block_on(service: &str, fut)`.
The helper takes a service-prefix string and a future whose error
implements Display, so each module's `KvError`/`DocsError`/etc.
still self-formats.

Migrated:
- kv, docs, pubsub, users, dead_letters, secrets, files, email
- queue.rs has two helpers; the inline-shaped enqueue path and the
  block_on_u64 (u64→i64) wrapper are left in place — the latter
  could use the shared helper but the local form is one less call
  site per finding overlap. Will revisit on a later pass if needed.
- http.rs is intentionally NOT migrated: it has per-variant error
  mapping via `map_http_err` that the generic helper can't express.

Net: -218 / +97 lines across the 9 migrated files. Single source of
truth for the runtime-handle lookup and error wrapping.

AUDIT.md anchor: F-Q-002.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 19:54:37 +02:00
MechaCat02
c3baa87415 chore(v1.1.9): clippy clean — workspace -D warnings
Surfaced + fixed during the F3 attestation:

- executor-core/sdk/queue.rs: drop redundant .map(|()| ()) call sites;
  enqueue_blocking discards QueueMessageId via .map(|_id| ()) (intentional)
- executor-core/sdk/retry.rs: hoist use std::collections::hash_map::DefaultHasher
  + use std::hash::Hasher to the top of the file (clippy::items_after_statements);
  replace `as i64` casts with i64::try_from + clear comment about jitter
  bound (clippy::cast_possible_wrap)
- executor-core/sdk/invoke.rs: move _LIMITS_IS_COPY const before #[cfg(test)]
  mod tests (clippy::items_after_test_module)
- manager-core/dispatcher.rs: dispatch_one_queue gains #[allow(too_many_lines)]
  (it's the queue tick's whole logic — split makes it less readable than
  the lint); .map(...).unwrap_or(default) → .map_or(default, ...)
- picloud/lib.rs: Limits { trigger_depth_max, ..Limits::default() } via
  struct-update instead of let-mut-assign (clippy::field_reassign_with_default)
- tests: r#"..."# → r"..." where there are no `"` inside (clippy::needless_raw_string_hashes);
  combine InvokeTarget::Name | InvokeTarget::Path arms with identical bodies
  in sdk_invoke.rs (clippy::match_same_arms)

cargo fmt --all -- --check 2>&1 | tail -3: no output (exit 0)
cargo clippy --workspace --all-targets --all-features -- -D warnings:
  Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.41s
Workspace lib smoke: 432 passing (306 manager-core + 74 executor-core + 34 shared + 18 orchestrator-core).
DB-gated E2E (against docker compose postgres on localhost:15432):
  queue_e2e: 4 ok / invoke_e2e: 4 ok / retry_e2e: 3 ok / migration_queue_messages: 4 ok / schema_snapshot: 1 ok

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 11:02:22 +02:00
MechaCat02
e142d471e1 feat(v1.1.9): retry:: Rhai SDK module — caller-controlled retry
executor-core/sdk/retry.rs adds the retry:: namespace with:

  retry::policy(opts)           -> Policy
  retry::on_codes(policy, codes) -> Policy
  retry::run(policy, closure)    -> Dynamic  (closure's return value)

NOTE: the brief specified retry::with(...), but `with` is a Rhai
reserved keyword (parse error before registration even matters). `call`
is also reserved. Shipping retry::run instead — flagged in HANDBACK §7.

Policy:
  - max_attempts clamped to [1, 20]
  - base_ms clamped to [1, 60_000]
  - jitter_pct clamped to [0, 100]
  - backoff ∈ {"exponential" | "linear" | "constant"} — bogus values
    surface a clear error
  - on_codes is an empty default ("retry any throw"); when non-empty,
    only error messages containing one of the codes retry — others
    surface immediately

retry::run uses NativeCallContext + FnPtr::call_within_context to
re-invoke the closure inside the caller's engine. Closures share the
caller's cx (Rhai semantics). If the closure calls invoke(), the inner
call gets a fresh cx with trigger_depth + 1 via the invoke bridge —
retry doesn't see or interfere with that.

Sleep between attempts: tokio::time::sleep through TokioHandle::block_on.
Safe because we're already inside the caller's spawn_blocking thread.

register_all gains retry::register positionally between queue and
secrets.

Unit tests (7): policy default, max_attempts clamping (0→1, 999→20),
base_ms + jitter clamping, bogus backoff reject, exponential doubles,
linear sums, constant.

Integration tests (sdk_retry.rs, 6 binaries):
  - succeeds first try (returns 42)
  - max_attempts exhausted surfaces last error ("boom")
  - on_codes filters — non-matching error throws immediately
  - jitter clamping is silent (script still runs)
  - bogus backoff string surfaces a clear error
  - "fails 2 then succeeds" — counter mutates across retries, 3rd
    attempt returns 3 (validates Rhai closure-capture semantics across
    re-invocations through NativeCallContext)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-06 22:32:06 +02:00
MechaCat02
302c1df577 feat(v1.1.9): invoke:: Rhai SDK module — sync re-entry + invoke_async
executor-core/sdk/invoke.rs adds `invoke()` and `invoke_async()` as
top-level Rhai helpers (no `::` namespace, mirroring the brief):

  invoke(target, args)        -> Dynamic  (sync, returns callee's value)
  invoke_async(target, args)  -> String   (fire-and-forget; returns execution_id)

Sync re-entry pattern:
  - bridge captures Arc<Engine> via Engine::self_arc()
  - calls engine.execute(&source, req) directly — same engine instance,
    same Services, same Limits; only SdkCallCx changes per call
  - runs inside the caller's spawn_blocking thread (no nested
    spawn_blocking, no gate re-admission — the outer execution already
    holds a permit)

Back-reference plumbing:
  - Engine gains self_weak: OnceLock<Weak<Engine>> + set_self_weak() +
    self_arc() accessor
  - picloud binary calls engine.set_self_weak(Arc::downgrade(&engine))
    right after construction
  - register_all extended with limits + Option<Arc<Engine>> params
  - Engine::execute_ast threads both through

Limits.trigger_depth_max added — mirrors TriggerConfig::max_trigger_depth.
Default 8; the picloud binary syncs from TriggerConfig::from_env() so
PICLOUD_MAX_TRIGGER_DEPTH governs both the dispatcher's fan-out cap and
the invoke depth bound.

Target parsing (string-first):
  - "/api/foo"             → InvokeTarget::Path
  - 36-char UUID-like      → InvokeTarget::Id  (uuid::Uuid parse-validated)
  - everything else        → InvokeTarget::Name

Cross-app + depth + FnPtr guards:
  - resolve() returns InvokeError::CrossApp if resolved.app_id != cx.app_id
  - bridge throws "invoke: depth limit exceeded (max N)" when
    cx.trigger_depth + 1 > limits.trigger_depth_max (checked BEFORE
    resolve to avoid a wasted DB round-trip)
  - args_to_json rejects FnPtr at any depth — closures don't survive
    invoke boundaries

invoke_async path:
  - calls services.invoke.enqueue_async() which writes an
    OutboxSourceKind::Invoke row (commit 10 wires the dispatcher arm)
  - returns the new ExecutionId as a string for caller tracking

Integration tests (sdk_invoke.rs, 6 binaries):
  - sync invoke returns callee's value (script returns body.x + 1)
  - cross-app invoke rejected (target in other app)
  - depth limit exceeded (recursive script that calls itself; throws
    before stack overflow)
  - callee receives incremented depth in cx (smoke — strict assertion
    needs ctx.trigger_depth surface, deferred to v1.2)
  - args_to_json rejects FnPtr in payload
  - invoke_async returns valid UUID string + queues args payload

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-06 19:56:05 +02:00
MechaCat02
36bf046791 feat(v1.1.9): InvokeService trait + InvokeTarget + ScriptRepository::get_by_name
shared/invoke.rs introduces the InvokeService trait + InvokeTarget enum:

  - InvokeTarget::Path(String)  — resolves via the route trie
  - InvokeTarget::Name(String)  — (cx.app_id, name) -> script_id via get_by_name
  - InvokeTarget::Id(ScriptId)  — direct lookup

InvokeService::resolve enforces same-app at the service layer
(InvokeError::CrossApp). InvokeService::enqueue_async writes a v1.1.9
OutboxSourceKind::Invoke row for fire-and-forget composition.

Errors: NotFound, CrossApp, DepthExceeded(u32), Forbidden, Rejected,
Unavailable. NoopInvokeService surfaces all calls as Unavailable for
harnesses that don't wire the service.

ScriptRepository gains get_by_name(app_id, name) backed by the existing
(app_id, name) uniqueness constraint. Postgres impl + in-memory mock +
the picloud crate's PostgresScriptRepoHandle delegate added.

Services::new gains invoke: Arc<dyn InvokeService> positionally after
queue. with_noop_services wires NoopInvokeService. 12 test sites
threaded through. picloud/lib.rs binds NoopInvokeService at this commit
boundary — the real InvokeResolver lands in commit 8.

Deviation from plan (flagged for HANDBACK §7): the plan called for
moving ExecutorClient + ScriptIdentity from orchestrator-core to shared
so the invoke bridge could call a re-entrant variant. Re-evaluated:
the bridge lives in executor-core which can hold Arc<Engine> directly,
making the trait move unnecessary. Engine::execute is sync, returns
ExecResponse, and shares the engine instance + per-call SdkCallCx
exactly as required. AST cache reuse across invokes is a v1.2 optimization
(invokes recompile each call — ms-scale cost, acceptable).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-06 19:44:43 +02:00
MechaCat02
cae2269932 feat(v1.1.9): queue:: Rhai SDK module
executor-core/sdk/queue.rs registers the queue:: namespace into the
per-call Rhai engine:

  queue::enqueue("name", message)               -> ()
  queue::enqueue("name", message, opts)         -> ()  // opts: Map
  queue::depth("name")                          -> i64
  queue::depth_pending("name")                  -> i64

No handle pattern (queues are the grouping unit, mirroring pubsub).
Opts map accepts optional delay_ms (i64) and max_attempts (u32);
unknown keys are silently ignored to match Rhai's Map semantics.

Bridge details:
- captures Arc<dyn QueueService> + Arc<SdkCallCx> per closure (cheap
  clone)
- block_on via TokioHandle::try_current() — same pattern pubsub/kv use
- message → JSON via a local message_to_json that base64-encodes Blobs
  at any depth (mirrors pubsub) and REJECTS FnPtr / closures with a
  clear error (queue messages must survive Postgres round-trip)
- depth / depth_pending clamp u64 → i64::MAX (Rhai's INT is i64)

register_all gains queue::register(...) positionally after pubsub.

Unit tests (executor-core --lib): blob → base64, nested-map round-trip,
FnPtr rejection, opts parsing (partial / max_attempts only / non-integer
delay rejection).

Integration tests (executor-core/tests/sdk_queue.rs, 6 binaries):
enqueue map payload, enqueue with opts threads through, blob base64
encodes, depth returns service value, FnPtr in payload throws, empty
queue_name throws.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-06 19:33:06 +02:00
MechaCat02
73e19ca626 feat(v1.1.9): QueueService trait + Postgres impl + Services bundle wiring
- shared/services.rs: Services::new gains queue: Arc<dyn QueueService>
  positionally after users (mirrors v1.1.8's append of users).
  with_noop_services adds NoopQueueService.
- manager-core/queue_service.rs: QueueServiceImpl wraps QueueRepo with
  script-as-gate authz on AppQueueEnqueue. enqueue clamps max_attempts
  to [1,20] and delay_ms to [0, 86_400_000ms]. depth/depth_pending are
  read-only — no authz check (scripts in the app can see their own
  queue depths). cx.principal threads through as enqueued_by_principal
  (forensic only).
- manager-core/authz.rs: AppQueueEnqueue(AppId) capability — script:write
  scope, granted to editor+ (same trust shape as AppPubsubPublish).
- picloud/lib.rs: wires PostgresQueueRepo + QueueServiceImpl into
  Services::new alongside the existing v1.1.7+ services.
- 11 sdk test binaries + manager-core/realtime_authority.rs updated to
  pass NoopQueueService to Services::new.

Unit tests cover empty queue_name reject, max_attempts clamping
(0/21 → invalid), delay_ms negative-reject, anonymous principal skips
authz, depth/depth_pending pass-through.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-06 19:30:34 +02:00
MechaCat02
9ce3c4c704 feat(v1.1.9): TriggerEvent::Queue + TriggerKind::Queue + types
Add the data-shape pieces for v1.1.9's queue surface; behavior lands in
later commits. Each piece compiles independently.

- shared/trigger_event.rs: TriggerEvent::Queue variant (queue_name,
  message, enqueued_at, attempt, message_id). source() returns "queue".
  Surfaced to scripts as ctx.event.queue with op = "receive".
- manager-core/trigger_repo.rs: TriggerKind::Queue + TriggerDetails::Queue +
  CreateQueueTrigger + ActiveQueueConsumer + QueueDetailRow + QueueConsumerRow.
  PostgresTriggerRepo gains create_queue_trigger (advisory-lock-then-SELECT
  enforces one-consumer-per-queue), list_active_queue_consumers (dispatcher
  scan), touch_queue_trigger_last_fired_at. hydrate_one hydrates the Queue arm.
- manager-core/outbox_repo.rs: OutboxSourceKind::Invoke for invoke_async()
  outbox rows.
- manager-core/dispatcher.rs: placeholder OutboxSourceKind::Invoke arm (logs +
  drops the row) so the workspace compiles; real arm lands in commit 10.
- manager-core/trigger_config.rs: queue_reclaim_interval_ms (30000) +
  queue_default_visibility_timeout_secs (30) env-overridable knobs.
- executor-core/engine.rs: trigger_event_to_dynamic handles Queue → builds
  ctx.event.queue map.
- manager-core/triggers_api.rs: in-memory mock TriggerRepo gains the three
  new methods (returns Default-ish values for tests).

Unit tests: TriggerEvent::Queue serde round-trip, TriggerKind::Queue wire
round-trip, advisory_lock_key stability per (app_id, queue_name) pair.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-06 19:08:21 +02:00
MechaCat02
7027e0dfb8 chore(v1.1.8): cargo fmt --all (reviewer)
8 files needed re-wrapping after the F2 clippy-fix pass landed
un-fmt'd. Pure cosmetic — no behavioral change. Re-runs of
`cargo fmt --all -- --check` now exit clean.

Reviewer-supplied per REVIEW.md §7; folded onto the branch
to avoid a NEEDS-CHANGES round-trip on a purely mechanical fix.
2026-06-06 18:44:52 +02:00
MechaCat02
7610a16a0b chore(v1.1.8): clippy --all-targets clean (F2)
Sweep the v1.1.8-introduced clippy warnings under
--workspace --all-targets --all-features -- -D warnings:

  * 12x map_unwrap_or in executor-core/sdk/users.rs — rewrite as
    map_or; mostly the Option<X> -> Dynamic conversion shape.
  * Doc-list-without-indentation in
    app_user_password_reset_repo.rs's module docstring — rewrap
    so a continuation line doesn't start with `+ `.
  * usize-as-i64 cast in app_user_repo.rs list — use
    i64::try_from(...).unwrap_or(i64::MAX) for the cursor cap.
  * 2x map_unwrap_or in users_service.rs env helpers — map_or.
  * single-pattern match in users_service.rs login — let-else
    rewrite so the dummy-Argon2 side-effect stays in the
    diverging branch (no semantic change).

Also updated the 10 executor-core integration-test bins
(sdk_email, sdk_kv, sdk_docs, sdk_files, sdk_pubsub,
sdk_secrets, sdk_http, sdk_subscriber_token,
module_redaction_logging, modules) to pass the new
NoopUsersService positional arg into Services::new().

NOTE — the brief specifies running this attestation under `cargo
clean` first. I skipped the clean step because an earlier
`cargo test --workspace` froze this host (the user explicitly
asked me to keep subsequent builds lighter). The incremental
cache was hot for every workspace crate when clippy ran; test
binaries appeared in the Checking output as expected. Flagged in
HANDBACK §6 F2 + §7.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-06 18:03:51 +02:00
MechaCat02
c855739559 feat(v1.1.8): email verification flow (migration 0028 + SDK)
migration 0028: app_user_email_verifications table — token_hash PK,
app_id + user_id FKs cascading, expires_at, consumed_at. Single-use
via atomic UPDATE WHERE consumed_at IS NULL.

UsersServiceImpl gains:
  * verifications: Arc<dyn AppUserVerificationRepo>
  * email: Arc<dyn EmailService>

users::send_verification_email(user_id, opts) mints a 32-byte token,
stores SHA-256(token), and calls EmailService::send with the body
template's {link} placeholder substituted by link_base + ?token=raw.
EmailError::NotConfigured propagates as UsersError::EmailNotConfigured
so scripts already handling email-disabled mode (v1.1.7 email::send)
don't need new branches.

users::verify_email(token) atomically consumes the one-shot token via
the verifications repo, marks the user's email_verified_at = NOW(),
and emits a "users::email_verified" event for future triggers.

Internal email send uses a synthesized SdkCallCx with principal=None
so the email-service AppEmailSend authz check is skipped (the
users::* surface has already gated on AppUsersWrite — the internal
hop isn't the script's direct call). Documented in HANDBACK §7.

EmailTemplateOpts now requires `from` (the v1.1.7 email service needs
an envelope sender). The brief example omitted it; deviation logged
in HANDBACK §7.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-06 12:04:09 +02:00
MechaCat02
36e5c5041a feat(v1.1.8): users::* SDK module + register hook
Rhai bridge for the v1.1.8 users::* namespace, wired through
sdk::register_all. Collection-less surface (mirrors email::/secrets::,
not kv::'s handle pattern) — app_id never appears on the script-side
signature; the service derives it from cx.app_id.

Eighteen functions bound:

  * CRUD: users::create / get / find_by_email / update / delete / list
  * Auth: users::login (returns session token string or ()),
          users::verify (returns user map or ()),
          users::logout
  * Email-tied: users::send_verification_email / verify_email /
          request_password_reset / complete_password_reset /
          invite / accept_invite (two arities: with/without
          display_name override)
  * Roles: users::add_role / remove_role / has_role

Bindings for methods whose service impl isn't in place yet (email
flows, roles, invitations) still route to the trait — the service
returns UsersError::Backend("not yet implemented") which surfaces
as a Rhai runtime error. Later v1.1.8 commits replace the service
stubs without re-touching the bridge.

User map shape: id, email, display_name, email_verified_at,
last_login_at, created_at (rfc3339), updated_at (rfc3339), roles (Vec).
Never password_hash. list returns #{ users: [...], next_cursor }
where next_cursor is the rfc3339 timestamp of the last row.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-06 11:59:54 +02:00
MechaCat02
2ea47eb05a chore(v1.1.7): fix clippy --all-targets warnings
Clears the workspace under `clippy --all-targets --all-features
-D warnings`. Four were pre-existing at v1.1.6 HEAD (latent finding,
see HANDBACK): double_must_use on realtime_router, map_unwrap_or in
pubsub_service, redundant_closure in topic_repo, needless_raw_string in
a subscriber-token test. The rest are v1.1.7 nits (needless_borrow +
semicolon in the dead-letter / realtime-migration code).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 22:38:11 +02:00
MechaCat02
1f78937dd2 feat(v1.1.7-email-inbound): webhook receiver + email:receive trigger
Inbound email: a provider POSTs a normalized JSON message to
POST /api/v1/email-inbound/{app_id}/{trigger_id}; the public receiver
verifies the optional HMAC signature, builds a TriggerEvent::Email, and
enqueues an outbox row the dispatcher delivers like any async trigger.
Handlers see ctx.event.email = #{from,to,cc,subject,text,html,
received_at,message_id}.

- migration 0024: widen triggers.kind + outbox.source_kind CHECKs to
  'email'; new email_trigger_details table.
- TriggerKind::Email, TriggerDetails::Email{has_inbound_secret},
  OutboxSourceKind::Email, TriggerEvent::Email; dispatcher routes the
  email row via the generic resolve_trigger path.
- Admin POST /apps/{id}/triggers/email (validate_trigger_target; module
  + cross-app rejection). inbound_secret is stored ENCRYPTED via the
  master key (deviation from the brief's plaintext default; decrypted
  per inbound request — see HANDBACK §7).
- Dashboard: email trigger form on the Triggers tab + webhook URL +
  expected-payload help.
- 8 DB-gated e2e tests (202/401/404/422/cross-app/handler-fire) +
  receiver unit tests (HMAC verify, secret round-trip, payload parse).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 22:24:35 +02:00
MechaCat02
8f2d2bc721 feat(v1.1.7-email-outbound): SMTP send/send_html
Outbound email reachable from scripts as email::send(#{...}) (plain
text) and email::send_html(#{...}) (multipart text + HTML). Backed by a
lettre SMTP relay configured from PICLOUD_SMTP_HOST/PORT/USER/PASSWORD/
TLS/TIMEOUT_SECS; if HOST/USER/PASSWORD aren't all set the service runs
in disabled mode (every send throws NotConfigured, warned at startup).

- EmailService trait + OutboundEmail DTO (picloud-shared);
  EmailServiceImpl + EmailTransport seam + lettre transport
  (manager-core), wired into the Services bundle and Rhai engine.
- Capability::AppEmailSend (→ script:write); seven-scope commitment held.
- Required-field + RFC5322-ish address validation; 25 MB per-message cap
  (PICLOUD_EMAIL_MAX_MESSAGE_BYTES). reply_to defaults to from.
- Per-call connection (pooling deferred to v1.2); no per-app from
  validation (operator's SMTP/SPF/DKIM concern).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 21:47:46 +02:00
MechaCat02
2d11090d1a feat(v1.1.7-secrets): secrets SDK + table + admin API + dashboard
Encrypted per-app secrets, reachable from scripts as
secrets::{get,set,delete,list}(name) and managed from the dashboard
Secrets tab. Values are AES-256-GCM-sealed with the process master key
(picloud_shared::crypto) before they touch Postgres; the repo only ever
sees ciphertext + nonce. JSON round-trip preserves Rhai types.

- migration 0023_secrets.sql (PRIMARY KEY (app_id, name)).
- SecretsService trait (picloud-shared) + SecretsServiceImpl + repo
  (manager-core), wired into the Services bundle and Rhai engine.
- Capability::AppSecretsRead/Write (→ script:read / script:write); no
  new Scope variants (seven-scope commitment).
- Admin API GET/POST/DELETE /apps/{id}/secrets (list returns names +
  updated_at, never values).
- build_app now takes a MasterKey, sourced from PICLOUD_SECRET_KEY in
  main.rs; test callers pass a fixed test key.
- 64 KB value cap (PICLOUD_SECRET_MAX_VALUE_BYTES); no ServiceEvent
  emission (secret writes don't fire triggers, by design).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 21:37:17 +02:00
MechaCat02
fcbcc576a2 feat(v1.1.6): realtime channels + v1.1.5 follow-ups + version bumps
Server-side realtime SSE on per-app pub/sub topics, plus the three
v1.1.5 follow-ups and the version bumps.

Realtime:
- topics registry (0021) + admin endpoints + Capability::AppTopicManage
  (-> app:admin; no new scope).
- GET /realtime/topics/{topic} SSE endpoint (orchestrator-core data
  plane): Host -> app, RealtimeAuthority gate (404 missing/internal,
  401 bad/absent token), broadcast::Receiver stream + heartbeat.
- RealtimeBroadcaster / RealtimeEvent / RealtimeAuthority traits
  (picloud-shared); InProcessBroadcaster + GC (orchestrator-core);
  DB-backed RealtimeAuthorityImpl (manager-core). Publish path fans out
  to in-process subscribers after the durable outbox commit (best-effort,
  panic-isolated).
- HMAC subscriber tokens (subscriber_token.rs) + app_secrets table (0022)
  + pubsub::subscriber_token SDK (schema 1.6 -> 1.7). TTL clamp + env
  overrides.
- Dashboard Topics tab (register/list/edit/delete, prominent external
  badge, flip confirmation).

v1.1.5 follow-ups:
- Empty blobs accepted (NewFile/FileUpdate::validate) + round-trip test.
- Orphan *.tmp.* sweeper (spawn_files_orphan_sweep).
- Dispatcher e2e tests, one per trigger kind (DATABASE_URL-gated).

Versions: workspace 1.1.6, SDK 1.7, dashboard 0.12.0. Schema-snapshot
golden re-blessed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 20:18:50 +02:00
MechaCat02
834c787ee1 feat(v1.1.5): pubsub::publish_durable SDK + pubsub:* triggers
Durable pub/sub through the universal outbox — the sixth trigger kind.

- `pubsub::publish_durable(topic, message)` Rhai SDK (no handle; topics
  ARE the grouping unit). Message JSON-encoded; Blobs base64 at any
  depth.
- `PubsubService` trait in picloud-shared with the topic matcher +
  validator (exact / `<prefix>.*` / `*`; mid-pattern wildcards
  rejected). `PostgresPubsubRepo` + `PubsubServiceImpl` in manager-core.
- Publish-time fan-out: one outbox row per matching enabled pubsub
  trigger, all in ONE transaction (no half-fan-out on crash). No
  matching trigger → publish succeeds silently, zero rows.
- `pubsub:*` trigger kind via Layout-E (0020: widen both CHECKs +
  pubsub_trigger_details + partial index), TriggerEvent::Pubsub +
  ctx.event.pubsub, dispatcher arm, admin endpoint POST /triggers/pubsub
  (validates topic pattern + reuses validate_trigger_target).
- AppPubsubPublish capability → script:write (seven-scope held).
- Dashboard Pub/Sub trigger form on the Triggers tab + list rendering.

publish_ephemeral stays deferred to v1.2. ~18 new tests (service
in-memory incl. transactional-rollback, shared matcher, bridge
encoding). No DB required for the suite.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-03 21:37:06 +02:00
MechaCat02
6e132b6ee0 feat(v1.1.5): files SDK + files:* triggers
Filesystem-backed blob storage as the fifth concrete trigger kind.

- `files::collection(c).{create,head,get,update,delete,list}` Rhai SDK
  (blob in/out; metadata maps; missing-field throws naming the field).
- `FilesService` trait in picloud-shared; `FsFilesRepo` (atomic
  write: temp→fsync→rename→fsync-dir→DB; single-pass SHA-256;
  checksum-verified reads → Corrupted) + `FilesServiceImpl` in
  manager-core. Metadata in Postgres (0018), bytes on disk under
  PICLOUD_FILES_ROOT with 0o700 shard dirs.
- `files:*` trigger kind via the Layout-E pattern (0019: widen both
  CHECKs + files_trigger_details), TriggerEvent::Files (metadata only,
  no bytes), emit_files fan-out, dispatcher arm, admin endpoint
  POST /triggers/files (reuses validate_trigger_target).
- AppFilesRead/AppFilesWrite capabilities → script:read/script:write
  (seven-scope commitment held). AppPubsubPublish reserved for v1.1.6.
- Admin files API (list + delete) + dashboard Files view per app.

Cross-app isolation keyed on cx.app_id at every layer. ~45 new tests
(service in-memory, fs tempdir, bridge integration). No DB required
for the suite. publish_ephemeral and the orphan sweep stay deferred.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-03 21:18:17 +02:00