`trigger_depth` bounds how DEEP a trigger/invoke chain runs, but nothing bounded
how WIDE one execution could fan out. A single anonymous request running
`for i in 0..1_000_000 { invoke_async("w", #{}) }` costs a few Rhai ops plus one
cheap outbox INSERT per iteration, so within the op / wall-clock budget it could
write ~10^5 durable rows — each dispatched as its own execution — flooding the
outbox and dispatcher (a durable-amplification DoS).
Add a per-execution ceiling on DURABLE emissions (`invoke_async`,
`pubsub::publish_durable`, `queue::enqueue`, and the shared-topic/-queue
variants), env `PICLOUD_MAX_EMISSIONS_PER_EXECUTION` (default 1000). It's a
thread-local counter wrapped by a re-entrancy-aware `EmissionBudgetScope` around
every `execute_ast`: the OUTERMOST scope resets it, so a fresh dispatched
handler (a new pooled-thread task) gets a full budget while a synchronous
`invoke()` / interceptor re-entry nested in the same call stack SHARES it (fan-
out counted across the whole synchronous chain). The outermost Drop re-zeroes
the counter so a pooled thread never leaks a count into the next task.
Pinned by an `emit_budget` unit test (outermost resets, nested shares, ceiling
trips); the invoke/queue/pubsub/workflow journeys (small counts) stay green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The per-element Rhai sandbox caps (max_string_size 64 KiB, max_array_size /
max_map_size 10 000) do NOT bound the *materialized* size: Rhai shares strings
and arrays by Rc, so a 10 000-element array of one aliased 64 KiB string is
cheap to build (~10 000 ops, far under the 1 M op budget) yet dealiases to
~640 MiB of distinct JSON. Every Dynamic→JSON conversion deep-copies the
aliases; the HTTP response body path had NO size cap at all, and the KV/docs
value caps run only AFTER full materialization — so a handful of anonymous
requests could OOM the node (32 concurrent × ~640 MiB ≈ 20 GiB) on the stated
consumer-hardware target.
Add a byte-budgeted materializer that bails the moment the budget is exceeded,
so the transient allocation is bounded regardless of aliasing:
- bridge.rs: `dynamic_to_json_capped(value, max) -> Result<Json, JsonSizeError>`
(charges a running budget) + `MAX_JSON_MATERIALIZE_BYTES` (16 MiB hard rail,
far above the 256 KiB business caps). `dynamic_to_json` stays infallible for
best-effort paths (logging) but is now internally bounded — it collapses an
over-limit value to a marker string instead of OOMing.
- Route every user-value boundary through the fail-closed capped form: KV
set/set_if (+ the CAS `expected`), docs create/update/find, secrets set,
http request body, workflow input, `json::stringify`, and the HTTP RESPONSE
body (previously the one fully-uncapped exit → now a 500). `invoke` args and
the pubsub/queue message materializers get the same byte budget in place.
- `impl From<JsonSizeError> for Box<EvalAltResult>` so SDK sites protect
themselves with a bare `?`.
Pinned by bridge unit tests: an aliased 10 000×64 KiB array errors on the
budget rather than materializing, and the infallible wrapper yields a marker
instead of OOMing. Workspace 914 + journeys 157/157 green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
An end-to-end audit of the (unmerged) interceptor slice surfaced seven ways a
KV allow/deny guard could be silently defeated or misbehave. Close all of them:
1. Shared-collection bypass — `kv::shared_collection(...).set/delete` skipped
the hook entirely, so any `(kv, set/delete)` guard was circumvented by
choosing the shared handle. `GroupKvHandle` now runs the same before-op hook.
2. Seal to the declaring owner — the marker resolved nearest-owner-wins but its
script name was then re-resolved on the CALLER's chain, so a descendant app
could shadow a group's mandatory guard with a same-named local script.
`resolve_before` now returns the script sealed to the owner that declared the
marker (one LEFT JOIN in manager-core), so the executor never re-resolves by
name. A descendant can only override with its OWN explicit marker.
3. Re-entrancy — an "allow + audit-log" guard that itself wrote KV re-triggered
itself to the depth cap and then DENIED the original write (plus ~8x
resolve/compile/execute amplification). A thread-local guard makes a write
performed by an interceptor bypass interception.
4. Fail-closed verdict — only `#{ allowed: false }` denied; a bare bool, a
typo'd key, a non-bool, or a bare unit all ALLOWED. Now allow ONLY on an
explicit `#{ allowed: true }`; every other shape denies.
5/6. Fail-closed edges — a dangling (missing/disabled) interceptor script and a
missing engine back-reference now DENY instead of allowing.
7. AppInvoke coupling — resolution no longer routes through `invoke.resolve`, so
installing a guard no longer denies writes to principals who hold KV-write
but not AppInvoke.
The seam (`InterceptorService::resolve_before`) now returns an
`InterceptorResolution { None | Dangling | Run(ResolvedScript) }`; the executor
runs the sealed script straight through the shared `run_resolved_blocking` core
and drops its `InvokeService` dependency. executor-core stays Postgres-free.
No migration change (0073 unchanged).
Pinned by three new journeys in tests/interceptors.rs: the seal (a same-named
app script does NOT shadow the group's guard), shared-collection coverage, and
the fail-closed verdict. Full journey suite 157/157.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Smallest honest vertical slice of §9.4: a `[[interceptors]]` block (app OR
group) binds a script to run BEFORE `kv::set`/`delete`; it reads the operation
context (`ctx.request.body`: service, action, collection, key, value, caller
ids) and returns `#{ allowed, reason }` — `allowed == false` denies the op (the
write never runs, the caller gets a runtime error).
Reuses two existing mechanisms rather than inventing new ones:
- Registration mirrors extension points (§5.5): a marker table
`0073_interceptors.sql` (owner-polymorphic app_id/group_id XOR, keyed
(service, op) → script), `interceptor_repo` (insert/delete/list + the
nearest-owner-wins `resolve_before` chain walk), reconciled through the
declarative apply exactly like `vars` (create/update/delete, prunable).
- Execution reuses the `invoke()` re-entry path: the new `InterceptorService`
(shared trait + Postgres-backed impl) only RESOLVES the script name (keeping
executor-core Postgres-free); the executor's `sdk::interceptor::run_before`
resolves that name and runs it via `run_resolved_blocking` (extracted from
`invoke_blocking` — shared depth bound + AST cache). An un-hooked write pays
one indexed `Ok(None)` resolve; no interceptor ⇒ zero overhead.
Nearest-owner-wins so an app overrides a group's interceptor, and a group
interceptor is inherited by every descendant app — the chain walk is the
isolation boundary (a sibling subtree never matches). `validate_bundle_for`
restricts the MVP to `service = "kv"`, `op ∈ {set, delete}`, one marker per
(service, op).
Deferred (documented in §9.4): the `data` transform return, services other
than kv, `after_*` hooks, chaining + circular-dependency guard, the timeout
policy, and a `pic interceptors ls` read surface (needs a server route).
Pinned by `tests/interceptors.rs` (deny blocks the write; allow passes;
group→app inheritance), schema snapshot re-blessed. 154/154 journeys pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`http.rs` hand-rolled its own `block_on` + `map_http_err`, and `secrets.rs` /
`email.rs` / `users.rs` each carried a byte-identical `runtime_err` — five
`#[allow(clippy::unnecessary_box_returns)]` across four files for one pattern.
Move `runtime_err` to `bridge.rs` beside `block_on` (single home, single allow)
and have the three bridges import it. `http.rs` now calls the shared
`bridge::block_on("http", …)` (same "http:"-prefixed error, so messages are
unchanged) and its `err` validation helper delegates to `bridge::runtime_err`.
Deletes 4 duplicate fns and 3 of the 5 allows; no behavior change (pinned by
the executor-core SDK contract tests).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The user-facing surface for the durable DAG engine. A workflow run can now be
started three ways, all resolving the workflow by name in the caller's app
(`cx.app_id` / the resolved app — never a passed-in arg, the isolation
boundary), seeding a run the orchestrator advances.
SDK seam (shared):
- `WorkflowService` trait + `NoopWorkflowService` + `WorkflowError` in
`shared::workflow` (mirrors `InvokeService`); added to `Services` as a
noop-defaulted `with_workflow` builder (all `Services::new` call sites
unchanged).
- `WorkflowServiceImpl` (manager-core): resolves + seeds a run; authenticated
callers gated on `AppInvoke` (anonymous skips, script-as-gate), the same gate
`invoke()` uses.
- `executor-core::sdk::workflow` — the Rhai bridge `workflow::start(name, input)`
/ `workflow::start(name)`, registered in `register_all`. Returns the run id.
Admin API (manager-core `workflows_api`, mounted in the binary):
- `GET /apps/{app}/workflows` — definitions (AppRead)
- `POST /apps/{app}/workflows/{name}/runs` — start a run (AppInvoke)
- `GET /apps/{app}/workflows/{name}/runs` — run history (AppRead)
- `GET /apps/{app}/workflow-runs/{run_id}` — one run + its steps (AppRead)
`app_id` scopes every query; a foreign run 404s.
CLI (`pic workflows`): `ls` · `run <name> [--input JSON]` · `runs <name>` ·
`run-status <run_id>` — all `--app`-scoped; run definitions stay declarative
(`[[workflows]]` → `pic apply`).
Also: `list_runs_for_workflow` repo reader.
Tests: `workflow_service_start_seeds_a_run` (DB-gated — the SDK/API-shared
entry) + `workflow_run_executes_end_to_end` CLI journey (apply → `pic workflows
run` → poll `run-status` → succeeded, through the real binary + orchestrator).
fmt + clippy -D warnings clean, 449 manager-core lib tests, 14 orchestrator DB
tests, 18 executor-core lib tests, 3 workflow CLI journeys green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A five-agent roadmap-verification pass found the v1.2 Hierarchies code fully and
correctly implemented, but the three source-of-truth docs carried stale
"deferred/remaining/v1.3" notes — and one self-contradiction — for features that
have since shipped (§3 M3 server-side approval gate, §6 group create/reparent, the
Track A closeout, and the 2026-07-11 audit remediation). Reconcile all three to
reflect the true code; no behavior change.
CLAUDE.md: retract the inline "groups pre-exist" / "per-env approval deferred" /
"shared-topic SSE deferred" / "group dead-letter store deferred" / "email v0/no-AAD"
notes → mark §6, §3 M3, and Track A M1–M6 shipped; add per-group KV/docs byte
quotas + set_if; note migration 0063; refresh the "Out of MVP" section.
design doc: fix the header ("Phases 4–6 remain" → all shipped); resolve the §11.6
self-contradiction (the "Deferred" list still named shared-topic SSE / byte quotas /
set_if / operator admin API, all shipped — line 1317 already said SSE shipped); mark
the D3 dead-letter store shipped; retract the stale "groups pre-exist / §6 deferred"
forward references.
blueprint (comprehensive sweep): dashboard Alpine.js → SvelteKit + CodeMirror
(diagram, §3.3, tech table); Docker-per-execution → embedded in-process Rhai
(diagram + data flow); §12 Phase 4 "current focus" → shipped through v1.1.9; Phase 5
"in active development" + "Remaining" block → Hierarchies complete; per-app RBAC
"v1.3+" → shipped Phase 3.5; SDK reference → handle-pattern notation note, S3 tag
v1.1 → v1.3+; MVP schema + docker-compose flagged non-authoritative; §9 header noted
Hierarchies-shipped / Workflows-future; top status line refreshed.
Also fix two behavior-neutral stale in-code comments (sdk/kv.rs `kv::shared` →
`kv::shared_collection`; dispatcher.rs `q_terminal` "no group dead-letter store yet"
→ dead-letters to group_dead_letters, Track A M2).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
No atomic CAS existed, so concurrent writers to the same KV key raced. Add
set_if(key, expected, new): writes only when the current value equals expected,
or (expected absent) only when the key is missing — the primitive for lock-free
counters / optimistic concurrency.
- KvService + GroupKvService traits gain set_if with a defaulted "unsupported"
impl (Noop + other impls untouched); the real services override it.
- kv_repo/group_kv_repo: atomic set_if — a single UPDATE ... WHERE value =
$expected (JSONB semantic equality) or INSERT ... ON CONFLICT DO NOTHING.
Atomic without a transaction; returns whether the write happened.
- Services enforce the same value-size cap + (group) writes-authed + row/byte
quotas as set; the event fires only on a successful swap.
- Executor Rhai SDK: kv::collection(c).set_if(key, expected, new) and the
shared_collection variant; Rhai () for expected means "only if absent".
Pinned by kv_service/group_kv_service set_if unit tests (CAS + fail-closed) +
sdk_kv::kv_set_if_compare_and_swap (through a real script). No migration.
docs/sdk-shape.md documents the primitive.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The runtime for shared topics:
- pubsub_repo: fan_out_publish gains `AND t.shared = FALSE` (a shared
trigger never fires on a per-app publish); new fan_out_shared_publish
matches `shared = true` pubsub triggers on the resolved owning group,
each delivery stamped with the writer app_id (M2 model).
- GroupPubsubService trait (shared) + GroupPubsubServiceImpl: resolve the
owning group (kind='topic') from cx.app_id's chain, require editor+
(GroupPubsubPublish, fails closed for anon), size-cap, fan out.
- SDK: `pubsub::shared_topic("name")` -> GroupTopicHandle with
`.publish(subtopic, msg)` / `.publish(msg)`; wired through Services +
the picloud binary (reuses the one PubsubRepo).
- authz: Capability::GroupPubsubPublish(GroupId), editor+ / script:write.
The owning-group chain walk is the isolation boundary; a sibling-subtree
app never resolves the topic.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Finish the §11.6 KV slice: declarative authoring, read-only inspection, the
runtime journey, and docs — plus a forced SDK-name fix.
- SDK name: `shared` is a Rhai RESERVED keyword, so `kv::shared(...)` is a parse
error. Renamed the handle constructor to `kv::shared_collection(...)`
(keeps the user's "shared" intent; unambiguous). Updated everywhere.
- CLI manifest: `collections = [...]` on `[group]` only (ManifestApp has no such
field + deny_unknown_fields → an app manifest carrying it is a hard error);
Manifest::collections() accessor; build_bundle emits it.
- plan/apply: a `collection` row group in `pic plan`; created/deleted counts in
the apply summary; collections threaded through PlanDto/NodePlanDto/
ApplyReportDto.
- read-only `pic collections ls --group` → GET /admin/groups/{id}/collections
(viewer+), backed by ApplyService::collection_report + CollectionInfo.
- journey: a group declares `catalog`; app A writes, app B (same subtree) reads
it back; a sibling-subtree app gets CollectionNotShared; `collections ls` +
re-apply NoOp. Full suite 114/114.
- docs: groups-and-project-tool §11.6 (KV-only MVP shipped + deferrals),
sdk-shape.md (the shared-collection handle + trust model), CLAUDE.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The runtime for shared group collections. A script reaches a shared store via
the explicit kv::shared("name") handle (distinct GroupKvHandle Rhai type, so
private-vs-shared scope is a deliberate choice). The service resolves the
owning group from cx.app_id's ancestor chain — the script never names a group,
and that walk is the isolation boundary (a foreign app's chain never contains
the owning group → CollectionNotShared).
- shared: GroupKvService trait + GroupKvError + NoopGroupKvService; threaded
into the Services bundle as a field defaulted to noop, opted into via a new
with_group_kv() (keeps the ~14 Services::new call sites unchanged).
- manager-core: GroupKvServiceImpl with the reads-open/writes-authed split —
reads use script_gate (anonymous public scripts skip), writes use the new
script_gate_require_principal (anonymous fails closed). Owner resolution
behind a GroupCollectionResolver trait (Postgres impl + injectable fake);
unit tests cover off-chain CollectionNotShared, anonymous-read-OK/
anonymous-write-Forbidden, authed-no-role-Forbidden.
- executor-core: GroupKvHandle + kv::shared constructor + get/set/has/delete/
list (Rhai dispatches by receiver type, reusing the block_on bridge).
- picloud: wire PostgresGroupKvRepo + resolver + GroupKvServiceImpl.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Carry the resolved script's owner (its defining node) from dispatch into
the executor so `import` resolution can be lexical (§5.5). The executing
app (`app_id`) stays the SDK isolation boundary; `script_owner` is a
separate axis — the lexical origin for imports.
- `ExecRequest.script_owner: Option<ScriptOwner>` (serde default; `None`
falls back to `App(app_id)` in the engine for old payloads / cluster wire).
- `ScriptOwner` gains `Serialize`/`Deserialize` for the wire.
- The engine computes `default_origin` and hands it to the resolver
(used fully in C3; the resolve() lookup already uses it).
- Every dispatch site sets it from the resolved `Script.owner()`:
the 4 dispatcher arms (queue/trigger/http/invoke_async, via a new
`ResolvedTrigger.script_owner`), the orchestrator id-bypass, and the
`invoke()` SDK path (new `ResolvedScript.owner`, so a group script
invoked by an app resolves imports from the group).
Behaviour-preserving: app scripts still resolve `App(app_id)`; group
scripts can't yet carry imports (lifted in C4).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Scripts can now read group-inherited config via vars::get(key) / vars::all().
- shared: VarsService trait (read-only get/all) + NoopVarsService; added as
the 14th field on the Services bundle.
- manager-core: VarsServiceImpl resolves the calling app's config via
config_resolver (derives app_id from cx.app_id; AppVarsRead-gated for
authed principals, script-as-gate for anon).
- executor-core: vars:: Rhai bridge (get/all), registered in the SDK.
- authz: AppVarsRead/Write, GroupVarsRead/Write, GroupSecretsRead/Write
capabilities (group caps resolve via the Phase-2 ancestor walk; the
GroupSecretsRead human-read gate is group_admin-only, distinct from an
app's runtime injection).
- wired VarsServiceImpl into the picloud binary; updated the executor-core
test Services::new call sites.
386 manager-core lib tests green; clippy clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses findings from an independent review of the gap-closing commits:
- R1 (regression): add #[serde(default)] to HttpDispatchPayload.method so
async-HTTP outbox rows enqueued before the field existed still decode
after upgrade instead of dead-lettering on "missing field `method`".
Adds a regression test for the missing-key path.
- method case: uppercase ctx.request.method at the orchestrator boundary
so it honors its documented "uppercased" contract for extension verbs.
Route matching is already case-insensitive, so matching is unaffected.
- CLI hardening: percent-encode free-form path segments (app slug, topic
name, ids, secret name) in the reqwest client so a value containing
`/ ? #` can't break out of its URL segment. Adds a `seg()` helper +
unit test and applies it uniformly across all path-interpolating
methods (new and pre-existing).
- S1 (docs): correct the users::email_available enumeration framing — it
adds no new vector vs. create's uniqueness error, but is cheaper and
unthrottled; there is no built-in throttle/CAPTCHA primitive, so the
honest mitigation is a kv-based counter. Updated SDK doc-comments,
trait docs, and stdlib-reference.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`users::find_by_email` requires an authenticated principal (F-S-003,
anti-enumeration), so the natural check-then-create register pattern 502s
for anonymous public scripts. Add `users::email_available(email) -> bool`,
gated like `create` (the registration write path) but without the
anonymous rejection: it leaks only a single boolean — no more than
`create`'s own uniqueness error already does — so self-serve register
scripts can pre-check. Also document find_by_email's principal
requirement in the SDK doc-comment.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The request map exposed path/headers/body/params/query/rest but not the
HTTP method, forcing one script per verb (the E2E app needed 7 scripts
where ~3 would do). Add a `method` field to ExecRequest (populated from
the HTTP method on the sync-execute bypass and the HTTP outbox payload;
empty for non-HTTP triggers) and surface it as `ctx.request.method`,
uppercased. A single script can now branch GET/POST on one route.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
regex::is_match, find, find_all, replace, replace_all, split, captures
all called Regex::new(pattern) per invocation. A script doing
`regex::is_match("\\d+", x)` in a tight loop paid the compile every
iteration. Regex compile dominates is_match on short strings.
Add a thread-local LruCache<String, Arc<Regex>> (cap 128). The
compile helper now memoises by pattern string, returning Arc<Regex>
on hit. Cap is well above what any sensible script uses but bounded
enough to keep memory predictable on long-running threads.
AUDIT.md anchor: F-P-014.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ORDER BY created_at DESC, id DESC but cursor was `WHERE created_at <
$2` — when two users were created at the same instant, pagination
could skip the second row at a page boundary or return it twice.
- Add ListCursor { created_at, id } with `<rfc3339>_<uuid>` encode /
decode helpers.
- Change WHERE predicate to `(created_at, id) < ($2, $3)` matching the
ORDER BY.
- Surface the opaque cursor string through UsersListOpts /
UsersListPage / users_admin_api ListUsersResponse instead of the raw
DateTime — keeps the wire format stable while the id half stops the
boundary bug.
Same shape as F-P-005 (execution_logs).
AUDIT.md anchor: F-P-012.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two paths bypassed the AST cache: LocalExecutorClient::execute (tests +
fallback) and the synchronous invoke() re-entry in the executor SDK.
The latter is the hot one — composed workflows multiplied parse cost
by depth, so a 4-deep invoke chain on a 200-line script paid the parse
budget × 4 per call.
Add a per-Engine HashMap<ScriptId, (updated_at, Arc<AST>)> + a
`compile_for_identity(script_id, updated_at, source)` helper that
behaves like LocalExecutorClient::get_or_compile but lives on the
Engine. Update the SDK invoke synchronous re-entry to:
resolved → compile_for_identity → execute_ast
The orchestrator-core LocalExecutorClient cache (HTTP-path dispatch)
is left untouched — it caches a different access pattern at a
different boundary.
AUDIT.md anchor: F-P-004.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Whitespace + import-grouping fixups in the 9 SDK modules touched by
F-Q-002. No behavior change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace ten near-identical `block_on` helpers (one per SDK module)
with a single shared `sdk::bridge::block_on(service: &str, fut)`.
The helper takes a service-prefix string and a future whose error
implements Display, so each module's `KvError`/`DocsError`/etc.
still self-formats.
Migrated:
- kv, docs, pubsub, users, dead_letters, secrets, files, email
- queue.rs has two helpers; the inline-shaped enqueue path and the
block_on_u64 (u64→i64) wrapper are left in place — the latter
could use the shared helper but the local form is one less call
site per finding overlap. Will revisit on a later pass if needed.
- http.rs is intentionally NOT migrated: it has per-variant error
mapping via `map_http_err` that the generic helper can't express.
Net: -218 / +97 lines across the 9 migrated files. Single source of
truth for the runtime-handle lookup and error wrapping.
AUDIT.md anchor: F-Q-002.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
executor-core/sdk/retry.rs adds the retry:: namespace with:
retry::policy(opts) -> Policy
retry::on_codes(policy, codes) -> Policy
retry::run(policy, closure) -> Dynamic (closure's return value)
NOTE: the brief specified retry::with(...), but `with` is a Rhai
reserved keyword (parse error before registration even matters). `call`
is also reserved. Shipping retry::run instead — flagged in HANDBACK §7.
Policy:
- max_attempts clamped to [1, 20]
- base_ms clamped to [1, 60_000]
- jitter_pct clamped to [0, 100]
- backoff ∈ {"exponential" | "linear" | "constant"} — bogus values
surface a clear error
- on_codes is an empty default ("retry any throw"); when non-empty,
only error messages containing one of the codes retry — others
surface immediately
retry::run uses NativeCallContext + FnPtr::call_within_context to
re-invoke the closure inside the caller's engine. Closures share the
caller's cx (Rhai semantics). If the closure calls invoke(), the inner
call gets a fresh cx with trigger_depth + 1 via the invoke bridge —
retry doesn't see or interfere with that.
Sleep between attempts: tokio::time::sleep through TokioHandle::block_on.
Safe because we're already inside the caller's spawn_blocking thread.
register_all gains retry::register positionally between queue and
secrets.
Unit tests (7): policy default, max_attempts clamping (0→1, 999→20),
base_ms + jitter clamping, bogus backoff reject, exponential doubles,
linear sums, constant.
Integration tests (sdk_retry.rs, 6 binaries):
- succeeds first try (returns 42)
- max_attempts exhausted surfaces last error ("boom")
- on_codes filters — non-matching error throws immediately
- jitter clamping is silent (script still runs)
- bogus backoff string surfaces a clear error
- "fails 2 then succeeds" — counter mutates across retries, 3rd
attempt returns 3 (validates Rhai closure-capture semantics across
re-invocations through NativeCallContext)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
8 files needed re-wrapping after the F2 clippy-fix pass landed
un-fmt'd. Pure cosmetic — no behavioral change. Re-runs of
`cargo fmt --all -- --check` now exit clean.
Reviewer-supplied per REVIEW.md §7; folded onto the branch
to avoid a NEEDS-CHANGES round-trip on a purely mechanical fix.
Sweep the v1.1.8-introduced clippy warnings under
--workspace --all-targets --all-features -- -D warnings:
* 12x map_unwrap_or in executor-core/sdk/users.rs — rewrite as
map_or; mostly the Option<X> -> Dynamic conversion shape.
* Doc-list-without-indentation in
app_user_password_reset_repo.rs's module docstring — rewrap
so a continuation line doesn't start with `+ `.
* usize-as-i64 cast in app_user_repo.rs list — use
i64::try_from(...).unwrap_or(i64::MAX) for the cursor cap.
* 2x map_unwrap_or in users_service.rs env helpers — map_or.
* single-pattern match in users_service.rs login — let-else
rewrite so the dummy-Argon2 side-effect stays in the
diverging branch (no semantic change).
Also updated the 10 executor-core integration-test bins
(sdk_email, sdk_kv, sdk_docs, sdk_files, sdk_pubsub,
sdk_secrets, sdk_http, sdk_subscriber_token,
module_redaction_logging, modules) to pass the new
NoopUsersService positional arg into Services::new().
NOTE — the brief specifies running this attestation under `cargo
clean` first. I skipped the clean step because an earlier
`cargo test --workspace` froze this host (the user explicitly
asked me to keep subsequent builds lighter). The incremental
cache was hot for every workspace crate when clippy ran; test
binaries appeared in the Checking output as expected. Flagged in
HANDBACK §6 F2 + §7.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
migration 0028: app_user_email_verifications table — token_hash PK,
app_id + user_id FKs cascading, expires_at, consumed_at. Single-use
via atomic UPDATE WHERE consumed_at IS NULL.
UsersServiceImpl gains:
* verifications: Arc<dyn AppUserVerificationRepo>
* email: Arc<dyn EmailService>
users::send_verification_email(user_id, opts) mints a 32-byte token,
stores SHA-256(token), and calls EmailService::send with the body
template's {link} placeholder substituted by link_base + ?token=raw.
EmailError::NotConfigured propagates as UsersError::EmailNotConfigured
so scripts already handling email-disabled mode (v1.1.7 email::send)
don't need new branches.
users::verify_email(token) atomically consumes the one-shot token via
the verifications repo, marks the user's email_verified_at = NOW(),
and emits a "users::email_verified" event for future triggers.
Internal email send uses a synthesized SdkCallCx with principal=None
so the email-service AppEmailSend authz check is skipped (the
users::* surface has already gated on AppUsersWrite — the internal
hop isn't the script's direct call). Documented in HANDBACK §7.
EmailTemplateOpts now requires `from` (the v1.1.7 email service needs
an envelope sender). The brief example omitted it; deviation logged
in HANDBACK §7.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Rhai bridge for the v1.1.8 users::* namespace, wired through
sdk::register_all. Collection-less surface (mirrors email::/secrets::,
not kv::'s handle pattern) — app_id never appears on the script-side
signature; the service derives it from cx.app_id.
Eighteen functions bound:
* CRUD: users::create / get / find_by_email / update / delete / list
* Auth: users::login (returns session token string or ()),
users::verify (returns user map or ()),
users::logout
* Email-tied: users::send_verification_email / verify_email /
request_password_reset / complete_password_reset /
invite / accept_invite (two arities: with/without
display_name override)
* Roles: users::add_role / remove_role / has_role
Bindings for methods whose service impl isn't in place yet (email
flows, roles, invitations) still route to the trait — the service
returns UsersError::Backend("not yet implemented") which surfaces
as a Rhai runtime error. Later v1.1.8 commits replace the service
stubs without re-touching the bridge.
User map shape: id, email, display_name, email_verified_at,
last_login_at, created_at (rfc3339), updated_at (rfc3339), roles (Vec).
Never password_hash. list returns #{ users: [...], next_cursor }
where next_cursor is the rfc3339 timestamp of the last row.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Outbound email reachable from scripts as email::send(#{...}) (plain
text) and email::send_html(#{...}) (multipart text + HTML). Backed by a
lettre SMTP relay configured from PICLOUD_SMTP_HOST/PORT/USER/PASSWORD/
TLS/TIMEOUT_SECS; if HOST/USER/PASSWORD aren't all set the service runs
in disabled mode (every send throws NotConfigured, warned at startup).
- EmailService trait + OutboundEmail DTO (picloud-shared);
EmailServiceImpl + EmailTransport seam + lettre transport
(manager-core), wired into the Services bundle and Rhai engine.
- Capability::AppEmailSend (→ script:write); seven-scope commitment held.
- Required-field + RFC5322-ish address validation; 25 MB per-message cap
(PICLOUD_EMAIL_MAX_MESSAGE_BYTES). reply_to defaults to from.
- Per-call connection (pooling deferred to v1.2); no per-app from
validation (operator's SMTP/SPF/DKIM concern).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Encrypted per-app secrets, reachable from scripts as
secrets::{get,set,delete,list}(name) and managed from the dashboard
Secrets tab. Values are AES-256-GCM-sealed with the process master key
(picloud_shared::crypto) before they touch Postgres; the repo only ever
sees ciphertext + nonce. JSON round-trip preserves Rhai types.
- migration 0023_secrets.sql (PRIMARY KEY (app_id, name)).
- SecretsService trait (picloud-shared) + SecretsServiceImpl + repo
(manager-core), wired into the Services bundle and Rhai engine.
- Capability::AppSecretsRead/Write (→ script:read / script:write); no
new Scope variants (seven-scope commitment).
- Admin API GET/POST/DELETE /apps/{id}/secrets (list returns names +
updated_at, never values).
- build_app now takes a MasterKey, sourced from PICLOUD_SECRET_KEY in
main.rs; test callers pass a fixed test key.
- 64 KB value cap (PICLOUD_SECRET_MAX_VALUE_BYTES); no ServiceEvent
emission (secret writes don't fire triggers, by design).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Durable pub/sub through the universal outbox — the sixth trigger kind.
- `pubsub::publish_durable(topic, message)` Rhai SDK (no handle; topics
ARE the grouping unit). Message JSON-encoded; Blobs base64 at any
depth.
- `PubsubService` trait in picloud-shared with the topic matcher +
validator (exact / `<prefix>.*` / `*`; mid-pattern wildcards
rejected). `PostgresPubsubRepo` + `PubsubServiceImpl` in manager-core.
- Publish-time fan-out: one outbox row per matching enabled pubsub
trigger, all in ONE transaction (no half-fan-out on crash). No
matching trigger → publish succeeds silently, zero rows.
- `pubsub:*` trigger kind via Layout-E (0020: widen both CHECKs +
pubsub_trigger_details + partial index), TriggerEvent::Pubsub +
ctx.event.pubsub, dispatcher arm, admin endpoint POST /triggers/pubsub
(validates topic pattern + reuses validate_trigger_target).
- AppPubsubPublish capability → script:write (seven-scope held).
- Dashboard Pub/Sub trigger form on the Triggers tab + list rendering.
publish_ephemeral stays deferred to v1.2. ~18 new tests (service
in-memory incl. transactional-rollback, shared matcher, bridge
encoding). No DB required for the suite.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Filesystem-backed blob storage as the fifth concrete trigger kind.
- `files::collection(c).{create,head,get,update,delete,list}` Rhai SDK
(blob in/out; metadata maps; missing-field throws naming the field).
- `FilesService` trait in picloud-shared; `FsFilesRepo` (atomic
write: temp→fsync→rename→fsync-dir→DB; single-pass SHA-256;
checksum-verified reads → Corrupted) + `FilesServiceImpl` in
manager-core. Metadata in Postgres (0018), bytes on disk under
PICLOUD_FILES_ROOT with 0o700 shard dirs.
- `files:*` trigger kind via the Layout-E pattern (0019: widen both
CHECKs + files_trigger_details), TriggerEvent::Files (metadata only,
no bytes), emit_files fan-out, dispatcher arm, admin endpoint
POST /triggers/files (reuses validate_trigger_target).
- AppFilesRead/AppFilesWrite capabilities → script:read/script:write
(seven-scope commitment held). AppPubsubPublish reserved for v1.1.6.
- Admin files API (list + delete) + dashboard Files view per app.
Cross-app isolation keyed on cx.app_id at every layer. ~45 new tests
(service in-memory, fs tempdir, bridge integration). No DB required
for the suite. publish_ephemeral and the orphan sweep stay deferred.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
HTTP (`http::*`):
- `HttpService` trait (picloud-shared) + reqwest-backed `HttpServiceImpl`
(manager-core), wired into the `Services` bundle.
- SSRF deny-list applied to the resolved IP via a custom reqwest
`dns_resolver` (covers every redirect hop + defeats DNS rebinding) plus
a literal-IP check at URL-parse time. Scheme/port restrictions, request
+ response body caps (stream-with-cap), layered timeout. Error reason is
a CIDR category, never the IP. `PICLOUD_HTTP_ALLOW_PRIVATE` dev override
(logs a startup warning).
- Rhai bridge with three-arg split `verb(url, body, opts)` (resolves the
brief's body-vs-opts contradiction; unknown opt keys throw). Body
dispatch by type; response `#{status,headers,body,body_raw}` with JSON
auto-parse; non-2xx does not throw.
- `Capability::AppHttpRequest` → existing `script:write` scope (no new
Scope variant). `SdkCallCx` gains `script_id` (attribution + User-Agent).
Cron triggers (4th trigger kind):
- Migration 0017 widens the kind/source_kind CHECKs and adds
`cron_trigger_details`. `cron`/`chrono-tz` parse + validate 6-field
schedules and IANA timezones.
- `spawn_cron_scheduler` polls due triggers and enqueues to the universal
outbox; the dispatcher delivers them (one-line match-arm extension).
Catch-up fires exactly once per trigger per tick, not once per missed
window. `ctx.event.cron` for handlers.
- `POST /api/v1/admin/apps/{id}/triggers/cron` reuses the v1.1.3
cross-app + kind!=module target check.
- Dashboard: admin-gated Triggers tab (cron create form + list).
Follow-ups: redact module backend errors at the resolver boundary (log
original at error level); pin `rhai = "=1.24"`; CHANGELOG incl. retroactive
v1.1.3 cross-app-trigger security note. Version bumps: workspace 1.1.4,
SDK 1.5, dashboard 0.10.0.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The docs:: SDK bridge mirrors kv::'s collection-handle pattern: a
custom Rhai type DocsHandle captures (collection, service, cx) once
via docs::collection(name), and methods bind via engine.register_fn
so scripts use dot-notation (users.create(...), users.find(...),
etc.). app_id never appears in the script-visible call shape — the
service derives it from cx.app_id, preserving cross-app isolation.
Methods registered: create, get, find, find_one, update, delete,
list (zero-arg and one-arg map-shaped overloads). The find filter
goes through dynamic_to_json -> DocsService::find -> docs_filter
parser; unsupported operators surface to Rhai with the parser's
verbatim error message (including the v1.2 pointer).
The doc envelope per Decision D:
#{ id: "uuid", data: #{...user data...},
created_at: "ISO-8601", updated_at: "ISO-8601" }
engine.rs trigger_event_to_dynamic gains a Docs arm that builds
ctx.event.docs = #{ collection, id, data, prev_data } where data
and prev_data follow the variant's Option<Value> -> () | map shape.
15 bridge integration tests under tests/sdk_docs.rs exercise the
round-trip via tokio::task::spawn_blocking. Covers create/get/find/
find_one/update/delete/list semantics, $in + $gt operators, the
unsupported-operator throw with v1.2 pointer, invalid-UUID rejection
on get/update/delete, the doc envelope's shape (id is string, data
is map, timestamps are strings), and the load-bearing cross-app
isolation guarantee. sdk_kv.rs is updated to take the new docs
field on Services::new.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`PostgresDeadLetterService` lands as the real `DeadLetterService`
impl, replacing `NoopDeadLetterService` in the picloud binary's
`Services` bundle. Both methods are gated by
`Capability::AppDeadLetterManage(AppId)` — public-HTTP scripts with
`principal: None` fail the check, per design notes §4.
- `dead_letters::replay(id)` (Rhai SDK + admin endpoint): re-inserts
the original event payload into the outbox with attempt_count=0,
reply_to=None. The DL row is marked `resolution='replayed'`.
- `dead_letters::resolve(id, reason)` (Rhai SDK + admin endpoint):
closes the row with `resolved_at = NOW()` and the given reason.
CHECK constraint on the column enforces the 4-value vocabulary.
- `dead_letters::list(filter)` is intentionally NOT shipped —
design notes §4 defers it to v1.2 to align with the eventual
`docs::find()` query DSL.
Admin endpoints under `/api/v1/admin/apps/{id}/dead_letters/*`:
- `GET /` (with `?unresolved=true`) → list view
- `GET /count` → unresolved-count badge
- `GET /{dl_id}` → row detail (full payload + error)
- `POST /{dl_id}/replay` → re-enqueue
- `POST /{dl_id}/resolve` body `{reason}` → close out
All cross-app-aware: the row's `app_id` is compared against the path
param so a caller with rights on app A cannot manipulate app B's
dead letters by id alone.
The Rhai bridge for `dead_letters::*` follows the same sync↔async
pattern as the `kv::` bridge (`Handle::current().block_on(...)`
inside the spawn_blocking-wrapped Rhai engine).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wires the KV store into Rhai scripts via the handle pattern:
let widgets = kv::collection("widgets");
widgets.set("k", #{ n: 1 });
let v = widgets.get("k"); // value or () if absent
widgets.has("k") / widgets.delete("k")
let page = widgets.list(); // cursor-style pagination
`KvHandle` is a custom Rhai type holding `Arc<dyn KvService>` + the
per-call `Arc<SdkCallCx>`. Methods route async service calls through
`tokio::Handle::current().block_on(...)` — works because
`LocalExecutorClient` runs the script under `spawn_blocking` so a
runtime is reachable. The bridge surfaces `app_id` exclusively
through `cx.app_id`; no public-facing argument can spoof an app.
`TriggerEvent` lands in `picloud-shared` as the wire shape the
dispatcher will emit (KV + DeadLetter variants — KV exercised now,
DL hooks up with the dispatcher in commit 5/8). `SdkCallCx` and
`ExecRequest` grow `is_dead_letter_handler: bool` and
`event: Option<TriggerEvent>`. `engine.rs::build_ctx_map` flattens
the event into `ctx.event` for triggered handlers; direct ingress
leaves the key absent so scripts can `if "event" in ctx`.
Tests:
- 7 `sdk_kv.rs` integration tests covering the full Rhai surface
(round-trip, missing-key unit, has bool, delete was-present,
empty-collection rejection, cursor pagination, cross-app
isolation through the bridge).
- 3 new `engine.rs` tests pinning `ctx.event` shape per
design notes §4 (KV insert with value, delete with unit value,
direct invocations have no `event` key).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the v1.1.0 user-visible stdlib: regex, random, time, json, base64,
hex, url — each exposed as a `::` namespace mirroring the existing
`log::` pattern. Modules register once at engine build via
`Engine::register_static_module`, distinct from the stateful service
modules (KV, docs, …) that hook into `sdk::register_all` per call.
- regex: linear-time, compile-per-call (no cache by design)
- random: OsRng only; bytes/string capped to prevent script-side blow-up
- time: UTC, ms-since-epoch as canonical i64; RFC 3339 strings for I/O
- json: parse/stringify via existing dynamic<->json bridge
- base64: standard + URL-safe alphabets, Blob and String inputs
- hex: lowercase output, case-insensitive decode
- url: RFC 3986 percent-encoding + encode_query for Maps
Stdlib registration runs unconditionally — including in the parse-only
validate path — so scripts get a uniform surface in both phases.
See docs/sdk-shape.md for the stateless-vs-stateful distinction.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- sdk/bridge.rs: drop #[must_use] on the bridge fns — `Dynamic` and
`serde_json::Value` are both #[must_use] already; the wrapper
attribute is double-must-use noise.
- api.rs IntoResponse: hoist `use ApiError as E;` above the early
Overloaded branch so `E::Exec(...)` works in the if-let too
(clippy::items_after_statements).
- gate.rs test: bind the returned permit with `let _ =` so the
OwnedSemaphorePermit doesn't trip unused-must-use.
No behaviour change. Caught by `cargo clippy --all-targets
--all-features -- -D warnings`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Hoist the json_to_dynamic / dynamic_to_json helpers out of engine.rs
into a new sdk/bridge.rs so the v1.1.x service modules (KV, docs, …)
can use them without engine.rs being the sole owner. No behavioural
change — the sdk_contract round-trip test pins the observable JSON
fidelity.
Also lands the structural shape that subsequent v1.1.x PRs hook into:
- sdk::register_all(engine, services, cx) — single per-call hook
every stateful service registers through. Body is a no-op for
v1.1.0; SdkCallCx construction inside Engine::execute lands in
the next commit alongside the new ExecRequest fields it reads.
- sdk::cx re-exports picloud_shared::SdkCallCx so SDK callers don't
cross-import shared for one type.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
