The request map exposed path/headers/body/params/query/rest but not the
HTTP method, forcing one script per verb (the E2E app needed 7 scripts
where ~3 would do). Add a `method` field to ExecRequest (populated from
the HTTP method on the sync-execute bypass and the HTTP outbox payload;
empty for non-HTTP triggers) and surface it as `ctx.request.method`,
uppercased. A single script can now branch GET/POST on one route.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
build_invoke_request already asserts `script.app_id == row.app_id`
(dispatcher.rs:752) before constructing an ExecRequest. The queue
dispatcher (`dispatch_one_queue`) and the HTTP outbox arm
(`build_http_request`) did not. validate_trigger_target blocks cross-
app registration at write time, so the gap was latent; but a hand-
edited row, a partial backup restore, or a script re-pointed across
apps post-hoc could otherwise execute one app's script under another
app's SdkCallCx, breaking the cross-app isolation boundary the SDK
relies on.
Adds the runtime guard at both sites:
* dispatch_one_queue — on mismatch, dead-letter the message so the
misfire is observable rather than silently dropped, and short-
circuit.
* build_http_request — on mismatch, return ResolveTrigger error; the
outer dispatch arm logs + drops the row (same path as decode
failures).
Audit ref: security_audit/02_authz_isolation.md (H-F1).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Stage 6 review Obs 1b: the 300s default lived as a literal in two
places (DEFAULT_ASYNC_EXEC_TIMEOUT in dispatcher.rs and
DEFAULT_SAFE_VISIBILITY_VS_EXEC_BUDGET_SECS in triggers_api.rs). A
future change to the dispatcher would silently leave the
visibility-timeout warn threshold out of sync.
Extract a single pub const DEFAULT_ASYNC_EXEC_TIMEOUT_SECS: u32 = 300
in dispatcher.rs, derive the existing Duration constant from it, and
re-export it under the triggers_api name so the validator's
self-contained semantics are preserved at the call site. Tests still
inject the value explicitly via TEST_SAFE_LIMIT — no behavior change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the queue DL fan-out gap (audit Medium) and surfaces the
KV/docs/files non-transactional emit gap to operators.
- handle_queue_failure previously called queue.dead_letter to write the
DL row but never invoked fan_out_dead_letter. The comment claimed
"the outbox arm fires registered dead_letter handlers off the new row"
— but queue DL rows are written via a separate path that bypasses the
outbox entirely, so handlers filtered on source="queue" sat idle
forever. Refactor fan_out_dead_letter to take a DeadLetterFanOutCtx
struct so both the outbox arm and the queue arm can call it; the
queue arm constructs a TriggerEvent::Queue from the claimed message
and passes it through.
- New integration test queue_dead_letter_fans_out_to_dead_letter_handler
registers a dead_letter trigger filtered on "queue", forces queue
exhaustion, asserts the handler fires with the correctly-shaped event.
- KV/docs/files services committed the data write then ran events.emit
as a separate operation, logging-and-swallowing on failure. Bump the
six call sites from tracing::warn to tracing::error with an
event_emit_failure=true marker so operators can grep them. The full
single-tx repo refactor (extending ServiceEventEmitter with
emit_in_tx + tx-aware *_repo methods) is documented in kv_service.rs
as a v1.2 follow-up — it's a meaningful redesign that deserves its
own pass (pubsub_service::fan_out_publish is the reference shape).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes 4 High-severity audit findings:
- describe_event broken for HTTP-async payloads: was reading source/op
JSON fields that HttpDispatchPayload doesn't carry, producing empty
source on DL rows. from_wire("") then fell back to Kv on replay, the
dispatcher tried to resolve a non-existent trigger, and replay no-op'd
silently. Now branches on OutboxSourceKind: HTTP rows emit
"<method> <path>" + source="http"; Invoke rows emit "invoke_async" +
source="invoke"; TriggerEvent payloads keep top-level field extraction.
- HTTP outbound Authorization leaked across cross-origin redirects.
Manual redirect loop (Policy::none) reused header_map on every hop
and only scrubbed Content-Type for POST->GET. Now compares
url::Url::origin() across hops and strips Authorization,
Proxy-Authorization, Cookie when origin changes — matching reqwest's
default policy.
- F-S-010 inbound email HMAC had no replay protection. Signature was
computed over body only with no timestamp or nonce, so captured POSTs
were replayable indefinitely. Adds X-Picloud-Timestamp header bound
into the HMAC input (signed string is ts || "." || body), 300s
tolerance window, and a process-local LRU keyed on
(ts, sha256(body)) with 600s TTL to catch within-window replays.
Breaking change: webhook senders must include the timestamp header.
- expected_schema.txt re-blessed for migrations 0036-0039 (the previous
snapshot stopped at 0035 so CI would have gone red on first run).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
cargo fmt regroups; three clippy fixes:
- F-S-006: hoist MAX_API_KEY_CANDIDATES to module scope to satisfy
clippy::items_after_statements.
- F-S-009: use map_or instead of map().unwrap_or() per
clippy::map_unwrap_or.
- F-P-012: replace `|c| c.encode()` closure with the method itself
per clippy::redundant_closure_for_method_calls.
No behavior change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
TICK_INTERVAL (100ms) and ASYNC_EXEC_TIMEOUT (300s) were `const`. Every
other timing knob in the file (cron tick, queue reclaim, retry policy)
is env-overridable via TriggerConfig::from_env. Operators on a
constrained Pi or a busier instance could tune retries but not
dispatcher cadence.
Add env knobs:
- PICLOUD_DISPATCHER_TICK_INTERVAL_MS (default 100)
- PICLOUD_DISPATCHER_ASYNC_EXEC_TIMEOUT_SEC (default 300)
Read via tick_interval_from_env() / async_exec_timeout_from_env() at
dispatcher startup. Invalid values fall back with a tracing-warn.
AUDIT.md anchor: F-Q-009.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
tick_queue_arm called list_active_queue_consumers() and then iterated
serially, awaiting one queue.claim(app, queue) per consumer. With N
consumers and a 100ms tick the worst-case throughput was
N × (claim + executor) / tick — one slow handler blocked every other
queue's progress on the dispatcher's task.
Replace the for-loop with `futures::stream::iter(consumers)
.for_each_concurrent(QUEUE_DISPATCH_PARALLELISM, …)`. The execution
gate already caps real script-concurrency to its permits, so this just
removes the dispatcher-side serialization.
QUEUE_DISPATCH_PARALLELISM = 32 (matches the default gate). Workspace
gains `futures = 0.3` so `for_each_concurrent` is available.
AUDIT.md anchor: F-P-007.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the commit 2 placeholder with the real OutboxSourceKind::Invoke
handler.
build_invoke_request(row) hydrates an (ResolvedTrigger, ExecRequest)
pair from a payload InvokeService::enqueue_async wrote. Validates:
- script_id present + parses as UUID
- script exists
- script.app_id == row.app_id (defensive — re-check the cross-app
boundary; same-app was already verified at enqueue time)
Synthesized ResolvedTrigger carries retry_max_attempts = 1 so the
existing handle_failure path skips the retry loop. invoke_async runs
exactly once; on throw the row is deleted + a dead_letters row written
(handle_failure already does this when attempt >= max_attempts), so a
caller who wants retry semantics wraps the call site in retry::with.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extends Dispatcher with the v1.1.9 queue path. The queue table IS the
outbox for queue semantics, so the dispatcher polls queue_messages
directly via FOR UPDATE SKIP LOCKED — no outbox indirection.
Per tick (every 100ms):
1) outbox arm — unchanged
2) queue arm: list_active_queue_consumers() → per (app_id, queue_name)
attempt one claim. Bounded by registered-consumer count so one busy
queue can't starve others.
Per claimed message:
- Build TriggerEvent::Queue + ExecRequest (executes as the trigger's
registering principal, matching design notes §4)
- dispatch through executor.execute_with_identity (reuses AST cache)
- success → queue.ack(id, claim_token) — DELETE WHERE id AND token
- throw + attempt < max_attempts → queue.nack(...) — clear claim, set
deliver_after = NOW() + compute_backoff(attempt, backoff, base_ms, jitter)
- throw + exhausted → queue.dead_letter(...) atomic move to dead_letters
+ DELETE in one transaction. fan_out_dead_letter on the outbox arm
fires registered dead_letter handlers off the new row without changes.
Visibility-timeout reclaim task: separate tokio::spawn ticking on
queue_reclaim_interval_ms (default 30000). UPDATE clears claim_token /
claimed_at on rows whose claimed_at exceeds the per-queue
visibility_timeout_secs (joined from queue_trigger_details). A crashed
consumer thus loses its lease and the message becomes claimable again.
Dispatcher gains queue: Arc<dyn QueueRepo>; picloud/lib.rs threads
queue_repo.clone() into the construction.
ActiveQueueConsumer extended with app_id so the queue claim can be
performed without a follow-up trigger lookup; list_active_queue_consumers
SQL extended to SELECT t.app_id.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Clears the workspace under `clippy --all-targets --all-features
-D warnings`. Four were pre-existing at v1.1.6 HEAD (latent finding,
see HANDBACK): double_must_use on realtime_router, map_unwrap_or in
pubsub_service, redundant_closure in topic_repo, needless_raw_string in
a subscriber-token test. The rest are v1.1.7 nits (needless_borrow +
semicolon in the dead-letter / realtime-migration code).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
dead_letter triggers have been registerable since v1.1.1 but their
handlers never fired: dispatcher::handle_failure wrote the dead_letters
row and stopped — list_matching_dead_letter had no production caller.
Any deploy v1.1.1–v1.1.6 with dead_letter triggers had silently
non-functional handlers.
The fix: after the dead-letter row is inserted on retry exhaustion, fan
out to matching dead_letter triggers (filtered by source / originating
trigger_id / script_id) and enqueue one outbox row per match carrying a
real-shape TriggerEvent::DeadLetter (the §6 brief field names were stale
— used the actual variant: dead_letter_id, original: Box<TriggerEvent>,
attempts, last_error, trigger_id, script_id, first/last_attempt_at).
The recursion-stop (a handler's own failure isn't re-dead-lettered)
is upheld by the existing is_dead_letter_handler short-circuit.
Tests (DB-gated): handler actually fires with the nested original event;
existing row-create test now also asserts handler-fire; source_filter
excludes non-matching; failing handler does not recurse.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Inbound email: a provider POSTs a normalized JSON message to
POST /api/v1/email-inbound/{app_id}/{trigger_id}; the public receiver
verifies the optional HMAC signature, builds a TriggerEvent::Email, and
enqueues an outbox row the dispatcher delivers like any async trigger.
Handlers see ctx.event.email = #{from,to,cc,subject,text,html,
received_at,message_id}.
- migration 0024: widen triggers.kind + outbox.source_kind CHECKs to
'email'; new email_trigger_details table.
- TriggerKind::Email, TriggerDetails::Email{has_inbound_secret},
OutboxSourceKind::Email, TriggerEvent::Email; dispatcher routes the
email row via the generic resolve_trigger path.
- Admin POST /apps/{id}/triggers/email (validate_trigger_target; module
+ cross-app rejection). inbound_secret is stored ENCRYPTED via the
master key (deviation from the brief's plaintext default; decrypted
per inbound request — see HANDBACK §7).
- Dashboard: email trigger form on the Triggers tab + webhook URL +
expected-payload help.
- 8 DB-gated e2e tests (202/401/404/422/cross-app/handler-fire) +
receiver unit tests (HMAC verify, secret round-trip, payload parse).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Durable pub/sub through the universal outbox — the sixth trigger kind.
- `pubsub::publish_durable(topic, message)` Rhai SDK (no handle; topics
ARE the grouping unit). Message JSON-encoded; Blobs base64 at any
depth.
- `PubsubService` trait in picloud-shared with the topic matcher +
validator (exact / `<prefix>.*` / `*`; mid-pattern wildcards
rejected). `PostgresPubsubRepo` + `PubsubServiceImpl` in manager-core.
- Publish-time fan-out: one outbox row per matching enabled pubsub
trigger, all in ONE transaction (no half-fan-out on crash). No
matching trigger → publish succeeds silently, zero rows.
- `pubsub:*` trigger kind via Layout-E (0020: widen both CHECKs +
pubsub_trigger_details + partial index), TriggerEvent::Pubsub +
ctx.event.pubsub, dispatcher arm, admin endpoint POST /triggers/pubsub
(validates topic pattern + reuses validate_trigger_target).
- AppPubsubPublish capability → script:write (seven-scope held).
- Dashboard Pub/Sub trigger form on the Triggers tab + list rendering.
publish_ephemeral stays deferred to v1.2. ~18 new tests (service
in-memory incl. transactional-rollback, shared matcher, bridge
encoding). No DB required for the suite.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Filesystem-backed blob storage as the fifth concrete trigger kind.
- `files::collection(c).{create,head,get,update,delete,list}` Rhai SDK
(blob in/out; metadata maps; missing-field throws naming the field).
- `FilesService` trait in picloud-shared; `FsFilesRepo` (atomic
write: temp→fsync→rename→fsync-dir→DB; single-pass SHA-256;
checksum-verified reads → Corrupted) + `FilesServiceImpl` in
manager-core. Metadata in Postgres (0018), bytes on disk under
PICLOUD_FILES_ROOT with 0o700 shard dirs.
- `files:*` trigger kind via the Layout-E pattern (0019: widen both
CHECKs + files_trigger_details), TriggerEvent::Files (metadata only,
no bytes), emit_files fan-out, dispatcher arm, admin endpoint
POST /triggers/files (reuses validate_trigger_target).
- AppFilesRead/AppFilesWrite capabilities → script:read/script:write
(seven-scope commitment held). AppPubsubPublish reserved for v1.1.6.
- Admin files API (list + delete) + dashboard Files view per app.
Cross-app isolation keyed on cx.app_id at every layer. ~45 new tests
(service in-memory, fs tempdir, bridge integration). No DB required
for the suite. publish_ephemeral and the orphan sweep stay deferred.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
HTTP (`http::*`):
- `HttpService` trait (picloud-shared) + reqwest-backed `HttpServiceImpl`
(manager-core), wired into the `Services` bundle.
- SSRF deny-list applied to the resolved IP via a custom reqwest
`dns_resolver` (covers every redirect hop + defeats DNS rebinding) plus
a literal-IP check at URL-parse time. Scheme/port restrictions, request
+ response body caps (stream-with-cap), layered timeout. Error reason is
a CIDR category, never the IP. `PICLOUD_HTTP_ALLOW_PRIVATE` dev override
(logs a startup warning).
- Rhai bridge with three-arg split `verb(url, body, opts)` (resolves the
brief's body-vs-opts contradiction; unknown opt keys throw). Body
dispatch by type; response `#{status,headers,body,body_raw}` with JSON
auto-parse; non-2xx does not throw.
- `Capability::AppHttpRequest` → existing `script:write` scope (no new
Scope variant). `SdkCallCx` gains `script_id` (attribution + User-Agent).
Cron triggers (4th trigger kind):
- Migration 0017 widens the kind/source_kind CHECKs and adds
`cron_trigger_details`. `cron`/`chrono-tz` parse + validate 6-field
schedules and IANA timezones.
- `spawn_cron_scheduler` polls due triggers and enqueues to the universal
outbox; the dispatcher delivers them (one-line match-arm extension).
Catch-up fires exactly once per trigger per tick, not once per missed
window. `ctx.event.cron` for handlers.
- `POST /api/v1/admin/apps/{id}/triggers/cron` reuses the v1.1.3
cross-app + kind!=module target check.
- Dashboard: admin-gated Triggers tab (cron create form + list).
Follow-ups: redact module backend errors at the resolver boundary (log
original at error level); pin `rhai = "=1.24"`; CHANGELOG incl. retroactive
v1.1.3 cross-app-trigger security note. Version bumps: workspace 1.1.4,
SDK 1.5, dashboard 0.10.0.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- New `ScriptIdentity { script_id, updated_at }` DTO.
- `ExecutorClient` trait gains an `execute_with_identity` method;
default impl forwards to `execute` so `RemoteExecutorClient` (and
cluster-mode transports later) keep working without bespoke caching.
- `LocalExecutorClient` overrides `execute_with_identity` to consult
an `LruCache<ScriptId, CachedScript>`. Cache hit only when the
cached entry's `updated_at` matches the caller's identity; mismatch
triggers a fresh `Engine::compile`. `Engine::execute_ast(&Arc<AST>, req)`
is called inside `spawn_blocking` exactly as `execute` does today.
- Cache size from `PICLOUD_SCRIPT_CACHE_SIZE` (default 256).
- Orchestrator's HTTP data-plane path and the dispatcher both switch
to `execute_with_identity`. `ResolvedTrigger` carries
`script_updated_at` for the dispatcher's identity construction.
Workspace builds; full test suite (~440 tests) green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The docs trigger kind hangs off the same Layout-E shape that v1.1.1
established for KV: a parent triggers row + a docs_trigger_details
row (collection_glob TEXT + ops TEXT[]) with the empty-array =
any-op semantic preserved.
- trigger_repo.rs adds TriggerKind::Docs + TriggerDetails::Docs +
CreateDocsTrigger + DocsTriggerMatch + PostgresTriggerRepo
implementations of create_docs_trigger and list_matching_docs.
list_matching_docs mirrors KV's Rust-side filter (does NOT push
ops membership into SQL — that would exclude empty-ops rows).
- outbox_repo.rs adds OutboxSourceKind::Docs to the enum + wire form.
- dispatcher.rs's generic Kv | DeadLetter routing arm extends to
Kv | DeadLetter | Docs. No kind-specific logic needed — the
resolve_trigger + build_exec_request path is already abstract.
- outbox_event_emitter.rs gains a "docs" arm in the emit match plus
emit_docs which builds TriggerEvent::Docs (carrying data +
prev_data) and fans out across matching triggers.
- triggers_api.rs adds CreateDocsTriggerRequest + create_docs_trigger
+ the POST /api/v1/admin/apps/{id}/triggers/docs route, all
guarded by Capability::AppManageTriggers (same as KV).
3 new triggers_api unit tests covering happy path, empty-glob
rejection, and capability denial. All existing trigger-related
tests still pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Routes gain `dispatch_mode TEXT NOT NULL DEFAULT 'sync'` (migration
0012). Existing routes default to sync so the migration is
non-breaking. `DispatchMode` enum lands in `picloud-shared`.
The user-routes orchestrator handler now branches:
- `dispatch_mode = async` → write outbox row with `reply_to = None`,
return `202 Accepted` + `{accepted_at, execution_id}`. Dispatcher
fires the script in the background; retries / dead-letters via
the framework from commit 5.
- `dispatch_mode = sync` → register an inbox channel
(`tokio::sync::oneshot`), write outbox row with `reply_to =
inbox_id`, `.await` on the receiver with a timeout =
script.timeout_seconds + 2s buffer. Dispatcher hands the result
back; orchestrator maps `InboxResult` into the HTTP response per
the design-notes §3 status-code table (422/502/503/504/507/500).
`InboxRegistry` (orchestrator-core/src/inbox.rs) is the in-process
implementation of `InboxResolver`. Lock-free HashMap of pending
oneshot senders keyed by `inbox_id`. Tests cover register/deliver
round-trip, unknown-id is abandoned, dropped-receiver is abandoned,
explicit cancel. Cluster mode (v1.3+) swaps this for
LISTEN/NOTIFY-keyed lookup behind the same trait.
`OutboxWriter` trait lives in `picloud-shared` so orchestrator-core
can write to the outbox without depending on manager-core (which
would invert the dependency arrow). `PostgresOutboxRepo` implements
both `OutboxRepo` (dispatcher surface) and `OutboxWriter`
(orchestrator surface); the picloud binary clones the same concrete
Arc into both trait views.
The dispatcher's HTTP arm (commit 5 had a stub) now decodes the
`HttpDispatchPayload` off the outbox row, looks up the script,
synthesizes an `ExecRequest`, and runs it through the executor.
Outcome routing reuses the same path as KV triggers — sync HTTP
flows through the inbox, async dispatch gets dropped after
success (or DL'd on exhaustion).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`OutboxEventEmitter` replaces `NoopEventEmitter` in the picloud
binary's `Services` bundle. KV mutations now fan out to the outbox
via `TriggerRepo::list_matching_kv` — one row per matching trigger,
carrying the serialized `TriggerEvent` payload + the matching
trigger's retry policy.
`Dispatcher` is the single tokio task that polls the outbox every
100ms, claims due rows via FOR UPDATE SKIP LOCKED (with a batch cap),
and routes each to the executor. Shares the `ExecutionGate` with
sync HTTP per design notes §2 — gate saturation reschedules the
row instead of dropping it.
Outcome handling matches design notes §3 and §4:
- reply_to.is_some() (sync HTTP): never retry. Deliver via
`InboxResolver`; if the receiver was dropped, write an
`abandoned_executions` row.
- is_dead_letter_handler == true: never retry, never DL. On
failure, annotate the original DL row with
`resolution = 'handler_failed'`. Stops the recursion that would
otherwise re-fire a broken handler script.
- Otherwise async: bump attempt_count, reschedule with exponential
backoff + ±jitter; once max_attempts is reached, write a
`dead_letters` row and drop from outbox.
- Trigger-depth limit: `cx.trigger_depth > max_trigger_depth` skips
execution entirely (log + future metric), NEVER dead-letters.
Loops are not retried via the DL chain — they're terminated.
`InboxResolver` trait lands in `picloud-shared` with a
`NoopInboxResolver` bootstrap that flags every delivery as
`Abandoned`. Commit 6 replaces the noop with the real
in-process registry in `orchestrator-core`.
`AdminPrincipalResolver` builds a `Principal` from a trigger's
`registered_by_principal` user id so the dispatched script executes
as the trigger registrant (design notes §4).
Unit tests cover backoff math (exponential/linear/constant) +
jitter range + ExecError → InboxFailureKind classification + the
status-code table mapping. Integration tests for the full
dispatcher loop need a real Postgres + executor; reviewer runs them
via the manual smoke flow in the plan / HANDBACK.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
