Compare commits
35 Commits
test/cli-j
...
feat/v1.1.
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2796f36fef | ||
|
|
5a95ff2d07 | ||
|
|
66b661f64c | ||
|
|
6b7ff78730 | ||
|
|
1795dfc98a | ||
|
|
20f1b5e64d | ||
|
|
77b2cb58bb | ||
|
|
6a2971ac70 | ||
|
|
2e92691ee1 | ||
|
|
545d863199 | ||
|
|
6b99f74c48 | ||
|
|
434fb63cd2 | ||
|
|
1efb350b54 | ||
|
|
10cfde9e40 | ||
|
|
bb88b024d2 | ||
|
|
9d01f42d5e | ||
|
|
1a6324078c | ||
|
|
54efe61167 | ||
|
|
1d2e99e42c | ||
|
|
9e54b7f875 | ||
|
|
a685674dbf | ||
|
|
a8aab22163 | ||
|
|
e375735796 | ||
|
|
098e18a989 | ||
|
|
9b4a834627 | ||
|
|
5302bd3192 | ||
|
|
902dd78027 | ||
|
|
dea776b2a3 | ||
|
|
fe1dd90836 | ||
|
|
aaba58dee1 | ||
|
|
2669714a51 | ||
|
|
662d5a2cf8 | ||
|
|
fc8d473416 | ||
|
|
c73e3c80c0 | ||
|
|
f147665157 |
88
CHANGELOG.md
Normal file
88
CHANGELOG.md
Normal file
@@ -0,0 +1,88 @@
|
||||
# PiCloud Changelog
|
||||
|
||||
## v1.1.1 — Storage & Events (unreleased)
|
||||
|
||||
The triggers framework — KV store + universal outbox + dispatcher +
|
||||
NATS-style sync HTTP + per-route async dispatch + dead-letter
|
||||
handling + dashboard surface. Every subsequent v1.1.x service module
|
||||
(docs, files, pubsub, …) hangs off the dispatcher built here.
|
||||
|
||||
### Added
|
||||
|
||||
- **KV store** — `kv_entries` table keyed `(app_id, collection, key)`
|
||||
with JSONB values. Rhai SDK exposes the handle pattern:
|
||||
`kv::collection(name).{get,set,has,delete,list}`. Cursor-style
|
||||
pagination with opaque base64 cursors. Cross-app isolation
|
||||
enforced via `cx.app_id` (never script-passed).
|
||||
- **Triggers framework (Layout E)** — parent `triggers` table +
|
||||
per-kind detail tables (`kv_trigger_details`,
|
||||
`dead_letter_trigger_details`). Trigger CRUD admin endpoints
|
||||
(`/api/v1/admin/apps/{id}/triggers/{kv,dead_letter}`) +
|
||||
`Capability::AppManageTriggers(AppId)`.
|
||||
- **Universal outbox + dispatcher** — single tokio task that polls
|
||||
the outbox via `FOR UPDATE SKIP LOCKED`, routes due rows to the
|
||||
executor through the shared `ExecutionGate`. Retry with
|
||||
exponential backoff + ±jitter; on exhaustion, dead-letter.
|
||||
- **NATS-style sync HTTP via outbox** — `InboxRegistry` (in-process
|
||||
oneshot map) lets the orchestrator await dispatcher delivery on
|
||||
every sync HTTP request. Cluster mode (v1.3+) swaps this for
|
||||
`LISTEN/NOTIFY` behind the same `InboxResolver` trait.
|
||||
- **`dispatch_mode: async` on routes** — `POST` to a route with
|
||||
`dispatch_mode = 'async'` returns `202 Accepted` immediately;
|
||||
the script runs via the dispatcher (with retries / dead-letter).
|
||||
- **Dead-letter handling** — separate `dead_letters` table per
|
||||
design notes §4. `dead_letters::{replay,resolve}` Rhai SDK +
|
||||
admin endpoints + `Capability::AppDeadLetterManage(AppId)`.
|
||||
Recursion-stop rule: dead-letter handler failures annotate the
|
||||
original row as `resolution = 'handler_failed'` and never produce
|
||||
a new dead-letter or retry.
|
||||
- **Dashboard surface for dead letters** — unresolved-count red
|
||||
badge on the apps list + per-app page; per-app dead-letters list
|
||||
view at `/admin/apps/{slug}/dead-letters` with Replay + Mark
|
||||
resolved per-row actions and expandable payload detail.
|
||||
- **`abandoned_executions` table** — forensic row written by the
|
||||
dispatcher when it tries to resolve an inbox the orchestrator
|
||||
already abandoned (timed out). Counter metric path reserved.
|
||||
- **Trigger-depth limit** — `cx.trigger_depth > max_trigger_depth`
|
||||
(default 8) skips execution + logs; does NOT dead-letter
|
||||
(depth-exceeded means "you built a loop").
|
||||
- **GC sweepers** — weekly retention sweeps for `dead_letters`
|
||||
(30 days) and `abandoned_executions` (7 days), both with
|
||||
`FOR UPDATE SKIP LOCKED` for cluster-mode safety.
|
||||
- **Env-overridable trigger config** — `TriggerConfig::from_env`
|
||||
reads `PICLOUD_MAX_TRIGGER_DEPTH`, `PICLOUD_TRIGGER_RETRY_*`,
|
||||
`PICLOUD_DEAD_LETTER_RETENTION_DAYS`,
|
||||
`PICLOUD_ABANDONED_EXECUTIONS_RETENTION_DAYS`.
|
||||
|
||||
### Changed
|
||||
|
||||
- **Workspace version**: `1.1.0` → `1.1.1`.
|
||||
- **Rhai SDK version**: `1.1` → `1.2` (additive — every v1.1 script
|
||||
still runs unchanged; new surfaces: `kv::*`, `dead_letters::*`,
|
||||
`ctx.event` for triggered handlers).
|
||||
- **Dashboard version**: `0.6.0` → `0.7.0` for the dead-letters UI.
|
||||
- **`Services` bundle** — replaces v1.1.0's no-arg `Services::new()`
|
||||
with explicit `Services::new(kv, dead_letters, events)`. Tests
|
||||
use `Services::default()` for an all-noop bundle.
|
||||
- **`SdkCallCx`** grows `is_dead_letter_handler: bool` and
|
||||
`event: Option<TriggerEvent>` fields.
|
||||
- **`ExecRequest`** mirrors the new `SdkCallCx` fields and grows
|
||||
`event` for serializable trigger payload transport.
|
||||
- **Routes table** grows `dispatch_mode TEXT NOT NULL DEFAULT 'sync'`
|
||||
(CHECK in {sync, async}).
|
||||
- **Schema version**: 6 → 12 (migrations 0007 through 0012).
|
||||
|
||||
### Migrations
|
||||
|
||||
- `0007_kv.sql` — `kv_entries` table + index
|
||||
- `0008_triggers.sql` — `triggers` + `kv_trigger_details` +
|
||||
`dead_letter_trigger_details`
|
||||
- `0009_outbox.sql` — universal `outbox` table + due-row partial index
|
||||
- `0010_dead_letters.sql` — `dead_letters` table + unresolved partial
|
||||
index + GC index
|
||||
- `0011_abandoned_executions.sql` — forensic table + GC index
|
||||
- `0012_routes_dispatch_mode.sql` — `routes.dispatch_mode` column
|
||||
|
||||
## v1.1.0 — Foundation & Standard Library
|
||||
|
||||
See `docs/v1.1.x-design-notes.md` §7 for the full v1.1.x roadmap.
|
||||
17
CLAUDE.md
17
CLAUDE.md
@@ -8,7 +8,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
|
||||
|
||||
Authoritative design: [serverless_cloud_blueprint.md](serverless_cloud_blueprint.md). The blueprint is a living document — when architecture decisions are made in conversation that contradict it, treat the latest decision as truth and update the blueprint.
|
||||
|
||||
**Current focus (Phase 4, v1.1):** data-plane SDKs — KV store, then document store, then HTTP client, then cron triggers. See blueprint §12. Phase 3 (admin auth + multi-app scoping) shipped; every v1.1+ table starts with `app_id UUID NOT NULL REFERENCES apps(id) ON DELETE CASCADE` and every Rhai SDK call resolves its app from the execution context.
|
||||
**Current focus (Phase 4, v1.1.0):** SDK foundation + stdlib utilities — the shape every v1.1.x service module hangs off, see [docs/sdk-shape.md](docs/sdk-shape.md). Stdlib reference at [docs/stdlib-reference.md](docs/stdlib-reference.md). Subsequent v1.1.x releases (KV in v1.1.1, docs in v1.1.2, …) fill it in; see blueprint §12 for the full table. Phase 3 shipped end-to-end: admin auth, multi-app scoping, and Phase 3.5 capability gating (`manager-core::authz::{can, require, Capability}` + migration `0006_users_authz.sql`). Every v1.1+ table starts with `app_id UUID NOT NULL REFERENCES apps(id) ON DELETE CASCADE` and every Rhai SDK call resolves its app from the execution context.
|
||||
|
||||
## Three-Service Architecture
|
||||
|
||||
@@ -48,7 +48,7 @@ Caddy fronts everything. Same Caddyfile shape works for single-node and cluster
|
||||
- **Rust 1.92+** workspace, pinned via `rust-toolchain.toml`
|
||||
- **Axum** for HTTP, **Tokio** async, **sqlx** for Postgres
|
||||
- **Rhai** embedded scripting (in `executor-core`)
|
||||
- **PostgreSQL 15+** with `pgcrypto` and (v1.1+) `hstore`
|
||||
- **PostgreSQL 15+** with `pgcrypto`. v1.1+ data-plane tables use JSONB for value columns (hstore was considered for KV and rejected — see blueprint §8.1).
|
||||
- **SvelteKit** dashboard, static adapter, CodeMirror 6 for the script editor
|
||||
- **Caddy 2** reverse proxy (auto-HTTPS in prod)
|
||||
- **Docker Compose** for dev and single-node prod
|
||||
@@ -103,9 +103,22 @@ docs/
|
||||
- **Honor the three-service boundary.** Don't reach across `*-core` crates. If `orchestrator-core` needs something from `manager-core`, define a trait in `shared` and inject the impl.
|
||||
- **`executor-core` has no Postgres dependency.** Data-plane services (kv, docs, users — v1.1+) come in via injected `ServiceProvider` traits.
|
||||
- **Database writes only from `manager-core`.** `orchestrator-core` reads scripts (cached); `executor-core` doesn't touch the DB.
|
||||
- **Stateful SDK services use the handle pattern + `SdkCallCx`.** Collection-scoped surfaces look like `kv::collection("x").get(k)`, not `kv::get("x", k)`. Every service trait method takes `&SdkCallCx` and **MUST** derive `app_id` from `cx.app_id` — never trust a script-passed `app_id`. That is the cross-app isolation boundary. See [docs/sdk-shape.md](docs/sdk-shape.md).
|
||||
- **MVP builds only the `picloud` all-in-one binary.** The three split binaries exist as skeletons so the crate boundaries stay honest; flesh them out only when cluster mode is being implemented.
|
||||
- **Trunk-based dev.** See [docs/git-workflow.md](docs/git-workflow.md). No long-lived branches. Feature flags for incomplete work.
|
||||
|
||||
## Runtime configuration
|
||||
|
||||
Environment variables consumed by the `picloud` binary:
|
||||
|
||||
| Variable | Default | Purpose |
|
||||
|---|---|---|
|
||||
| `PICLOUD_BIND` | `0.0.0.0:8080` | HTTP listen address. Port 8080 is owned by another process on this host — override locally. |
|
||||
| `PICLOUD_MAX_CONCURRENT_EXECUTIONS` | `32` | Global concurrency cap on data-plane script executions. Overflow returns HTTP 503 with `Retry-After: 1` immediately (no queue). |
|
||||
| `DATABASE_URL` | — | Required. Postgres connection string. |
|
||||
| `PICLOUD_SESSION_TTL_HOURS` | `24` | Sliding-window session lifetime. |
|
||||
| `PICLOUD_SANDBOX_MAX_*` | conservative defaults | Per-knob admin ceilings on Rhai sandbox overrides. See `manager-core::sandbox::SandboxCeiling`. |
|
||||
|
||||
## Out of MVP
|
||||
|
||||
Queue triggers, cron triggers, SMTP ingress, KV / docs / email / users / HTTP SDKs in scripts, interceptors, workflows, function-to-function `invoke()`, secrets, metrics dashboard. All deferred to v1.1+ per the blueprint. Don't pre-build for them — but don't make decisions that close the door on them either.
|
||||
|
||||
27
Cargo.lock
generated
27
Cargo.lock
generated
@@ -1505,7 +1505,7 @@ checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220"
|
||||
|
||||
[[package]]
|
||||
name = "picloud"
|
||||
version = "0.6.0"
|
||||
version = "1.1.1"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"async-trait",
|
||||
@@ -1531,10 +1531,11 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "picloud-cli"
|
||||
version = "0.6.0"
|
||||
version = "1.1.1"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"assert_cmd",
|
||||
"chrono",
|
||||
"clap",
|
||||
"directories",
|
||||
"libc",
|
||||
@@ -1551,7 +1552,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "picloud-executor"
|
||||
version = "0.6.0"
|
||||
version = "1.1.1"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"picloud-executor-core",
|
||||
@@ -1563,21 +1564,28 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "picloud-executor-core"
|
||||
version = "0.6.0"
|
||||
version = "1.1.1"
|
||||
dependencies = [
|
||||
"async-trait",
|
||||
"base64",
|
||||
"chrono",
|
||||
"hex",
|
||||
"percent-encoding",
|
||||
"picloud-shared",
|
||||
"rand 0.8.6",
|
||||
"regex",
|
||||
"rhai",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"thiserror 1.0.69",
|
||||
"tokio",
|
||||
"tracing",
|
||||
"uuid",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "picloud-manager"
|
||||
version = "0.6.0"
|
||||
version = "1.1.1"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"picloud-manager-core",
|
||||
@@ -1589,7 +1597,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "picloud-manager-core"
|
||||
version = "0.6.0"
|
||||
version = "1.1.1"
|
||||
dependencies = [
|
||||
"argon2",
|
||||
"async-trait",
|
||||
@@ -1597,6 +1605,7 @@ dependencies = [
|
||||
"base64",
|
||||
"chrono",
|
||||
"data-encoding",
|
||||
"picloud-executor-core",
|
||||
"picloud-orchestrator-core",
|
||||
"picloud-shared",
|
||||
"rand 0.8.6",
|
||||
@@ -1613,7 +1622,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "picloud-orchestrator"
|
||||
version = "0.6.0"
|
||||
version = "1.1.1"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"picloud-orchestrator-core",
|
||||
@@ -1625,7 +1634,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "picloud-orchestrator-core"
|
||||
version = "0.6.0"
|
||||
version = "1.1.1"
|
||||
dependencies = [
|
||||
"async-trait",
|
||||
"axum",
|
||||
@@ -1644,7 +1653,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "picloud-shared"
|
||||
version = "0.6.0"
|
||||
version = "1.1.1"
|
||||
dependencies = [
|
||||
"async-trait",
|
||||
"chrono",
|
||||
|
||||
@@ -13,7 +13,7 @@ members = [
|
||||
]
|
||||
|
||||
[workspace.package]
|
||||
version = "0.6.0"
|
||||
version = "1.1.1"
|
||||
edition = "2021"
|
||||
rust-version = "1.92"
|
||||
license = "MIT OR Apache-2.0"
|
||||
@@ -74,6 +74,12 @@ sha2 = "0.10"
|
||||
base64 = "0.22"
|
||||
data-encoding = "2.6"
|
||||
|
||||
# Stdlib utility crates (v1.1.0 stdlib PR — registered into the
|
||||
# Rhai engine as the regex::/random::/etc. namespaces)
|
||||
regex = "1"
|
||||
hex = "0.4"
|
||||
percent-encoding = "2"
|
||||
|
||||
[workspace.lints.rust]
|
||||
unsafe_code = "forbid"
|
||||
|
||||
|
||||
340
HANDBACK.md
Normal file
340
HANDBACK.md
Normal file
@@ -0,0 +1,340 @@
|
||||
# v1.1.1 Implementation HANDBACK
|
||||
|
||||
## 1. Branch + commit count
|
||||
|
||||
- Branch: `feat/v1.1.1-storage-and-events`
|
||||
- Base: `main`
|
||||
- 11 commits ahead of `main`. Branch is **not pushed**, **not merged**.
|
||||
|
||||
```
|
||||
66b661f chore(release): bump workspace to v1.1.1 + CHANGELOG
|
||||
6b7ff78 feat(v1.1.1-gc): dead-letter + abandoned-executions retention sweepers
|
||||
1795dfc feat(v1.1.1-dead-letters): dashboard badge + list view
|
||||
20f1b5e feat(v1.1.1-dead-letters): service + Rhai SDK + admin endpoints
|
||||
77b2cb5 feat(v1.1.1-routes): outbox-routed sync HTTP + dispatch_mode=async
|
||||
6a2971a feat(v1.1.1-dispatcher): dispatcher loop + retry + depth limit + outbox emitter
|
||||
2e92691 feat(v1.1.1-triggers): trigger CRUD admin endpoints
|
||||
545d863 feat(v1.1.1-triggers): triggers + outbox schema + repos
|
||||
6b99f74 feat(v1.1.1-kv): Rhai kv:: SDK module + ctx.event wiring
|
||||
434fb63 feat(v1.1.1-kv): migrations + KvService trait + Postgres impl
|
||||
1efb350 docs(v1.1.x): resolve in-flight decisions as Decided 2026-06-01
|
||||
```
|
||||
|
||||
The first commit (`1efb350`) absorbed working-tree edits to
|
||||
`docs/v1.1.x-design-notes.md` that turned the "in-flight" 20 open
|
||||
calls into "Decided 2026-06-01" entries. Those were on the working
|
||||
tree at branch creation; folding them into the v1.1.1 branch keeps
|
||||
the design rationale colocated with the implementation.
|
||||
|
||||
## 2. Scope coverage (Done / Partial / Skipped)
|
||||
|
||||
| Scope item | Status | Notes |
|
||||
|---|---|---|
|
||||
| **1. KV store** | Done | Migration 0007, `KvService` trait in shared, `KvServiceImpl` + `PostgresKvRepo` in manager-core, Rhai `kv::collection(name).{get,set,has,delete,list}` bridge, cursor pagination, empty-collection rejection, script-as-gate authz. |
|
||||
| **2. Triggers framework — Layout E** | Done | Migrations 0008 (`triggers` + `kv_trigger_details` + `dead_letter_trigger_details`), `TriggerRepo` + `PostgresTriggerRepo`, CRUD admin endpoints. `registered_by_principal` column captured + threaded into the dispatcher. Depth-limit enforced in the dispatcher (default 8). |
|
||||
| **3. Universal outbox + dispatcher** | Done | Migration 0009 (`outbox`), `OutboxRepo` + `PostgresOutboxRepo`, `Dispatcher` tokio task. Polls every 100ms, claims 8 rows/tick via `FOR UPDATE SKIP LOCKED`, gate-bounds dispatch, retries with backoff+jitter, dead-letters on exhaustion, late-completion → `abandoned_executions`. |
|
||||
| **4. NATS-style sync HTTP** | Done | `InboxRegistry` in orchestrator-core (in-process `Mutex<HashMap<Uuid, oneshot::Sender>>`), `InboxResolver` trait in shared. Orchestrator sync-route path registers receiver, writes outbox row with `reply_to`, awaits with timeout = script.timeout + 2s buffer. Status mapping per design notes §3 (422/502/503/504/507/500). |
|
||||
| **5. `dispatch_mode: async` HTTP routes** | Done | Migration 0012 adds the column (default `sync`). `DispatchMode` enum in shared. Route admin payload + RouteRepository serialize it. Compiled routes carry it; the matcher returns it in `Matched`. Orchestrator branches: async → outbox + 202; sync → outbox + inbox. |
|
||||
| **6. Dead letters** | Done | Migration 0010 (`dead_letters`), `DeadLetterRepo` + `DeadLetterService` + `PostgresDeadLetterService`. Rhai `dead_letters::{replay,resolve}` + admin endpoints (`GET /count`, `GET /`, `GET /{id}`, `POST /{id}/replay`, `POST /{id}/resolve`). `Capability::AppDeadLetterManage(AppId)` enforced. List intentionally NOT shipped (deferred to v1.2). Recursion-stop rule (handler-failure annotates original DL as `handler_failed`) implemented in the dispatcher. |
|
||||
| **7. Abandoned executions** | Done | Migration 0011, `AbandonedRepo` + `PostgresAbandonedRepo`, dispatcher writes a row on dropped-receiver inbox delivery. Metric path reserved (`TODO(metrics)` markers in dispatcher.rs). |
|
||||
| **8. Retry policy defaults** | Done | `TriggerConfig::from_env` (new module). Env vars: `PICLOUD_MAX_TRIGGER_DEPTH`, `PICLOUD_TRIGGER_RETRY_{MAX_ATTEMPTS,BACKOFF,BASE_MS,JITTER_PCT}`, `PICLOUD_DEAD_LETTER_RETENTION_DAYS`, `PICLOUD_ABANDONED_EXECUTIONS_RETENTION_DAYS`. Per-trigger overrides applied at trigger-creation time. |
|
||||
| **9. `ctx.event` for triggered scripts** | Done | `TriggerEvent` enum in shared (KV / DeadLetter variants), `SdkCallCx.event: Option<TriggerEvent>` + `is_dead_letter_handler: bool`. `engine.rs::build_ctx_map` flattens the event into `ctx.event` for triggered handlers; direct invocations leave the key absent. Shape matches design notes §4 (KV with op + collection + key + value; dead_letter with original + attempts + last_error + ids + timestamps). |
|
||||
| **10. Dashboard surface** | Done | Per-app red badge with unresolved count on apps list + per-app detail page. New `apps/[slug]/dead-letters/+page.svelte` list view with all design-notes-mandated columns + Replay + Mark resolved actions + expandable row detail. svelte-check passes (369 files, 0 errors, 0 warnings). |
|
||||
| **11. Workspace version bump** | Done | Workspace `1.1.0` → `1.1.1`, SDK `1.1` → `1.2`, dashboard `0.6.0` → `0.7.0`. CHANGELOG.md created at repo root. |
|
||||
|
||||
## 3. Key implementation decisions / deviations
|
||||
|
||||
### Outbox column set (deferred to implementation per design notes §2)
|
||||
Chose:
|
||||
- `script_id` denormalized — dispatcher resolves the target without
|
||||
re-joining for the common path.
|
||||
- `trigger_id` polymorphic (no DB FK) — references `triggers.id` for
|
||||
`source_kind IN {kv, dead_letter}`, `routes.id` for
|
||||
`source_kind = 'http'`. Discrimination in Rust at dispatch time.
|
||||
- `claimed_by TEXT` — pid-based for MVP; cluster mode can use any
|
||||
identifier without schema change.
|
||||
- `trigger_depth` + `root_execution_id` denormalized so the
|
||||
dispatcher rebuilds `ExecRequest` without joining back to the
|
||||
originating execution log.
|
||||
- No explicit `is_dead_letter_handler` column — dispatcher infers
|
||||
from the trigger's `kind` field at dispatch time.
|
||||
|
||||
### KV pagination
|
||||
- **Cursor-style**, opaque base64-encoded last-key.
|
||||
- Page-size cap of 1000 with default 100 (enforced in repo).
|
||||
- Documented in `crates/shared/src/kv.rs` and the SDK function
|
||||
comment.
|
||||
|
||||
### KV TTL
|
||||
- Blueprint §8.1 reserved an `expires_at` column. v1.1.1 design notes
|
||||
don't surface TTL through the SDK (`set(k,v)` has no TTL argument)
|
||||
so the column is **omitted from migration 0007**. Adding it later
|
||||
is a non-breaking forward migration. Recorded in CHANGELOG as a
|
||||
deferred item.
|
||||
|
||||
### Authz scope mapping (seven-scope commitment)
|
||||
The four new capabilities map onto existing scopes — **no new scope
|
||||
variants** to honour the `Scope` enum's "exactly seven values"
|
||||
contract (`crates/shared/src/auth.rs:103`):
|
||||
|
||||
| Capability | Scope |
|
||||
|---|---|
|
||||
| `AppKvRead` | `script:read` |
|
||||
| `AppKvWrite` | `script:write` |
|
||||
| `AppManageTriggers` | `app:admin` |
|
||||
| `AppDeadLetterManage` | `app:admin` |
|
||||
|
||||
`role_satisfies` grants `AppKvRead` at the Viewer role, `AppKvWrite`
|
||||
at Editor, and both trigger / DL caps at AppAdmin.
|
||||
|
||||
### Script-as-gate authz for SDK calls
|
||||
- `KvServiceImpl` runs `authz::require` only when
|
||||
`cx.principal.is_some()`. Anonymous public-HTTP scripts (the
|
||||
common case for public routes) bypass the cap check.
|
||||
- Cross-app isolation is **independent** of this — enforced by
|
||||
`cx.app_id` being the only source of `app_id` on every query.
|
||||
- `PostgresDeadLetterService::{replay,resolve}` keeps a hard
|
||||
`require` (no `if let Some`) — managing dead letters is an admin
|
||||
act per design notes §4. Public scripts with `principal: None`
|
||||
fail the check, which is correct.
|
||||
|
||||
### Trait split: `OutboxRepo` vs `OutboxWriter`
|
||||
orchestrator-core can't depend on manager-core (would invert the
|
||||
dependency arrow). Defined a small `OutboxWriter` trait in
|
||||
`picloud-shared` with a single `enqueue_http` method.
|
||||
`PostgresOutboxRepo` implements both `OutboxRepo` (dispatcher
|
||||
surface) and `OutboxWriter` (orchestrator surface); the picloud
|
||||
binary clones one concrete Arc into both trait views — mirrors the
|
||||
existing `members_concrete` / `AuthzRepo` pattern.
|
||||
|
||||
### `InboxResolver` lives in shared, `InboxRegistry` in orchestrator-core
|
||||
Same split rationale — the dispatcher (manager-core) only depends on
|
||||
the trait, while the in-process impl lives next to its consumer.
|
||||
Cluster mode (v1.3+) swaps the impl for `LISTEN/NOTIFY` behind the
|
||||
unchanged trait.
|
||||
|
||||
### manager-core now depends on executor-core
|
||||
Previously manager-core only depended on orchestrator-core. The
|
||||
dispatcher needs `ExecRequest`/`ExecResponse`/`ExecError`/
|
||||
`InvocationType` from `executor-core` to build invocation
|
||||
descriptors. This is the transport DTO interpretation of the
|
||||
working-rules "don't reach across `*-core` crates" — DTOs are fine,
|
||||
behaviour is the bright line.
|
||||
|
||||
### Sync HTTP via outbox is the default for the user-routes path
|
||||
The orchestrator's user-route handler is fully on the NATS-style
|
||||
path now — every sync HTTP request writes to the outbox and awaits
|
||||
inbox delivery. Adds ~2-5ms per request per design notes §3 latency
|
||||
budget. `/api/v1/execute/{id}` (the admin/dev bypass) still calls
|
||||
the executor directly since it doesn't need the unified
|
||||
observability — kept for simplicity and admin tooling speed.
|
||||
|
||||
### Trigger-depth check is on the outbox row, not in the executor
|
||||
Dispatcher rejects depth-exceeded rows **before** trying to
|
||||
execute. The `cx.trigger_depth` field is informational on the
|
||||
executor side. Rejection writes a log + (reserved) metric and
|
||||
deletes the row — no DL, per design notes §4.
|
||||
|
||||
## 4. Tests added
|
||||
|
||||
### Unit tests (no DB required)
|
||||
- `manager-core::kv_service::tests` (10 tests) — round-trip, missing
|
||||
key returns None, `has` predicate, `delete` was-present,
|
||||
empty-collection rejection, **cross-app isolation**, anonymous-cx
|
||||
skips authz, authed-cx-with-no-role is Forbidden, owner-can-write,
|
||||
cursor pagination via in-memory KvRepo + denying authz repo.
|
||||
- `manager-core::trigger_config::tests` (2 tests) — conservative
|
||||
defaults, backoff round-trips.
|
||||
- `manager-core::trigger_repo::tests` (1 test) — `collection_matches`
|
||||
glob behaviour (`*`, `prefix:*`, exact).
|
||||
- `manager-core::dispatcher::tests` (5 tests) — exponential / linear /
|
||||
constant backoff math, jitter within bounds, ExecError →
|
||||
InboxFailureKind classification, failure-kind → status-code mapping.
|
||||
- `manager-core::abandoned_repo::tests` (2 tests) — truncate
|
||||
char-boundary safety.
|
||||
- `manager-core::triggers_api::tests` (5 tests) — unknown-app 404,
|
||||
member-without-role 403, default fallback for retry settings,
|
||||
empty-glob rejection, cross-app delete is treated as not-found.
|
||||
- `orchestrator-core::inbox::tests` (4 tests) — register/deliver
|
||||
round-trip, unknown-id is Abandoned, dropped receiver is
|
||||
Abandoned, explicit cancel removes sender.
|
||||
- `executor-core::engine::tests` (3 new) — `ctx.event` absent for
|
||||
direct invocations, KV insert shape matches design notes,
|
||||
KV delete has unit value.
|
||||
- `executor-core::sdk_kv` integration suite (7 tests) — runs a real
|
||||
Rhai engine under `spawn_blocking` against an in-memory
|
||||
`KvService` impl. Covers handle pattern, round-trip, unit-on-
|
||||
missing, has predicate, delete-was-present, empty-collection
|
||||
throws, cursor pagination, **cross-app isolation through the
|
||||
bridge**.
|
||||
|
||||
**Total: 47 new tests across the workspace.** Workspace test counts
|
||||
after v1.1.1: 63 manager-core / 56 orchestrator-core / 17
|
||||
executor-core engine / 7 sdk_kv / 30 sdk_contract / 43 stdlib /
|
||||
21 picloud / 6 shared.
|
||||
|
||||
### Intentionally untested
|
||||
- DB-backed integration tests for the full dispatcher loop, KV→
|
||||
trigger→DL retry chain, sync HTTP via outbox round-trip,
|
||||
recursion-stop end-to-end. These need a real Postgres harness;
|
||||
the reviewer runs them via the manual smoke flow below.
|
||||
- Postgres-specific repo behaviour (sqlx query correctness). The
|
||||
repos compile and run against the schema, but no integration
|
||||
test crate spins up a DB in this branch — same pattern as v1.1.0
|
||||
(see existing `ignored, needs DATABASE_URL` test markers).
|
||||
|
||||
## 5. Open questions for the reviewer
|
||||
|
||||
1. **Outbox `claimed_at` clearing on success.** The dispatcher
|
||||
`delete`s the outbox row after success / DL. For failures it
|
||||
reschedules (which sets `claimed_at = NULL`). Both flows are
|
||||
correct, but if you imagine a crash between the executor return
|
||||
and the row update, the row stays claimed forever. Cluster mode
|
||||
should add a periodic "unstick stale claims" sweep. Not in
|
||||
v1.1.1 scope but worth surfacing.
|
||||
2. **Sync HTTP overhead.** Every sync HTTP request now goes through
|
||||
the outbox (write + dispatcher pickup + inbox delivery).
|
||||
Measured overhead expected ~2-5ms per design notes §3. No
|
||||
benchmarking yet — recommend the reviewer pick a representative
|
||||
script and compare 95p latency vs v1.1.0 if performance matters.
|
||||
3. **HTTP outbox rows don't run as a principal.** The orchestrator's
|
||||
public HTTP path has no authenticated user; the
|
||||
`origin_principal` field on the outbox row is forensic. The
|
||||
resulting `ExecRequest.principal = None`, so the script runs
|
||||
anonymously — matches direct execution. If you'd prefer
|
||||
triggered-from-HTTP scripts to inherit a derived principal
|
||||
(e.g. the route's app's owner), that's an additive change.
|
||||
4. **Dispatcher uses `ASYNC_EXEC_TIMEOUT = 300s` for async rows.**
|
||||
Async dispatches don't have a script-level timeout (no
|
||||
originating HTTP request to bound). Picked the same platform
|
||||
cap as `LocalExecutorClient`. If async needs a different cap,
|
||||
easy to thread through `TriggerConfig`.
|
||||
5. **Dispatcher tick cadence is 100ms.** Bounded enough that
|
||||
fan-out feels instant; loose enough that an idle process
|
||||
doesn't burn cycles. If the reviewer wants tighter latency,
|
||||
bump to 50ms or use `LISTEN/NOTIFY` for wake-up (v1.3+ work).
|
||||
6. **CHANGELOG.md is new.** Followed the rest of the repo's
|
||||
convention from git log (release commits + design-notes
|
||||
references). If a different format is preferred, easy to swap.
|
||||
|
||||
## 6. Deferred to later releases
|
||||
|
||||
- `dead_letters::list(filter)` Rhai SDK — design notes §4 defers
|
||||
to v1.2 to align with `docs::find()` query DSL.
|
||||
- KV TTL (`set(k, v, ttl_secs)`) — blueprint reserved it; v1.1.1
|
||||
SDK doesn't surface it. Forward-compat (no schema cost).
|
||||
- Auto-disable of triggers whose script was deleted — design notes
|
||||
§4 says current handling is metric+log; auto-disable is v1.2.
|
||||
- Per-app dead-letter retention — design notes §4 says env-only in
|
||||
v1.1.1.
|
||||
- Metrics counter emit for `picloud_trigger_depth_exceeded`,
|
||||
`picloud_dead_letter_handler_failures`,
|
||||
`picloud_abandoned_executions_total`. Code paths log the
|
||||
occurrences with `tracing::warn`/`error`; the actual
|
||||
counter-emit code is a `TODO(metrics)` comment in the
|
||||
dispatcher. Metrics surface is v1.1.7+ per the roadmap.
|
||||
- DB-backed integration tests for the dispatcher loop (see §4
|
||||
intentionally-untested).
|
||||
- Sync HTTP performance benchmarks comparing v1.1.0 direct path vs
|
||||
v1.1.1 outbox path.
|
||||
|
||||
## 7. How to verify locally
|
||||
|
||||
### Static checks (all green on this branch)
|
||||
```sh
|
||||
cargo fmt --all -- --check
|
||||
cargo clippy --all-targets --all-features -- -D warnings
|
||||
cargo test --workspace
|
||||
cd dashboard && npm run check && npm run build
|
||||
```
|
||||
|
||||
### Migration integrity
|
||||
```sh
|
||||
docker compose down -v && docker compose up -d postgres
|
||||
cargo run -p picloud # applies 0001..0012 from empty
|
||||
```
|
||||
Then start from `main` (v1.1.0 schema state) and switch to this
|
||||
branch; restart `picloud` to apply 0007..0012 on top.
|
||||
|
||||
### Manual end-to-end smoke (reviewer should run)
|
||||
```sh
|
||||
docker compose up -d
|
||||
# 1. Bootstrap an owner user via the existing flow + create app A.
|
||||
# 2. Create a script in A whose body is: throw "boom"
|
||||
# 3. POST /api/v1/admin/apps/{A}/triggers/kv with
|
||||
# {"script_id": "<broken>", "collection_glob": "*", "ops": ["insert"]}
|
||||
# 4. From another script (or a public HTTP route):
|
||||
# kv::collection("widgets").set("k1", #{n:1})
|
||||
# 5. Wait ~7 seconds (3 attempts × ~1/2/4s backoff with ±20% jitter).
|
||||
# 6. Open the dashboard at /admin.
|
||||
# 7. Apps list shows a red "1" badge next to app A.
|
||||
# 8. Click into app A → "Dead letters" tab link → row visible.
|
||||
# 9. Click row → full payload + error history.
|
||||
# 10. Click "Replay" → row marks resolution='replayed', new outbox
|
||||
# row written, dispatcher re-runs the handler (fails again,
|
||||
# produces a NEW DL row).
|
||||
# 11. Click "Mark resolved" on the original DL → resolution='ignored'.
|
||||
```
|
||||
|
||||
### Async route smoke
|
||||
```sh
|
||||
# Create a route via POST /api/v1/admin/scripts/{id}/routes with
|
||||
# {"host_kind":"any","path_kind":"exact","path":"/work","dispatch_mode":"async"}
|
||||
curl -X POST -d '{"work":"thing"}' http://localhost:8080/work
|
||||
# Expect: HTTP 202 + {"accepted_at":"...","execution_id":"..."}
|
||||
# Then tail execution_logs — the script ran later (not synchronously).
|
||||
```
|
||||
|
||||
### Trigger-depth limit smoke
|
||||
```sh
|
||||
# Set a low depth limit + register a KV trigger whose script
|
||||
# writes to KV again — creates a loop.
|
||||
PICLOUD_MAX_TRIGGER_DEPTH=3 cargo run -p picloud
|
||||
# kv.set(...) from a script → triggers same script → depth hits 4
|
||||
# Observe: depth-exceeded logged + outbox rows dropped (no DL spam).
|
||||
```
|
||||
|
||||
## 8. Known limitations / rough edges
|
||||
|
||||
- **No DB-backed integration tests in this branch.** Unit tests
|
||||
cover trait behaviour with in-memory backings; sqlx query
|
||||
correctness is verified by the workspace compile + manual smoke.
|
||||
- **Dispatcher concurrency is in-process serial-per-tick.** Up to
|
||||
8 rows claimed per tick, processed one at a time. Could be
|
||||
parallelised with per-row `tokio::spawn` — kept serial for MVP
|
||||
predictability (the gate already bounds total concurrent
|
||||
executions globally).
|
||||
- **Metric emission is TODO** at the three spots noted in
|
||||
Open Questions §5. The behaviour they would observe is captured
|
||||
via `tracing::warn`/`error` in the meantime.
|
||||
- **`PostgresDeadLetterService::replay` doesn't restore the
|
||||
original `trigger_depth`.** Replays start at depth 0. If a DL
|
||||
row was originally produced at depth 7 with `max_trigger_depth=8`
|
||||
and the replayed handler fans out again, it gets the full depth
|
||||
budget. Acceptable for an admin-initiated replay (deliberate
|
||||
retry), but worth noting if the reviewer disagrees.
|
||||
- **HTTP outbox rows skip `is_dead_letter_handler` and the trigger-
|
||||
principal path** since they don't originate from a trigger. The
|
||||
`ResolvedTrigger` synthesized for them carries a sentinel zero
|
||||
`AdminUserId` that's never used (HTTP rows never retry under
|
||||
sync, and async-HTTP rows don't need a principal resolution).
|
||||
- **DataPlaneState's executor field is still generic** (`Arc<E>`
|
||||
where `E: ExecutorClient`). The dispatcher uses `Arc<dyn
|
||||
ExecutorClient>` directly. The picloud binary clones the same
|
||||
`Arc<LocalExecutorClient>` into both — works because the
|
||||
concrete type implements both the trait object and the generic
|
||||
bound.
|
||||
- **dispatcher always sets `principal: None` for HTTP rows.** As
|
||||
noted in Open Question §3, HTTP outbox rows don't resolve a
|
||||
principal. Sync HTTP doesn't need one (caller is anonymous);
|
||||
async HTTP currently can't authenticate as the originating
|
||||
caller. If that's not the intent, additive change.
|
||||
- **Cluster-mode crash recovery for claimed rows.** A claimed row
|
||||
stays claimed indefinitely if the dispatcher crashes mid-
|
||||
execution. v1.1.1 has one dispatcher per process so this is
|
||||
rare; cluster mode (v1.3+) needs a stale-claim sweeper.
|
||||
|
||||
---
|
||||
|
||||
Branch ready for review. Reviewer reads this report + audits the
|
||||
diff. Do not merge to main until the audit clears.
|
||||
151
REVIEW.md
Normal file
151
REVIEW.md
Normal file
@@ -0,0 +1,151 @@
|
||||
# v1.1.1 Audit & Review
|
||||
|
||||
**Branch:** `feat/v1.1.1-storage-and-events`
|
||||
**Base:** `main` (v1.1.0)
|
||||
**Commits ahead:** 12
|
||||
**Audited by:** reviewer (this report)
|
||||
**Audited against:** `docs/v1.1.x-design-notes.md` §1–4 (Decided 2026-06-01) + the original v1.1.1 dispatch prompt
|
||||
|
||||
## Verdict
|
||||
|
||||
**APPROVE — ready to merge to `main` as v1.1.1.**
|
||||
|
||||
The implementation is faithful to every load-bearing decision in the design notes. Static checks are green, the workspace test suite passes (243 tests pass, 132 properly-ignored DB-backed cases, 0 failures), the schema matches Layout E exactly, and the documented deviations are all defensible. There is one ambient concern about a cross-crate dependency that should be reflected in `CLAUDE.md` after the merge, but it is not a merge blocker.
|
||||
|
||||
---
|
||||
|
||||
## 1. Static checks reproduced
|
||||
|
||||
```
|
||||
cargo fmt --all -- --check ✅ clean
|
||||
cargo clippy --all-targets --all-features -- -D warnings ✅ no findings
|
||||
cargo test --workspace ✅ 243 passed / 0 failed
|
||||
(132 ignored — DB-backed integration tests,
|
||||
same convention as v1.1.0; documented in HANDBACK §4)
|
||||
```
|
||||
|
||||
Test distribution per crate matches HANDBACK §4:
|
||||
- manager-core: 63
|
||||
- orchestrator-core: 56
|
||||
- stdlib: 43
|
||||
- sdk_contract: 30
|
||||
- picloud: 21
|
||||
- executor-core (engine): 17
|
||||
- sdk_kv: 7
|
||||
- shared: 6
|
||||
|
||||
47 of these are new in v1.1.1; the rest are v1.1.0's existing suite still passing.
|
||||
|
||||
## 2. Design-notes conformance (spot-checks)
|
||||
|
||||
| Decision | Where it lives | Verdict |
|
||||
|---|---|---|
|
||||
| Layout E trigger storage (parent + per-kind detail) | [0008_triggers.sql:22-72](crates/manager-core/migrations/0008_triggers.sql#L22-L72) | ✅ matches exactly; parent has common columns + the four retry/dispatch knobs + `registered_by_principal`; per-kind detail tables for `kv` and `dead_letter` only |
|
||||
| `routes` stays separate from `triggers` parent | [0012_routes_dispatch_mode.sql](crates/manager-core/migrations/0012_routes_dispatch_mode.sql), [0009_outbox.sql:13-18](crates/manager-core/migrations/0009_outbox.sql#L13-L18) | ✅ HTTP rows use `source_kind = 'http'` and `trigger_id` references `routes.id`; non-HTTP references `triggers.id`; polymorphism in Rust per the design-notes deferral of the column-set refinement |
|
||||
| Sync HTTP via outbox + NATS-style inbox | [inbox.rs:30-89](crates/orchestrator-core/src/inbox.rs#L30-L89), [dispatcher.rs:359-394](crates/manager-core/src/dispatcher.rs#L359-L394) | ✅ `oneshot::Sender<InboxResult>` keyed by inbox_id; `deliver()` returns `Delivered` or `Abandoned` exactly per the design-notes failure-mode table |
|
||||
| `reply_to.is_some()` never retries | [dispatcher.rs:376-394](crates/manager-core/src/dispatcher.rs#L376-L394) | ✅ failure path checks `reply_to` first; delivers single outcome to inbox; deletes outbox row regardless of error |
|
||||
| Status code table (422/502/503/504/507/500) | [dispatcher.rs:555-564](crates/manager-core/src/dispatcher.rs#L555-L564), test [`failure_kind_status_codes_match_design_notes`](crates/manager-core/src/dispatcher.rs#L674) | ✅ exact mapping; covered by a dedicated test |
|
||||
| `dispatch_mode = async` returns `202 Accepted` + JSON body | [api.rs:325-332](crates/orchestrator-core/src/api.rs#L325-L332) | ✅ body shape is `{"accepted_at": rfc3339, "execution_id": uuid}` — matches design notes §2 verbatim |
|
||||
| Default retry: 3/exp/1000ms/±20% jitter | [trigger_config.rs](crates/manager-core/src/trigger_config.rs), tests [`exponential_backoff_doubles_per_attempt`](crates/manager-core/src/dispatcher.rs#L621), [`jitter_within_pct_of_base`](crates/manager-core/src/dispatcher.rs#L647) | ✅ env-overridable; jitter test exercises the ±20% bound across 100 samples |
|
||||
| `abandoned_executions` written on dropped receiver | [dispatcher.rs:480-509](crates/manager-core/src/dispatcher.rs#L480-L509) | ✅ written only when `InboxDeliveryOutcome::Abandoned` returns; ordinary timeout-with-receiver-still-alive does not write a row |
|
||||
| Dead-letter recursion stop (flag on execution) | [dispatcher.rs:396-425](crates/manager-core/src/dispatcher.rs#L396-L425), [trigger_repo.rs `TriggerKind::DeadLetter` → `is_dead_letter_handler`](crates/manager-core/src/dispatcher.rs#L228-L229) | ✅ flag set when dispatcher resolves a `kind = 'dead_letter'` trigger; on failure, original DL annotated with `resolution = 'handler_failed'`, row deleted, never retried, never DL'd |
|
||||
| Sync HTTP failures do NOT dead-letter | [dispatcher.rs:378-394](crates/manager-core/src/dispatcher.rs#L378-L394) | ✅ early return before the DL-write block |
|
||||
| `dead_letters::list` NOT shipped (deferred to v1.2) | [executor-core/src/sdk/dead_letters.rs:13](crates/executor-core/src/sdk/dead_letters.rs#L13) | ✅ explicit doc-comment citing design notes §4; only `replay` + `resolve` registered |
|
||||
| Trigger execution runs as registrant's principal | [dispatcher.rs:249-253](crates/manager-core/src/dispatcher.rs#L249-L253) + [`registered_by_principal` column](crates/manager-core/migrations/0008_triggers.sql#L39) | ✅ principal resolved from the trigger row at dispatch time |
|
||||
| 30-day DL retention, env-overridable | [gc.rs](crates/manager-core/src/gc.rs) | ✅ |
|
||||
| 7-day abandoned-executions retention | [gc.rs](crates/manager-core/src/gc.rs) | ✅ |
|
||||
| Trigger-depth limit (default 8); depth-exceeded does NOT dead-letter | [dispatcher.rs:122-137](crates/manager-core/src/dispatcher.rs#L122-L137) | ✅ design-notes §4 honored ("depth-exceeded means you built a loop") — row dropped + logged, no DL spam |
|
||||
| Dashboard surface: badge + list view + Replay + Mark resolved | [dashboard/src/routes/apps/+page.svelte](dashboard/src/routes/apps/+page.svelte), [dashboard/src/routes/apps/\[slug\]/dead-letters/+page.svelte](dashboard/src/routes/apps/[slug]/dead-letters/+page.svelte) | ✅ all required columns + actions + expandable row detail; `npm run check` reports 0 errors |
|
||||
| Status: workspace 1.1.0 → 1.1.1, SDK 1.1 → 1.2, dashboard 0.6.0 → 0.7.0, CHANGELOG.md created | last commit `66b661f` | ✅ |
|
||||
| `ctx.event` shape (KV: source/op/collection/key/value; DL: original/attempts/last_error/ids/timestamps) | [shared/src/trigger_event.rs](crates/shared/src/trigger_event.rs), [executor-core engine tests](crates/executor-core/src/engine.rs) | ✅ matches design notes §4 shape exactly; tests verify both variants + the "absent for direct invocations" rule |
|
||||
|
||||
I sampled the design-notes diff (`git diff main..HEAD -- docs/v1.1.x-design-notes.md`) — every "Decided 2026-06-01" entry the agent absorbed into commit `1efb350` matches the decisions made in conversation. No drift.
|
||||
|
||||
## 3. Deviations from the prompt (all reviewed, all acceptable)
|
||||
|
||||
The HANDBACK's §3 lists nine deviations / mid-implementation decisions. My take on each:
|
||||
|
||||
1. **Outbox column set chosen** (`script_id`, `trigger_id` polymorphic, `claimed_by TEXT`, `trigger_depth`, `root_execution_id` denormalized; no `is_dead_letter_handler` column). The design notes explicitly deferred this set to implementation. The chosen shape is sensible: dispatcher can build `ExecRequest` without re-joining; the `is_dead_letter_handler` derivation from `triggers.kind` at dispatch time is cleaner than storing redundant state. ✅
|
||||
|
||||
2. **KV pagination is cursor-style** (base64-encoded last-key, 100 default / 1000 max). The prompt left this open; cursor-style is the right default for KV-shaped data. ✅
|
||||
|
||||
3. **KV TTL deferred**. Blueprint §8.1 reserved `expires_at` but v1.1.1 SDK doesn't surface TTL. Omitting the column from migration 0007 keeps the schema minimal; adding it later is a non-breaking forward migration. ✅ (CHANGELOG records the deferral.)
|
||||
|
||||
4. **Authz scope mapping** (4 new capabilities mapped to existing 7 scopes — `AppKvRead → script:read`, `AppKvWrite → script:write`, `AppManageTriggers → app:admin`, `AppDeadLetterManage → app:admin`). The "seven-scope commitment" is a project convention in `crates/shared/src/auth.rs:103` the prompt didn't mention; honoring it is correct. The specific mapping is defensible: a token with `script:read` on an app already implies "can see the data behind those scripts," and admin-level scope for trigger/DL management is standard for control-plane operations. ✅
|
||||
|
||||
5. **Script-as-gate authz** (`if cx.principal.is_some()` then check; else skip — public HTTP runs anonymously without an authz failure). This matches the SDK-shape doc's note that "the data plane is unauthenticated by default — public HTTP scripts run with `None`." Cross-app isolation is preserved regardless (every query keyed by `cx.app_id`). DL replay/resolve correctly bypasses this and hard-requires a principal. ✅
|
||||
|
||||
6. **Trait split `OutboxRepo` vs `OutboxWriter`**. Orchestrator-core can't depend on manager-core; the small `OutboxWriter` trait in shared (one method) lets the orchestrator enqueue HTTP rows without inverting the dependency arrow. ✅ Pattern mirrors the existing `members_concrete`/`AuthzRepo` split.
|
||||
|
||||
7. **`InboxResolver` in shared, `InboxRegistry` in orchestrator-core**. Same split rationale. Cluster mode (v1.3+) swaps the impl behind the unchanged trait. ✅
|
||||
|
||||
8. **manager-core now depends on executor-core**. ⚠️ **See §4 below — flagged, accepted, but should be reflected in `CLAUDE.md`.**
|
||||
|
||||
9. **Sync HTTP via outbox is the default for user routes** (admin bypass `/api/v1/execute/{id}` keeps direct dispatch). Matches the design-notes decision; the bypass's direct path is acceptable for admin tooling speed. ✅
|
||||
|
||||
## 4. The one concern worth surfacing: manager-core → executor-core
|
||||
|
||||
`CLAUDE.md` working rules say:
|
||||
|
||||
> Honor the three-service boundary. Don't reach across `*-core` crates. If
|
||||
> orchestrator-core needs something from manager-core, define a trait in
|
||||
> shared and inject the impl.
|
||||
|
||||
The dispatcher in manager-core directly imports `ExecRequest`, `ExecResponse`, `ExecError`, and `InvocationType` from `executor-core`:
|
||||
|
||||
```rust
|
||||
// crates/manager-core/src/dispatcher.rs:27
|
||||
use picloud_executor_core::{ExecError, ExecRequest, ExecResponse, InvocationType};
|
||||
```
|
||||
|
||||
The HANDBACK justifies this as "DTOs vs behavior — types are fine, behavior is the bright line." That's a defensible interpretation, but not what `CLAUDE.md` actually says.
|
||||
|
||||
**Two options the project can pick:**
|
||||
|
||||
- **(a) Accept the dependency and update `CLAUDE.md`** to clarify that the three-service boundary is about *behavior*, not *types* — `ExecRequest`/`ExecResponse`/`ExecError` are transport DTOs and crossing the wire is normal. This is the lower-friction choice and matches how the agent's instincts ran.
|
||||
- **(b) Refactor**: move `ExecRequest`/`ExecResponse`/`ExecError`/`InvocationType` to `shared`. About 200 lines of moves; would land cleanly as a follow-up PR.
|
||||
|
||||
**My recommendation: (a)**. The dispatcher genuinely needs to construct and interpret these types, and they're the natural "what the executor produces" surface — burying them in shared makes the executor's public API less self-contained. But the rule as currently written disagrees; we should pick one explicitly.
|
||||
|
||||
This is **not a merge blocker** for v1.1.1 — the implementation already exists and works. The CLAUDE.md update can land as a small commit on `main` after the merge.
|
||||
|
||||
## 5. Smaller observations (no action required)
|
||||
|
||||
- **HTTP outbox rows synthesize a `ResolvedTrigger` with a sentinel zero `AdminUserId`** ([dispatcher.rs:342](crates/manager-core/src/dispatcher.rs#L342)). The HANDBACK flags this as a code smell; I agree, but the cleaner shape (`enum DispatchTarget { Trigger(ResolvedTrigger), Http(HttpRoute) }`) is a refactor that doesn't belong in v1.1.1. Worth doing in v1.1.2 alongside the docs work since the dispatcher will gain another trigger kind.
|
||||
- **Triggers parent `dispatch_mode` defaults to `'async'`** ([0008_triggers.sql:30](crates/manager-core/migrations/0008_triggers.sql#L30)) with `sync` allowed by the CHECK constraint but unsupported in v1.1.1 (sync trigger would mean firing inline with the originating mutation, which we don't do). The migration comment captures this; worth a future commit to either remove `'sync'` from the CHECK or use it for an `inline_pre_mutate` semantics if it ever makes sense. Not v1.1.1's problem.
|
||||
- **Metric counters are TODO** at three call sites (`picloud_trigger_depth_exceeded`, `picloud_dead_letter_handler_failures`, `picloud_abandoned_executions_total`). The events are logged via `tracing::warn`/`error` in the meantime. Per the prompt and roadmap, metrics surface is v1.1.7+. ✅
|
||||
- **Dispatcher tick cadence is 100ms with `CLAIM_BATCH = 8`**, serial per tick. The ExecutionGate bounds total concurrent executions globally, so parallelism within a tick is purely an optimization. Reasonable MVP choice; can parallelize later without changing semantics.
|
||||
- **Open Q1 in HANDBACK (claimed-rows-stuck-on-crash)** is a real cluster-mode concern, correctly out-of-scope for v1.1.1 (single dispatcher per process). Cluster mode adds a stale-claim sweeper — track for v1.3+.
|
||||
- **Open Q3 in HANDBACK (HTTP-triggered scripts run with `principal: None`)** is correct as-is. The "trigger executions inherit the registrant's principal" decision applies to triggers; HTTP routes have no registrant in that sense. Public HTTP is anonymous by design.
|
||||
|
||||
## 6. Versioning audit
|
||||
|
||||
| File | Before | After | Status |
|
||||
|---|---|---|---|
|
||||
| Workspace `Cargo.toml` (workspace.package.version) | 1.1.0 | 1.1.1 | ✅ |
|
||||
| SDK schema version (`shared/src/version.rs`) | 1.1 | 1.2 | ✅ correctly bumped — the SDK surface added `KvService` + `DeadLetterService` + `TriggerEvent` |
|
||||
| Dashboard `package.json` | 0.6.0 | 0.7.0 | ✅ |
|
||||
| Migrations | 0001..0006 | 0007..0012 added | ✅ sequential, no skips |
|
||||
| CHANGELOG.md | not present | created at repo root | ✅ first entry covers v1.1.1 |
|
||||
|
||||
## 7. Manual smoke recommendation
|
||||
|
||||
The reviewer (you) does **not** need to run the manual end-to-end smoke before merging — the automated tests + the static review above cover the contracts. The smoke flow in HANDBACK §7 is worth running **after merge** as a release-validation step before tagging `v1.1.1` (if the project tags releases). Specifically:
|
||||
|
||||
1. `docker compose up -d` (fresh DB)
|
||||
2. `cargo run -p picloud`
|
||||
3. Create app + script-that-throws + KV trigger
|
||||
4. Trigger a KV write → wait ~7s → confirm DL row appears
|
||||
5. Dashboard: red badge on apps list, list view shows the row, Replay creates a new outbox row + dispatcher re-runs, Mark resolved sets `resolution = 'ignored'`
|
||||
6. Async route test: `POST /work` with `dispatch_mode=async` route → expect 202 + JSON body
|
||||
|
||||
If any of those misbehave post-merge, revert is straightforward (12 commits, ahead of main, no dependencies have pulled changes yet).
|
||||
|
||||
## 8. Recommended next steps (post-merge)
|
||||
|
||||
1. **Merge** `feat/v1.1.1-storage-and-events` into `main` (fast-forward; branch is linear ahead).
|
||||
2. **Tag** `v1.1.1` if release tagging is the project convention (git log shows v1.1.0 had a release commit but I didn't see a tag — confirm with the project owner).
|
||||
3. **Small CLAUDE.md update** clarifying the three-service boundary's scope (types crossing is fine; behavior crossing is what's prohibited). One-paragraph change.
|
||||
4. **Pause** before dispatching the v1.1.2 (Documents) agent — the v1.1.1 work shipped substantial infrastructure that v1.1.2 will lean on, and there may be small lessons from the v1.1.1 implementation to fold into the v1.1.2 prompt (e.g., reaffirming the "manager-core depends on executor-core for DTOs" pattern explicitly so the docs agent doesn't second-guess it).
|
||||
|
||||
Branch is ready for merge. Verdict: **APPROVE**.
|
||||
@@ -14,7 +14,18 @@ picloud-shared.workspace = true
|
||||
serde.workspace = true
|
||||
serde_json.workspace = true
|
||||
thiserror.workspace = true
|
||||
tokio.workspace = true
|
||||
tracing.workspace = true
|
||||
uuid.workspace = true
|
||||
chrono.workspace = true
|
||||
rhai.workspace = true
|
||||
|
||||
# Stdlib utility modules — see crates/executor-core/src/sdk/stdlib/.
|
||||
regex.workspace = true
|
||||
rand.workspace = true
|
||||
base64.workspace = true
|
||||
hex.workspace = true
|
||||
percent-encoding.workspace = true
|
||||
|
||||
[dev-dependencies]
|
||||
async-trait.workspace = true
|
||||
|
||||
@@ -3,30 +3,40 @@ use std::sync::{Arc, Mutex};
|
||||
use std::time::Instant;
|
||||
|
||||
use chrono::Utc;
|
||||
use picloud_shared::{ScriptValidator, ValidationError, SDK_VERSION};
|
||||
use picloud_shared::{
|
||||
ScriptValidator, SdkCallCx, Services, TriggerEvent, ValidationError, SDK_VERSION,
|
||||
};
|
||||
use rhai::{Dynamic, Engine as RhaiEngine, EvalAltResult, Map, Module, Scope};
|
||||
use serde_json::Value as Json;
|
||||
|
||||
use crate::sandbox::Limits;
|
||||
use crate::sdk;
|
||||
use crate::sdk::bridge::{dynamic_to_json, json_to_dynamic};
|
||||
use crate::types::{
|
||||
ExecError, ExecRequest, ExecResponse, ExecStats, InvocationType, LogEntry, LogLevel,
|
||||
};
|
||||
|
||||
/// Preconfigured Rhai engine with sandbox limits applied.
|
||||
/// Preconfigured Rhai engine with sandbox limits applied and the SDK
|
||||
/// `Services` bundle attached.
|
||||
///
|
||||
/// One `Engine` is constructed at process startup and reused across
|
||||
/// invocations. `execute` is **synchronous** — it owns the per-call
|
||||
/// scope and log buffer. Wall-clock timeouts and offloading off the
|
||||
/// async runtime belong to the caller (orchestrator-core's
|
||||
/// `LocalExecutorClient` wraps this with `spawn_blocking` + `timeout`).
|
||||
///
|
||||
/// The `Services` bundle is empty in v1.1.0; subsequent v1.1.x PRs add
|
||||
/// service handles (KV, docs, …) and `sdk::register_all` wires them
|
||||
/// into each per-call Rhai engine.
|
||||
pub struct Engine {
|
||||
limits: Limits,
|
||||
services: Services,
|
||||
}
|
||||
|
||||
impl Engine {
|
||||
#[must_use]
|
||||
pub fn new(limits: Limits) -> Self {
|
||||
Self { limits }
|
||||
pub fn new(limits: Limits, services: Services) -> Self {
|
||||
Self { limits, services }
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
@@ -55,7 +65,22 @@ impl Engine {
|
||||
pub fn execute(&self, source: &str, req: ExecRequest) -> Result<ExecResponse, ExecError> {
|
||||
let effective_limits = self.limits.with_overrides(&req.sandbox_overrides);
|
||||
let logs: Arc<Mutex<Vec<LogEntry>>> = Arc::new(Mutex::new(Vec::new()));
|
||||
let engine = build_engine(effective_limits, Some(logs.clone()));
|
||||
let mut engine = build_engine(effective_limits, Some(logs.clone()));
|
||||
|
||||
// Per-call context handed to every stateful SDK service via the
|
||||
// `sdk::register_all` hook. The Arc lets future service closures
|
||||
// capture cheap clones of the cx for use at script-call time.
|
||||
let cx = Arc::new(SdkCallCx {
|
||||
app_id: req.app_id,
|
||||
principal: req.principal.clone(),
|
||||
execution_id: req.execution_id,
|
||||
request_id: req.request_id,
|
||||
trigger_depth: req.trigger_depth,
|
||||
root_execution_id: req.root_execution_id,
|
||||
is_dead_letter_handler: req.is_dead_letter_handler,
|
||||
event: req.event.clone(),
|
||||
});
|
||||
sdk::register_all(&mut engine, &self.services, cx);
|
||||
|
||||
let ast = engine
|
||||
.compile(source)
|
||||
@@ -122,6 +147,11 @@ fn build_engine(limits: Limits, logs: Option<Arc<Mutex<Vec<LogEntry>>>>) -> Rhai
|
||||
engine.register_static_module("log", build_log_module(logs).into());
|
||||
}
|
||||
|
||||
// Stateless utility modules — regex::/random::/time::/json::/base64::/
|
||||
// hex::/url::. Always registered, including in the parse-only validate
|
||||
// path, so script authors get consistent surface in both phases.
|
||||
sdk::stdlib::register_stdlib(&mut engine);
|
||||
|
||||
engine
|
||||
}
|
||||
|
||||
@@ -213,9 +243,82 @@ fn build_ctx_map(req: &ExecRequest) -> Map {
|
||||
request.insert("rest".into(), req.rest.clone().into());
|
||||
|
||||
ctx.insert("request".into(), request.into());
|
||||
|
||||
// Triggered invocations: surface the originating event as
|
||||
// `ctx.event`. Direct ingress (HTTP request, manual run) leaves
|
||||
// the key absent so scripts can test `if "event" in ctx`.
|
||||
if let Some(event) = req.event.as_ref() {
|
||||
ctx.insert("event".into(), trigger_event_to_dynamic(event));
|
||||
}
|
||||
|
||||
ctx
|
||||
}
|
||||
|
||||
/// Convert a `TriggerEvent` into the `ctx.event` Rhai shape defined in
|
||||
/// `docs/v1.1.x-design-notes.md` §4 (the dead-letter sub-shape) and
|
||||
/// §2/blueprint §9 (KV). Each variant becomes a Rhai map with a
|
||||
/// `source` discriminant plus per-source fields.
|
||||
fn trigger_event_to_dynamic(event: &TriggerEvent) -> Dynamic {
|
||||
let mut m = Map::new();
|
||||
m.insert("source".into(), event.source().into());
|
||||
match event {
|
||||
TriggerEvent::Kv {
|
||||
op,
|
||||
collection,
|
||||
key,
|
||||
value,
|
||||
} => {
|
||||
m.insert("op".into(), op.as_str().into());
|
||||
let mut kv_map = Map::new();
|
||||
kv_map.insert("collection".into(), collection.clone().into());
|
||||
kv_map.insert("key".into(), key.clone().into());
|
||||
kv_map.insert(
|
||||
"value".into(),
|
||||
value.clone().map_or(Dynamic::UNIT, json_to_dynamic),
|
||||
);
|
||||
m.insert("kv".into(), kv_map.into());
|
||||
}
|
||||
TriggerEvent::DeadLetter {
|
||||
dead_letter_id,
|
||||
original,
|
||||
attempts,
|
||||
last_error,
|
||||
trigger_id,
|
||||
script_id,
|
||||
first_attempt_at,
|
||||
last_attempt_at,
|
||||
} => {
|
||||
let mut dl = Map::new();
|
||||
dl.insert("id".into(), dead_letter_id.to_string().into());
|
||||
dl.insert("original".into(), trigger_event_to_dynamic(original));
|
||||
dl.insert("attempts".into(), i64::from(*attempts).into());
|
||||
dl.insert("last_error".into(), last_error.clone().into());
|
||||
dl.insert(
|
||||
"trigger_id".into(),
|
||||
trigger_id
|
||||
.map(|id| Dynamic::from(id.to_string()))
|
||||
.unwrap_or(Dynamic::UNIT),
|
||||
);
|
||||
dl.insert(
|
||||
"script_id".into(),
|
||||
script_id
|
||||
.map(|id| Dynamic::from(id.to_string()))
|
||||
.unwrap_or(Dynamic::UNIT),
|
||||
);
|
||||
dl.insert(
|
||||
"first_attempt_at".into(),
|
||||
first_attempt_at.to_rfc3339().into(),
|
||||
);
|
||||
dl.insert(
|
||||
"last_attempt_at".into(),
|
||||
last_attempt_at.to_rfc3339().into(),
|
||||
);
|
||||
m.insert("dead_letter".into(), dl.into());
|
||||
}
|
||||
}
|
||||
m.into()
|
||||
}
|
||||
|
||||
fn invocation_type_str(it: InvocationType) -> &'static str {
|
||||
match it {
|
||||
InvocationType::Http => "http",
|
||||
@@ -265,69 +368,6 @@ fn parse_structured_response(map: Map) -> Result<(u16, BTreeMap<String, String>,
|
||||
Ok((status_code, headers, body))
|
||||
}
|
||||
|
||||
// ----------------------------------------------------------------------------
|
||||
// Rhai ↔ serde_json bridges
|
||||
// ----------------------------------------------------------------------------
|
||||
|
||||
fn json_to_dynamic(value: Json) -> Dynamic {
|
||||
match value {
|
||||
Json::Null => Dynamic::UNIT,
|
||||
Json::Bool(b) => b.into(),
|
||||
Json::Number(n) => {
|
||||
if let Some(i) = n.as_i64() {
|
||||
i.into()
|
||||
} else if let Some(f) = n.as_f64() {
|
||||
f.into()
|
||||
} else {
|
||||
n.to_string().into()
|
||||
}
|
||||
}
|
||||
Json::String(s) => s.into(),
|
||||
Json::Array(arr) => arr
|
||||
.into_iter()
|
||||
.map(json_to_dynamic)
|
||||
.collect::<Vec<Dynamic>>()
|
||||
.into(),
|
||||
Json::Object(obj) => {
|
||||
let mut m = Map::new();
|
||||
for (k, v) in obj {
|
||||
m.insert(k.into(), json_to_dynamic(v));
|
||||
}
|
||||
Dynamic::from(m)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn dynamic_to_json(value: &Dynamic) -> Json {
|
||||
if value.is_unit() {
|
||||
return Json::Null;
|
||||
}
|
||||
if let Ok(b) = value.as_bool() {
|
||||
return Json::Bool(b);
|
||||
}
|
||||
if let Ok(i) = value.as_int() {
|
||||
return Json::Number(i.into());
|
||||
}
|
||||
if let Ok(f) = value.as_float() {
|
||||
return serde_json::Number::from_f64(f).map_or(Json::Null, Json::Number);
|
||||
}
|
||||
if value.is_string() {
|
||||
return Json::String(value.clone().into_string().unwrap_or_default());
|
||||
}
|
||||
if let Some(arr) = value.clone().try_cast::<rhai::Array>() {
|
||||
return Json::Array(arr.iter().map(dynamic_to_json).collect());
|
||||
}
|
||||
if let Some(map) = value.clone().try_cast::<Map>() {
|
||||
let mut out = serde_json::Map::new();
|
||||
for (k, v) in map {
|
||||
out.insert(k.to_string(), dynamic_to_json(&v));
|
||||
}
|
||||
return Json::Object(out);
|
||||
}
|
||||
// Anything else (timestamps, custom types) — best-effort string form.
|
||||
Json::String(value.to_string())
|
||||
}
|
||||
|
||||
// ----------------------------------------------------------------------------
|
||||
// Error mapping
|
||||
// ----------------------------------------------------------------------------
|
||||
|
||||
@@ -8,6 +8,7 @@ pub mod context;
|
||||
pub mod engine;
|
||||
pub mod logging;
|
||||
pub mod sandbox;
|
||||
pub mod sdk;
|
||||
pub mod types;
|
||||
|
||||
pub use engine::Engine;
|
||||
|
||||
77
crates/executor-core/src/sdk/bridge.rs
Normal file
77
crates/executor-core/src/sdk/bridge.rs
Normal file
@@ -0,0 +1,77 @@
|
||||
//! JSON ↔ Rhai `Dynamic` value bridge.
|
||||
//!
|
||||
//! Originally inline in `engine.rs`; moved here for v1.1.0 so future
|
||||
//! service modules (KV in v1.1.1, docs in v1.1.2, …) can convert
|
||||
//! values without `engine.rs` being the only owner of the conversions.
|
||||
//! Behaviour is unchanged from the pre-extraction implementation —
|
||||
//! `sdk_contract.rs::json_round_trip_preserves_nested_shapes` pins the
|
||||
//! observable round-trip.
|
||||
|
||||
use rhai::{Dynamic, Map};
|
||||
use serde_json::Value as Json;
|
||||
|
||||
/// Convert a `serde_json::Value` into a Rhai `Dynamic` suitable for
|
||||
/// pushing into a script's scope. Numbers prefer the narrowest type
|
||||
/// (`i64` over `f64`); anything that can't round-trip falls back to a
|
||||
/// string so the script always sees a defined value.
|
||||
pub fn json_to_dynamic(value: Json) -> Dynamic {
|
||||
match value {
|
||||
Json::Null => Dynamic::UNIT,
|
||||
Json::Bool(b) => b.into(),
|
||||
Json::Number(n) => {
|
||||
if let Some(i) = n.as_i64() {
|
||||
i.into()
|
||||
} else if let Some(f) = n.as_f64() {
|
||||
f.into()
|
||||
} else {
|
||||
n.to_string().into()
|
||||
}
|
||||
}
|
||||
Json::String(s) => s.into(),
|
||||
Json::Array(arr) => arr
|
||||
.into_iter()
|
||||
.map(json_to_dynamic)
|
||||
.collect::<Vec<Dynamic>>()
|
||||
.into(),
|
||||
Json::Object(obj) => {
|
||||
let mut m = Map::new();
|
||||
for (k, v) in obj {
|
||||
m.insert(k.into(), json_to_dynamic(v));
|
||||
}
|
||||
Dynamic::from(m)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Convert a Rhai `Dynamic` back to a `serde_json::Value`. Custom Rhai
|
||||
/// types (timestamps, user-registered modules) fall back to their
|
||||
/// `Display` form so they appear as strings in JSON output rather than
|
||||
/// failing the response build.
|
||||
pub fn dynamic_to_json(value: &Dynamic) -> Json {
|
||||
if value.is_unit() {
|
||||
return Json::Null;
|
||||
}
|
||||
if let Ok(b) = value.as_bool() {
|
||||
return Json::Bool(b);
|
||||
}
|
||||
if let Ok(i) = value.as_int() {
|
||||
return Json::Number(i.into());
|
||||
}
|
||||
if let Ok(f) = value.as_float() {
|
||||
return serde_json::Number::from_f64(f).map_or(Json::Null, Json::Number);
|
||||
}
|
||||
if value.is_string() {
|
||||
return Json::String(value.clone().into_string().unwrap_or_default());
|
||||
}
|
||||
if let Some(arr) = value.clone().try_cast::<rhai::Array>() {
|
||||
return Json::Array(arr.iter().map(dynamic_to_json).collect());
|
||||
}
|
||||
if let Some(map) = value.clone().try_cast::<Map>() {
|
||||
let mut out = serde_json::Map::new();
|
||||
for (k, v) in map {
|
||||
out.insert(k.to_string(), dynamic_to_json(&v));
|
||||
}
|
||||
return Json::Object(out);
|
||||
}
|
||||
Json::String(value.to_string())
|
||||
}
|
||||
10
crates/executor-core/src/sdk/cx.rs
Normal file
10
crates/executor-core/src/sdk/cx.rs
Normal file
@@ -0,0 +1,10 @@
|
||||
//! Re-export of `picloud_shared::SdkCallCx`.
|
||||
//!
|
||||
//! The type itself lives in `picloud-shared` because future stateful
|
||||
//! service impls live in `manager-core` (which `executor-core` must
|
||||
//! not depend on) and need to reference the same cx shape. This
|
||||
//! re-export lets executor-side code write
|
||||
//! `use picloud_executor_core::sdk::SdkCallCx;` instead of reaching
|
||||
//! into `picloud_shared` for one type.
|
||||
|
||||
pub use picloud_shared::SdkCallCx;
|
||||
84
crates/executor-core/src/sdk/dead_letters.rs
Normal file
84
crates/executor-core/src/sdk/dead_letters.rs
Normal file
@@ -0,0 +1,84 @@
|
||||
//! `dead_letters::` Rhai bridge.
|
||||
//!
|
||||
//! ```rhai
|
||||
//! dead_letters::replay("01234567-..."); // re-enqueue + mark replayed
|
||||
//! dead_letters::resolve("01234567-...", "ignored"); // close out the row
|
||||
//! ```
|
||||
//!
|
||||
//! Sync↔async via `Handle::current().block_on(...)` — same pattern as
|
||||
//! the `kv::` bridge (works because `LocalExecutorClient` runs the
|
||||
//! script under `spawn_blocking`).
|
||||
//!
|
||||
//! `dead_letters::list(filter)` is intentionally NOT shipped — design
|
||||
//! notes §4 defers it to v1.2 to align with the `docs::find()` query
|
||||
//! DSL.
|
||||
|
||||
use std::str::FromStr;
|
||||
use std::sync::Arc;
|
||||
|
||||
use picloud_shared::{DeadLetterError, DeadLetterId, SdkCallCx, Services};
|
||||
use rhai::{Engine as RhaiEngine, EvalAltResult, Module};
|
||||
use tokio::runtime::Handle as TokioHandle;
|
||||
use uuid::Uuid;
|
||||
|
||||
pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<SdkCallCx>) {
|
||||
let svc = services.dead_letters.clone();
|
||||
let mut module = Module::new();
|
||||
{
|
||||
let svc = svc.clone();
|
||||
let cx = cx.clone();
|
||||
module.set_native_fn(
|
||||
"replay",
|
||||
move |id: &str| -> Result<(), Box<EvalAltResult>> {
|
||||
let dl_id = parse_dl_id(id)?;
|
||||
let svc = svc.clone();
|
||||
let cx = cx.clone();
|
||||
block_on(async move { svc.replay(&cx, dl_id).await })
|
||||
},
|
||||
);
|
||||
}
|
||||
{
|
||||
let svc = svc.clone();
|
||||
let cx = cx.clone();
|
||||
module.set_native_fn(
|
||||
"resolve",
|
||||
move |id: &str, reason: &str| -> Result<(), Box<EvalAltResult>> {
|
||||
let dl_id = parse_dl_id(id)?;
|
||||
let reason = reason.to_string();
|
||||
let svc = svc.clone();
|
||||
let cx = cx.clone();
|
||||
block_on(async move { svc.resolve(&cx, dl_id, &reason).await })
|
||||
},
|
||||
);
|
||||
}
|
||||
engine.register_static_module("dead_letters", module.into());
|
||||
}
|
||||
|
||||
fn parse_dl_id(s: &str) -> Result<DeadLetterId, Box<EvalAltResult>> {
|
||||
Uuid::from_str(s)
|
||||
.map(DeadLetterId::from)
|
||||
.map_err(|e| -> Box<EvalAltResult> {
|
||||
EvalAltResult::ErrorRuntime(
|
||||
format!("dead_letters: invalid id {s:?}: {e}").into(),
|
||||
rhai::Position::NONE,
|
||||
)
|
||||
.into()
|
||||
})
|
||||
}
|
||||
|
||||
fn block_on<F>(fut: F) -> Result<(), Box<EvalAltResult>>
|
||||
where
|
||||
F: std::future::Future<Output = Result<(), DeadLetterError>> + Send,
|
||||
{
|
||||
let handle = TokioHandle::try_current().map_err(|e| -> Box<EvalAltResult> {
|
||||
EvalAltResult::ErrorRuntime(
|
||||
format!("dead_letters: no tokio runtime available: {e}").into(),
|
||||
rhai::Position::NONE,
|
||||
)
|
||||
.into()
|
||||
})?;
|
||||
handle.block_on(fut).map_err(|err| -> Box<EvalAltResult> {
|
||||
EvalAltResult::ErrorRuntime(format!("dead_letters: {err}").into(), rhai::Position::NONE)
|
||||
.into()
|
||||
})
|
||||
}
|
||||
193
crates/executor-core/src/sdk/kv.rs
Normal file
193
crates/executor-core/src/sdk/kv.rs
Normal file
@@ -0,0 +1,193 @@
|
||||
//! `kv::` Rhai bridge — collection-scoped handle pattern.
|
||||
//!
|
||||
//! ```rhai
|
||||
//! let widgets = kv::collection("widgets");
|
||||
//! widgets.set("k", #{ n: 1 });
|
||||
//! let v = widgets.get("k"); // value or () if absent
|
||||
//! if widgets.has("k") { ... }
|
||||
//! widgets.delete("k"); // bool (was-present)
|
||||
//! let page = widgets.list(); // returns #{ keys: [...], next_cursor: () }
|
||||
//! ```
|
||||
//!
|
||||
//! The `KvHandle` custom Rhai type captures the collection name once
|
||||
//! and routes each call through the injected `Arc<dyn KvService>` with
|
||||
//! the per-call `Arc<SdkCallCx>`. **The service derives `app_id` from
|
||||
//! `cx.app_id` — `app_id` never appears in any function signature
|
||||
//! script-side, preserving cross-app isolation.**
|
||||
//!
|
||||
//! Sync↔async bridge: Rhai is synchronous; the underlying service is
|
||||
//! async. Closures wrap each call in `Handle::current().block_on(...)`
|
||||
//! — safe because `LocalExecutorClient` runs the script under
|
||||
//! `spawn_blocking`, so a runtime handle is reachable and blocking on
|
||||
//! it doesn't park an async worker.
|
||||
//!
|
||||
//! Error convention (per `docs/sdk-shape.md`):
|
||||
//! - throw on failure (Rhai runtime error string)
|
||||
//! - `()` for absent values (`get` on a missing key)
|
||||
//! - `bool` for predicates (`has`; also `delete` returns was-present)
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use picloud_shared::{KvError, KvService, SdkCallCx, Services};
|
||||
use rhai::{Array, Dynamic, Engine as RhaiEngine, EvalAltResult, Map, Module};
|
||||
use tokio::runtime::Handle as TokioHandle;
|
||||
|
||||
use super::bridge::{dynamic_to_json, json_to_dynamic};
|
||||
|
||||
/// Per-call handle captured by the Rhai SDK. Cheap to clone (two Arcs
|
||||
/// plus an owned string).
|
||||
#[derive(Clone)]
|
||||
pub struct KvHandle {
|
||||
collection: String,
|
||||
service: Arc<dyn KvService>,
|
||||
cx: Arc<SdkCallCx>,
|
||||
}
|
||||
|
||||
pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<SdkCallCx>) {
|
||||
let kv_service = services.kv.clone();
|
||||
|
||||
// `kv::collection(name)` — handle constructor lives in the `kv`
|
||||
// static module so the script-visible call is `kv::collection(...)`.
|
||||
let mut module = Module::new();
|
||||
{
|
||||
let kv_service = kv_service.clone();
|
||||
let cx = cx.clone();
|
||||
module.set_native_fn(
|
||||
"collection",
|
||||
move |name: &str| -> Result<KvHandle, Box<EvalAltResult>> {
|
||||
if name.is_empty() {
|
||||
return Err("kv::collection name must not be empty".into());
|
||||
}
|
||||
Ok(KvHandle {
|
||||
collection: name.to_string(),
|
||||
service: kv_service.clone(),
|
||||
cx: cx.clone(),
|
||||
})
|
||||
},
|
||||
);
|
||||
}
|
||||
engine.register_static_module("kv", module.into());
|
||||
|
||||
// Methods on KvHandle — `register_fn` with `&mut KvHandle` first
|
||||
// argument lets Rhai dispatch them as `handle.get(k)` /
|
||||
// `handle.set(k, v)` / etc. through the dot-notation.
|
||||
engine.register_type_with_name::<KvHandle>("KvHandle");
|
||||
|
||||
register_get(engine);
|
||||
register_set(engine);
|
||||
register_has(engine);
|
||||
register_delete(engine);
|
||||
register_list(engine);
|
||||
}
|
||||
|
||||
fn register_get(engine: &mut RhaiEngine) {
|
||||
engine.register_fn(
|
||||
"get",
|
||||
|handle: &mut KvHandle, key: &str| -> Result<Dynamic, Box<EvalAltResult>> {
|
||||
let h = handle.clone();
|
||||
block_on(async move { h.service.get(&h.cx, &h.collection, key).await })
|
||||
.map(|opt| opt.map_or(Dynamic::UNIT, json_to_dynamic))
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
fn register_set(engine: &mut RhaiEngine) {
|
||||
engine.register_fn(
|
||||
"set",
|
||||
|handle: &mut KvHandle, key: &str, value: Dynamic| -> Result<(), Box<EvalAltResult>> {
|
||||
let h = handle.clone();
|
||||
let json = dynamic_to_json(&value);
|
||||
block_on(async move { h.service.set(&h.cx, &h.collection, key, json).await })
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
fn register_has(engine: &mut RhaiEngine) {
|
||||
engine.register_fn(
|
||||
"has",
|
||||
|handle: &mut KvHandle, key: &str| -> Result<bool, Box<EvalAltResult>> {
|
||||
let h = handle.clone();
|
||||
block_on(async move { h.service.has(&h.cx, &h.collection, key).await })
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
fn register_delete(engine: &mut RhaiEngine) {
|
||||
engine.register_fn(
|
||||
"delete",
|
||||
|handle: &mut KvHandle, key: &str| -> Result<bool, Box<EvalAltResult>> {
|
||||
let h = handle.clone();
|
||||
block_on(async move { h.service.delete(&h.cx, &h.collection, key).await })
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
fn register_list(engine: &mut RhaiEngine) {
|
||||
// Zero-arg form — full page, no cursor.
|
||||
engine.register_fn(
|
||||
"list",
|
||||
|handle: &mut KvHandle| -> Result<Map, Box<EvalAltResult>> { list_call(handle, None, 0) },
|
||||
);
|
||||
|
||||
// One-arg form — cursor only.
|
||||
engine.register_fn(
|
||||
"list",
|
||||
|handle: &mut KvHandle, cursor: &str| -> Result<Map, Box<EvalAltResult>> {
|
||||
list_call(handle, Some(cursor.to_string()), 0)
|
||||
},
|
||||
);
|
||||
|
||||
// Two-arg form — cursor + limit.
|
||||
engine.register_fn(
|
||||
"list",
|
||||
|handle: &mut KvHandle, cursor: &str, limit: i64| -> Result<Map, Box<EvalAltResult>> {
|
||||
let limit = u32::try_from(limit.max(0)).unwrap_or(0);
|
||||
list_call(handle, Some(cursor.to_string()), limit)
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
fn list_call(
|
||||
handle: &KvHandle,
|
||||
cursor: Option<String>,
|
||||
limit: u32,
|
||||
) -> Result<Map, Box<EvalAltResult>> {
|
||||
let h = handle.clone();
|
||||
let page = block_on(async move {
|
||||
h.service
|
||||
.list(&h.cx, &h.collection, cursor.as_deref(), limit)
|
||||
.await
|
||||
})?;
|
||||
let mut m = Map::new();
|
||||
let keys: Array = page.keys.into_iter().map(Dynamic::from).collect();
|
||||
m.insert("keys".into(), keys.into());
|
||||
m.insert(
|
||||
"next_cursor".into(),
|
||||
page.next_cursor.map_or(Dynamic::UNIT, Dynamic::from),
|
||||
);
|
||||
Ok(m)
|
||||
}
|
||||
|
||||
/// Run an async future inside the synchronous Rhai context.
|
||||
///
|
||||
/// `LocalExecutorClient` wraps script execution in `spawn_blocking`, so
|
||||
/// the current Tokio runtime is reachable via `Handle::current()`. We
|
||||
/// block on it directly; we are NOT calling this from an async task,
|
||||
/// so blocking is the correct primitive (`block_in_place` would also
|
||||
/// work, but we're already on a blocking worker).
|
||||
fn block_on<F, T>(fut: F) -> Result<T, Box<EvalAltResult>>
|
||||
where
|
||||
F: std::future::Future<Output = Result<T, KvError>> + Send,
|
||||
T: Send,
|
||||
{
|
||||
let handle = TokioHandle::try_current().map_err(|e| -> Box<EvalAltResult> {
|
||||
EvalAltResult::ErrorRuntime(
|
||||
format!("kv: no tokio runtime available: {e}").into(),
|
||||
rhai::Position::NONE,
|
||||
)
|
||||
.into()
|
||||
})?;
|
||||
handle.block_on(fut).map_err(|err| -> Box<EvalAltResult> {
|
||||
EvalAltResult::ErrorRuntime(format!("kv: {err}").into(), rhai::Position::NONE).into()
|
||||
})
|
||||
}
|
||||
37
crates/executor-core/src/sdk/mod.rs
Normal file
37
crates/executor-core/src/sdk/mod.rs
Normal file
@@ -0,0 +1,37 @@
|
||||
//! SDK plumbing — types and the per-call registration entry point.
|
||||
//!
|
||||
//! `executor-core` is responsible for building the per-invocation Rhai
|
||||
//! engine and wiring stateful services into it. v1.1.0 ships the
|
||||
//! shapes (`Services` bundle, `SdkCallCx`, `register_all` entry point)
|
||||
//! but no actual services — subsequent v1.1.x PRs (KV in v1.1.1,
|
||||
//! docs in v1.1.2, …) extend `register_all` rather than re-threading
|
||||
//! plumbing through `engine.rs`.
|
||||
//!
|
||||
//! Bridge functions (`json_to_dynamic` / `dynamic_to_json`) also live
|
||||
//! here so service modules can convert values without `engine.rs`
|
||||
//! being the only home for the conversion logic.
|
||||
|
||||
pub mod bridge;
|
||||
pub mod cx;
|
||||
pub mod dead_letters;
|
||||
pub mod kv;
|
||||
pub mod stdlib;
|
||||
|
||||
pub use bridge::{dynamic_to_json, json_to_dynamic};
|
||||
pub use cx::SdkCallCx;
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use picloud_shared::Services;
|
||||
use rhai::Engine as RhaiEngine;
|
||||
|
||||
/// Single hook every v1.1.x stateful service registers into. Called
|
||||
/// once per invocation, just after `build_engine` constructs the
|
||||
/// sandboxed Rhai engine and just before script compilation.
|
||||
///
|
||||
/// v1.1.1 wires the first stateful service (KV). Subsequent PRs add a
|
||||
/// single `<service>::register(...)` line per service.
|
||||
pub fn register_all(engine: &mut RhaiEngine, services: &Services, cx: Arc<SdkCallCx>) {
|
||||
kv::register(engine, services, cx.clone());
|
||||
dead_letters::register(engine, services, cx);
|
||||
}
|
||||
48
crates/executor-core/src/sdk/stdlib/base64.rs
Normal file
48
crates/executor-core/src/sdk/stdlib/base64.rs
Normal file
@@ -0,0 +1,48 @@
|
||||
//! `base64::` — standard and URL-safe Base64.
|
||||
//!
|
||||
//! Two encoders are exposed: standard alphabet with padding (`encode`/
|
||||
//! `decode`) and URL-safe alphabet without padding (`encode_url`/
|
||||
//! `decode_url`). Each encoder accepts both `String` and `Blob` inputs
|
||||
//! as separate Rhai overloads; decoders always return `Blob` — the
|
||||
//! caller knows whether the original bytes were textual.
|
||||
|
||||
use base64::engine::general_purpose::{STANDARD, URL_SAFE_NO_PAD};
|
||||
use base64::Engine as _;
|
||||
use rhai::{Blob, Engine as RhaiEngine, EvalAltResult, Module};
|
||||
|
||||
pub fn register(engine: &mut RhaiEngine) {
|
||||
let mut module = Module::new();
|
||||
|
||||
module.set_native_fn("encode", |s: &str| -> Result<String, Box<EvalAltResult>> {
|
||||
Ok(STANDARD.encode(s.as_bytes()))
|
||||
});
|
||||
module.set_native_fn("encode", |b: Blob| -> Result<String, Box<EvalAltResult>> {
|
||||
Ok(STANDARD.encode(&b))
|
||||
});
|
||||
module.set_native_fn("decode", |s: &str| -> Result<Blob, Box<EvalAltResult>> {
|
||||
STANDARD
|
||||
.decode(s)
|
||||
.map_err(|e| format!("base64::decode: {e}").into())
|
||||
});
|
||||
|
||||
module.set_native_fn(
|
||||
"encode_url",
|
||||
|s: &str| -> Result<String, Box<EvalAltResult>> {
|
||||
Ok(URL_SAFE_NO_PAD.encode(s.as_bytes()))
|
||||
},
|
||||
);
|
||||
module.set_native_fn(
|
||||
"encode_url",
|
||||
|b: Blob| -> Result<String, Box<EvalAltResult>> { Ok(URL_SAFE_NO_PAD.encode(&b)) },
|
||||
);
|
||||
module.set_native_fn(
|
||||
"decode_url",
|
||||
|s: &str| -> Result<Blob, Box<EvalAltResult>> {
|
||||
URL_SAFE_NO_PAD
|
||||
.decode(s)
|
||||
.map_err(|e| format!("base64::decode_url: {e}").into())
|
||||
},
|
||||
);
|
||||
|
||||
engine.register_static_module("base64", module.into());
|
||||
}
|
||||
21
crates/executor-core/src/sdk/stdlib/hex.rs
Normal file
21
crates/executor-core/src/sdk/stdlib/hex.rs
Normal file
@@ -0,0 +1,21 @@
|
||||
//! `hex::` — hexadecimal encode/decode (lowercase output, case-
|
||||
//! insensitive input). String and Blob inputs are both accepted on
|
||||
//! encode; decode always returns `Blob`.
|
||||
|
||||
use rhai::{Blob, Engine as RhaiEngine, EvalAltResult, Module};
|
||||
|
||||
pub fn register(engine: &mut RhaiEngine) {
|
||||
let mut module = Module::new();
|
||||
|
||||
module.set_native_fn("encode", |s: &str| -> Result<String, Box<EvalAltResult>> {
|
||||
Ok(hex::encode(s.as_bytes()))
|
||||
});
|
||||
module.set_native_fn("encode", |b: Blob| -> Result<String, Box<EvalAltResult>> {
|
||||
Ok(hex::encode(&b))
|
||||
});
|
||||
module.set_native_fn("decode", |s: &str| -> Result<Blob, Box<EvalAltResult>> {
|
||||
hex::decode(s).map_err(|e| format!("hex::decode: {e}").into())
|
||||
});
|
||||
|
||||
engine.register_static_module("hex", module.into());
|
||||
}
|
||||
43
crates/executor-core/src/sdk/stdlib/json.rs
Normal file
43
crates/executor-core/src/sdk/stdlib/json.rs
Normal file
@@ -0,0 +1,43 @@
|
||||
//! `json::` — JSON parse and stringify. Reuses the bridge functions in
|
||||
//! `crate::sdk::bridge` so script-visible JSON has the same shape
|
||||
//! (numbers, maps, arrays, nulls) as `ctx.request.body` already does.
|
||||
|
||||
use rhai::{Dynamic, Engine as RhaiEngine, EvalAltResult, Module};
|
||||
|
||||
use crate::sdk::bridge::{dynamic_to_json, json_to_dynamic};
|
||||
|
||||
pub fn register(engine: &mut RhaiEngine) {
|
||||
let mut module = Module::new();
|
||||
register_parse(&mut module);
|
||||
register_stringify(&mut module);
|
||||
register_stringify_pretty(&mut module);
|
||||
engine.register_static_module("json", module.into());
|
||||
}
|
||||
|
||||
fn register_parse(module: &mut Module) {
|
||||
module.set_native_fn("parse", |s: &str| -> Result<Dynamic, Box<EvalAltResult>> {
|
||||
let value: serde_json::Value =
|
||||
serde_json::from_str(s).map_err(|e| format!("json::parse: {e}"))?;
|
||||
Ok(json_to_dynamic(value))
|
||||
});
|
||||
}
|
||||
|
||||
fn register_stringify(module: &mut Module) {
|
||||
module.set_native_fn(
|
||||
"stringify",
|
||||
|v: Dynamic| -> Result<String, Box<EvalAltResult>> {
|
||||
serde_json::to_string(&dynamic_to_json(&v))
|
||||
.map_err(|e| format!("json::stringify: {e}").into())
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
fn register_stringify_pretty(module: &mut Module) {
|
||||
module.set_native_fn(
|
||||
"stringify_pretty",
|
||||
|v: Dynamic| -> Result<String, Box<EvalAltResult>> {
|
||||
serde_json::to_string_pretty(&dynamic_to_json(&v))
|
||||
.map_err(|e| format!("json::stringify_pretty: {e}").into())
|
||||
},
|
||||
);
|
||||
}
|
||||
25
crates/executor-core/src/sdk/stdlib/mod.rs
Normal file
25
crates/executor-core/src/sdk/stdlib/mod.rs
Normal file
@@ -0,0 +1,25 @@
|
||||
//! Stateless utility modules registered once at engine build via
|
||||
//! `Engine::register_static_module`. They have no per-call state, no
|
||||
//! cross-app sensitivity, and no `SdkCallCx` — distinguishing them
|
||||
//! from stateful service modules (KV, docs, …) which hook into
|
||||
//! `sdk::register_all` instead. See [docs/sdk-shape.md](../../../../../docs/sdk-shape.md).
|
||||
|
||||
use rhai::Engine as RhaiEngine;
|
||||
|
||||
pub mod base64;
|
||||
pub mod hex;
|
||||
pub mod json;
|
||||
pub mod random;
|
||||
pub mod regex;
|
||||
pub mod time;
|
||||
pub mod url;
|
||||
|
||||
pub fn register_stdlib(engine: &mut RhaiEngine) {
|
||||
regex::register(engine);
|
||||
random::register(engine);
|
||||
time::register(engine);
|
||||
json::register(engine);
|
||||
base64::register(engine);
|
||||
hex::register(engine);
|
||||
url::register(engine);
|
||||
}
|
||||
70
crates/executor-core/src/sdk/stdlib/random.rs
Normal file
70
crates/executor-core/src/sdk/stdlib/random.rs
Normal file
@@ -0,0 +1,70 @@
|
||||
//! `random::` — CSPRNG primitives (`rand::rngs::OsRng`).
|
||||
//!
|
||||
//! Only the OS RNG is exposed. No "fast non-crypto" variant — scripts
|
||||
//! should not pick between secure and insecure entropy. Output sizes
|
||||
//! are capped to keep a single script call from blowing host memory.
|
||||
|
||||
use rand::distributions::{Alphanumeric, DistString};
|
||||
use rand::{rngs::OsRng, Rng, RngCore};
|
||||
use rhai::{Blob, Engine as RhaiEngine, EvalAltResult, Module};
|
||||
use uuid::Uuid;
|
||||
|
||||
const MAX_BYTES: i64 = 65_536;
|
||||
const MAX_STRING: i64 = 4_096;
|
||||
|
||||
pub fn register(engine: &mut RhaiEngine) {
|
||||
let mut module = Module::new();
|
||||
register_int(&mut module);
|
||||
register_float(&mut module);
|
||||
register_bytes(&mut module);
|
||||
register_string(&mut module);
|
||||
register_uuid(&mut module);
|
||||
engine.register_static_module("random", module.into());
|
||||
}
|
||||
|
||||
fn register_int(module: &mut Module) {
|
||||
module.set_native_fn(
|
||||
"int",
|
||||
|min: i64, max: i64| -> Result<i64, Box<EvalAltResult>> {
|
||||
if min > max {
|
||||
return Err(format!("random::int: min ({min}) > max ({max})").into());
|
||||
}
|
||||
Ok(OsRng.gen_range(min..=max))
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
fn register_float(module: &mut Module) {
|
||||
module.set_native_fn("float", || -> Result<f64, Box<EvalAltResult>> {
|
||||
Ok(OsRng.gen::<f64>())
|
||||
});
|
||||
}
|
||||
|
||||
fn register_bytes(module: &mut Module) {
|
||||
module.set_native_fn("bytes", |n: i64| -> Result<Blob, Box<EvalAltResult>> {
|
||||
if !(0..=MAX_BYTES).contains(&n) {
|
||||
return Err(format!("random::bytes: n must be in 0..={MAX_BYTES}, got {n}").into());
|
||||
}
|
||||
// Safe: n is non-negative and bounded by MAX_BYTES, which fits in usize.
|
||||
let len = usize::try_from(n).expect("n bounded above by MAX_BYTES");
|
||||
let mut buf = vec![0u8; len];
|
||||
OsRng.fill_bytes(&mut buf);
|
||||
Ok(buf)
|
||||
});
|
||||
}
|
||||
|
||||
fn register_string(module: &mut Module) {
|
||||
module.set_native_fn("string", |n: i64| -> Result<String, Box<EvalAltResult>> {
|
||||
if !(0..=MAX_STRING).contains(&n) {
|
||||
return Err(format!("random::string: n must be in 0..={MAX_STRING}, got {n}").into());
|
||||
}
|
||||
let len = usize::try_from(n).expect("n bounded above by MAX_STRING");
|
||||
Ok(Alphanumeric.sample_string(&mut OsRng, len))
|
||||
});
|
||||
}
|
||||
|
||||
fn register_uuid(module: &mut Module) {
|
||||
module.set_native_fn("uuid", || -> Result<String, Box<EvalAltResult>> {
|
||||
Ok(Uuid::new_v4().to_string())
|
||||
});
|
||||
}
|
||||
105
crates/executor-core/src/sdk/stdlib/regex.rs
Normal file
105
crates/executor-core/src/sdk/stdlib/regex.rs
Normal file
@@ -0,0 +1,105 @@
|
||||
//! `regex::` — non-backtracking regular expressions (Rust `regex` crate).
|
||||
//!
|
||||
//! Patterns compile per call. No cache: premature for v1.1.0, and the
|
||||
//! `regex` crate's linear-time guarantees keep per-call cost bounded.
|
||||
//! Catastrophic patterns are rejected at compile time by the crate
|
||||
//! itself; no extra defense needed.
|
||||
|
||||
use regex::Regex;
|
||||
use rhai::{Array, Dynamic, Engine as RhaiEngine, EvalAltResult, Module};
|
||||
|
||||
pub fn register(engine: &mut RhaiEngine) {
|
||||
let mut module = Module::new();
|
||||
register_is_match(&mut module);
|
||||
register_find(&mut module);
|
||||
register_find_all(&mut module);
|
||||
register_replace(&mut module);
|
||||
register_replace_all(&mut module);
|
||||
register_split(&mut module);
|
||||
register_captures(&mut module);
|
||||
engine.register_static_module("regex", module.into());
|
||||
}
|
||||
|
||||
fn compile(pattern: &str) -> Result<Regex, Box<EvalAltResult>> {
|
||||
Regex::new(pattern).map_err(|e| format!("invalid regex: {e}").into())
|
||||
}
|
||||
|
||||
fn register_is_match(module: &mut Module) {
|
||||
module.set_native_fn(
|
||||
"is_match",
|
||||
|pattern: &str, text: &str| -> Result<bool, Box<EvalAltResult>> {
|
||||
Ok(compile(pattern)?.is_match(text))
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
fn register_find(module: &mut Module) {
|
||||
module.set_native_fn(
|
||||
"find",
|
||||
|pattern: &str, text: &str| -> Result<Dynamic, Box<EvalAltResult>> {
|
||||
Ok(compile(pattern)?
|
||||
.find(text)
|
||||
.map_or(Dynamic::UNIT, |m| Dynamic::from(m.as_str().to_string())))
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
fn register_find_all(module: &mut Module) {
|
||||
module.set_native_fn(
|
||||
"find_all",
|
||||
|pattern: &str, text: &str| -> Result<Array, Box<EvalAltResult>> {
|
||||
Ok(compile(pattern)?
|
||||
.find_iter(text)
|
||||
.map(|m| Dynamic::from(m.as_str().to_string()))
|
||||
.collect())
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
fn register_replace(module: &mut Module) {
|
||||
module.set_native_fn(
|
||||
"replace",
|
||||
|pattern: &str, text: &str, replacement: &str| -> Result<String, Box<EvalAltResult>> {
|
||||
Ok(compile(pattern)?.replace(text, replacement).into_owned())
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
fn register_replace_all(module: &mut Module) {
|
||||
module.set_native_fn(
|
||||
"replace_all",
|
||||
|pattern: &str, text: &str, replacement: &str| -> Result<String, Box<EvalAltResult>> {
|
||||
Ok(compile(pattern)?
|
||||
.replace_all(text, replacement)
|
||||
.into_owned())
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
fn register_split(module: &mut Module) {
|
||||
module.set_native_fn(
|
||||
"split",
|
||||
|pattern: &str, text: &str| -> Result<Array, Box<EvalAltResult>> {
|
||||
Ok(compile(pattern)?
|
||||
.split(text)
|
||||
.map(|s| Dynamic::from(s.to_string()))
|
||||
.collect())
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
fn register_captures(module: &mut Module) {
|
||||
module.set_native_fn(
|
||||
"captures",
|
||||
|pattern: &str, text: &str| -> Result<Dynamic, Box<EvalAltResult>> {
|
||||
let re = compile(pattern)?;
|
||||
Ok(re.captures(text).map_or(Dynamic::UNIT, |caps| {
|
||||
let arr: Array = caps
|
||||
.iter()
|
||||
.map(|m| m.map_or(Dynamic::UNIT, |m| Dynamic::from(m.as_str().to_string())))
|
||||
.collect();
|
||||
Dynamic::from(arr)
|
||||
}))
|
||||
},
|
||||
);
|
||||
}
|
||||
68
crates/executor-core/src/sdk/stdlib/time.rs
Normal file
68
crates/executor-core/src/sdk/stdlib/time.rs
Normal file
@@ -0,0 +1,68 @@
|
||||
//! `time::` — UTC time. The canonical "time value" is milliseconds
|
||||
//! since the Unix epoch as `i64`. ISO 8601 strings are for parsing and
|
||||
//! display only. UTC only — no timezone support in v1.1.0 (would pull
|
||||
//! in chrono-tz, deferred until a real use case demands it).
|
||||
|
||||
use chrono::{DateTime, SecondsFormat, Utc};
|
||||
use rhai::{Engine as RhaiEngine, EvalAltResult, Module};
|
||||
|
||||
pub fn register(engine: &mut RhaiEngine) {
|
||||
let mut module = Module::new();
|
||||
register_now(&mut module);
|
||||
register_now_ms(&mut module);
|
||||
register_parse(&mut module);
|
||||
register_format(&mut module);
|
||||
register_add_seconds(&mut module);
|
||||
register_diff_seconds(&mut module);
|
||||
engine.register_static_module("time", module.into());
|
||||
}
|
||||
|
||||
fn register_now(module: &mut Module) {
|
||||
module.set_native_fn("now", || -> Result<String, Box<EvalAltResult>> {
|
||||
Ok(Utc::now().to_rfc3339_opts(SecondsFormat::Millis, true))
|
||||
});
|
||||
}
|
||||
|
||||
fn register_now_ms(module: &mut Module) {
|
||||
module.set_native_fn("now_ms", || -> Result<i64, Box<EvalAltResult>> {
|
||||
Ok(Utc::now().timestamp_millis())
|
||||
});
|
||||
}
|
||||
|
||||
fn register_parse(module: &mut Module) {
|
||||
module.set_native_fn("parse", |iso: &str| -> Result<i64, Box<EvalAltResult>> {
|
||||
DateTime::parse_from_rfc3339(iso)
|
||||
.map(|dt| dt.timestamp_millis())
|
||||
.map_err(|e| format!("time::parse: invalid ISO 8601 / RFC 3339: {e}").into())
|
||||
});
|
||||
}
|
||||
|
||||
fn register_format(module: &mut Module) {
|
||||
module.set_native_fn("format", |ms: i64| -> Result<String, Box<EvalAltResult>> {
|
||||
DateTime::<Utc>::from_timestamp_millis(ms)
|
||||
.map(|dt| dt.to_rfc3339_opts(SecondsFormat::Millis, true))
|
||||
.ok_or_else(|| format!("time::format: ms ({ms}) out of representable range").into())
|
||||
});
|
||||
}
|
||||
|
||||
fn register_add_seconds(module: &mut Module) {
|
||||
module.set_native_fn(
|
||||
"add_seconds",
|
||||
|ms: i64, secs: i64| -> Result<i64, Box<EvalAltResult>> {
|
||||
secs.checked_mul(1000)
|
||||
.and_then(|delta| ms.checked_add(delta))
|
||||
.ok_or_else(|| format!("time::add_seconds: overflow (ms={ms}, secs={secs})").into())
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
fn register_diff_seconds(module: &mut Module) {
|
||||
module.set_native_fn(
|
||||
"diff_seconds",
|
||||
|a_ms: i64, b_ms: i64| -> Result<i64, Box<EvalAltResult>> {
|
||||
b_ms.checked_sub(a_ms)
|
||||
.map(|d| d / 1000)
|
||||
.ok_or_else(|| format!("time::diff_seconds: overflow (a={a_ms}, b={b_ms})").into())
|
||||
},
|
||||
);
|
||||
}
|
||||
64
crates/executor-core/src/sdk/stdlib/url.rs
Normal file
64
crates/executor-core/src/sdk/stdlib/url.rs
Normal file
@@ -0,0 +1,64 @@
|
||||
//! `url::` — RFC 3986 percent-encoding.
|
||||
//!
|
||||
//! `encode`/`decode` operate on opaque component values; `encode_query`
|
||||
//! builds an `application/x-www-form-urlencoded`-style query string
|
||||
//! from a Rhai `Map`. Key ordering is the map's natural order (Rhai's
|
||||
//! `Map` is a `BTreeMap`, so keys come out alphabetically — fine for
|
||||
//! query strings, which RFC 3986 leaves unordered).
|
||||
|
||||
use percent_encoding::{percent_decode_str, utf8_percent_encode, AsciiSet, NON_ALPHANUMERIC};
|
||||
use rhai::{Engine as RhaiEngine, EvalAltResult, Map, Module};
|
||||
|
||||
/// RFC 3986 unreserved set: `A-Z / a-z / 0-9 / - / _ / . / ~`.
|
||||
/// Everything outside this set gets percent-encoded.
|
||||
const UNRESERVED: &AsciiSet = &NON_ALPHANUMERIC
|
||||
.remove(b'-')
|
||||
.remove(b'_')
|
||||
.remove(b'.')
|
||||
.remove(b'~');
|
||||
|
||||
pub fn register(engine: &mut RhaiEngine) {
|
||||
let mut module = Module::new();
|
||||
register_encode(&mut module);
|
||||
register_decode(&mut module);
|
||||
register_encode_query(&mut module);
|
||||
engine.register_static_module("url", module.into());
|
||||
}
|
||||
|
||||
fn register_encode(module: &mut Module) {
|
||||
module.set_native_fn("encode", |s: &str| -> Result<String, Box<EvalAltResult>> {
|
||||
Ok(utf8_percent_encode(s, UNRESERVED).to_string())
|
||||
});
|
||||
}
|
||||
|
||||
fn register_decode(module: &mut Module) {
|
||||
module.set_native_fn("decode", |s: &str| -> Result<String, Box<EvalAltResult>> {
|
||||
percent_decode_str(s)
|
||||
.decode_utf8()
|
||||
.map(std::borrow::Cow::into_owned)
|
||||
.map_err(|e| format!("url::decode: invalid UTF-8: {e}").into())
|
||||
});
|
||||
}
|
||||
|
||||
fn register_encode_query(module: &mut Module) {
|
||||
module.set_native_fn(
|
||||
"encode_query",
|
||||
|m: Map| -> Result<String, Box<EvalAltResult>> {
|
||||
let mut out = String::new();
|
||||
for (k, v) in m {
|
||||
if !out.is_empty() {
|
||||
out.push('&');
|
||||
}
|
||||
out.push_str(&utf8_percent_encode(&k, UNRESERVED).to_string());
|
||||
out.push('=');
|
||||
// Coerce values via `to_string` rather than throwing on
|
||||
// non-strings — scripts commonly pass numbers/bools here
|
||||
// and a forced cast at the call site is friction with
|
||||
// no upside.
|
||||
let value = v.to_string();
|
||||
out.push_str(&utf8_percent_encode(&value, UNRESERVED).to_string());
|
||||
}
|
||||
Ok(out)
|
||||
},
|
||||
);
|
||||
}
|
||||
@@ -1,7 +1,9 @@
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
use chrono::{DateTime, Utc};
|
||||
use picloud_shared::{ExecutionId, RequestId, ScriptId, ScriptSandbox};
|
||||
use picloud_shared::{
|
||||
AppId, ExecutionId, Principal, RequestId, ScriptId, ScriptSandbox, TriggerEvent,
|
||||
};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use thiserror::Error;
|
||||
|
||||
@@ -50,6 +52,49 @@ pub struct ExecRequest {
|
||||
/// override) before the Rhai engine is built.
|
||||
#[serde(default)]
|
||||
pub sandbox_overrides: ScriptSandbox,
|
||||
|
||||
/// Owning application. Source of truth for every `(app_id, …)`
|
||||
/// storage lookup the script makes via stateful SDK services.
|
||||
/// Internal-only; not surfaced via `ctx` (which the script sees).
|
||||
pub app_id: AppId,
|
||||
|
||||
/// Caller identity, when authenticated. `None` for unauthenticated
|
||||
/// data-plane HTTP requests (the common case for public scripts);
|
||||
/// `Some` when a bearer token or session cookie was resolved.
|
||||
/// Internal-only — exposed via `SdkCallCx` to service trait impls.
|
||||
///
|
||||
/// `#[serde(skip)]`: `ExecRequest` is serializable so cluster mode
|
||||
/// (v1.3+) can ship invocations to remote executors over HTTP, but
|
||||
/// `Principal` has no wire derivation today. Skipping here keeps
|
||||
/// v1.1.0 compiling; the cluster-mode PR will introduce a wire-safe
|
||||
/// snapshot then.
|
||||
#[serde(skip)]
|
||||
pub principal: Option<Principal>,
|
||||
|
||||
/// Triggers-framework depth. `0` for direct invocations. The
|
||||
/// dispatcher (v1.1.1) increments on each indirection to bound
|
||||
/// runaway feedback loops.
|
||||
#[serde(default)]
|
||||
pub trigger_depth: u32,
|
||||
|
||||
/// Originating execution id of a trigger chain. Equal to
|
||||
/// `execution_id` for direct invocations; preserves the root
|
||||
/// across fan-out for audit log grouping.
|
||||
pub root_execution_id: ExecutionId,
|
||||
|
||||
/// `true` only when the dispatcher resolved this invocation
|
||||
/// against a `dead_letter` trigger. The retry / dead-letter
|
||||
/// machinery short-circuits when this is set so handler failures
|
||||
/// cannot themselves be dead-lettered (design notes §4
|
||||
/// recursion-stop rule).
|
||||
#[serde(default)]
|
||||
pub is_dead_letter_handler: bool,
|
||||
|
||||
/// The originating event for a triggered invocation. `None` for
|
||||
/// direct ingress (sync HTTP, manual admin run). Flattened into
|
||||
/// `ctx.event` by the executor's per-call ctx builder.
|
||||
#[serde(default)]
|
||||
pub event: Option<TriggerEvent>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
@@ -100,4 +145,11 @@ pub enum ExecError {
|
||||
|
||||
#[error("script runtime error: {0}")]
|
||||
Runtime(String),
|
||||
|
||||
/// Concurrency gate (orchestrator-core::ExecutionGate) refused
|
||||
/// admission. Surfaced as HTTP 503 with a `Retry-After` header.
|
||||
/// The gate enforces a global cap so a script storm can't park
|
||||
/// every blocking thread.
|
||||
#[error("execution declined: server at capacity (retry after {retry_after_secs}s)")]
|
||||
Overloaded { retry_after_secs: u32 },
|
||||
}
|
||||
|
||||
@@ -1,12 +1,15 @@
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
use picloud_executor_core::{Engine, ExecError, ExecRequest, InvocationType, Limits, LogLevel};
|
||||
use picloud_shared::{ExecutionId, RequestId, ScriptId, ScriptSandbox};
|
||||
use picloud_shared::{
|
||||
AppId, ExecutionId, KvEventOp, RequestId, ScriptId, ScriptSandbox, Services, TriggerEvent,
|
||||
};
|
||||
use serde_json::json;
|
||||
|
||||
fn req(body: serde_json::Value) -> ExecRequest {
|
||||
let execution_id = ExecutionId::new();
|
||||
ExecRequest {
|
||||
execution_id: ExecutionId::new(),
|
||||
execution_id,
|
||||
request_id: RequestId::new(),
|
||||
script_id: ScriptId::new(),
|
||||
script_name: "test".into(),
|
||||
@@ -18,11 +21,17 @@ fn req(body: serde_json::Value) -> ExecRequest {
|
||||
query: BTreeMap::new(),
|
||||
rest: String::new(),
|
||||
sandbox_overrides: ScriptSandbox::default(),
|
||||
app_id: AppId::new(),
|
||||
principal: None,
|
||||
trigger_depth: 0,
|
||||
root_execution_id: execution_id,
|
||||
is_dead_letter_handler: false,
|
||||
event: None,
|
||||
}
|
||||
}
|
||||
|
||||
fn engine() -> Engine {
|
||||
Engine::new(Limits::default())
|
||||
Engine::new(Limits::default(), Services::default())
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -121,7 +130,7 @@ fn enforces_operation_budget() {
|
||||
max_operations: 1_000,
|
||||
..Limits::default()
|
||||
};
|
||||
let engine = Engine::new(limits);
|
||||
let engine = Engine::new(limits, Services::default());
|
||||
// 10_000 iterations vastly exceeds 1_000 ops.
|
||||
let src = r"let n = 0; for i in 0..10000 { n += 1; } n";
|
||||
let err = engine
|
||||
@@ -230,3 +239,67 @@ fn body_passes_through_nested_json_round_trip() {
|
||||
let resp = engine().execute(src, req(body.clone())).unwrap();
|
||||
assert_eq!(resp.body, body);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ctx_event_absent_for_direct_invocations() {
|
||||
// Scripts not fired through the triggers framework see no
|
||||
// `ctx.event` key — they can use `"event" in ctx` to detect.
|
||||
let src = r#"
|
||||
if "event" in ctx { #{ statusCode: 500, body: "should be absent" } }
|
||||
else { "absent" }
|
||||
"#;
|
||||
let resp = engine().execute(src, req(json!(null))).unwrap();
|
||||
assert_eq!(resp.body, json!("absent"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ctx_event_kv_shape_matches_design_notes() {
|
||||
// Build an ExecRequest mimicking what the dispatcher hands a
|
||||
// KV-triggered handler — `event = Some(TriggerEvent::Kv { … })`.
|
||||
let mut r = req(json!(null));
|
||||
r.event = Some(TriggerEvent::Kv {
|
||||
op: KvEventOp::Insert,
|
||||
collection: "widgets".into(),
|
||||
key: "k1".into(),
|
||||
value: Some(json!({ "n": 1 })),
|
||||
});
|
||||
let src = r"
|
||||
#{
|
||||
source: ctx.event.source,
|
||||
op: ctx.event.op,
|
||||
collection: ctx.event.kv.collection,
|
||||
key: ctx.event.kv.key,
|
||||
value: ctx.event.kv.value
|
||||
}
|
||||
";
|
||||
let resp = engine().execute(src, r).unwrap();
|
||||
assert_eq!(
|
||||
resp.body,
|
||||
json!({
|
||||
"source": "kv",
|
||||
"op": "insert",
|
||||
"collection": "widgets",
|
||||
"key": "k1",
|
||||
"value": { "n": 1 }
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ctx_event_kv_delete_has_unit_value() {
|
||||
let mut r = req(json!(null));
|
||||
r.event = Some(TriggerEvent::Kv {
|
||||
op: KvEventOp::Delete,
|
||||
collection: "widgets".into(),
|
||||
key: "k1".into(),
|
||||
value: None,
|
||||
});
|
||||
let src = r"
|
||||
#{
|
||||
op: ctx.event.op,
|
||||
value_is_unit: ctx.event.kv.value == ()
|
||||
}
|
||||
";
|
||||
let resp = engine().execute(src, r).unwrap();
|
||||
assert_eq!(resp.body, json!({ "op": "delete", "value_is_unit": true }));
|
||||
}
|
||||
|
||||
@@ -23,7 +23,7 @@
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
use picloud_executor_core::{Engine, ExecRequest, InvocationType, Limits, LogLevel};
|
||||
use picloud_shared::{ExecutionId, RequestId, ScriptId, ScriptSandbox};
|
||||
use picloud_shared::{AppId, ExecutionId, RequestId, ScriptId, ScriptSandbox, Services};
|
||||
use serde_json::{json, Value};
|
||||
|
||||
// ----------------------------------------------------------------------------
|
||||
@@ -31,12 +31,13 @@ use serde_json::{json, Value};
|
||||
// ----------------------------------------------------------------------------
|
||||
|
||||
fn engine() -> Engine {
|
||||
Engine::new(Limits::default())
|
||||
Engine::new(Limits::default(), Services::default())
|
||||
}
|
||||
|
||||
fn baseline_request() -> ExecRequest {
|
||||
let execution_id = ExecutionId::new();
|
||||
ExecRequest {
|
||||
execution_id: ExecutionId::new(),
|
||||
execution_id,
|
||||
request_id: RequestId::new(),
|
||||
script_id: ScriptId::new(),
|
||||
script_name: "contract".into(),
|
||||
@@ -48,6 +49,12 @@ fn baseline_request() -> ExecRequest {
|
||||
query: BTreeMap::new(),
|
||||
rest: String::new(),
|
||||
sandbox_overrides: ScriptSandbox::default(),
|
||||
app_id: AppId::new(),
|
||||
principal: None,
|
||||
trigger_depth: 0,
|
||||
root_execution_id: execution_id,
|
||||
is_dead_letter_handler: false,
|
||||
event: None,
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
260
crates/executor-core/tests/sdk_kv.rs
Normal file
260
crates/executor-core/tests/sdk_kv.rs
Normal file
@@ -0,0 +1,260 @@
|
||||
//! `kv::` SDK bridge integration tests — runs a real Rhai engine
|
||||
//! against an in-memory `KvService` impl. Mirrors how
|
||||
//! `orchestrator-core::LocalExecutorClient` invokes the engine: under
|
||||
//! `tokio::task::spawn_blocking` so the bridge's `block_on` has a
|
||||
//! reachable runtime.
|
||||
|
||||
use std::collections::{BTreeMap, HashMap};
|
||||
use std::sync::Arc;
|
||||
|
||||
use async_trait::async_trait;
|
||||
use picloud_executor_core::{Engine, ExecRequest, InvocationType, Limits};
|
||||
use picloud_shared::{
|
||||
AppId, ExecutionId, KvError, KvListPage, KvService, NoopDeadLetterService, NoopEventEmitter,
|
||||
RequestId, ScriptId, ScriptSandbox, SdkCallCx, Services,
|
||||
};
|
||||
use serde_json::{json, Value};
|
||||
use tokio::sync::Mutex;
|
||||
|
||||
#[derive(Default)]
|
||||
struct InMemoryKv {
|
||||
data: Mutex<HashMap<(AppId, String, String), Value>>,
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl KvService for InMemoryKv {
|
||||
async fn get(
|
||||
&self,
|
||||
cx: &SdkCallCx,
|
||||
collection: &str,
|
||||
key: &str,
|
||||
) -> Result<Option<Value>, KvError> {
|
||||
Ok(self
|
||||
.data
|
||||
.lock()
|
||||
.await
|
||||
.get(&(cx.app_id, collection.to_string(), key.to_string()))
|
||||
.cloned())
|
||||
}
|
||||
|
||||
async fn set(
|
||||
&self,
|
||||
cx: &SdkCallCx,
|
||||
collection: &str,
|
||||
key: &str,
|
||||
value: Value,
|
||||
) -> Result<(), KvError> {
|
||||
self.data
|
||||
.lock()
|
||||
.await
|
||||
.insert((cx.app_id, collection.to_string(), key.to_string()), value);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn delete(&self, cx: &SdkCallCx, collection: &str, key: &str) -> Result<bool, KvError> {
|
||||
Ok(self
|
||||
.data
|
||||
.lock()
|
||||
.await
|
||||
.remove(&(cx.app_id, collection.to_string(), key.to_string()))
|
||||
.is_some())
|
||||
}
|
||||
|
||||
async fn has(&self, cx: &SdkCallCx, collection: &str, key: &str) -> Result<bool, KvError> {
|
||||
Ok(self.data.lock().await.contains_key(&(
|
||||
cx.app_id,
|
||||
collection.to_string(),
|
||||
key.to_string(),
|
||||
)))
|
||||
}
|
||||
|
||||
async fn list(
|
||||
&self,
|
||||
cx: &SdkCallCx,
|
||||
collection: &str,
|
||||
cursor: Option<&str>,
|
||||
limit: u32,
|
||||
) -> Result<KvListPage, KvError> {
|
||||
let data = self.data.lock().await;
|
||||
let mut keys: Vec<String> = data
|
||||
.iter()
|
||||
.filter(|((a, c, _), _)| *a == cx.app_id && c == collection)
|
||||
.map(|((_, _, k), _)| k.clone())
|
||||
.filter(|k| cursor.is_none_or(|c| k.as_str() > c))
|
||||
.collect();
|
||||
keys.sort();
|
||||
let take = if limit == 0 {
|
||||
usize::MAX
|
||||
} else {
|
||||
limit as usize
|
||||
};
|
||||
let next_cursor = if keys.len() > take {
|
||||
keys.truncate(take);
|
||||
keys.last().cloned()
|
||||
} else {
|
||||
None
|
||||
};
|
||||
Ok(KvListPage { keys, next_cursor })
|
||||
}
|
||||
}
|
||||
|
||||
fn make_engine() -> Arc<Engine> {
|
||||
let services = Services::new(
|
||||
Arc::new(InMemoryKv::default()),
|
||||
Arc::new(NoopDeadLetterService),
|
||||
Arc::new(NoopEventEmitter),
|
||||
);
|
||||
Arc::new(Engine::new(Limits::default(), services))
|
||||
}
|
||||
|
||||
fn baseline_request(app_id: AppId) -> ExecRequest {
|
||||
let execution_id = ExecutionId::new();
|
||||
ExecRequest {
|
||||
execution_id,
|
||||
request_id: RequestId::new(),
|
||||
script_id: ScriptId::new(),
|
||||
script_name: "kv-test".into(),
|
||||
invocation_type: InvocationType::Http,
|
||||
path: "/kv-test".into(),
|
||||
headers: BTreeMap::new(),
|
||||
body: Value::Null,
|
||||
params: BTreeMap::new(),
|
||||
query: BTreeMap::new(),
|
||||
rest: String::new(),
|
||||
sandbox_overrides: ScriptSandbox::default(),
|
||||
app_id,
|
||||
principal: None,
|
||||
trigger_depth: 0,
|
||||
root_execution_id: execution_id,
|
||||
is_dead_letter_handler: false,
|
||||
event: None,
|
||||
}
|
||||
}
|
||||
|
||||
async fn run_script(engine: Arc<Engine>, src: &str, req: ExecRequest) -> Value {
|
||||
let src = src.to_string();
|
||||
tokio::task::spawn_blocking(move || engine.execute(&src, req))
|
||||
.await
|
||||
.expect("spawn_blocking should not panic")
|
||||
.expect("script execution should succeed")
|
||||
.body
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn kv_set_then_get_round_trip() {
|
||||
let engine = make_engine();
|
||||
let app = AppId::new();
|
||||
let src = r#"
|
||||
let widgets = kv::collection("widgets");
|
||||
widgets.set("k1", #{ n: 1 });
|
||||
widgets.get("k1")
|
||||
"#;
|
||||
let body = run_script(engine, src, baseline_request(app)).await;
|
||||
assert_eq!(body, json!({ "n": 1 }));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn kv_get_missing_returns_unit() {
|
||||
let engine = make_engine();
|
||||
let app = AppId::new();
|
||||
let src = r#"
|
||||
let c = kv::collection("widgets");
|
||||
let v = c.get("nope");
|
||||
v == ()
|
||||
"#;
|
||||
let body = run_script(engine, src, baseline_request(app)).await;
|
||||
assert_eq!(body, json!(true));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn kv_has_returns_bool() {
|
||||
let engine = make_engine();
|
||||
let app = AppId::new();
|
||||
let src = r#"
|
||||
let c = kv::collection("widgets");
|
||||
let before = c.has("k");
|
||||
c.set("k", "v");
|
||||
let after = c.has("k");
|
||||
#{ before: before, after: after }
|
||||
"#;
|
||||
let body = run_script(engine, src, baseline_request(app)).await;
|
||||
assert_eq!(body, json!({ "before": false, "after": true }));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn kv_delete_returns_was_present() {
|
||||
let engine = make_engine();
|
||||
let app = AppId::new();
|
||||
let src = r#"
|
||||
let c = kv::collection("widgets");
|
||||
let nope = c.delete("missing");
|
||||
c.set("k", 1);
|
||||
let yep = c.delete("k");
|
||||
#{ nope: nope, yep: yep }
|
||||
"#;
|
||||
let body = run_script(engine, src, baseline_request(app)).await;
|
||||
assert_eq!(body, json!({ "nope": false, "yep": true }));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn kv_empty_collection_name_throws() {
|
||||
let engine = make_engine();
|
||||
let app = AppId::new();
|
||||
let src = r#"kv::collection("")"#;
|
||||
let req = baseline_request(app);
|
||||
let err = tokio::task::spawn_blocking(move || engine.execute(src, req))
|
||||
.await
|
||||
.unwrap()
|
||||
.expect_err("empty collection should throw");
|
||||
assert!(format!("{err:?}").contains("kv::collection"));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn kv_list_pages_via_cursor() {
|
||||
let engine = make_engine();
|
||||
let app = AppId::new();
|
||||
let src = r#"
|
||||
let c = kv::collection("widgets");
|
||||
for i in 0..5 { c.set(`k${i}`, i); }
|
||||
let p1 = c.list("", 2);
|
||||
let p2 = c.list(p1.next_cursor, 2);
|
||||
#{
|
||||
p1_keys: p1.keys,
|
||||
p1_cursor: p1.next_cursor,
|
||||
p2_keys: p2.keys,
|
||||
}
|
||||
"#;
|
||||
let body = run_script(engine, src, baseline_request(app)).await;
|
||||
let obj = body.as_object().unwrap();
|
||||
let p1_keys = obj["p1_keys"].as_array().unwrap();
|
||||
let p2_keys = obj["p2_keys"].as_array().unwrap();
|
||||
assert_eq!(p1_keys.len(), 2);
|
||||
assert_eq!(p2_keys.len(), 2);
|
||||
assert!(obj["p1_cursor"].is_string());
|
||||
}
|
||||
|
||||
/// Cross-app isolation via `cx.app_id` — script with `app_id = A`
|
||||
/// cannot see entries from `app_id = B`. The kv:: bridge never
|
||||
/// surfaces `app_id` to the script, so this is enforced purely by the
|
||||
/// service deriving it from the captured `Arc<SdkCallCx>`.
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn kv_bridge_preserves_cross_app_isolation() {
|
||||
let engine = make_engine();
|
||||
let app_a = AppId::new();
|
||||
let app_b = AppId::new();
|
||||
|
||||
let writer = r#"
|
||||
let c = kv::collection("shared");
|
||||
c.set("k", "from-a");
|
||||
"ok"
|
||||
"#;
|
||||
let _ = run_script(engine.clone(), writer, baseline_request(app_a)).await;
|
||||
|
||||
// App B sees nothing under the same collection/key.
|
||||
let reader = r#"
|
||||
let c = kv::collection("shared");
|
||||
c.get("k")
|
||||
"#;
|
||||
let body = run_script(engine, reader, baseline_request(app_b)).await;
|
||||
assert_eq!(body, Value::Null);
|
||||
}
|
||||
384
crates/executor-core/tests/stdlib.rs
Normal file
384
crates/executor-core/tests/stdlib.rs
Normal file
@@ -0,0 +1,384 @@
|
||||
//! Integration tests for the v1.1.0 stdlib utility modules.
|
||||
//!
|
||||
//! These exist alongside `sdk_contract.rs` rather than inside it
|
||||
//! because the stateless utilities aren't part of the same versioned
|
||||
//! SDK contract surface — `sdk_contract.rs` covers things that bump
|
||||
//! `SDK_VERSION` when they change; stdlib additions don't.
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
use picloud_executor_core::{Engine, ExecError, ExecRequest, InvocationType, Limits};
|
||||
use picloud_shared::{AppId, ExecutionId, RequestId, ScriptId, ScriptSandbox, Services};
|
||||
use serde_json::{json, Value};
|
||||
|
||||
// ----------------------------------------------------------------------------
|
||||
// Test harness — duplicated from sdk_contract.rs (each integration test
|
||||
// crate has its own; there is no tests/common/).
|
||||
// ----------------------------------------------------------------------------
|
||||
|
||||
fn engine() -> Engine {
|
||||
Engine::new(Limits::default(), Services::default())
|
||||
}
|
||||
|
||||
fn baseline_request() -> ExecRequest {
|
||||
let execution_id = ExecutionId::new();
|
||||
ExecRequest {
|
||||
execution_id,
|
||||
request_id: RequestId::new(),
|
||||
script_id: ScriptId::new(),
|
||||
script_name: "stdlib".into(),
|
||||
invocation_type: InvocationType::Http,
|
||||
path: "/stdlib-test".into(),
|
||||
headers: BTreeMap::new(),
|
||||
body: Value::Null,
|
||||
params: BTreeMap::new(),
|
||||
query: BTreeMap::new(),
|
||||
rest: String::new(),
|
||||
sandbox_overrides: ScriptSandbox::default(),
|
||||
app_id: AppId::new(),
|
||||
principal: None,
|
||||
trigger_depth: 0,
|
||||
root_execution_id: execution_id,
|
||||
is_dead_letter_handler: false,
|
||||
event: None,
|
||||
}
|
||||
}
|
||||
|
||||
fn run(source: &str) -> Value {
|
||||
engine()
|
||||
.execute(source, baseline_request())
|
||||
.expect("stdlib test should execute cleanly")
|
||||
.body
|
||||
}
|
||||
|
||||
fn run_err(source: &str) -> ExecError {
|
||||
engine()
|
||||
.execute(source, baseline_request())
|
||||
.expect_err("stdlib test expected to throw")
|
||||
}
|
||||
|
||||
fn assert_runtime_err(err: ExecError, needle: &str) {
|
||||
match err {
|
||||
ExecError::Runtime(msg) => assert!(
|
||||
msg.contains(needle),
|
||||
"runtime error did not contain `{needle}`: {msg}"
|
||||
),
|
||||
other => panic!("expected Runtime error containing `{needle}`, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// regex
|
||||
// ============================================================================
|
||||
|
||||
#[test]
|
||||
fn regex_is_match_true_and_false() {
|
||||
assert_eq!(run(r#"regex::is_match("^h", "hello")"#), json!(true));
|
||||
assert_eq!(run(r#"regex::is_match("^x", "hello")"#), json!(false));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn regex_find_returns_first_match() {
|
||||
assert_eq!(run(r#"regex::find("\\d+", "abc 42 def 99")"#), json!("42"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn regex_find_returns_unit_when_no_match() {
|
||||
// () serializes to JSON null via dynamic_to_json.
|
||||
assert_eq!(run(r#"regex::find("\\d+", "abc")"#), Value::Null);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn regex_find_all_returns_array() {
|
||||
assert_eq!(
|
||||
run(r#"regex::find_all("\\d+", "a1 b22 c333")"#),
|
||||
json!(["1", "22", "333"])
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn regex_replace_first_only() {
|
||||
assert_eq!(
|
||||
run(r#"regex::replace("a", "banana", "X")"#),
|
||||
json!("bXnana")
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn regex_replace_all() {
|
||||
assert_eq!(
|
||||
run(r#"regex::replace_all("a", "banana", "X")"#),
|
||||
json!("bXnXnX")
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn regex_split() {
|
||||
assert_eq!(
|
||||
run(r#"regex::split(",\\s*", "a, b,c, d")"#),
|
||||
json!(["a", "b", "c", "d"])
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn regex_captures_extracts_groups() {
|
||||
assert_eq!(
|
||||
run(r#"regex::captures("(\\d+)-(\\w+)", "42-abc")"#),
|
||||
json!(["42-abc", "42", "abc"])
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn regex_captures_returns_unit_when_no_match() {
|
||||
assert_eq!(run(r#"regex::captures("(\\d+)", "abc")"#), Value::Null);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn regex_invalid_pattern_throws() {
|
||||
assert_runtime_err(run_err(r#"regex::is_match("(", "x")"#), "invalid regex");
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// random
|
||||
// ============================================================================
|
||||
|
||||
#[test]
|
||||
fn random_int_within_range() {
|
||||
// Run a few times to exercise the bounds — each call is independent.
|
||||
let body = run(r"
|
||||
let n = random::int(10, 20);
|
||||
n >= 10 && n <= 20
|
||||
");
|
||||
assert_eq!(body, json!(true));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn random_int_throws_when_min_greater_than_max() {
|
||||
assert_runtime_err(run_err("random::int(20, 10)"), "min");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn random_float_in_unit_interval() {
|
||||
let body = run(r"
|
||||
let f = random::float();
|
||||
f >= 0.0 && f < 1.0
|
||||
");
|
||||
assert_eq!(body, json!(true));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn random_bytes_returns_blob_of_correct_length() {
|
||||
assert_eq!(run("random::bytes(16).len()"), json!(16));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn random_bytes_rejects_negative() {
|
||||
assert_runtime_err(run_err("random::bytes(-1)"), "random::bytes");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn random_bytes_rejects_oversize() {
|
||||
assert_runtime_err(run_err("random::bytes(70000)"), "random::bytes");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn random_string_produces_alphanumeric_of_correct_length() {
|
||||
let body = run(r#"
|
||||
let s = random::string(32);
|
||||
s.len == 32 && regex::is_match("^[A-Za-z0-9]+$", s)
|
||||
"#);
|
||||
assert_eq!(body, json!(true));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn random_uuid_has_canonical_format() {
|
||||
let body = run(
|
||||
r#"regex::is_match("^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$", random::uuid())"#,
|
||||
);
|
||||
assert_eq!(body, json!(true));
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// time
|
||||
// ============================================================================
|
||||
|
||||
#[test]
|
||||
fn time_now_ms_is_positive() {
|
||||
let body = run("time::now_ms() > 0");
|
||||
assert_eq!(body, json!(true));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn time_now_string_looks_like_iso() {
|
||||
let body = run(r#"regex::is_match("^\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2}:\\d{2}", time::now())"#);
|
||||
assert_eq!(body, json!(true));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn time_parse_format_round_trip() {
|
||||
let body = run(r"
|
||||
let ms = 1700000000000;
|
||||
time::parse(time::format(ms)) == ms
|
||||
");
|
||||
assert_eq!(body, json!(true));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn time_add_seconds() {
|
||||
assert_eq!(run("time::add_seconds(0, 60)"), json!(60_000));
|
||||
assert_eq!(run("time::add_seconds(1000, -1)"), json!(0));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn time_diff_seconds_truncates() {
|
||||
assert_eq!(run("time::diff_seconds(0, 65_500)"), json!(65));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn time_parse_rejects_garbage() {
|
||||
assert_runtime_err(run_err(r#"time::parse("nonsense")"#), "time::parse");
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// json
|
||||
// ============================================================================
|
||||
|
||||
#[test]
|
||||
fn json_parse_then_stringify_round_trip() {
|
||||
let body = run(r#"
|
||||
let src = `{"a":1,"b":"x"}`;
|
||||
json::stringify(json::parse(src)) == src
|
||||
"#);
|
||||
assert_eq!(body, json!(true));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn json_stringify_compact() {
|
||||
assert_eq!(run(r"json::stringify(#{ a: 1 })"), json!(r#"{"a":1}"#));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn json_stringify_pretty_has_newlines() {
|
||||
let body = run(r#"json::stringify_pretty(#{ a: 1 }).contains("\n")"#);
|
||||
assert_eq!(body, json!(true));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn json_parse_invalid_throws() {
|
||||
assert_runtime_err(run_err(r#"json::parse("not json")"#), "json::parse");
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// base64
|
||||
// ============================================================================
|
||||
|
||||
#[test]
|
||||
fn base64_encode_string() {
|
||||
assert_eq!(run(r#"base64::encode("hi")"#), json!("aGk="));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn base64_decode_then_re_encode_round_trip() {
|
||||
assert_eq!(
|
||||
run(r#"base64::encode(base64::decode("aGVsbG8="))"#),
|
||||
json!("aGVsbG8=")
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn base64_encode_url_has_no_padding() {
|
||||
let body = run(r#"
|
||||
let s = base64::encode_url("hello world!?");
|
||||
!s.contains("=") && !s.contains("+") && !s.contains("/")
|
||||
"#);
|
||||
assert_eq!(body, json!(true));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn base64_decode_url_round_trip() {
|
||||
assert_eq!(
|
||||
run(r#"base64::encode_url(base64::decode_url("aGVsbG8"))"#),
|
||||
json!("aGVsbG8")
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn base64_decode_invalid_throws() {
|
||||
assert_runtime_err(run_err(r#"base64::decode("!!!")"#), "base64::decode");
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// hex
|
||||
// ============================================================================
|
||||
|
||||
#[test]
|
||||
fn hex_encode_produces_lowercase() {
|
||||
assert_eq!(run(r#"hex::encode("Z")"#), json!("5a"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn hex_decode_then_re_encode_round_trip() {
|
||||
// mixed-case input → lowercase output proves both case-insensitive
|
||||
// decode and lowercase encode.
|
||||
assert_eq!(
|
||||
run(r#"hex::encode(hex::decode("DeAdBeEf"))"#),
|
||||
json!("deadbeef")
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn hex_decode_returns_correct_length() {
|
||||
assert_eq!(run(r#"hex::decode("deadbeef").len()"#), json!(4));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn hex_decode_invalid_throws() {
|
||||
assert_runtime_err(run_err(r#"hex::decode("xyz")"#), "hex::decode");
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// url
|
||||
// ============================================================================
|
||||
|
||||
#[test]
|
||||
fn url_encode_basic() {
|
||||
assert_eq!(run(r#"url::encode("hello world")"#), json!("hello%20world"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn url_encode_preserves_unreserved() {
|
||||
assert_eq!(
|
||||
run(r#"url::encode("abcXYZ123-_.~")"#),
|
||||
json!("abcXYZ123-_.~")
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn url_decode_round_trip() {
|
||||
assert_eq!(
|
||||
run(r#"url::decode(url::encode("hello world!?"))"#),
|
||||
json!("hello world!?")
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn url_encode_query_basic() {
|
||||
// Map keys come out alphabetically (Rhai's Map is a BTreeMap).
|
||||
assert_eq!(
|
||||
run(r#"url::encode_query(#{ a: "1", b: "x y" })"#),
|
||||
json!("a=1&b=x%20y")
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn url_encode_query_coerces_non_strings() {
|
||||
// Numbers and bools shouldn't throw; they coerce via to_string().
|
||||
let body = run(r"url::encode_query(#{ n: 42, b: true })");
|
||||
// Order is alphabetical: b before n.
|
||||
assert_eq!(body, json!("b=true&n=42"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn url_decode_rejects_invalid_utf8() {
|
||||
assert_runtime_err(run_err(r#"url::decode("%FF%FE%80")"#), "url::decode");
|
||||
}
|
||||
@@ -10,13 +10,16 @@ workspace = true
|
||||
|
||||
[dependencies]
|
||||
picloud-shared.workspace = true
|
||||
picloud-executor-core.workspace = true
|
||||
picloud-orchestrator-core.workspace = true
|
||||
|
||||
async-trait.workspace = true
|
||||
axum.workspace = true
|
||||
rand.workspace = true
|
||||
serde.workspace = true
|
||||
serde_json.workspace = true
|
||||
thiserror.workspace = true
|
||||
tokio.workspace = true
|
||||
tracing.workspace = true
|
||||
uuid.workspace = true
|
||||
chrono.workspace = true
|
||||
@@ -24,7 +27,6 @@ sqlx.workspace = true
|
||||
url.workspace = true
|
||||
|
||||
argon2.workspace = true
|
||||
rand.workspace = true
|
||||
sha2.workspace = true
|
||||
base64.workspace = true
|
||||
data-encoding.workspace = true
|
||||
|
||||
28
crates/manager-core/migrations/0007_kv.sql
Normal file
28
crates/manager-core/migrations/0007_kv.sql
Normal file
@@ -0,0 +1,28 @@
|
||||
-- v1.1.1: Key-value store — see blueprint §8.1 + docs/sdk-shape.md.
|
||||
--
|
||||
-- Identity tuple `(app_id, collection, key)`. `app_id` is first in the
|
||||
-- primary key so the implicit index is always per-app; cross-app reads
|
||||
-- cannot happen even with a buggy query. Collections are a required
|
||||
-- namespace inside an app — the same key can live in different
|
||||
-- collections without collision.
|
||||
--
|
||||
-- `value` is JSONB so scripts can store nested structures without
|
||||
-- a separate serialization step. No TTL column in v1.1.1; deferred
|
||||
-- until a concrete need surfaces (the blueprint reserved one but the
|
||||
-- v1.1.1 SDK surface — get/set/has/delete/list — doesn't expose TTL).
|
||||
|
||||
CREATE TABLE kv_entries (
|
||||
app_id UUID NOT NULL REFERENCES apps(id) ON DELETE CASCADE,
|
||||
collection TEXT NOT NULL,
|
||||
key TEXT NOT NULL,
|
||||
value JSONB NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
PRIMARY KEY (app_id, collection, key)
|
||||
);
|
||||
|
||||
-- Supports list-by-collection (keyset pagination) and per-collection
|
||||
-- triggers' fan-out scans. The PK already covers (app_id, collection)
|
||||
-- as a prefix but spelling out the explicit index makes intent clear
|
||||
-- for the planner.
|
||||
CREATE INDEX idx_kv_entries_app_collection ON kv_entries (app_id, collection);
|
||||
72
crates/manager-core/migrations/0008_triggers.sql
Normal file
72
crates/manager-core/migrations/0008_triggers.sql
Normal file
@@ -0,0 +1,72 @@
|
||||
-- v1.1.1: Trigger framework — Layout E (design notes §2 + §7).
|
||||
--
|
||||
-- A parent `triggers` table holds the common columns (script_id, retry
|
||||
-- config, dispatch_mode, registered-by principal); per-kind detail
|
||||
-- tables hold the kind-specific filter columns. v1.1.1 ships two
|
||||
-- kinds: KV (collection_glob + ops) and dead_letter (source / trigger
|
||||
-- / script filters). Future kinds (cron, pubsub, queue, email) extend
|
||||
-- the parent and add their own detail table.
|
||||
--
|
||||
-- `registered_by_principal` captures the admin user that registered
|
||||
-- the trigger. The dispatcher resolves this back to a `Principal` at
|
||||
-- execution time so the trigger runs as the user that set it up
|
||||
-- (design notes §4: "a trigger execution runs as the principal that
|
||||
-- registered the trigger").
|
||||
--
|
||||
-- HTTP routes stay in their own `routes` table for now (Phase 3
|
||||
-- production schema with its own trie-index columns); the dispatcher
|
||||
-- discriminates HTTP outbox rows by `source_kind = 'http'` and
|
||||
-- `trigger_id` referencing `routes.id`. Folding routes into triggers
|
||||
-- is a v1.2 cleanup, not a v1.1.1 requirement.
|
||||
|
||||
CREATE TABLE triggers (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
app_id UUID NOT NULL REFERENCES apps(id) ON DELETE CASCADE,
|
||||
script_id UUID NOT NULL REFERENCES scripts(id) ON DELETE CASCADE,
|
||||
kind TEXT NOT NULL CHECK (kind IN ('kv', 'dead_letter')),
|
||||
enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
-- Async by default — sync would mean the trigger fires inline with
|
||||
-- the originating mutation, which v1.1.1 doesn't support.
|
||||
dispatch_mode TEXT NOT NULL DEFAULT 'async'
|
||||
CHECK (dispatch_mode IN ('sync', 'async')),
|
||||
-- Defaults applied at write time so the row is auditable on its
|
||||
-- own. Per-trigger overrides set on create; the env-defined
|
||||
-- defaults provide the fallback values.
|
||||
retry_max_attempts INT NOT NULL,
|
||||
retry_backoff TEXT NOT NULL
|
||||
CHECK (retry_backoff IN ('exponential', 'linear', 'constant')),
|
||||
retry_base_ms INT NOT NULL,
|
||||
registered_by_principal UUID NOT NULL REFERENCES admin_users(id) ON DELETE CASCADE,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- The dispatcher's hot lookup: "all enabled triggers for app X of
|
||||
-- kind Y". Indexed only when enabled = TRUE so disabled rows don't
|
||||
-- pollute the index.
|
||||
CREATE INDEX idx_triggers_app_kind_enabled
|
||||
ON triggers (app_id, kind)
|
||||
WHERE enabled = TRUE;
|
||||
|
||||
-- One row per KV trigger. `collection_glob` accepts:
|
||||
-- "*" — any collection in the app
|
||||
-- "widgets" — exact match
|
||||
-- "users:*" — prefix wildcard (matched in Rust, not SQL)
|
||||
-- `ops` is the subset of {insert, update, delete} this trigger
|
||||
-- subscribes to. Empty array means "any op" (the trigger fires on
|
||||
-- every mutation; admin endpoint validates this).
|
||||
CREATE TABLE kv_trigger_details (
|
||||
trigger_id UUID PRIMARY KEY REFERENCES triggers(id) ON DELETE CASCADE,
|
||||
collection_glob TEXT NOT NULL,
|
||||
ops TEXT[] NOT NULL
|
||||
);
|
||||
|
||||
-- One row per dead-letter trigger. All three filter columns are
|
||||
-- nullable — NULL means "no filter on this dimension". A trigger
|
||||
-- with all three nullable filters fires on every dead-letter row.
|
||||
CREATE TABLE dead_letter_trigger_details (
|
||||
trigger_id UUID PRIMARY KEY REFERENCES triggers(id) ON DELETE CASCADE,
|
||||
source_filter TEXT,
|
||||
trigger_id_filter UUID,
|
||||
script_id_filter UUID
|
||||
);
|
||||
64
crates/manager-core/migrations/0009_outbox.sql
Normal file
64
crates/manager-core/migrations/0009_outbox.sql
Normal file
@@ -0,0 +1,64 @@
|
||||
-- v1.1.1: Universal trigger outbox — design notes §2.
|
||||
--
|
||||
-- One table for every async dispatch in the system. KV/cron/pubsub/
|
||||
-- queue/email/dead-letter all write rows in this shape; the dispatcher
|
||||
-- claims due rows with `FOR UPDATE SKIP LOCKED` and routes them to
|
||||
-- the executor.
|
||||
--
|
||||
-- Sync HTTP also writes here (NATS-style inbox, design notes §3) —
|
||||
-- `reply_to` carries an `inbox_id` that the orchestrator awaits on a
|
||||
-- oneshot channel. `reply_to.is_some()` is the "don't retry" signal:
|
||||
-- one attempt, surface the result via the inbox.
|
||||
--
|
||||
-- `trigger_id` is a polymorphic reference discriminated by
|
||||
-- `source_kind`: for `source_kind='http'` it references `routes.id`;
|
||||
-- otherwise it references `triggers.id`. Polymorphism handled in
|
||||
-- Rust (the dispatcher); no DB-level FK because Postgres doesn't
|
||||
-- support polymorphic FKs cleanly. NULL is allowed because direct
|
||||
-- admin-replay paths may not have a triggering row at all.
|
||||
--
|
||||
-- `script_id` denormalized so the dispatcher resolves the target
|
||||
-- script without an extra round-trip per row.
|
||||
|
||||
CREATE TABLE outbox (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
app_id UUID NOT NULL REFERENCES apps(id) ON DELETE CASCADE,
|
||||
source_kind TEXT NOT NULL
|
||||
CHECK (source_kind IN ('http', 'kv', 'dead_letter')),
|
||||
-- Polymorphic — see comment above. No FK constraint.
|
||||
trigger_id UUID,
|
||||
-- Pre-resolved at write time so the dispatcher doesn't re-look it up.
|
||||
script_id UUID,
|
||||
-- NULL = async (retry per policy). Some(inbox_id) = sync HTTP
|
||||
-- (never retry; resolve the inbox with the result).
|
||||
reply_to UUID,
|
||||
-- ServiceEvent + ExecRequest scaffold serialized as JSONB.
|
||||
payload JSONB NOT NULL,
|
||||
-- Forensic field — the principal that triggered the originating
|
||||
-- event. NOT the execution principal for trigger fan-out (that
|
||||
-- comes from `triggers.registered_by_principal`).
|
||||
origin_principal UUID,
|
||||
-- Trigger-depth as the dispatcher will hand it to the executor.
|
||||
-- Read out into ExecRequest.trigger_depth at dispatch time.
|
||||
trigger_depth INT NOT NULL DEFAULT 0,
|
||||
-- Originating execution id (for audit log grouping). Equals the
|
||||
-- root for direct invocations; preserved across fan-out chains.
|
||||
root_execution_id UUID,
|
||||
attempt_count INT NOT NULL DEFAULT 0,
|
||||
next_attempt_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
-- Set inside the SELECT FOR UPDATE SKIP LOCKED transaction so
|
||||
-- the dispatcher can't double-pick a row across concurrent loop
|
||||
-- iterations.
|
||||
claimed_at TIMESTAMPTZ,
|
||||
claimed_by TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- Hot index: the dispatcher's `WHERE next_attempt_at <= NOW() AND
|
||||
-- claimed_at IS NULL` claim query. Partial index keeps the hot set
|
||||
-- small even if the table grows large.
|
||||
CREATE INDEX idx_outbox_due
|
||||
ON outbox (next_attempt_at)
|
||||
WHERE claimed_at IS NULL;
|
||||
|
||||
CREATE INDEX idx_outbox_app ON outbox (app_id);
|
||||
50
crates/manager-core/migrations/0010_dead_letters.sql
Normal file
50
crates/manager-core/migrations/0010_dead_letters.sql
Normal file
@@ -0,0 +1,50 @@
|
||||
-- v1.1.1: dead_letters — design notes §4.
|
||||
--
|
||||
-- Async invocations that exhaust their retry policy land here. Each
|
||||
-- row carries the original event payload verbatim plus the attempt
|
||||
-- history so handlers (registered via `dead_letter` triggers) and the
|
||||
-- dashboard can decide what to do.
|
||||
--
|
||||
-- Schema mirrors design notes §4. The CHECK constraint on
|
||||
-- `resolution` enforces the closed vocabulary used by both the SDK
|
||||
-- (`dead_letters::resolve(id, reason)`) and the recursion-stop rule
|
||||
-- (`handler_failed`). Sync HTTP failures (`reply_to.is_some()`) never
|
||||
-- land here — they're served via the inbox channel.
|
||||
--
|
||||
-- Indexes:
|
||||
-- - partial index on unresolved rows: the dashboard's
|
||||
-- unresolved-count badge query (`COUNT(*) WHERE app_id = $1 AND
|
||||
-- resolved_at IS NULL`).
|
||||
-- - GC index on `created_at`: the weekly retention sweep.
|
||||
|
||||
CREATE TABLE dead_letters (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
app_id UUID NOT NULL REFERENCES apps(id) ON DELETE CASCADE,
|
||||
-- The outbox.id row that exhausted retries. The outbox row itself
|
||||
-- has been deleted at this point.
|
||||
original_event_id UUID NOT NULL,
|
||||
source TEXT NOT NULL,
|
||||
op TEXT NOT NULL,
|
||||
-- Nullable because direct admin replays may have no trigger row.
|
||||
trigger_id UUID,
|
||||
script_id UUID,
|
||||
payload JSONB NOT NULL,
|
||||
attempt_count INT NOT NULL,
|
||||
first_attempt_at TIMESTAMPTZ NOT NULL,
|
||||
last_attempt_at TIMESTAMPTZ NOT NULL,
|
||||
last_error TEXT NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
resolved_at TIMESTAMPTZ,
|
||||
resolution TEXT
|
||||
CHECK (resolution IN
|
||||
('replayed', 'ignored', 'handled_by_script', 'handler_failed'))
|
||||
);
|
||||
|
||||
-- Dashboard unresolved-count badge — partial index on the predicate
|
||||
-- the query uses.
|
||||
CREATE INDEX idx_dead_letters_app_unresolved
|
||||
ON dead_letters (app_id)
|
||||
WHERE resolved_at IS NULL;
|
||||
|
||||
-- GC sweep scans by creation time.
|
||||
CREATE INDEX idx_dead_letters_gc ON dead_letters (created_at);
|
||||
31
crates/manager-core/migrations/0011_abandoned_executions.sql
Normal file
31
crates/manager-core/migrations/0011_abandoned_executions.sql
Normal file
@@ -0,0 +1,31 @@
|
||||
-- v1.1.1: abandoned_executions — design notes §3 #9.
|
||||
--
|
||||
-- Forensic table for the "dispatcher tried to resolve a oneshot inbox
|
||||
-- but the receiver was already dropped" edge case. The orchestrator
|
||||
-- timed out (returned 504 to the caller) and gave up on the channel,
|
||||
-- but then the dispatcher's execution succeeded later. The caller
|
||||
-- never sees the result; the row exists so the operator can
|
||||
-- correlate when the abandoned-counter metric spikes.
|
||||
--
|
||||
-- Only the dispatcher-after-orchestrator-timeout edge case writes
|
||||
-- here; ordinary "script timed out, caller got 504" stays uneventful.
|
||||
--
|
||||
-- 7-day retention, GC by `created_at`, sweep alongside dead_letters.
|
||||
|
||||
CREATE TABLE abandoned_executions (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
app_id UUID NOT NULL REFERENCES apps(id) ON DELETE CASCADE,
|
||||
-- Original outbox row id (the row itself has been deleted).
|
||||
outbox_id UUID NOT NULL,
|
||||
script_id UUID,
|
||||
-- The inbox channel id the dispatcher tried to resolve.
|
||||
inbox_id UUID NOT NULL,
|
||||
-- The HTTP status code the dispatcher attempted to send back.
|
||||
status_code INT NOT NULL,
|
||||
-- Truncated body / error description (capped at write time —
|
||||
-- the dispatcher doesn't need to ship megabytes here).
|
||||
result_summary TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
|
||||
);
|
||||
|
||||
CREATE INDEX idx_abandoned_executions_gc ON abandoned_executions (created_at);
|
||||
16
crates/manager-core/migrations/0012_routes_dispatch_mode.sql
Normal file
16
crates/manager-core/migrations/0012_routes_dispatch_mode.sql
Normal file
@@ -0,0 +1,16 @@
|
||||
-- v1.1.1: per-route dispatch mode (design notes §2 + §3).
|
||||
--
|
||||
-- `sync` (default): orchestrator awaits the executor inline and
|
||||
-- returns the response in the same HTTP request — current MVP
|
||||
-- behaviour.
|
||||
-- `async`: orchestrator writes the request to the trigger outbox,
|
||||
-- returns `202 Accepted` immediately. The dispatcher runs the
|
||||
-- script in the background and surfaces failures via the
|
||||
-- retry / dead-letter machinery — same shape as any other async
|
||||
-- event.
|
||||
--
|
||||
-- Existing routes default to `sync` so the migration is non-breaking.
|
||||
|
||||
ALTER TABLE routes
|
||||
ADD COLUMN dispatch_mode TEXT NOT NULL DEFAULT 'sync'
|
||||
CHECK (dispatch_mode IN ('sync', 'async'));
|
||||
128
crates/manager-core/src/abandoned_repo.rs
Normal file
128
crates/manager-core/src/abandoned_repo.rs
Normal file
@@ -0,0 +1,128 @@
|
||||
//! `AbandonedExecutionsRepo` — forensic table written by the
|
||||
//! dispatcher when it tries to resolve a sync-HTTP inbox channel
|
||||
//! that's already been dropped (orchestrator timed out and gave up).
|
||||
//!
|
||||
//! Schema: see `migrations/0011_abandoned_executions.sql`.
|
||||
//!
|
||||
//! Tiny surface: insert + GC. Reading happens via direct SQL when
|
||||
//! correlating the metric counter spike.
|
||||
|
||||
use async_trait::async_trait;
|
||||
use chrono::{DateTime, Utc};
|
||||
use picloud_shared::{AppId, ScriptId};
|
||||
use sqlx::PgPool;
|
||||
use uuid::Uuid;
|
||||
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub enum AbandonedRepoError {
|
||||
#[error("database error: {0}")]
|
||||
Db(#[from] sqlx::Error),
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct NewAbandonedExecution {
|
||||
pub app_id: AppId,
|
||||
pub outbox_id: Uuid,
|
||||
pub script_id: Option<ScriptId>,
|
||||
pub inbox_id: Uuid,
|
||||
pub status_code: u16,
|
||||
pub result_summary: Option<String>,
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
pub trait AbandonedRepo: Send + Sync {
|
||||
async fn insert(&self, row: NewAbandonedExecution) -> Result<Uuid, AbandonedRepoError>;
|
||||
|
||||
/// Retention sweep — deletes rows older than `older_than` up to
|
||||
/// `limit` at a time.
|
||||
async fn gc(&self, older_than: DateTime<Utc>, limit: i64) -> Result<u64, AbandonedRepoError>;
|
||||
}
|
||||
|
||||
pub struct PostgresAbandonedRepo {
|
||||
pool: PgPool,
|
||||
}
|
||||
|
||||
impl PostgresAbandonedRepo {
|
||||
#[must_use]
|
||||
pub fn new(pool: PgPool) -> Self {
|
||||
Self { pool }
|
||||
}
|
||||
}
|
||||
|
||||
const SUMMARY_CAP_BYTES: usize = 4096;
|
||||
|
||||
#[async_trait]
|
||||
impl AbandonedRepo for PostgresAbandonedRepo {
|
||||
async fn insert(&self, row: NewAbandonedExecution) -> Result<Uuid, AbandonedRepoError> {
|
||||
// Truncate the summary at write-time. The forensic table
|
||||
// doesn't need megabytes; the original outbox row may have
|
||||
// been arbitrary size but we lose nothing useful by clipping.
|
||||
let summary = row.result_summary.map(|s| truncate(s, SUMMARY_CAP_BYTES));
|
||||
let (id,): (Uuid,) = sqlx::query_as(
|
||||
"INSERT INTO abandoned_executions ( \
|
||||
app_id, outbox_id, script_id, inbox_id, status_code, result_summary \
|
||||
) VALUES ($1, $2, $3, $4, $5, $6) \
|
||||
RETURNING id",
|
||||
)
|
||||
.bind(row.app_id.into_inner())
|
||||
.bind(row.outbox_id)
|
||||
.bind(row.script_id.map(ScriptId::into_inner))
|
||||
.bind(row.inbox_id)
|
||||
.bind(i32::from(row.status_code))
|
||||
.bind(summary)
|
||||
.fetch_one(&self.pool)
|
||||
.await?;
|
||||
Ok(id)
|
||||
}
|
||||
|
||||
async fn gc(&self, older_than: DateTime<Utc>, limit: i64) -> Result<u64, AbandonedRepoError> {
|
||||
let res = sqlx::query(
|
||||
"DELETE FROM abandoned_executions \
|
||||
WHERE id IN ( \
|
||||
SELECT id FROM abandoned_executions \
|
||||
WHERE created_at < $1 \
|
||||
FOR UPDATE SKIP LOCKED \
|
||||
LIMIT $2 \
|
||||
)",
|
||||
)
|
||||
.bind(older_than)
|
||||
.bind(limit)
|
||||
.execute(&self.pool)
|
||||
.await?;
|
||||
Ok(res.rows_affected())
|
||||
}
|
||||
}
|
||||
|
||||
fn truncate(mut s: String, max_bytes: usize) -> String {
|
||||
if s.len() <= max_bytes {
|
||||
return s;
|
||||
}
|
||||
// Walk back from `max_bytes` to a UTF-8 char boundary so we never
|
||||
// panic on `truncate` mid-codepoint.
|
||||
let mut cut = max_bytes;
|
||||
while cut > 0 && !s.is_char_boundary(cut) {
|
||||
cut -= 1;
|
||||
}
|
||||
s.truncate(cut);
|
||||
s
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn truncate_respects_char_boundaries() {
|
||||
// 3-byte UTF-8 chars; cap inside the middle char should walk
|
||||
// back to the start.
|
||||
let s = "héllo".to_string();
|
||||
let t = truncate(s, 2);
|
||||
assert!(t.is_char_boundary(t.len()));
|
||||
assert_eq!(t, "h");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn truncate_passthrough_for_short_strings() {
|
||||
assert_eq!(truncate("ok".into(), 100), "ok");
|
||||
}
|
||||
}
|
||||
@@ -82,6 +82,7 @@ async fn seed_into(
|
||||
// Accept any method so both `curl /hello` and
|
||||
// `curl -d '{"name":"X"}' /hello` work out of the box.
|
||||
method: None,
|
||||
dispatch_mode: picloud_shared::DispatchMode::Sync,
|
||||
})
|
||||
.await?;
|
||||
|
||||
|
||||
@@ -100,6 +100,35 @@ pub async fn require_admin(state: State<AuthState>, req: Request<Body>, next: Ne
|
||||
require_authenticated(state, req, next).await
|
||||
}
|
||||
|
||||
/// Opportunistic data-plane variant: always inserts an
|
||||
/// `Extension<Option<Principal>>` and forwards the request. Used on
|
||||
/// `/execute/{id}` and the user-route fallback, where most invocations
|
||||
/// are anonymous public HTTP and the few authed ones (dashboard
|
||||
/// test-runs, API keys) should still let scripts see the caller via
|
||||
/// `cx.principal` once services consume it.
|
||||
///
|
||||
/// Failure modes — all degrade to `None` rather than rejecting:
|
||||
/// * No bearer / cookie → `None`.
|
||||
/// * Malformed or unknown token → `None`.
|
||||
/// * DB blip while resolving → `None` (fail-open; the data plane
|
||||
/// should not 500 on transient infra failures for an *optional*
|
||||
/// identity check).
|
||||
///
|
||||
/// Admin-side routes that REQUIRE an identity keep using
|
||||
/// `require_authenticated`.
|
||||
pub async fn attach_principal_if_present(
|
||||
State(state): State<AuthState>,
|
||||
mut req: Request<Body>,
|
||||
next: Next,
|
||||
) -> Response {
|
||||
let principal: Option<Principal> = match extract_token(&req) {
|
||||
Some(token) => resolve_principal(&state, &token).await.unwrap_or(None),
|
||||
None => None,
|
||||
};
|
||||
req.extensions_mut().insert(principal);
|
||||
next.run(req).await
|
||||
}
|
||||
|
||||
/// Decide whether the token is an API key (pic_ prefix) or a session
|
||||
/// token, then resolve the corresponding `Principal`. `Ok(None)`
|
||||
/// means the token was structurally valid but didn't match any active
|
||||
|
||||
@@ -57,6 +57,21 @@ pub enum Capability {
|
||||
AppAdmin(AppId),
|
||||
/// Read execution logs for scripts in this app.
|
||||
AppLogRead(AppId),
|
||||
/// Read entries from this app's KV store (v1.1.1). Granted to
|
||||
/// `viewer`+ in the per-app role table. Maps to `script:read` on
|
||||
/// API keys — the seven-scope vocabulary stays locked.
|
||||
AppKvRead(AppId),
|
||||
/// Write entries to this app's KV store (v1.1.1). Granted to
|
||||
/// `editor`+. Maps to `script:write` on API keys.
|
||||
AppKvWrite(AppId),
|
||||
/// Create / list / delete triggers for this app (v1.1.1). Maps to
|
||||
/// `app:admin` on API keys — triggers are app-configuration acts
|
||||
/// rather than data-plane access. Granted to `app_admin`+.
|
||||
AppManageTriggers(AppId),
|
||||
/// Replay / resolve dead-letter rows for this app (v1.1.1). Maps
|
||||
/// to `app:admin` on API keys. Public-HTTP scripts (principal None)
|
||||
/// fail this check — managing dead letters is an admin act.
|
||||
AppDeadLetterManage(AppId),
|
||||
}
|
||||
|
||||
impl Capability {
|
||||
@@ -73,7 +88,11 @@ impl Capability {
|
||||
| Self::AppWriteRoute(id)
|
||||
| Self::AppManageDomains(id)
|
||||
| Self::AppAdmin(id)
|
||||
| Self::AppLogRead(id) => Some(id),
|
||||
| Self::AppLogRead(id)
|
||||
| Self::AppKvRead(id)
|
||||
| Self::AppKvWrite(id)
|
||||
| Self::AppManageTriggers(id)
|
||||
| Self::AppDeadLetterManage(id) => Some(id),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -88,11 +107,13 @@ impl Capability {
|
||||
Self::InstanceCreateApp | Self::InstanceManageUsers | Self::InstanceManageSettings => {
|
||||
Scope::InstanceAdmin
|
||||
}
|
||||
Self::AppRead(_) => Scope::ScriptRead,
|
||||
Self::AppWriteScript(_) => Scope::ScriptWrite,
|
||||
Self::AppRead(_) | Self::AppKvRead(_) => Scope::ScriptRead,
|
||||
Self::AppWriteScript(_) | Self::AppKvWrite(_) => Scope::ScriptWrite,
|
||||
Self::AppWriteRoute(_) => Scope::RouteWrite,
|
||||
Self::AppManageDomains(_) => Scope::DomainManage,
|
||||
Self::AppAdmin(_) => Scope::AppAdmin,
|
||||
Self::AppAdmin(_) | Self::AppManageTriggers(_) | Self::AppDeadLetterManage(_) => {
|
||||
Scope::AppAdmin
|
||||
}
|
||||
Self::AppLogRead(_) => Scope::LogRead,
|
||||
}
|
||||
}
|
||||
@@ -230,16 +251,24 @@ async fn member_grants(
|
||||
/// domain claims, and delete. Roles form a strict subset chain, so
|
||||
/// the check is "is this capability in the role's set?".
|
||||
const fn role_satisfies(role: AppRole, cap: Capability) -> bool {
|
||||
let in_viewer = matches!(cap, Capability::AppRead(_) | Capability::AppLogRead(_));
|
||||
let in_viewer = matches!(
|
||||
cap,
|
||||
Capability::AppRead(_) | Capability::AppLogRead(_) | Capability::AppKvRead(_)
|
||||
);
|
||||
let in_editor = in_viewer
|
||||
|| matches!(
|
||||
cap,
|
||||
Capability::AppWriteScript(_) | Capability::AppWriteRoute(_)
|
||||
Capability::AppWriteScript(_)
|
||||
| Capability::AppWriteRoute(_)
|
||||
| Capability::AppKvWrite(_)
|
||||
);
|
||||
let in_app_admin = in_editor
|
||||
|| matches!(
|
||||
cap,
|
||||
Capability::AppManageDomains(_) | Capability::AppAdmin(_)
|
||||
Capability::AppManageDomains(_)
|
||||
| Capability::AppAdmin(_)
|
||||
| Capability::AppManageTriggers(_)
|
||||
| Capability::AppDeadLetterManage(_)
|
||||
);
|
||||
match role {
|
||||
AppRole::Viewer => in_viewer,
|
||||
|
||||
261
crates/manager-core/src/dead_letter_repo.rs
Normal file
261
crates/manager-core/src/dead_letter_repo.rs
Normal file
@@ -0,0 +1,261 @@
|
||||
//! `DeadLetterRepo` — CRUD over the `dead_letters` table.
|
||||
//!
|
||||
//! The dispatcher writes new rows when an async trigger exhausts its
|
||||
//! retry policy. Admin endpoints (commit 8) read for the dashboard
|
||||
//! list view and write to mark rows resolved or replay them. The GC
|
||||
//! sweeper (commit 10) deletes expired rows by `created_at`.
|
||||
|
||||
use async_trait::async_trait;
|
||||
use chrono::{DateTime, Utc};
|
||||
use picloud_shared::{AppId, DeadLetterId, ScriptId, TriggerId};
|
||||
use sqlx::PgPool;
|
||||
use uuid::Uuid;
|
||||
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub enum DeadLetterRepoError {
|
||||
#[error("database error: {0}")]
|
||||
Db(#[from] sqlx::Error),
|
||||
|
||||
#[error("dead-letter row not found: {0}")]
|
||||
NotFound(DeadLetterId),
|
||||
|
||||
#[error("invalid resolution {0:?}")]
|
||||
InvalidResolution(String),
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct NewDeadLetter {
|
||||
pub app_id: AppId,
|
||||
/// `outbox.id` that exhausted retries. Outbox row deleted at the
|
||||
/// same time.
|
||||
pub original_event_id: Uuid,
|
||||
pub source: String,
|
||||
pub op: String,
|
||||
pub trigger_id: Option<TriggerId>,
|
||||
pub script_id: Option<ScriptId>,
|
||||
pub payload: serde_json::Value,
|
||||
pub attempt_count: u32,
|
||||
pub first_attempt_at: DateTime<Utc>,
|
||||
pub last_attempt_at: DateTime<Utc>,
|
||||
pub last_error: String,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct DeadLetterRow {
|
||||
pub id: DeadLetterId,
|
||||
pub app_id: AppId,
|
||||
pub original_event_id: Uuid,
|
||||
pub source: String,
|
||||
pub op: String,
|
||||
pub trigger_id: Option<TriggerId>,
|
||||
pub script_id: Option<ScriptId>,
|
||||
pub payload: serde_json::Value,
|
||||
pub attempt_count: u32,
|
||||
pub first_attempt_at: DateTime<Utc>,
|
||||
pub last_attempt_at: DateTime<Utc>,
|
||||
pub last_error: String,
|
||||
pub created_at: DateTime<Utc>,
|
||||
pub resolved_at: Option<DateTime<Utc>>,
|
||||
pub resolution: Option<String>,
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
pub trait DeadLetterRepo: Send + Sync {
|
||||
/// Insert a new dead-letter row. Returns the assigned id.
|
||||
async fn insert(&self, row: NewDeadLetter) -> Result<DeadLetterId, DeadLetterRepoError>;
|
||||
|
||||
async fn get(&self, id: DeadLetterId) -> Result<Option<DeadLetterRow>, DeadLetterRepoError>;
|
||||
|
||||
/// Lookup for the dashboard list view. `unresolved_only=true`
|
||||
/// filters to `resolved_at IS NULL`.
|
||||
async fn list_for_app(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
unresolved_only: bool,
|
||||
limit: i64,
|
||||
offset: i64,
|
||||
) -> Result<Vec<DeadLetterRow>, DeadLetterRepoError>;
|
||||
|
||||
/// Hot path for the dashboard's per-app unresolved-count badge.
|
||||
async fn unresolved_count(&self, app_id: AppId) -> Result<i64, DeadLetterRepoError>;
|
||||
|
||||
/// Mark the row resolved with the given reason. The reason MUST
|
||||
/// be one of the four CHECK-constraint values
|
||||
/// (`replayed`, `ignored`, `handled_by_script`, `handler_failed`).
|
||||
async fn resolve(&self, id: DeadLetterId, reason: &str) -> Result<(), DeadLetterRepoError>;
|
||||
|
||||
/// Retention sweep. Deletes rows with `created_at < older_than`
|
||||
/// up to `limit` at a time, using FOR UPDATE SKIP LOCKED to play
|
||||
/// nicely with concurrent dispatchers. Returns the count deleted.
|
||||
async fn gc(&self, older_than: DateTime<Utc>, limit: i64) -> Result<u64, DeadLetterRepoError>;
|
||||
}
|
||||
|
||||
pub struct PostgresDeadLetterRepo {
|
||||
pool: PgPool,
|
||||
}
|
||||
|
||||
impl PostgresDeadLetterRepo {
|
||||
#[must_use]
|
||||
pub fn new(pool: PgPool) -> Self {
|
||||
Self { pool }
|
||||
}
|
||||
}
|
||||
|
||||
const ALLOWED_RESOLUTIONS: &[&str] =
|
||||
&["replayed", "ignored", "handled_by_script", "handler_failed"];
|
||||
|
||||
#[async_trait]
|
||||
impl DeadLetterRepo for PostgresDeadLetterRepo {
|
||||
async fn insert(&self, row: NewDeadLetter) -> Result<DeadLetterId, DeadLetterRepoError> {
|
||||
let (id,): (Uuid,) = sqlx::query_as(
|
||||
"INSERT INTO dead_letters ( \
|
||||
app_id, original_event_id, source, op, trigger_id, script_id, \
|
||||
payload, attempt_count, first_attempt_at, last_attempt_at, last_error \
|
||||
) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11) \
|
||||
RETURNING id",
|
||||
)
|
||||
.bind(row.app_id.into_inner())
|
||||
.bind(row.original_event_id)
|
||||
.bind(row.source)
|
||||
.bind(row.op)
|
||||
.bind(row.trigger_id.map(TriggerId::into_inner))
|
||||
.bind(row.script_id.map(ScriptId::into_inner))
|
||||
.bind(row.payload)
|
||||
.bind(i32::try_from(row.attempt_count).unwrap_or(0))
|
||||
.bind(row.first_attempt_at)
|
||||
.bind(row.last_attempt_at)
|
||||
.bind(row.last_error)
|
||||
.fetch_one(&self.pool)
|
||||
.await?;
|
||||
Ok(id.into())
|
||||
}
|
||||
|
||||
async fn get(&self, id: DeadLetterId) -> Result<Option<DeadLetterRow>, DeadLetterRepoError> {
|
||||
let row: Option<DeadLetterRowRaw> = sqlx::query_as(
|
||||
"SELECT id, app_id, original_event_id, source, op, trigger_id, script_id, \
|
||||
payload, attempt_count, first_attempt_at, last_attempt_at, \
|
||||
last_error, created_at, resolved_at, resolution \
|
||||
FROM dead_letters WHERE id = $1",
|
||||
)
|
||||
.bind(id.into_inner())
|
||||
.fetch_optional(&self.pool)
|
||||
.await?;
|
||||
Ok(row.map(DeadLetterRowRaw::into_row))
|
||||
}
|
||||
|
||||
async fn list_for_app(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
unresolved_only: bool,
|
||||
limit: i64,
|
||||
offset: i64,
|
||||
) -> Result<Vec<DeadLetterRow>, DeadLetterRepoError> {
|
||||
let rows: Vec<DeadLetterRowRaw> = sqlx::query_as(
|
||||
"SELECT id, app_id, original_event_id, source, op, trigger_id, script_id, \
|
||||
payload, attempt_count, first_attempt_at, last_attempt_at, \
|
||||
last_error, created_at, resolved_at, resolution \
|
||||
FROM dead_letters \
|
||||
WHERE app_id = $1 \
|
||||
AND ($2::bool = FALSE OR resolved_at IS NULL) \
|
||||
ORDER BY created_at DESC \
|
||||
LIMIT $3 OFFSET $4",
|
||||
)
|
||||
.bind(app_id.into_inner())
|
||||
.bind(unresolved_only)
|
||||
.bind(limit)
|
||||
.bind(offset)
|
||||
.fetch_all(&self.pool)
|
||||
.await?;
|
||||
Ok(rows.into_iter().map(DeadLetterRowRaw::into_row).collect())
|
||||
}
|
||||
|
||||
async fn unresolved_count(&self, app_id: AppId) -> Result<i64, DeadLetterRepoError> {
|
||||
let (count,): (i64,) = sqlx::query_as(
|
||||
"SELECT COUNT(*) FROM dead_letters \
|
||||
WHERE app_id = $1 AND resolved_at IS NULL",
|
||||
)
|
||||
.bind(app_id.into_inner())
|
||||
.fetch_one(&self.pool)
|
||||
.await?;
|
||||
Ok(count)
|
||||
}
|
||||
|
||||
async fn resolve(&self, id: DeadLetterId, reason: &str) -> Result<(), DeadLetterRepoError> {
|
||||
if !ALLOWED_RESOLUTIONS.contains(&reason) {
|
||||
return Err(DeadLetterRepoError::InvalidResolution(reason.to_string()));
|
||||
}
|
||||
let res = sqlx::query(
|
||||
"UPDATE dead_letters \
|
||||
SET resolution = $2, resolved_at = NOW() \
|
||||
WHERE id = $1",
|
||||
)
|
||||
.bind(id.into_inner())
|
||||
.bind(reason)
|
||||
.execute(&self.pool)
|
||||
.await?;
|
||||
if res.rows_affected() == 0 {
|
||||
return Err(DeadLetterRepoError::NotFound(id));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn gc(&self, older_than: DateTime<Utc>, limit: i64) -> Result<u64, DeadLetterRepoError> {
|
||||
// Tombstones picked under FOR UPDATE SKIP LOCKED so concurrent
|
||||
// sweepers (cluster mode) don't fight each other.
|
||||
let res = sqlx::query(
|
||||
"DELETE FROM dead_letters \
|
||||
WHERE id IN ( \
|
||||
SELECT id FROM dead_letters \
|
||||
WHERE created_at < $1 \
|
||||
FOR UPDATE SKIP LOCKED \
|
||||
LIMIT $2 \
|
||||
)",
|
||||
)
|
||||
.bind(older_than)
|
||||
.bind(limit)
|
||||
.execute(&self.pool)
|
||||
.await?;
|
||||
Ok(res.rows_affected())
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(sqlx::FromRow)]
|
||||
struct DeadLetterRowRaw {
|
||||
id: Uuid,
|
||||
app_id: Uuid,
|
||||
original_event_id: Uuid,
|
||||
source: String,
|
||||
op: String,
|
||||
trigger_id: Option<Uuid>,
|
||||
script_id: Option<Uuid>,
|
||||
payload: serde_json::Value,
|
||||
attempt_count: i32,
|
||||
first_attempt_at: DateTime<Utc>,
|
||||
last_attempt_at: DateTime<Utc>,
|
||||
last_error: String,
|
||||
created_at: DateTime<Utc>,
|
||||
resolved_at: Option<DateTime<Utc>>,
|
||||
resolution: Option<String>,
|
||||
}
|
||||
|
||||
impl DeadLetterRowRaw {
|
||||
fn into_row(self) -> DeadLetterRow {
|
||||
DeadLetterRow {
|
||||
id: self.id.into(),
|
||||
app_id: self.app_id.into(),
|
||||
original_event_id: self.original_event_id,
|
||||
source: self.source,
|
||||
op: self.op,
|
||||
trigger_id: self.trigger_id.map(Into::into),
|
||||
script_id: self.script_id.map(Into::into),
|
||||
payload: self.payload,
|
||||
attempt_count: u32::try_from(self.attempt_count).unwrap_or(0),
|
||||
first_attempt_at: self.first_attempt_at,
|
||||
last_attempt_at: self.last_attempt_at,
|
||||
last_error: self.last_error,
|
||||
created_at: self.created_at,
|
||||
resolved_at: self.resolved_at,
|
||||
resolution: self.resolution,
|
||||
}
|
||||
}
|
||||
}
|
||||
118
crates/manager-core/src/dead_letter_service.rs
Normal file
118
crates/manager-core/src/dead_letter_service.rs
Normal file
@@ -0,0 +1,118 @@
|
||||
//! `PostgresDeadLetterService` — replaces `NoopDeadLetterService` in
|
||||
//! v1.1.1's `Services` bundle. Implements `replay` (re-enqueue the
|
||||
//! original event into the outbox + mark the DL row replayed) and
|
||||
//! `resolve` (close the row out with a reason).
|
||||
//!
|
||||
//! Both methods are gated by `Capability::AppDeadLetterManage(AppId)`
|
||||
//! evaluated against `cx.principal`. Public-HTTP scripts with
|
||||
//! `principal: None` fail the check — design notes §4: managing
|
||||
//! dead letters is an admin act.
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use async_trait::async_trait;
|
||||
use picloud_shared::{DeadLetterError, DeadLetterId, DeadLetterService, SdkCallCx};
|
||||
|
||||
use crate::authz::{self, AuthzRepo, Capability};
|
||||
use crate::dead_letter_repo::{DeadLetterRepo, DeadLetterRepoError, DeadLetterRow};
|
||||
use crate::outbox_repo::{NewOutboxRow, OutboxRepo, OutboxSourceKind};
|
||||
|
||||
pub struct PostgresDeadLetterService {
|
||||
repo: Arc<dyn DeadLetterRepo>,
|
||||
outbox: Arc<dyn OutboxRepo>,
|
||||
authz: Arc<dyn AuthzRepo>,
|
||||
}
|
||||
|
||||
impl PostgresDeadLetterService {
|
||||
#[must_use]
|
||||
pub fn new(
|
||||
repo: Arc<dyn DeadLetterRepo>,
|
||||
outbox: Arc<dyn OutboxRepo>,
|
||||
authz: Arc<dyn AuthzRepo>,
|
||||
) -> Self {
|
||||
Self {
|
||||
repo,
|
||||
outbox,
|
||||
authz,
|
||||
}
|
||||
}
|
||||
|
||||
async fn require_dl_capability(&self, cx: &SdkCallCx) -> Result<(), DeadLetterError> {
|
||||
let Some(ref principal) = cx.principal else {
|
||||
return Err(DeadLetterError::Forbidden);
|
||||
};
|
||||
authz::require(
|
||||
&*self.authz,
|
||||
principal,
|
||||
Capability::AppDeadLetterManage(cx.app_id),
|
||||
)
|
||||
.await
|
||||
.map_err(|_| DeadLetterError::Forbidden)
|
||||
}
|
||||
|
||||
async fn load_row(&self, id: DeadLetterId) -> Result<DeadLetterRow, DeadLetterError> {
|
||||
self.repo
|
||||
.get(id)
|
||||
.await
|
||||
.map_err(map_repo_err)?
|
||||
.ok_or(DeadLetterError::NotFound)
|
||||
}
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl DeadLetterService for PostgresDeadLetterService {
|
||||
async fn replay(&self, cx: &SdkCallCx, id: DeadLetterId) -> Result<(), DeadLetterError> {
|
||||
self.require_dl_capability(cx).await?;
|
||||
let row = self.load_row(id).await?;
|
||||
if row.app_id != cx.app_id {
|
||||
// Cross-app — treat as not-found to avoid leaking
|
||||
// information about other apps' dead letters.
|
||||
return Err(DeadLetterError::NotFound);
|
||||
}
|
||||
|
||||
let source_kind = OutboxSourceKind::from_wire(&row.source).unwrap_or(OutboxSourceKind::Kv);
|
||||
self.outbox
|
||||
.insert(NewOutboxRow {
|
||||
app_id: row.app_id,
|
||||
source_kind,
|
||||
trigger_id: row.trigger_id,
|
||||
script_id: row.script_id,
|
||||
reply_to: None,
|
||||
payload: row.payload.clone(),
|
||||
origin_principal: None,
|
||||
trigger_depth: 0,
|
||||
root_execution_id: None,
|
||||
})
|
||||
.await
|
||||
.map_err(|e| DeadLetterError::Backend(e.to_string()))?;
|
||||
|
||||
self.repo
|
||||
.resolve(id, "replayed")
|
||||
.await
|
||||
.map_err(map_repo_err)?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn resolve(
|
||||
&self,
|
||||
cx: &SdkCallCx,
|
||||
id: DeadLetterId,
|
||||
reason: &str,
|
||||
) -> Result<(), DeadLetterError> {
|
||||
self.require_dl_capability(cx).await?;
|
||||
let row = self.load_row(id).await?;
|
||||
if row.app_id != cx.app_id {
|
||||
return Err(DeadLetterError::NotFound);
|
||||
}
|
||||
self.repo.resolve(id, reason).await.map_err(map_repo_err)?;
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
fn map_repo_err(e: DeadLetterRepoError) -> DeadLetterError {
|
||||
match e {
|
||||
DeadLetterRepoError::NotFound(_) => DeadLetterError::NotFound,
|
||||
DeadLetterRepoError::InvalidResolution(s) => DeadLetterError::InvalidResolution(s),
|
||||
DeadLetterRepoError::Db(e) => DeadLetterError::Backend(e.to_string()),
|
||||
}
|
||||
}
|
||||
316
crates/manager-core/src/dead_letters_api.rs
Normal file
316
crates/manager-core/src/dead_letters_api.rs
Normal file
@@ -0,0 +1,316 @@
|
||||
//! `/api/v1/admin/apps/{id}/dead_letters/*` — dashboard surface for
|
||||
//! the no-default-handler model (design notes §4).
|
||||
//!
|
||||
//! Endpoints:
|
||||
//! - `GET /apps/{id}/dead_letters?unresolved=true` — list view
|
||||
//! - `GET /apps/{id}/dead_letters/count` — badge count
|
||||
//! - `GET /apps/{id}/dead_letters/{dl_id}` — row detail
|
||||
//! - `POST /apps/{id}/dead_letters/{dl_id}/replay` — re-enqueue
|
||||
//! - `POST /apps/{id}/dead_letters/{dl_id}/resolve` — mark resolved
|
||||
//!
|
||||
//! All gated on `Capability::AppDeadLetterManage(app_id)`.
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use axum::extract::{Path, Query, State};
|
||||
use axum::http::StatusCode;
|
||||
use axum::response::{IntoResponse, Json, Response};
|
||||
use axum::routing::{get, post};
|
||||
use axum::{Extension, Router};
|
||||
use picloud_shared::{AppId, DeadLetterId, DeadLetterService, Principal, SdkCallCx};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use serde_json::json;
|
||||
|
||||
use crate::app_repo::AppRepository;
|
||||
use crate::authz::{require, AuthzDenied, AuthzError, AuthzRepo, Capability};
|
||||
use crate::dead_letter_repo::{DeadLetterRepo, DeadLetterRepoError, DeadLetterRow};
|
||||
|
||||
#[derive(Clone)]
|
||||
pub struct DeadLettersState {
|
||||
pub repo: Arc<dyn DeadLetterRepo>,
|
||||
pub service: Arc<dyn DeadLetterService>,
|
||||
pub apps: Arc<dyn AppRepository>,
|
||||
pub authz: Arc<dyn AuthzRepo>,
|
||||
}
|
||||
|
||||
pub fn dead_letters_router(state: DeadLettersState) -> Router {
|
||||
Router::new()
|
||||
.route("/apps/{app_id}/dead_letters", get(list))
|
||||
.route("/apps/{app_id}/dead_letters/count", get(count))
|
||||
.route("/apps/{app_id}/dead_letters/{dl_id}", get(detail))
|
||||
.route("/apps/{app_id}/dead_letters/{dl_id}/replay", post(replay))
|
||||
.route("/apps/{app_id}/dead_letters/{dl_id}/resolve", post(resolve))
|
||||
.with_state(state)
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize)]
|
||||
pub struct ListQuery {
|
||||
#[serde(default)]
|
||||
pub unresolved: bool,
|
||||
#[serde(default = "default_limit")]
|
||||
pub limit: i64,
|
||||
#[serde(default)]
|
||||
pub offset: i64,
|
||||
}
|
||||
|
||||
const fn default_limit() -> i64 {
|
||||
50
|
||||
}
|
||||
|
||||
#[derive(Debug, Serialize)]
|
||||
pub struct ListResponse {
|
||||
pub dead_letters: Vec<DeadLetterDto>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Serialize)]
|
||||
pub struct CountResponse {
|
||||
pub unresolved: i64,
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize)]
|
||||
pub struct ResolveBody {
|
||||
pub reason: String,
|
||||
}
|
||||
|
||||
#[derive(Debug, Serialize)]
|
||||
pub struct DeadLetterDto {
|
||||
pub id: DeadLetterId,
|
||||
pub app_id: AppId,
|
||||
pub source: String,
|
||||
pub op: String,
|
||||
pub trigger_id: Option<picloud_shared::TriggerId>,
|
||||
pub script_id: Option<picloud_shared::ScriptId>,
|
||||
pub payload: serde_json::Value,
|
||||
pub attempt_count: u32,
|
||||
pub first_attempt_at: chrono::DateTime<chrono::Utc>,
|
||||
pub last_attempt_at: chrono::DateTime<chrono::Utc>,
|
||||
pub last_error: String,
|
||||
pub created_at: chrono::DateTime<chrono::Utc>,
|
||||
pub resolved_at: Option<chrono::DateTime<chrono::Utc>>,
|
||||
pub resolution: Option<String>,
|
||||
}
|
||||
|
||||
impl From<DeadLetterRow> for DeadLetterDto {
|
||||
fn from(r: DeadLetterRow) -> Self {
|
||||
Self {
|
||||
id: r.id,
|
||||
app_id: r.app_id,
|
||||
source: r.source,
|
||||
op: r.op,
|
||||
trigger_id: r.trigger_id,
|
||||
script_id: r.script_id,
|
||||
payload: r.payload,
|
||||
attempt_count: r.attempt_count,
|
||||
first_attempt_at: r.first_attempt_at,
|
||||
last_attempt_at: r.last_attempt_at,
|
||||
last_error: r.last_error,
|
||||
created_at: r.created_at,
|
||||
resolved_at: r.resolved_at,
|
||||
resolution: r.resolution,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async fn list(
|
||||
State(s): State<DeadLettersState>,
|
||||
Extension(principal): Extension<Principal>,
|
||||
Path(app_id): Path<AppId>,
|
||||
Query(q): Query<ListQuery>,
|
||||
) -> Result<Json<ListResponse>, DeadLettersApiError> {
|
||||
ensure_app(&*s.apps, app_id).await?;
|
||||
require(
|
||||
s.authz.as_ref(),
|
||||
&principal,
|
||||
Capability::AppDeadLetterManage(app_id),
|
||||
)
|
||||
.await?;
|
||||
let rows = s
|
||||
.repo
|
||||
.list_for_app(app_id, q.unresolved, q.limit.clamp(1, 200), q.offset.max(0))
|
||||
.await?;
|
||||
Ok(Json(ListResponse {
|
||||
dead_letters: rows.into_iter().map(Into::into).collect(),
|
||||
}))
|
||||
}
|
||||
|
||||
async fn count(
|
||||
State(s): State<DeadLettersState>,
|
||||
Extension(principal): Extension<Principal>,
|
||||
Path(app_id): Path<AppId>,
|
||||
) -> Result<Json<CountResponse>, DeadLettersApiError> {
|
||||
ensure_app(&*s.apps, app_id).await?;
|
||||
require(
|
||||
s.authz.as_ref(),
|
||||
&principal,
|
||||
Capability::AppDeadLetterManage(app_id),
|
||||
)
|
||||
.await?;
|
||||
let n = s.repo.unresolved_count(app_id).await?;
|
||||
Ok(Json(CountResponse { unresolved: n }))
|
||||
}
|
||||
|
||||
async fn detail(
|
||||
State(s): State<DeadLettersState>,
|
||||
Extension(principal): Extension<Principal>,
|
||||
Path((app_id, dl_id)): Path<(AppId, DeadLetterId)>,
|
||||
) -> Result<Json<DeadLetterDto>, DeadLettersApiError> {
|
||||
ensure_app(&*s.apps, app_id).await?;
|
||||
require(
|
||||
s.authz.as_ref(),
|
||||
&principal,
|
||||
Capability::AppDeadLetterManage(app_id),
|
||||
)
|
||||
.await?;
|
||||
let row = s
|
||||
.repo
|
||||
.get(dl_id)
|
||||
.await?
|
||||
.ok_or(DeadLettersApiError::NotFound(dl_id))?;
|
||||
if row.app_id != app_id {
|
||||
return Err(DeadLettersApiError::NotFound(dl_id));
|
||||
}
|
||||
Ok(Json(row.into()))
|
||||
}
|
||||
|
||||
async fn replay(
|
||||
State(s): State<DeadLettersState>,
|
||||
Extension(principal): Extension<Principal>,
|
||||
Path((app_id, dl_id)): Path<(AppId, DeadLetterId)>,
|
||||
) -> Result<StatusCode, DeadLettersApiError> {
|
||||
ensure_app(&*s.apps, app_id).await?;
|
||||
// Authz handled inside the service via SdkCallCx.
|
||||
let cx = admin_cx(app_id, &principal);
|
||||
s.service
|
||||
.replay(&cx, dl_id)
|
||||
.await
|
||||
.map_err(map_service_err)?;
|
||||
Ok(StatusCode::NO_CONTENT)
|
||||
}
|
||||
|
||||
async fn resolve(
|
||||
State(s): State<DeadLettersState>,
|
||||
Extension(principal): Extension<Principal>,
|
||||
Path((app_id, dl_id)): Path<(AppId, DeadLetterId)>,
|
||||
Json(body): Json<ResolveBody>,
|
||||
) -> Result<StatusCode, DeadLettersApiError> {
|
||||
ensure_app(&*s.apps, app_id).await?;
|
||||
let cx = admin_cx(app_id, &principal);
|
||||
s.service
|
||||
.resolve(&cx, dl_id, &body.reason)
|
||||
.await
|
||||
.map_err(map_service_err)?;
|
||||
Ok(StatusCode::NO_CONTENT)
|
||||
}
|
||||
|
||||
/// Synthesize an `SdkCallCx` for the admin path. The service layer
|
||||
/// reads `cx.app_id` + `cx.principal` and ignores the trigger /
|
||||
/// execution fields, so the per-call ids are arbitrary.
|
||||
fn admin_cx(app_id: AppId, principal: &Principal) -> SdkCallCx {
|
||||
SdkCallCx {
|
||||
app_id,
|
||||
principal: Some(principal.clone()),
|
||||
execution_id: picloud_shared::ExecutionId::new(),
|
||||
request_id: picloud_shared::RequestId::new(),
|
||||
trigger_depth: 0,
|
||||
root_execution_id: picloud_shared::ExecutionId::new(),
|
||||
is_dead_letter_handler: false,
|
||||
event: None,
|
||||
}
|
||||
}
|
||||
|
||||
async fn ensure_app(apps: &dyn AppRepository, app_id: AppId) -> Result<(), DeadLettersApiError> {
|
||||
apps.get_by_id(app_id)
|
||||
.await
|
||||
.map_err(|e| DeadLettersApiError::Backend(e.to_string()))?
|
||||
.ok_or_else(|| DeadLettersApiError::AppNotFound(app_id.to_string()))?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn map_service_err(e: picloud_shared::DeadLetterError) -> DeadLettersApiError {
|
||||
match e {
|
||||
picloud_shared::DeadLetterError::NotFound => {
|
||||
DeadLettersApiError::NotFound(DeadLetterId::new())
|
||||
}
|
||||
picloud_shared::DeadLetterError::Forbidden => DeadLettersApiError::Forbidden,
|
||||
picloud_shared::DeadLetterError::InvalidResolution(s) => {
|
||||
DeadLettersApiError::Invalid(format!("invalid resolution: {s}"))
|
||||
}
|
||||
picloud_shared::DeadLetterError::Backend(s) => DeadLettersApiError::Backend(s),
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub enum DeadLettersApiError {
|
||||
#[error("app not found: {0}")]
|
||||
AppNotFound(String),
|
||||
|
||||
#[error("dead-letter not found: {0}")]
|
||||
NotFound(DeadLetterId),
|
||||
|
||||
#[error("invalid: {0}")]
|
||||
Invalid(String),
|
||||
|
||||
#[error("forbidden")]
|
||||
Forbidden,
|
||||
|
||||
#[error("authorization repo error: {0}")]
|
||||
AuthzRepo(String),
|
||||
|
||||
#[error("dead-letter backend: {0}")]
|
||||
Backend(String),
|
||||
}
|
||||
|
||||
impl From<AuthzDenied> for DeadLettersApiError {
|
||||
fn from(d: AuthzDenied) -> Self {
|
||||
match d {
|
||||
AuthzDenied::Denied => Self::Forbidden,
|
||||
AuthzDenied::Repo(e) => Self::AuthzRepo(e.to_string()),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl From<AuthzError> for DeadLettersApiError {
|
||||
fn from(e: AuthzError) -> Self {
|
||||
Self::AuthzRepo(e.to_string())
|
||||
}
|
||||
}
|
||||
|
||||
impl From<DeadLetterRepoError> for DeadLettersApiError {
|
||||
fn from(e: DeadLetterRepoError) -> Self {
|
||||
match e {
|
||||
DeadLetterRepoError::NotFound(id) => Self::NotFound(id),
|
||||
DeadLetterRepoError::InvalidResolution(s) => Self::Invalid(s),
|
||||
DeadLetterRepoError::Db(e) => Self::Backend(e.to_string()),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl IntoResponse for DeadLettersApiError {
|
||||
fn into_response(self) -> Response {
|
||||
let (status, body) = match &self {
|
||||
Self::AppNotFound(_) | Self::NotFound(_) => {
|
||||
(StatusCode::NOT_FOUND, json!({ "error": self.to_string() }))
|
||||
}
|
||||
Self::Invalid(_) => (
|
||||
StatusCode::UNPROCESSABLE_ENTITY,
|
||||
json!({ "error": self.to_string() }),
|
||||
),
|
||||
Self::Forbidden => (StatusCode::FORBIDDEN, json!({ "error": self.to_string() })),
|
||||
Self::AuthzRepo(e) => {
|
||||
tracing::error!(error = %e, "dead_letters authz repo error");
|
||||
(
|
||||
StatusCode::INTERNAL_SERVER_ERROR,
|
||||
json!({ "error": "internal error" }),
|
||||
)
|
||||
}
|
||||
Self::Backend(e) => {
|
||||
tracing::error!(error = %e, "dead_letters api backend error");
|
||||
(
|
||||
StatusCode::INTERNAL_SERVER_ERROR,
|
||||
json!({ "error": "internal error" }),
|
||||
)
|
||||
}
|
||||
};
|
||||
(status, Json(body)).into_response()
|
||||
}
|
||||
}
|
||||
685
crates/manager-core/src/dispatcher.rs
Normal file
685
crates/manager-core/src/dispatcher.rs
Normal file
@@ -0,0 +1,685 @@
|
||||
//! The triggers-framework dispatcher.
|
||||
//!
|
||||
//! Single tokio task that polls the outbox, claims due rows
|
||||
//! (`FOR UPDATE SKIP LOCKED`), and routes each to the executor.
|
||||
//! Shares the `ExecutionGate` with sync HTTP — they compete for the
|
||||
//! same permit budget, matching design notes §2.
|
||||
//!
|
||||
//! Outcome handling per design notes §3 and §4:
|
||||
//! - reply_to.is_some() (sync HTTP): never retry. Deliver to inbox
|
||||
//! (or write `abandoned_executions` if the receiver dropped).
|
||||
//! - is_dead_letter_handler == true: never retry, never DL. Failure
|
||||
//! just annotates the original DL row with `resolution =
|
||||
//! 'handler_failed'` and bumps a metric.
|
||||
//! - Otherwise on failure: if `attempt_count + 1 < max_attempts`,
|
||||
//! reschedule with backoff + jitter. Else, write a `dead_letters`
|
||||
//! row and delete from outbox.
|
||||
//!
|
||||
//! Depth-limit: `trigger_depth > max_trigger_depth` skips execution
|
||||
//! entirely (log + metric) and deletes the row — does NOT dead-letter
|
||||
//! (design notes §4: depth-exceeded means "you built a loop", and
|
||||
//! dead-lettering would just re-fire the same loop).
|
||||
|
||||
use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
|
||||
use chrono::Utc;
|
||||
use picloud_executor_core::{ExecError, ExecRequest, ExecResponse, InvocationType};
|
||||
use picloud_orchestrator_core::{ExecutionGate, ExecutorClient};
|
||||
use picloud_shared::{
|
||||
ExecResponseSummary, ExecutionId, HttpDispatchPayload, InboxDeliveryOutcome, InboxFailureKind,
|
||||
InboxResolver, InboxResult, RequestId, ScriptId, ScriptSandbox, TriggerEvent,
|
||||
};
|
||||
use rand::Rng;
|
||||
use uuid::Uuid;
|
||||
|
||||
use crate::abandoned_repo::{AbandonedRepo, NewAbandonedExecution};
|
||||
use crate::dead_letter_repo::{DeadLetterRepo, NewDeadLetter};
|
||||
use crate::outbox_repo::{OutboxRepo, OutboxRow, OutboxSourceKind};
|
||||
use crate::principal_resolver::PrincipalResolver;
|
||||
use crate::repo::ScriptRepository;
|
||||
use crate::trigger_config::{BackoffShape, TriggerConfig};
|
||||
use crate::trigger_repo::{TriggerKind, TriggerRepo};
|
||||
|
||||
/// Bundle the dispatcher reads from. Each handle is `Arc<dyn …>` so
|
||||
/// tests can substitute in-memory backings.
|
||||
pub struct Dispatcher {
|
||||
pub outbox: Arc<dyn OutboxRepo>,
|
||||
pub triggers: Arc<dyn TriggerRepo>,
|
||||
pub scripts: Arc<dyn ScriptRepository>,
|
||||
pub dead_letters: Arc<dyn DeadLetterRepo>,
|
||||
pub abandoned: Arc<dyn AbandonedRepo>,
|
||||
pub principals: Arc<dyn PrincipalResolver>,
|
||||
pub executor: Arc<dyn ExecutorClient>,
|
||||
pub gate: Arc<ExecutionGate>,
|
||||
pub inbox: Arc<dyn InboxResolver>,
|
||||
pub config: TriggerConfig,
|
||||
/// Stable id for this dispatcher instance — written into
|
||||
/// `outbox.claimed_by` for forensics. In MVP this is the host's
|
||||
/// pid; cluster mode (v1.3+) uses node identity.
|
||||
pub instance_id: String,
|
||||
}
|
||||
|
||||
/// How many outbox rows the dispatcher tries to claim per tick.
|
||||
/// Bounded to keep the working set small even if there's a flood.
|
||||
const CLAIM_BATCH: i64 = 8;
|
||||
|
||||
/// Polling cadence. Short enough that fan-out feels instant; long
|
||||
/// enough that an idle dispatcher doesn't burn cycles.
|
||||
const TICK_INTERVAL: Duration = Duration::from_millis(100);
|
||||
|
||||
/// Hard cap on the wall-clock budget passed to the executor for an
|
||||
/// async-dispatched script. Sync HTTP gets a per-script timeout via
|
||||
/// the orchestrator path; async rows don't have one, so we apply a
|
||||
/// platform-wide ceiling here. Matches `LocalExecutorClient`'s own
|
||||
/// 5-minute cap.
|
||||
const ASYNC_EXEC_TIMEOUT: Duration = Duration::from_secs(300);
|
||||
|
||||
impl Dispatcher {
|
||||
/// Spawn the dispatcher loop as a detached `tokio::task`. The
|
||||
/// returned `JoinHandle` is dropped — the loop runs for the
|
||||
/// process lifetime.
|
||||
pub fn spawn(self) {
|
||||
tokio::spawn(async move {
|
||||
self.run().await;
|
||||
});
|
||||
}
|
||||
|
||||
async fn run(self) {
|
||||
let mut ticker = tokio::time::interval(TICK_INTERVAL);
|
||||
// Skip the immediate first fire so we don't race startup.
|
||||
ticker.tick().await;
|
||||
loop {
|
||||
ticker.tick().await;
|
||||
if let Err(err) = self.tick().await {
|
||||
tracing::warn!(?err, "dispatcher tick errored");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async fn tick(&self) -> Result<(), DispatcherError> {
|
||||
// Cheap gate sample so we don't claim rows we can't dispatch.
|
||||
// The exact permit budget is reapplied per-row below.
|
||||
let rows = self
|
||||
.outbox
|
||||
.claim_due(&self.instance_id, CLAIM_BATCH)
|
||||
.await
|
||||
.map_err(|e| DispatcherError::Outbox(e.to_string()))?;
|
||||
if rows.is_empty() {
|
||||
return Ok(());
|
||||
}
|
||||
for row in rows {
|
||||
// Process serially within a tick — the outer ticker is the
|
||||
// pacing mechanism. Concurrent dispatchers are a cluster-
|
||||
// mode concern; v1.1.1 MVP has one.
|
||||
if let Err(err) = self.dispatch_one(row).await {
|
||||
tracing::warn!(?err, "dispatch one errored");
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn dispatch_one(&self, row: OutboxRow) -> Result<(), DispatcherError> {
|
||||
// Depth-limit check — design notes §4: loops aren't DL'd.
|
||||
if row.trigger_depth > self.config.max_trigger_depth {
|
||||
tracing::warn!(
|
||||
outbox_id = %row.id,
|
||||
app_id = %row.app_id,
|
||||
trigger_depth = row.trigger_depth,
|
||||
"trigger depth exceeded; dropping row"
|
||||
);
|
||||
// TODO(metrics): bump `picloud_trigger_depth_exceeded{app_id,trigger_id}`.
|
||||
self.outbox
|
||||
.delete(row.id)
|
||||
.await
|
||||
.map_err(|e| DispatcherError::Outbox(e.to_string()))?;
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
// Gate admission — non-blocking. If the gate is saturated,
|
||||
// release the claim by rescheduling so another tick can pick
|
||||
// it up. The row stays "due" essentially immediately.
|
||||
let Ok(permit) = self.gate.try_acquire() else {
|
||||
let next = Utc::now() + chrono::Duration::milliseconds(100);
|
||||
self.outbox
|
||||
.reschedule(row.id, row.attempt_count, next)
|
||||
.await
|
||||
.map_err(|e| DispatcherError::Outbox(e.to_string()))?;
|
||||
return Ok(());
|
||||
};
|
||||
|
||||
// Resolve the trigger config (KV / DL) or pull the HTTP
|
||||
// payload directly off the outbox row.
|
||||
let (resolved, exec_req) = match row.source_kind {
|
||||
OutboxSourceKind::Http => match self.build_http_request(&row).await {
|
||||
Ok(pair) => pair,
|
||||
Err(err) => {
|
||||
tracing::warn!(outbox_id = %row.id, ?err, "http exec build failed; dropping");
|
||||
self.outbox
|
||||
.delete(row.id)
|
||||
.await
|
||||
.map_err(|e| DispatcherError::Outbox(e.to_string()))?;
|
||||
drop(permit);
|
||||
return Ok(());
|
||||
}
|
||||
},
|
||||
OutboxSourceKind::Kv | OutboxSourceKind::DeadLetter => {
|
||||
let resolved = self.resolve_trigger(&row).await?;
|
||||
let req = match self.build_exec_request(&row, &resolved).await {
|
||||
Ok(req) => req,
|
||||
Err(err) => {
|
||||
tracing::warn!(outbox_id = %row.id, ?err, "exec request build failed; dropping row");
|
||||
self.outbox
|
||||
.delete(row.id)
|
||||
.await
|
||||
.map_err(|e| DispatcherError::Outbox(e.to_string()))?;
|
||||
drop(permit);
|
||||
return Ok(());
|
||||
}
|
||||
};
|
||||
(resolved, req)
|
||||
}
|
||||
};
|
||||
|
||||
// The gate permit auto-releases when this scope ends or when
|
||||
// the executor finishes. We hand control to the executor and
|
||||
// wait synchronously here — sync HTTP and dispatcher share the
|
||||
// semaphore so this is intentional.
|
||||
let source = resolved.script_source.clone();
|
||||
let outcome = self
|
||||
.executor
|
||||
.execute(&source, exec_req, ASYNC_EXEC_TIMEOUT)
|
||||
.await;
|
||||
drop(permit);
|
||||
|
||||
match outcome {
|
||||
Ok(resp) => self.handle_success(&row, &resolved, resp).await,
|
||||
Err(err) => self.handle_failure(&row, &resolved, err).await,
|
||||
}
|
||||
}
|
||||
|
||||
async fn resolve_trigger(&self, row: &OutboxRow) -> Result<ResolvedTrigger, DispatcherError> {
|
||||
// For KV and DL kinds, the outbox carries `trigger_id`. Use it
|
||||
// to look up the trigger row, then resolve the script.
|
||||
let Some(trigger_id) = row.trigger_id else {
|
||||
return Err(DispatcherError::ResolveTrigger(
|
||||
"outbox row missing trigger_id".into(),
|
||||
));
|
||||
};
|
||||
let trigger = self
|
||||
.triggers
|
||||
.get(trigger_id)
|
||||
.await
|
||||
.map_err(|e| DispatcherError::ResolveTrigger(e.to_string()))?
|
||||
.ok_or_else(|| {
|
||||
DispatcherError::ResolveTrigger(format!("trigger {trigger_id} not found"))
|
||||
})?;
|
||||
|
||||
let script = self
|
||||
.scripts
|
||||
.get(trigger.script_id)
|
||||
.await
|
||||
.map_err(|e| DispatcherError::ResolveTrigger(e.to_string()))?
|
||||
.ok_or_else(|| {
|
||||
DispatcherError::ResolveTrigger(format!("script {} not found", trigger.script_id))
|
||||
})?;
|
||||
|
||||
Ok(ResolvedTrigger {
|
||||
trigger_kind: trigger.kind,
|
||||
is_dead_letter_handler: matches!(trigger.kind, TriggerKind::DeadLetter),
|
||||
script_id: script.id,
|
||||
script_source: script.source,
|
||||
script_name: script.name,
|
||||
sandbox_overrides: script.sandbox,
|
||||
registered_by_principal: trigger.registered_by_principal,
|
||||
retry_max_attempts: trigger.retry_max_attempts,
|
||||
retry_backoff: trigger.retry_backoff,
|
||||
retry_base_ms: trigger.retry_base_ms,
|
||||
})
|
||||
}
|
||||
|
||||
async fn build_exec_request(
|
||||
&self,
|
||||
row: &OutboxRow,
|
||||
resolved: &ResolvedTrigger,
|
||||
) -> Result<ExecRequest, DispatcherError> {
|
||||
let trigger_event: TriggerEvent = serde_json::from_value(row.payload.clone())
|
||||
.map_err(|e| DispatcherError::ResolveTrigger(format!("decode payload: {e}")))?;
|
||||
|
||||
let principal = self
|
||||
.principals
|
||||
.resolve(resolved.registered_by_principal)
|
||||
.await
|
||||
.map_err(|e| DispatcherError::ResolveTrigger(e.to_string()))?;
|
||||
|
||||
let execution_id = ExecutionId::new();
|
||||
Ok(ExecRequest {
|
||||
execution_id,
|
||||
request_id: RequestId::new(),
|
||||
script_id: resolved.script_id,
|
||||
script_name: resolved.script_name.clone(),
|
||||
invocation_type: InvocationType::Function,
|
||||
path: format!("/trigger/{}", trigger_event.source()),
|
||||
headers: std::collections::BTreeMap::new(),
|
||||
body: serde_json::Value::Null,
|
||||
params: std::collections::BTreeMap::new(),
|
||||
query: std::collections::BTreeMap::new(),
|
||||
rest: String::new(),
|
||||
sandbox_overrides: resolved.sandbox_overrides,
|
||||
app_id: row.app_id,
|
||||
principal: Some(principal),
|
||||
trigger_depth: row.trigger_depth,
|
||||
root_execution_id: row.root_execution_id.unwrap_or(execution_id),
|
||||
is_dead_letter_handler: resolved.is_dead_letter_handler,
|
||||
event: Some(trigger_event),
|
||||
})
|
||||
}
|
||||
|
||||
/// Build an `(ResolvedTrigger, ExecRequest)` for an HTTP outbox
|
||||
/// row. HTTP rows don't have a backing `triggers` row (the
|
||||
/// `trigger_id` references `routes.id` instead). We pull the
|
||||
/// script id off the outbox row, the request shape off the
|
||||
/// payload, and synthesize a `ResolvedTrigger` with retry
|
||||
/// settings irrelevant for HTTP (sync HTTP is never retried;
|
||||
/// async HTTP uses default policy from `TriggerConfig`).
|
||||
async fn build_http_request(
|
||||
&self,
|
||||
row: &OutboxRow,
|
||||
) -> Result<(ResolvedTrigger, ExecRequest), DispatcherError> {
|
||||
let Some(script_id) = row.script_id else {
|
||||
return Err(DispatcherError::ResolveTrigger(
|
||||
"HTTP outbox row missing script_id".into(),
|
||||
));
|
||||
};
|
||||
let script = self
|
||||
.scripts
|
||||
.get(script_id)
|
||||
.await
|
||||
.map_err(|e| DispatcherError::ResolveTrigger(e.to_string()))?
|
||||
.ok_or_else(|| {
|
||||
DispatcherError::ResolveTrigger(format!("script {script_id} not found"))
|
||||
})?;
|
||||
|
||||
let payload: HttpDispatchPayload = serde_json::from_value(row.payload.clone())
|
||||
.map_err(|e| DispatcherError::ResolveTrigger(format!("decode http payload: {e}")))?;
|
||||
|
||||
let execution_id = ExecutionId::new();
|
||||
let req = ExecRequest {
|
||||
execution_id,
|
||||
request_id: RequestId::new(),
|
||||
script_id,
|
||||
script_name: payload.script_name.clone(),
|
||||
invocation_type: InvocationType::Http,
|
||||
path: payload.path.clone(),
|
||||
headers: payload.headers,
|
||||
body: payload.body,
|
||||
params: payload.params,
|
||||
query: payload.query,
|
||||
rest: payload.rest,
|
||||
sandbox_overrides: script.sandbox,
|
||||
app_id: row.app_id,
|
||||
// HTTP outbox rows don't run as the trigger registrant —
|
||||
// they run with no principal (public ingress) or the
|
||||
// attached one (origin_principal forensic field is not
|
||||
// promoted to execution principal in this MVP).
|
||||
principal: None,
|
||||
trigger_depth: row.trigger_depth,
|
||||
root_execution_id: row.root_execution_id.unwrap_or(execution_id),
|
||||
is_dead_letter_handler: false,
|
||||
event: None,
|
||||
};
|
||||
|
||||
let resolved = ResolvedTrigger {
|
||||
trigger_kind: TriggerKind::Kv, // placeholder; HTTP doesn't have a kind
|
||||
is_dead_letter_handler: false,
|
||||
script_id,
|
||||
script_source: script.source,
|
||||
script_name: payload.script_name,
|
||||
sandbox_overrides: script.sandbox,
|
||||
// HTTP outbox rows don't carry a registered_by_principal
|
||||
// — use a sentinel zero UUID since this field isn't used
|
||||
// downstream for HTTP (no retries, no inbox principal).
|
||||
registered_by_principal: picloud_shared::AdminUserId::from(uuid::Uuid::nil()),
|
||||
// Async HTTP uses the platform default retry policy from
|
||||
// TriggerConfig. Sync HTTP (reply_to.is_some) never retries
|
||||
// regardless.
|
||||
retry_max_attempts: self.config.retry_max_attempts,
|
||||
retry_backoff: self.config.retry_backoff,
|
||||
retry_base_ms: self.config.retry_base_ms,
|
||||
};
|
||||
Ok((resolved, req))
|
||||
}
|
||||
|
||||
async fn handle_success(
|
||||
&self,
|
||||
row: &OutboxRow,
|
||||
_resolved: &ResolvedTrigger,
|
||||
resp: ExecResponse,
|
||||
) -> Result<(), DispatcherError> {
|
||||
if let Some(inbox_id) = row.reply_to {
|
||||
self.deliver_inbox(row, inbox_id, InboxResult::Success(summarize(&resp)))
|
||||
.await;
|
||||
}
|
||||
self.outbox
|
||||
.delete(row.id)
|
||||
.await
|
||||
.map_err(|e| DispatcherError::Outbox(e.to_string()))?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn handle_failure(
|
||||
&self,
|
||||
row: &OutboxRow,
|
||||
resolved: &ResolvedTrigger,
|
||||
err: ExecError,
|
||||
) -> Result<(), DispatcherError> {
|
||||
// Sync HTTP: always single-attempt. Always deliver outcome
|
||||
// (success-or-failure) to the inbox. Never retry, never DL.
|
||||
if let Some(inbox_id) = row.reply_to {
|
||||
let (kind, message) = classify_exec_error(&err);
|
||||
self.deliver_inbox(
|
||||
row,
|
||||
inbox_id,
|
||||
InboxResult::Failure {
|
||||
kind,
|
||||
message: message.clone(),
|
||||
},
|
||||
)
|
||||
.await;
|
||||
self.outbox
|
||||
.delete(row.id)
|
||||
.await
|
||||
.map_err(|e| DispatcherError::Outbox(e.to_string()))?;
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
// Dead-letter handler: never retry, never DL. Failure
|
||||
// annotates the original DL row + bumps a metric.
|
||||
if resolved.is_dead_letter_handler {
|
||||
tracing::error!(
|
||||
outbox_id = %row.id,
|
||||
app_id = %row.app_id,
|
||||
?err,
|
||||
"dead-letter handler failed; not retrying"
|
||||
);
|
||||
// TODO(metrics): bump `picloud_dead_letter_handler_failures{app_id}`.
|
||||
// Annotate the original DL row (id is `row.payload.dead_letter.id`
|
||||
// when the payload is a DeadLetter TriggerEvent). Best-effort:
|
||||
// if the payload doesn't decode, just log and move on.
|
||||
if let Ok(TriggerEvent::DeadLetter { dead_letter_id, .. }) =
|
||||
serde_json::from_value::<TriggerEvent>(row.payload.clone())
|
||||
{
|
||||
if let Err(e) = self
|
||||
.dead_letters
|
||||
.resolve(dead_letter_id, "handler_failed")
|
||||
.await
|
||||
{
|
||||
tracing::warn!(?e, "could not annotate DL row as handler_failed");
|
||||
}
|
||||
}
|
||||
self.outbox
|
||||
.delete(row.id)
|
||||
.await
|
||||
.map_err(|e| DispatcherError::Outbox(e.to_string()))?;
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
// Async event: retry per policy, then dead-letter.
|
||||
let attempt = row.attempt_count + 1;
|
||||
if attempt < resolved.retry_max_attempts {
|
||||
let delay = compute_backoff(
|
||||
attempt,
|
||||
resolved.retry_backoff,
|
||||
resolved.retry_base_ms,
|
||||
self.config.retry_jitter_pct,
|
||||
);
|
||||
let next = Utc::now() + chrono::Duration::milliseconds(i64::from(delay));
|
||||
tracing::info!(
|
||||
outbox_id = %row.id,
|
||||
attempt,
|
||||
max_attempts = resolved.retry_max_attempts,
|
||||
retry_in_ms = delay,
|
||||
"rescheduling outbox row"
|
||||
);
|
||||
self.outbox
|
||||
.reschedule(row.id, attempt, next)
|
||||
.await
|
||||
.map_err(|e| DispatcherError::Outbox(e.to_string()))?;
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
// Exhausted retries → dead-letter.
|
||||
let (op, source) = describe_event(&row.payload);
|
||||
let now = Utc::now();
|
||||
if let Err(e) = self
|
||||
.dead_letters
|
||||
.insert(NewDeadLetter {
|
||||
app_id: row.app_id,
|
||||
original_event_id: row.id,
|
||||
source,
|
||||
op,
|
||||
trigger_id: row.trigger_id,
|
||||
script_id: Some(resolved.script_id),
|
||||
payload: row.payload.clone(),
|
||||
attempt_count: attempt,
|
||||
first_attempt_at: row.created_at,
|
||||
last_attempt_at: now,
|
||||
last_error: err.to_string(),
|
||||
})
|
||||
.await
|
||||
{
|
||||
tracing::error!(?e, "failed to write dead-letter row");
|
||||
}
|
||||
self.outbox
|
||||
.delete(row.id)
|
||||
.await
|
||||
.map_err(|e| DispatcherError::Outbox(e.to_string()))?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn deliver_inbox(&self, row: &OutboxRow, inbox_id: Uuid, result: InboxResult) {
|
||||
match self.inbox.deliver(inbox_id, result.clone()).await {
|
||||
InboxDeliveryOutcome::Delivered => {}
|
||||
InboxDeliveryOutcome::Abandoned => {
|
||||
// Receiver was dropped — record forensic row + bump
|
||||
// metric.
|
||||
let (status_code, summary) = match &result {
|
||||
InboxResult::Success(s) => (s.status_code, None),
|
||||
InboxResult::Failure { kind, message } => {
|
||||
(failure_kind_to_status(*kind), Some(message.clone()))
|
||||
}
|
||||
};
|
||||
if let Err(e) = self
|
||||
.abandoned
|
||||
.insert(NewAbandonedExecution {
|
||||
app_id: row.app_id,
|
||||
outbox_id: row.id,
|
||||
script_id: row.script_id,
|
||||
inbox_id,
|
||||
status_code,
|
||||
result_summary: summary,
|
||||
})
|
||||
.await
|
||||
{
|
||||
tracing::warn!(?e, "abandoned_executions insert failed");
|
||||
}
|
||||
// TODO(metrics): bump `picloud_abandoned_executions_total{app_id}`.
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
pub struct ResolvedTrigger {
|
||||
pub trigger_kind: TriggerKind,
|
||||
pub is_dead_letter_handler: bool,
|
||||
pub script_id: ScriptId,
|
||||
pub script_source: String,
|
||||
pub script_name: String,
|
||||
pub sandbox_overrides: ScriptSandbox,
|
||||
pub registered_by_principal: picloud_shared::AdminUserId,
|
||||
pub retry_max_attempts: u32,
|
||||
pub retry_backoff: BackoffShape,
|
||||
pub retry_base_ms: u32,
|
||||
}
|
||||
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub enum DispatcherError {
|
||||
#[error("outbox: {0}")]
|
||||
Outbox(String),
|
||||
#[error("resolve trigger: {0}")]
|
||||
ResolveTrigger(String),
|
||||
}
|
||||
|
||||
fn summarize(resp: &ExecResponse) -> ExecResponseSummary {
|
||||
ExecResponseSummary {
|
||||
status_code: resp.status_code,
|
||||
headers: resp.headers.clone(),
|
||||
body: resp.body.clone(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Map `ExecError` onto the design-notes §3 status-code table.
|
||||
fn classify_exec_error(err: &ExecError) -> (InboxFailureKind, String) {
|
||||
match err {
|
||||
ExecError::Parse(s) | ExecError::InvalidResponse(s) => {
|
||||
(InboxFailureKind::Validation, s.clone())
|
||||
}
|
||||
ExecError::Timeout(_) => (InboxFailureKind::Timeout, err.to_string()),
|
||||
ExecError::OperationBudgetExceeded => (InboxFailureKind::OperationBudget, err.to_string()),
|
||||
ExecError::Overloaded { .. } => (InboxFailureKind::Overloaded, err.to_string()),
|
||||
ExecError::Runtime(s) => (InboxFailureKind::Runtime, s.clone()),
|
||||
}
|
||||
}
|
||||
|
||||
fn failure_kind_to_status(k: InboxFailureKind) -> u16 {
|
||||
match k {
|
||||
InboxFailureKind::Validation => 422,
|
||||
InboxFailureKind::Runtime => 502,
|
||||
InboxFailureKind::Overloaded => 503,
|
||||
InboxFailureKind::Timeout => 504,
|
||||
InboxFailureKind::OperationBudget => 507,
|
||||
InboxFailureKind::Platform => 500,
|
||||
}
|
||||
}
|
||||
|
||||
/// `(op, source)` extracted from the outbox payload. Used to seed the
|
||||
/// `dead_letters` row when retries exhaust.
|
||||
fn describe_event(payload: &serde_json::Value) -> (String, String) {
|
||||
let source = payload
|
||||
.get("source")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("")
|
||||
.to_string();
|
||||
let op = payload
|
||||
.get("op")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("")
|
||||
.to_string();
|
||||
(op, source)
|
||||
}
|
||||
|
||||
/// Compute backoff (ms) for the given attempt + policy + jitter.
|
||||
/// Attempt is 1-indexed (first retry = attempt 1).
|
||||
#[must_use]
|
||||
pub fn compute_backoff(attempt: u32, backoff: BackoffShape, base_ms: u32, jitter_pct: u32) -> u32 {
|
||||
let base_ms = u64::from(base_ms);
|
||||
let attempt = u64::from(attempt.saturating_sub(1));
|
||||
let raw = match backoff {
|
||||
BackoffShape::Constant => base_ms,
|
||||
BackoffShape::Linear => base_ms * (attempt + 1),
|
||||
// 1x base, 2x base, 4x base, … (saturating).
|
||||
BackoffShape::Exponential => base_ms.saturating_mul(1u64 << attempt.min(20)),
|
||||
};
|
||||
let raw = u32::try_from(raw.min(u64::from(u32::MAX))).unwrap_or(u32::MAX);
|
||||
apply_jitter(raw, jitter_pct)
|
||||
}
|
||||
|
||||
fn apply_jitter(raw: u32, pct: u32) -> u32 {
|
||||
if pct == 0 {
|
||||
return raw;
|
||||
}
|
||||
let pct = pct.min(100);
|
||||
// ±span% — bounded by raw itself so we can't underflow when
|
||||
// raw + offset goes below zero.
|
||||
let span = u64::from(raw) * u64::from(pct) / 100;
|
||||
if span == 0 {
|
||||
return raw;
|
||||
}
|
||||
let span_i64 = i64::try_from(span).unwrap_or(i64::MAX);
|
||||
let mut rng = rand::thread_rng();
|
||||
let offset = rng.gen_range(-span_i64..=span_i64);
|
||||
let signed = i64::from(raw).saturating_add(offset).max(0);
|
||||
u32::try_from(signed.min(i64::from(u32::MAX))).unwrap_or(u32::MAX)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn exponential_backoff_doubles_per_attempt() {
|
||||
// No jitter (pct=0) for a deterministic check.
|
||||
assert_eq!(compute_backoff(1, BackoffShape::Exponential, 1000, 0), 1000);
|
||||
assert_eq!(compute_backoff(2, BackoffShape::Exponential, 1000, 0), 2000);
|
||||
assert_eq!(compute_backoff(3, BackoffShape::Exponential, 1000, 0), 4000);
|
||||
assert_eq!(compute_backoff(4, BackoffShape::Exponential, 1000, 0), 8000);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn linear_backoff_scales_with_attempt() {
|
||||
assert_eq!(compute_backoff(1, BackoffShape::Linear, 100, 0), 100);
|
||||
assert_eq!(compute_backoff(2, BackoffShape::Linear, 100, 0), 200);
|
||||
assert_eq!(compute_backoff(5, BackoffShape::Linear, 100, 0), 500);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn constant_backoff_returns_base() {
|
||||
for attempt in 1..=5 {
|
||||
assert_eq!(
|
||||
compute_backoff(attempt, BackoffShape::Constant, 750, 0),
|
||||
750
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn jitter_within_pct_of_base() {
|
||||
for _ in 0..100 {
|
||||
let v = compute_backoff(1, BackoffShape::Constant, 1000, 20);
|
||||
// ±20% of 1000 = 800..=1200.
|
||||
assert!((800..=1200).contains(&v), "jitter out of range: {v}");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn classify_exec_error_covers_every_variant() {
|
||||
let parse = classify_exec_error(&ExecError::Parse("nope".into()));
|
||||
assert!(matches!(parse.0, InboxFailureKind::Validation));
|
||||
let invalid = classify_exec_error(&ExecError::InvalidResponse("bad".into()));
|
||||
assert!(matches!(invalid.0, InboxFailureKind::Validation));
|
||||
let timeout = classify_exec_error(&ExecError::Timeout(30));
|
||||
assert!(matches!(timeout.0, InboxFailureKind::Timeout));
|
||||
let budget = classify_exec_error(&ExecError::OperationBudgetExceeded);
|
||||
assert!(matches!(budget.0, InboxFailureKind::OperationBudget));
|
||||
let runtime = classify_exec_error(&ExecError::Runtime("threw".into()));
|
||||
assert!(matches!(runtime.0, InboxFailureKind::Runtime));
|
||||
let overload = classify_exec_error(&ExecError::Overloaded {
|
||||
retry_after_secs: 1,
|
||||
});
|
||||
assert!(matches!(overload.0, InboxFailureKind::Overloaded));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn failure_kind_status_codes_match_design_notes() {
|
||||
assert_eq!(failure_kind_to_status(InboxFailureKind::Validation), 422);
|
||||
assert_eq!(failure_kind_to_status(InboxFailureKind::Runtime), 502);
|
||||
assert_eq!(failure_kind_to_status(InboxFailureKind::Overloaded), 503);
|
||||
assert_eq!(failure_kind_to_status(InboxFailureKind::Timeout), 504);
|
||||
assert_eq!(
|
||||
failure_kind_to_status(InboxFailureKind::OperationBudget),
|
||||
507
|
||||
);
|
||||
assert_eq!(failure_kind_to_status(InboxFailureKind::Platform), 500);
|
||||
}
|
||||
}
|
||||
95
crates/manager-core/src/gc.rs
Normal file
95
crates/manager-core/src/gc.rs
Normal file
@@ -0,0 +1,95 @@
|
||||
//! Weekly retention sweepers for `dead_letters` + `abandoned_executions`.
|
||||
//!
|
||||
//! Both use the `FOR UPDATE SKIP LOCKED` claim pattern so concurrent
|
||||
//! sweepers (cluster mode v1.3+) don't fight each other. Defaults
|
||||
//! match design notes §3 / §4: 30 days for DL, 7 days for abandoned.
|
||||
//! Both env-overridable via `PICLOUD_DEAD_LETTER_RETENTION_DAYS` and
|
||||
//! `PICLOUD_ABANDONED_EXECUTIONS_RETENTION_DAYS` (loaded by
|
||||
//! `TriggerConfig::from_env`).
|
||||
//!
|
||||
//! Spawned from `build_app` alongside `spawn_session_pruner`.
|
||||
|
||||
use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
|
||||
use chrono::Utc;
|
||||
|
||||
use crate::abandoned_repo::AbandonedRepo;
|
||||
use crate::dead_letter_repo::DeadLetterRepo;
|
||||
|
||||
/// Weekly sweep cadence — matches `spawn_session_pruner` shape.
|
||||
const SWEEP_INTERVAL: Duration = Duration::from_secs(7 * 24 * 60 * 60);
|
||||
|
||||
/// Per-tick batch cap so we don't try to delete millions of rows in
|
||||
/// one transaction. The loop keeps deleting batches until a tick
|
||||
/// returns 0 rows affected.
|
||||
const SWEEP_BATCH: i64 = 5_000;
|
||||
|
||||
pub fn spawn_dead_letter_gc(repo: Arc<dyn DeadLetterRepo>, retention_days: u32) {
|
||||
tokio::spawn(async move {
|
||||
let mut ticker = tokio::time::interval(SWEEP_INTERVAL);
|
||||
// Skip the immediate first fire — don't sweep at process start.
|
||||
ticker.tick().await;
|
||||
loop {
|
||||
ticker.tick().await;
|
||||
sweep_dead_letters(&*repo, retention_days).await;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
pub fn spawn_abandoned_gc(repo: Arc<dyn AbandonedRepo>, retention_days: u32) {
|
||||
tokio::spawn(async move {
|
||||
let mut ticker = tokio::time::interval(SWEEP_INTERVAL);
|
||||
ticker.tick().await;
|
||||
loop {
|
||||
ticker.tick().await;
|
||||
sweep_abandoned(&*repo, retention_days).await;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
async fn sweep_dead_letters(repo: &dyn DeadLetterRepo, retention_days: u32) {
|
||||
let cutoff = Utc::now() - chrono::Duration::days(i64::from(retention_days));
|
||||
let mut total: u64 = 0;
|
||||
loop {
|
||||
match repo.gc(cutoff, SWEEP_BATCH).await {
|
||||
Ok(0) => break,
|
||||
Ok(n) => {
|
||||
total += n;
|
||||
if n < SWEEP_BATCH as u64 {
|
||||
break;
|
||||
}
|
||||
}
|
||||
Err(e) => {
|
||||
tracing::warn!(?e, "dead_letters GC sweep errored");
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
if total > 0 {
|
||||
tracing::info!(swept = total, "dead_letters GC swept");
|
||||
}
|
||||
}
|
||||
|
||||
async fn sweep_abandoned(repo: &dyn AbandonedRepo, retention_days: u32) {
|
||||
let cutoff = Utc::now() - chrono::Duration::days(i64::from(retention_days));
|
||||
let mut total: u64 = 0;
|
||||
loop {
|
||||
match repo.gc(cutoff, SWEEP_BATCH).await {
|
||||
Ok(0) => break,
|
||||
Ok(n) => {
|
||||
total += n;
|
||||
if n < SWEEP_BATCH as u64 {
|
||||
break;
|
||||
}
|
||||
}
|
||||
Err(e) => {
|
||||
tracing::warn!(?e, "abandoned_executions GC sweep errored");
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
if total > 0 {
|
||||
tracing::info!(swept = total, "abandoned_executions GC swept");
|
||||
}
|
||||
}
|
||||
223
crates/manager-core/src/kv_repo.rs
Normal file
223
crates/manager-core/src/kv_repo.rs
Normal file
@@ -0,0 +1,223 @@
|
||||
//! Low-level Postgres CRUD over `kv_entries`. Stays storage-only;
|
||||
//! authorization, event emission, and empty-collection validation live
|
||||
//! one layer up in `KvServiceImpl`.
|
||||
|
||||
use async_trait::async_trait;
|
||||
use base64::engine::general_purpose::URL_SAFE_NO_PAD;
|
||||
use base64::Engine as _;
|
||||
use picloud_shared::{AppId, KvListPage};
|
||||
use sqlx::PgPool;
|
||||
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub enum KvRepoError {
|
||||
#[error("database error: {0}")]
|
||||
Db(#[from] sqlx::Error),
|
||||
|
||||
#[error("invalid pagination cursor")]
|
||||
InvalidCursor,
|
||||
}
|
||||
|
||||
/// Repo surface. The trait is exposed so tests can substitute an
|
||||
/// in-memory backing without spinning up Postgres.
|
||||
#[async_trait]
|
||||
pub trait KvRepo: Send + Sync {
|
||||
async fn get(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
collection: &str,
|
||||
key: &str,
|
||||
) -> Result<Option<serde_json::Value>, KvRepoError>;
|
||||
|
||||
/// Upserts the row. Returns the previous value (if any) so callers
|
||||
/// can determine whether this was an `insert` or an `update` for
|
||||
/// the emitted `ServiceEvent`.
|
||||
async fn set(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
collection: &str,
|
||||
key: &str,
|
||||
value: serde_json::Value,
|
||||
) -> Result<Option<serde_json::Value>, KvRepoError>;
|
||||
|
||||
/// Returns the deleted value if present, `None` if the row didn't
|
||||
/// exist. The caller turns the `bool was-present` part into the
|
||||
/// SDK's return value; the `Option<value>` part feeds the
|
||||
/// `old_payload` field of the emitted delete event.
|
||||
async fn delete(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
collection: &str,
|
||||
key: &str,
|
||||
) -> Result<Option<serde_json::Value>, KvRepoError>;
|
||||
|
||||
async fn has(&self, app_id: AppId, collection: &str, key: &str) -> Result<bool, KvRepoError>;
|
||||
|
||||
async fn list(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
collection: &str,
|
||||
cursor: Option<&str>,
|
||||
limit: u32,
|
||||
) -> Result<KvListPage, KvRepoError>;
|
||||
}
|
||||
|
||||
pub struct PostgresKvRepo {
|
||||
pool: PgPool,
|
||||
}
|
||||
|
||||
impl PostgresKvRepo {
|
||||
#[must_use]
|
||||
pub fn new(pool: PgPool) -> Self {
|
||||
Self { pool }
|
||||
}
|
||||
}
|
||||
|
||||
/// Hard ceiling on `list` page size — scripts that pass anything larger
|
||||
/// silently get clamped to this. Cursor-style pagination keeps a single
|
||||
/// request bounded; clients fetch the next page via the returned cursor.
|
||||
const KV_LIST_MAX_LIMIT: u32 = 1_000;
|
||||
const KV_LIST_DEFAULT_LIMIT: u32 = 100;
|
||||
|
||||
#[async_trait]
|
||||
impl KvRepo for PostgresKvRepo {
|
||||
async fn get(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
collection: &str,
|
||||
key: &str,
|
||||
) -> Result<Option<serde_json::Value>, KvRepoError> {
|
||||
let row: Option<(serde_json::Value,)> = sqlx::query_as(
|
||||
"SELECT value FROM kv_entries \
|
||||
WHERE app_id = $1 AND collection = $2 AND key = $3",
|
||||
)
|
||||
.bind(app_id.into_inner())
|
||||
.bind(collection)
|
||||
.bind(key)
|
||||
.fetch_optional(&self.pool)
|
||||
.await?;
|
||||
Ok(row.map(|(v,)| v))
|
||||
}
|
||||
|
||||
async fn set(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
collection: &str,
|
||||
key: &str,
|
||||
value: serde_json::Value,
|
||||
) -> Result<Option<serde_json::Value>, KvRepoError> {
|
||||
// `RETURNING` after `ON CONFLICT DO UPDATE` exposes the old
|
||||
// value via the `xmax`/old-row trick: capture the prior value
|
||||
// with a CTE so callers know whether this was insert vs update.
|
||||
let row: Option<(Option<serde_json::Value>,)> = sqlx::query_as(
|
||||
"WITH prev AS (\
|
||||
SELECT value FROM kv_entries \
|
||||
WHERE app_id = $1 AND collection = $2 AND key = $3\
|
||||
), \
|
||||
upserted AS (\
|
||||
INSERT INTO kv_entries (app_id, collection, key, value) \
|
||||
VALUES ($1, $2, $3, $4) \
|
||||
ON CONFLICT (app_id, collection, key) DO UPDATE \
|
||||
SET value = EXCLUDED.value, updated_at = NOW() \
|
||||
RETURNING 1\
|
||||
) \
|
||||
SELECT (SELECT value FROM prev) FROM upserted",
|
||||
)
|
||||
.bind(app_id.into_inner())
|
||||
.bind(collection)
|
||||
.bind(key)
|
||||
.bind(value)
|
||||
.fetch_optional(&self.pool)
|
||||
.await?;
|
||||
Ok(row.and_then(|(v,)| v))
|
||||
}
|
||||
|
||||
async fn delete(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
collection: &str,
|
||||
key: &str,
|
||||
) -> Result<Option<serde_json::Value>, KvRepoError> {
|
||||
let row: Option<(serde_json::Value,)> = sqlx::query_as(
|
||||
"DELETE FROM kv_entries \
|
||||
WHERE app_id = $1 AND collection = $2 AND key = $3 \
|
||||
RETURNING value",
|
||||
)
|
||||
.bind(app_id.into_inner())
|
||||
.bind(collection)
|
||||
.bind(key)
|
||||
.fetch_optional(&self.pool)
|
||||
.await?;
|
||||
Ok(row.map(|(v,)| v))
|
||||
}
|
||||
|
||||
async fn has(&self, app_id: AppId, collection: &str, key: &str) -> Result<bool, KvRepoError> {
|
||||
let row: Option<(i64,)> = sqlx::query_as(
|
||||
"SELECT 1 FROM kv_entries \
|
||||
WHERE app_id = $1 AND collection = $2 AND key = $3",
|
||||
)
|
||||
.bind(app_id.into_inner())
|
||||
.bind(collection)
|
||||
.bind(key)
|
||||
.fetch_optional(&self.pool)
|
||||
.await?;
|
||||
Ok(row.is_some())
|
||||
}
|
||||
|
||||
async fn list(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
collection: &str,
|
||||
cursor: Option<&str>,
|
||||
limit: u32,
|
||||
) -> Result<KvListPage, KvRepoError> {
|
||||
let limit = if limit == 0 {
|
||||
KV_LIST_DEFAULT_LIMIT
|
||||
} else {
|
||||
limit.min(KV_LIST_MAX_LIMIT)
|
||||
};
|
||||
|
||||
let last_key = match cursor {
|
||||
Some(c) => Some(decode_cursor(c)?),
|
||||
None => None,
|
||||
};
|
||||
|
||||
// Keyset pagination: rows beyond `last_key` ordered by key.
|
||||
// `+1` to detect a "more pages" condition without a separate
|
||||
// COUNT query.
|
||||
let take = i64::from(limit) + 1;
|
||||
let rows: Vec<(String,)> = sqlx::query_as(
|
||||
"SELECT key FROM kv_entries \
|
||||
WHERE app_id = $1 AND collection = $2 \
|
||||
AND ($3::text IS NULL OR key > $3) \
|
||||
ORDER BY key ASC \
|
||||
LIMIT $4",
|
||||
)
|
||||
.bind(app_id.into_inner())
|
||||
.bind(collection)
|
||||
.bind(last_key.as_deref())
|
||||
.bind(take)
|
||||
.fetch_all(&self.pool)
|
||||
.await?;
|
||||
|
||||
let mut keys: Vec<String> = rows.into_iter().map(|(k,)| k).collect();
|
||||
let next_cursor = if keys.len() > limit as usize {
|
||||
keys.truncate(limit as usize);
|
||||
keys.last().map(|k| encode_cursor(k))
|
||||
} else {
|
||||
None
|
||||
};
|
||||
|
||||
Ok(KvListPage { keys, next_cursor })
|
||||
}
|
||||
}
|
||||
|
||||
fn encode_cursor(last_key: &str) -> String {
|
||||
URL_SAFE_NO_PAD.encode(last_key.as_bytes())
|
||||
}
|
||||
|
||||
fn decode_cursor(cursor: &str) -> Result<String, KvRepoError> {
|
||||
let bytes = URL_SAFE_NO_PAD
|
||||
.decode(cursor)
|
||||
.map_err(|_| KvRepoError::InvalidCursor)?;
|
||||
String::from_utf8(bytes).map_err(|_| KvRepoError::InvalidCursor)
|
||||
}
|
||||
525
crates/manager-core/src/kv_service.rs
Normal file
525
crates/manager-core/src/kv_service.rs
Normal file
@@ -0,0 +1,525 @@
|
||||
//! `KvServiceImpl` — wires the `KvRepo` underneath the
|
||||
//! `picloud_shared::KvService` trait that scripts see via the Rhai
|
||||
//! bridge.
|
||||
//!
|
||||
//! Layers added here (vs the raw repo):
|
||||
//!
|
||||
//! 1. Empty-collection rejection at the SDK boundary
|
||||
//! (`docs/sdk-shape.md`).
|
||||
//! 2. **Script-as-gate authz**: when `cx.principal.is_some()` we run
|
||||
//! `authz::require(...)`; when it's `None` (public unauthenticated
|
||||
//! HTTP — the common case for public routes) we skip the check.
|
||||
//! Cross-app isolation isn't affected — every query is keyed by
|
||||
//! `cx.app_id`, never an argument.
|
||||
//! 3. `ServiceEvent` emission after each mutation (`insert` / `update`
|
||||
//! / `delete`). v1.1.0 ships a `NoopEventEmitter` so this is a
|
||||
//! no-op until the outbox emitter lands later in v1.1.1.
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use async_trait::async_trait;
|
||||
use picloud_shared::{
|
||||
KvError, KvListPage, KvService, SdkCallCx, ServiceEvent, ServiceEventEmitter,
|
||||
};
|
||||
|
||||
use crate::authz::{self, AuthzRepo, Capability};
|
||||
use crate::kv_repo::{KvRepo, KvRepoError};
|
||||
|
||||
pub struct KvServiceImpl {
|
||||
repo: Arc<dyn KvRepo>,
|
||||
authz: Arc<dyn AuthzRepo>,
|
||||
events: Arc<dyn ServiceEventEmitter>,
|
||||
}
|
||||
|
||||
impl KvServiceImpl {
|
||||
#[must_use]
|
||||
pub fn new(
|
||||
repo: Arc<dyn KvRepo>,
|
||||
authz: Arc<dyn AuthzRepo>,
|
||||
events: Arc<dyn ServiceEventEmitter>,
|
||||
) -> Self {
|
||||
Self {
|
||||
repo,
|
||||
authz,
|
||||
events,
|
||||
}
|
||||
}
|
||||
|
||||
async fn check_read(&self, cx: &SdkCallCx) -> Result<(), KvError> {
|
||||
if let Some(ref principal) = cx.principal {
|
||||
authz::require(&*self.authz, principal, Capability::AppKvRead(cx.app_id))
|
||||
.await
|
||||
.map_err(|_| KvError::Forbidden)?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn check_write(&self, cx: &SdkCallCx) -> Result<(), KvError> {
|
||||
if let Some(ref principal) = cx.principal {
|
||||
authz::require(&*self.authz, principal, Capability::AppKvWrite(cx.app_id))
|
||||
.await
|
||||
.map_err(|_| KvError::Forbidden)?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
fn validate_collection(collection: &str) -> Result<(), KvError> {
|
||||
if collection.is_empty() {
|
||||
return Err(KvError::InvalidCollection);
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
impl From<KvRepoError> for KvError {
|
||||
fn from(e: KvRepoError) -> Self {
|
||||
Self::Backend(e.to_string())
|
||||
}
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl KvService for KvServiceImpl {
|
||||
async fn get(
|
||||
&self,
|
||||
cx: &SdkCallCx,
|
||||
collection: &str,
|
||||
key: &str,
|
||||
) -> Result<Option<serde_json::Value>, KvError> {
|
||||
validate_collection(collection)?;
|
||||
self.check_read(cx).await?;
|
||||
Ok(self.repo.get(cx.app_id, collection, key).await?)
|
||||
}
|
||||
|
||||
async fn set(
|
||||
&self,
|
||||
cx: &SdkCallCx,
|
||||
collection: &str,
|
||||
key: &str,
|
||||
value: serde_json::Value,
|
||||
) -> Result<(), KvError> {
|
||||
validate_collection(collection)?;
|
||||
self.check_write(cx).await?;
|
||||
let previous = self
|
||||
.repo
|
||||
.set(cx.app_id, collection, key, value.clone())
|
||||
.await?;
|
||||
let op = if previous.is_some() {
|
||||
"update"
|
||||
} else {
|
||||
"insert"
|
||||
};
|
||||
// Emit unconditionally; the noop emitter drops it, the outbox
|
||||
// emitter persists it. Best-effort: a failed emit is logged
|
||||
// but does not roll back the write.
|
||||
if let Err(e) = self
|
||||
.events
|
||||
.emit(
|
||||
cx,
|
||||
ServiceEvent {
|
||||
source: "kv",
|
||||
op,
|
||||
collection: Some(collection.to_string()),
|
||||
key: Some(key.to_string()),
|
||||
payload: Some(value),
|
||||
old_payload: previous,
|
||||
},
|
||||
)
|
||||
.await
|
||||
{
|
||||
tracing::warn!(error = %e, source = "kv", op, "event emit failed");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn delete(&self, cx: &SdkCallCx, collection: &str, key: &str) -> Result<bool, KvError> {
|
||||
validate_collection(collection)?;
|
||||
self.check_write(cx).await?;
|
||||
let previous = self.repo.delete(cx.app_id, collection, key).await?;
|
||||
let was_present = previous.is_some();
|
||||
if was_present {
|
||||
if let Err(e) = self
|
||||
.events
|
||||
.emit(
|
||||
cx,
|
||||
ServiceEvent {
|
||||
source: "kv",
|
||||
op: "delete",
|
||||
collection: Some(collection.to_string()),
|
||||
key: Some(key.to_string()),
|
||||
payload: None,
|
||||
old_payload: previous,
|
||||
},
|
||||
)
|
||||
.await
|
||||
{
|
||||
tracing::warn!(error = %e, source = "kv", op = "delete", "event emit failed");
|
||||
}
|
||||
}
|
||||
Ok(was_present)
|
||||
}
|
||||
|
||||
async fn has(&self, cx: &SdkCallCx, collection: &str, key: &str) -> Result<bool, KvError> {
|
||||
validate_collection(collection)?;
|
||||
self.check_read(cx).await?;
|
||||
Ok(self.repo.has(cx.app_id, collection, key).await?)
|
||||
}
|
||||
|
||||
async fn list(
|
||||
&self,
|
||||
cx: &SdkCallCx,
|
||||
collection: &str,
|
||||
cursor: Option<&str>,
|
||||
limit: u32,
|
||||
) -> Result<KvListPage, KvError> {
|
||||
validate_collection(collection)?;
|
||||
self.check_read(cx).await?;
|
||||
Ok(self.repo.list(cx.app_id, collection, cursor, limit).await?)
|
||||
}
|
||||
}
|
||||
|
||||
// ----------------------------------------------------------------------------
|
||||
// Tests — in-memory KvRepo so unit tests don't need Postgres.
|
||||
// ----------------------------------------------------------------------------
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::authz::{AuthzError, AuthzRepo};
|
||||
use async_trait::async_trait;
|
||||
use picloud_shared::{
|
||||
AdminUserId, AppId, AppRole, ExecutionId, InstanceRole, NoopEventEmitter, Principal,
|
||||
RequestId, UserId,
|
||||
};
|
||||
use std::collections::{BTreeMap, HashMap};
|
||||
use tokio::sync::Mutex;
|
||||
|
||||
#[derive(Default)]
|
||||
struct InMemoryKvRepo {
|
||||
data: Mutex<BTreeMap<(AppId, String, String), serde_json::Value>>,
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl KvRepo for InMemoryKvRepo {
|
||||
async fn get(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
collection: &str,
|
||||
key: &str,
|
||||
) -> Result<Option<serde_json::Value>, KvRepoError> {
|
||||
Ok(self
|
||||
.data
|
||||
.lock()
|
||||
.await
|
||||
.get(&(app_id, collection.to_string(), key.to_string()))
|
||||
.cloned())
|
||||
}
|
||||
|
||||
async fn set(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
collection: &str,
|
||||
key: &str,
|
||||
value: serde_json::Value,
|
||||
) -> Result<Option<serde_json::Value>, KvRepoError> {
|
||||
Ok(self
|
||||
.data
|
||||
.lock()
|
||||
.await
|
||||
.insert((app_id, collection.to_string(), key.to_string()), value))
|
||||
}
|
||||
|
||||
async fn delete(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
collection: &str,
|
||||
key: &str,
|
||||
) -> Result<Option<serde_json::Value>, KvRepoError> {
|
||||
Ok(self
|
||||
.data
|
||||
.lock()
|
||||
.await
|
||||
.remove(&(app_id, collection.to_string(), key.to_string())))
|
||||
}
|
||||
|
||||
async fn has(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
collection: &str,
|
||||
key: &str,
|
||||
) -> Result<bool, KvRepoError> {
|
||||
Ok(self.data.lock().await.contains_key(&(
|
||||
app_id,
|
||||
collection.to_string(),
|
||||
key.to_string(),
|
||||
)))
|
||||
}
|
||||
|
||||
async fn list(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
collection: &str,
|
||||
cursor: Option<&str>,
|
||||
limit: u32,
|
||||
) -> Result<KvListPage, KvRepoError> {
|
||||
let data = self.data.lock().await;
|
||||
let last_key = cursor.map(std::string::ToString::to_string);
|
||||
let mut keys: Vec<String> = data
|
||||
.iter()
|
||||
.filter(|((a, c, _), _)| *a == app_id && c == collection)
|
||||
.map(|((_, _, k), _)| k.clone())
|
||||
.filter(|k| last_key.as_ref().is_none_or(|lk| k > lk))
|
||||
.collect();
|
||||
keys.sort();
|
||||
let take = (limit as usize).max(1);
|
||||
let next_cursor = if keys.len() > take {
|
||||
keys.truncate(take);
|
||||
keys.last().cloned()
|
||||
} else {
|
||||
None
|
||||
};
|
||||
Ok(KvListPage { keys, next_cursor })
|
||||
}
|
||||
}
|
||||
|
||||
/// AuthzRepo that always denies — used to confirm the service
|
||||
/// short-circuits on cx.principal.is_some() with a denial, and
|
||||
/// that it does NOT call into authz when cx.principal is None.
|
||||
#[derive(Default)]
|
||||
struct DenyingAuthzRepo;
|
||||
|
||||
#[async_trait]
|
||||
impl AuthzRepo for DenyingAuthzRepo {
|
||||
async fn membership(
|
||||
&self,
|
||||
_user_id: UserId,
|
||||
_app_id: AppId,
|
||||
) -> Result<Option<AppRole>, AuthzError> {
|
||||
Ok(None)
|
||||
}
|
||||
}
|
||||
|
||||
fn anon_cx(app_id: AppId) -> SdkCallCx {
|
||||
SdkCallCx {
|
||||
app_id,
|
||||
principal: None,
|
||||
execution_id: ExecutionId::new(),
|
||||
request_id: RequestId::new(),
|
||||
trigger_depth: 0,
|
||||
root_execution_id: ExecutionId::new(),
|
||||
is_dead_letter_handler: false,
|
||||
event: None,
|
||||
}
|
||||
}
|
||||
|
||||
fn owner_cx(app_id: AppId) -> SdkCallCx {
|
||||
SdkCallCx {
|
||||
app_id,
|
||||
principal: Some(Principal {
|
||||
user_id: AdminUserId::new(),
|
||||
instance_role: InstanceRole::Owner,
|
||||
scopes: None,
|
||||
app_binding: None,
|
||||
}),
|
||||
execution_id: ExecutionId::new(),
|
||||
request_id: RequestId::new(),
|
||||
trigger_depth: 0,
|
||||
root_execution_id: ExecutionId::new(),
|
||||
is_dead_letter_handler: false,
|
||||
event: None,
|
||||
}
|
||||
}
|
||||
|
||||
fn member_no_role_cx(app_id: AppId) -> SdkCallCx {
|
||||
SdkCallCx {
|
||||
app_id,
|
||||
principal: Some(Principal {
|
||||
user_id: AdminUserId::new(),
|
||||
instance_role: InstanceRole::Member,
|
||||
scopes: None,
|
||||
app_binding: None,
|
||||
}),
|
||||
execution_id: ExecutionId::new(),
|
||||
request_id: RequestId::new(),
|
||||
trigger_depth: 0,
|
||||
root_execution_id: ExecutionId::new(),
|
||||
is_dead_letter_handler: false,
|
||||
event: None,
|
||||
}
|
||||
}
|
||||
|
||||
fn svc() -> KvServiceImpl {
|
||||
KvServiceImpl::new(
|
||||
Arc::new(InMemoryKvRepo::default()),
|
||||
Arc::new(DenyingAuthzRepo),
|
||||
Arc::new(NoopEventEmitter),
|
||||
)
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn set_then_get_round_trips() {
|
||||
let kv = svc();
|
||||
let cx = anon_cx(AppId::new());
|
||||
kv.set(&cx, "widgets", "k1", serde_json::json!({"n": 1}))
|
||||
.await
|
||||
.unwrap();
|
||||
let v = kv.get(&cx, "widgets", "k1").await.unwrap();
|
||||
assert_eq!(v, Some(serde_json::json!({"n": 1})));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn get_missing_returns_none() {
|
||||
let kv = svc();
|
||||
let cx = anon_cx(AppId::new());
|
||||
let v = kv.get(&cx, "widgets", "nope").await.unwrap();
|
||||
assert_eq!(v, None);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn has_returns_bool() {
|
||||
let kv = svc();
|
||||
let cx = anon_cx(AppId::new());
|
||||
assert!(!kv.has(&cx, "widgets", "k1").await.unwrap());
|
||||
kv.set(&cx, "widgets", "k1", serde_json::json!(true))
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(kv.has(&cx, "widgets", "k1").await.unwrap());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn delete_returns_was_present() {
|
||||
let kv = svc();
|
||||
let cx = anon_cx(AppId::new());
|
||||
assert!(!kv.delete(&cx, "widgets", "missing").await.unwrap());
|
||||
kv.set(&cx, "widgets", "k1", serde_json::json!(1))
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(kv.delete(&cx, "widgets", "k1").await.unwrap());
|
||||
// Idempotent — second delete returns false.
|
||||
assert!(!kv.delete(&cx, "widgets", "k1").await.unwrap());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn empty_collection_rejected() {
|
||||
let kv = svc();
|
||||
let cx = anon_cx(AppId::new());
|
||||
let err = kv.get(&cx, "", "k1").await.unwrap_err();
|
||||
assert!(matches!(err, KvError::InvalidCollection));
|
||||
}
|
||||
|
||||
/// Load-bearing: a script with `cx.app_id = A` must NOT see
|
||||
/// entries inserted under `cx.app_id = B`. This is the cross-app
|
||||
/// isolation boundary; getting this wrong is a security
|
||||
/// vulnerability.
|
||||
#[tokio::test]
|
||||
async fn cross_app_isolation_via_cx_app_id() {
|
||||
let kv = svc();
|
||||
let app_a = AppId::new();
|
||||
let app_b = AppId::new();
|
||||
let cx_a = anon_cx(app_a);
|
||||
let cx_b = anon_cx(app_b);
|
||||
|
||||
kv.set(&cx_a, "shared", "k", serde_json::json!("from-a"))
|
||||
.await
|
||||
.unwrap();
|
||||
kv.set(&cx_b, "shared", "k", serde_json::json!("from-b"))
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
assert_eq!(
|
||||
kv.get(&cx_a, "shared", "k").await.unwrap(),
|
||||
Some(serde_json::json!("from-a"))
|
||||
);
|
||||
assert_eq!(
|
||||
kv.get(&cx_b, "shared", "k").await.unwrap(),
|
||||
Some(serde_json::json!("from-b"))
|
||||
);
|
||||
}
|
||||
|
||||
/// Script-as-gate: an `anon_cx` (principal = None) skips the
|
||||
/// capability check entirely. Even with a denying authz repo,
|
||||
/// the write succeeds.
|
||||
#[tokio::test]
|
||||
async fn anonymous_cx_skips_authz() {
|
||||
let kv = svc();
|
||||
let cx = anon_cx(AppId::new());
|
||||
kv.set(&cx, "widgets", "k", serde_json::json!(1))
|
||||
.await
|
||||
.unwrap();
|
||||
// No panic, no Forbidden.
|
||||
}
|
||||
|
||||
/// Authenticated principal with no role on the app: the
|
||||
/// `DenyingAuthzRepo` returns no membership, so the capability
|
||||
/// check denies. Set must surface KvError::Forbidden.
|
||||
#[tokio::test]
|
||||
async fn authed_cx_with_no_role_is_forbidden() {
|
||||
let kv = svc();
|
||||
let cx = member_no_role_cx(AppId::new());
|
||||
let err = kv
|
||||
.set(&cx, "widgets", "k", serde_json::json!(1))
|
||||
.await
|
||||
.unwrap_err();
|
||||
assert!(matches!(err, KvError::Forbidden));
|
||||
}
|
||||
|
||||
/// Owner principal: instance-role grants kick in inside `authz::can`
|
||||
/// (Owner -> implicit AppAdmin which covers KvWrite).
|
||||
#[tokio::test]
|
||||
async fn owner_principal_can_write() {
|
||||
let kv = svc();
|
||||
let cx = owner_cx(AppId::new());
|
||||
kv.set(&cx, "widgets", "k", serde_json::json!(1))
|
||||
.await
|
||||
.unwrap();
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn list_cursor_pagination() {
|
||||
let kv = svc();
|
||||
let cx = anon_cx(AppId::new());
|
||||
for i in 0..5 {
|
||||
kv.set(
|
||||
&cx,
|
||||
"widgets",
|
||||
&format!("k{i:02}"),
|
||||
serde_json::json!({"i": i}),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
}
|
||||
// page 1 — 2 keys
|
||||
let p1 = kv.list(&cx, "widgets", None, 2).await.unwrap();
|
||||
assert_eq!(p1.keys, vec!["k00".to_string(), "k01".to_string()]);
|
||||
assert!(p1.next_cursor.is_some());
|
||||
// page 2 — 2 keys
|
||||
let p2 = kv
|
||||
.list(&cx, "widgets", p1.next_cursor.as_deref(), 2)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(p2.keys, vec!["k02".to_string(), "k03".to_string()]);
|
||||
// final page — 1 key, no cursor
|
||||
let p3 = kv
|
||||
.list(&cx, "widgets", p2.next_cursor.as_deref(), 2)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(p3.keys, vec!["k04".to_string()]);
|
||||
assert!(p3.next_cursor.is_none());
|
||||
}
|
||||
|
||||
/// Pinning the v1.1.0 contract: services hold the emitter as a
|
||||
/// dyn Arc and call `emit().await` unconditionally. This test
|
||||
/// proves the call site doesn't blow up against the noop impl —
|
||||
/// the outbox emitter (v1.1.1) drops in transparently.
|
||||
#[tokio::test]
|
||||
async fn noop_emitter_does_not_block_mutations() {
|
||||
let kv = svc();
|
||||
let cx = anon_cx(AppId::new());
|
||||
kv.set(&cx, "widgets", "k", serde_json::json!(1))
|
||||
.await
|
||||
.unwrap();
|
||||
kv.delete(&cx, "widgets", "k").await.unwrap();
|
||||
// Reaching here means emit() returned Ok and didn't panic.
|
||||
// Suppress unused-import warning when run alone:
|
||||
let _ = HashMap::<String, String>::new();
|
||||
}
|
||||
}
|
||||
@@ -4,6 +4,7 @@
|
||||
//! the same DB for now; once we add caching and per-node ingress, the
|
||||
//! manager will publish change events.
|
||||
|
||||
pub mod abandoned_repo;
|
||||
pub mod admin_session_repo;
|
||||
pub mod admin_user_repo;
|
||||
pub mod admin_users_api;
|
||||
@@ -21,14 +22,30 @@ pub mod auth_api;
|
||||
pub mod auth_bootstrap;
|
||||
pub mod auth_middleware;
|
||||
pub mod authz;
|
||||
pub mod dead_letter_repo;
|
||||
pub mod dead_letter_service;
|
||||
pub mod dead_letters_api;
|
||||
pub mod dispatcher;
|
||||
pub mod gc;
|
||||
pub mod kv_repo;
|
||||
pub mod kv_service;
|
||||
pub mod log_sink;
|
||||
pub mod migrations;
|
||||
pub mod outbox_event_emitter;
|
||||
pub mod outbox_repo;
|
||||
pub mod principal_resolver;
|
||||
pub mod repo;
|
||||
pub mod route_admin;
|
||||
pub mod route_repo;
|
||||
pub mod sandbox;
|
||||
pub mod scheduler;
|
||||
pub mod trigger_config;
|
||||
pub mod trigger_repo;
|
||||
pub mod triggers_api;
|
||||
|
||||
pub use abandoned_repo::{
|
||||
AbandonedRepo, AbandonedRepoError, NewAbandonedExecution, PostgresAbandonedRepo,
|
||||
};
|
||||
pub use admin_session_repo::{
|
||||
AdminSessionLookup, AdminSessionRepository, AdminSessionRepositoryError,
|
||||
PostgresAdminSessionRepository,
|
||||
@@ -59,11 +76,25 @@ pub use auth_bootstrap::{
|
||||
};
|
||||
#[allow(deprecated)]
|
||||
pub use auth_middleware::{
|
||||
require_admin, require_authenticated, AuthState, AuthedAdmin, API_KEY_PREFIX,
|
||||
API_KEY_PREFIX_LEN, SESSION_COOKIE,
|
||||
attach_principal_if_present, require_admin, require_authenticated, AuthState, AuthedAdmin,
|
||||
API_KEY_PREFIX, API_KEY_PREFIX_LEN, SESSION_COOKIE,
|
||||
};
|
||||
pub use authz::{can, require, AuthzDenied, AuthzError, AuthzRepo, Capability, Decision};
|
||||
pub use dead_letter_repo::{
|
||||
DeadLetterRepo, DeadLetterRepoError, DeadLetterRow, NewDeadLetter, PostgresDeadLetterRepo,
|
||||
};
|
||||
pub use dead_letter_service::PostgresDeadLetterService;
|
||||
pub use dead_letters_api::{dead_letters_router, DeadLettersApiError, DeadLettersState};
|
||||
pub use dispatcher::{compute_backoff, Dispatcher, DispatcherError};
|
||||
pub use gc::{spawn_abandoned_gc, spawn_dead_letter_gc};
|
||||
pub use kv_repo::{KvRepo, KvRepoError, PostgresKvRepo};
|
||||
pub use kv_service::KvServiceImpl;
|
||||
pub use log_sink::PostgresExecutionLogSink;
|
||||
pub use outbox_event_emitter::OutboxEventEmitter;
|
||||
pub use outbox_repo::{
|
||||
NewOutboxRow, OutboxRepo, OutboxRepoError, OutboxRow, OutboxSourceKind, PostgresOutboxRepo,
|
||||
};
|
||||
pub use principal_resolver::{AdminPrincipalResolver, PrincipalResolver, PrincipalResolverError};
|
||||
pub use repo::{
|
||||
ExecutionLogRepository, NewScript, PostgresExecutionLogRepository, PostgresScriptRepository,
|
||||
RepoResolver, ScriptPatch, ScriptRepository, ScriptRepositoryError,
|
||||
@@ -71,3 +102,10 @@ pub use repo::{
|
||||
pub use route_admin::{compile_routes, route_admin_router, RouteAdminState};
|
||||
pub use route_repo::{NewRoute, PostgresRouteRepository, RouteRepository};
|
||||
pub use sandbox::{CeilingError, SandboxCeiling};
|
||||
pub use trigger_config::{BackoffShape, TriggerConfig};
|
||||
pub use trigger_repo::{
|
||||
collection_matches, CreateDeadLetterTrigger, CreateKvTrigger, DeadLetterTriggerMatch,
|
||||
KvTriggerMatch, PostgresTriggerRepo, Trigger, TriggerDetails, TriggerDispatchMode, TriggerKind,
|
||||
TriggerRepo, TriggerRepoError,
|
||||
};
|
||||
pub use triggers_api::{triggers_router, TriggersApiError, TriggersState};
|
||||
|
||||
103
crates/manager-core/src/outbox_event_emitter.rs
Normal file
103
crates/manager-core/src/outbox_event_emitter.rs
Normal file
@@ -0,0 +1,103 @@
|
||||
//! `OutboxEventEmitter` — the real `ServiceEventEmitter` that replaces
|
||||
//! v1.1.0's `NoopEventEmitter` once the triggers framework lands.
|
||||
//!
|
||||
//! On each `emit` (a KV mutation, future doc/file/pubsub event, etc.):
|
||||
//! 1. Look up matching triggers for the event's (app_id, source, op,
|
||||
//! collection) tuple via `TriggerRepo::list_matching_*`.
|
||||
//! 2. For each match, write one outbox row carrying the event payload
|
||||
//! serialized as a `TriggerEvent`.
|
||||
//!
|
||||
//! Defaults applied at write time so `OutboxRow.payload` carries
|
||||
//! everything the dispatcher needs to reconstruct the executor
|
||||
//! invocation without joining back to the trigger row.
|
||||
//!
|
||||
//! Non-KV `ServiceEvent` sources are silently dropped in v1.1.1 — the
|
||||
//! dispatcher only knows how to fire KV triggers this release. Future
|
||||
//! sources (docs/files/pubsub) add their own dispatch arm.
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use async_trait::async_trait;
|
||||
use picloud_shared::{
|
||||
EmitError, KvEventOp, SdkCallCx, ServiceEvent, ServiceEventEmitter, TriggerEvent,
|
||||
};
|
||||
|
||||
use crate::outbox_repo::{NewOutboxRow, OutboxRepo, OutboxSourceKind};
|
||||
use crate::trigger_repo::TriggerRepo;
|
||||
|
||||
pub struct OutboxEventEmitter {
|
||||
triggers: Arc<dyn TriggerRepo>,
|
||||
outbox: Arc<dyn OutboxRepo>,
|
||||
}
|
||||
|
||||
impl OutboxEventEmitter {
|
||||
#[must_use]
|
||||
pub fn new(triggers: Arc<dyn TriggerRepo>, outbox: Arc<dyn OutboxRepo>) -> Self {
|
||||
Self { triggers, outbox }
|
||||
}
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl ServiceEventEmitter for OutboxEventEmitter {
|
||||
async fn emit(&self, cx: &SdkCallCx, event: ServiceEvent) -> Result<(), EmitError> {
|
||||
match event.source {
|
||||
"kv" => self.emit_kv(cx, event).await,
|
||||
// Future sources land here. For now, silently drop — the
|
||||
// SDK calls `events.emit(...)` unconditionally for forward
|
||||
// compat, so swallowing without an error is correct.
|
||||
_ => Ok(()),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl OutboxEventEmitter {
|
||||
async fn emit_kv(&self, cx: &SdkCallCx, event: ServiceEvent) -> Result<(), EmitError> {
|
||||
let Some(op) = KvEventOp::from_wire(event.op) else {
|
||||
return Ok(()); // unknown op — drop quietly
|
||||
};
|
||||
let Some(collection) = event.collection.clone() else {
|
||||
return Ok(()); // KV events always carry a collection — defensively skip
|
||||
};
|
||||
let key = event.key.clone().unwrap_or_default();
|
||||
|
||||
let matches = self
|
||||
.triggers
|
||||
.list_matching_kv(cx.app_id, &collection, op)
|
||||
.await
|
||||
.map_err(|e| EmitError::Unavailable(format!("trigger lookup: {e}")))?;
|
||||
|
||||
if matches.is_empty() {
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
// Serialize the originating event as a TriggerEvent so the
|
||||
// dispatcher can hand it to the script as `ctx.event` without
|
||||
// round-tripping back to the trigger row.
|
||||
let trigger_event = TriggerEvent::Kv {
|
||||
op,
|
||||
collection,
|
||||
key,
|
||||
value: event.payload.clone(),
|
||||
};
|
||||
let payload = serde_json::to_value(&trigger_event)
|
||||
.map_err(|e| EmitError::Rejected(format!("event serialize: {e}")))?;
|
||||
|
||||
for m in matches {
|
||||
self.outbox
|
||||
.insert(NewOutboxRow {
|
||||
app_id: cx.app_id,
|
||||
source_kind: OutboxSourceKind::Kv,
|
||||
trigger_id: Some(m.trigger_id),
|
||||
script_id: Some(m.script_id),
|
||||
reply_to: None,
|
||||
payload: payload.clone(),
|
||||
origin_principal: cx.principal.as_ref().map(|p| p.user_id),
|
||||
trigger_depth: cx.trigger_depth.saturating_add(1),
|
||||
root_execution_id: Some(cx.root_execution_id),
|
||||
})
|
||||
.await
|
||||
.map_err(|e| EmitError::Unavailable(format!("outbox insert: {e}")))?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
258
crates/manager-core/src/outbox_repo.rs
Normal file
258
crates/manager-core/src/outbox_repo.rs
Normal file
@@ -0,0 +1,258 @@
|
||||
//! `OutboxRepo` — universal trigger outbox CRUD. Hot writes come from
|
||||
//! the `OutboxEventEmitter` (KV mutations fan out via this) and the
|
||||
//! sync-HTTP path. Hot reads come from the dispatcher, which claims
|
||||
//! due rows via `FOR UPDATE SKIP LOCKED`.
|
||||
|
||||
use async_trait::async_trait;
|
||||
use chrono::{DateTime, Utc};
|
||||
use picloud_shared::{
|
||||
AdminUserId, AppId, ExecutionId, NewHttpOutbox, OutboxWriter, OutboxWriterError, ScriptId,
|
||||
TriggerId,
|
||||
};
|
||||
use sqlx::PgPool;
|
||||
use uuid::Uuid;
|
||||
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub enum OutboxRepoError {
|
||||
#[error("database error: {0}")]
|
||||
Db(#[from] sqlx::Error),
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum OutboxSourceKind {
|
||||
Http,
|
||||
Kv,
|
||||
DeadLetter,
|
||||
}
|
||||
|
||||
impl OutboxSourceKind {
|
||||
#[must_use]
|
||||
pub const fn as_str(self) -> &'static str {
|
||||
match self {
|
||||
Self::Http => "http",
|
||||
Self::Kv => "kv",
|
||||
Self::DeadLetter => "dead_letter",
|
||||
}
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
pub fn from_wire(s: &str) -> Option<Self> {
|
||||
match s {
|
||||
"http" => Some(Self::Http),
|
||||
"kv" => Some(Self::Kv),
|
||||
"dead_letter" => Some(Self::DeadLetter),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Insert payload — what each event source writes when fanning out
|
||||
/// to the outbox. `payload` is the serialized `TriggerEvent` (plus
|
||||
/// any extra context the dispatcher needs to reconstruct an
|
||||
/// `ExecRequest`).
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct NewOutboxRow {
|
||||
pub app_id: AppId,
|
||||
pub source_kind: OutboxSourceKind,
|
||||
pub trigger_id: Option<TriggerId>,
|
||||
pub script_id: Option<ScriptId>,
|
||||
pub reply_to: Option<Uuid>,
|
||||
pub payload: serde_json::Value,
|
||||
pub origin_principal: Option<AdminUserId>,
|
||||
pub trigger_depth: u32,
|
||||
pub root_execution_id: Option<ExecutionId>,
|
||||
}
|
||||
|
||||
/// Row as the dispatcher sees it after a claim.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct OutboxRow {
|
||||
pub id: Uuid,
|
||||
pub app_id: AppId,
|
||||
pub source_kind: OutboxSourceKind,
|
||||
pub trigger_id: Option<TriggerId>,
|
||||
pub script_id: Option<ScriptId>,
|
||||
pub reply_to: Option<Uuid>,
|
||||
pub payload: serde_json::Value,
|
||||
pub origin_principal: Option<AdminUserId>,
|
||||
pub trigger_depth: u32,
|
||||
pub root_execution_id: Option<ExecutionId>,
|
||||
pub attempt_count: u32,
|
||||
pub next_attempt_at: DateTime<Utc>,
|
||||
pub created_at: DateTime<Utc>,
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
pub trait OutboxRepo: Send + Sync {
|
||||
async fn insert(&self, row: NewOutboxRow) -> Result<Uuid, OutboxRepoError>;
|
||||
|
||||
/// Claim up to `limit` due rows. Wraps the claim in a single
|
||||
/// transaction so two concurrent dispatchers (cluster mode) can't
|
||||
/// double-pick a row. Empty Vec when nothing is due.
|
||||
async fn claim_due(
|
||||
&self,
|
||||
claimed_by: &str,
|
||||
limit: i64,
|
||||
) -> Result<Vec<OutboxRow>, OutboxRepoError>;
|
||||
|
||||
/// Remove a row after a terminal outcome (success or dead-letter).
|
||||
async fn delete(&self, id: Uuid) -> Result<(), OutboxRepoError>;
|
||||
|
||||
/// Failure path: bump attempt_count, clear the claim, set the
|
||||
/// next attempt time. The dispatcher computes the delay (with
|
||||
/// backoff + jitter) and passes it in.
|
||||
async fn reschedule(
|
||||
&self,
|
||||
id: Uuid,
|
||||
attempt_count: u32,
|
||||
next_attempt_at: DateTime<Utc>,
|
||||
) -> Result<(), OutboxRepoError>;
|
||||
}
|
||||
|
||||
pub struct PostgresOutboxRepo {
|
||||
pool: PgPool,
|
||||
}
|
||||
|
||||
impl PostgresOutboxRepo {
|
||||
#[must_use]
|
||||
pub fn new(pool: PgPool) -> Self {
|
||||
Self { pool }
|
||||
}
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl OutboxRepo for PostgresOutboxRepo {
|
||||
async fn insert(&self, row: NewOutboxRow) -> Result<Uuid, OutboxRepoError> {
|
||||
let (id,): (Uuid,) = sqlx::query_as(
|
||||
"INSERT INTO outbox ( \
|
||||
app_id, source_kind, trigger_id, script_id, reply_to, \
|
||||
payload, origin_principal, trigger_depth, root_execution_id \
|
||||
) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9) \
|
||||
RETURNING id",
|
||||
)
|
||||
.bind(row.app_id.into_inner())
|
||||
.bind(row.source_kind.as_str())
|
||||
.bind(row.trigger_id.map(TriggerId::into_inner))
|
||||
.bind(row.script_id.map(ScriptId::into_inner))
|
||||
.bind(row.reply_to)
|
||||
.bind(row.payload)
|
||||
.bind(row.origin_principal.map(AdminUserId::into_inner))
|
||||
.bind(i32::try_from(row.trigger_depth).unwrap_or(0))
|
||||
.bind(row.root_execution_id.map(ExecutionId::into_inner))
|
||||
.fetch_one(&self.pool)
|
||||
.await?;
|
||||
Ok(id)
|
||||
}
|
||||
|
||||
async fn claim_due(
|
||||
&self,
|
||||
claimed_by: &str,
|
||||
limit: i64,
|
||||
) -> Result<Vec<OutboxRow>, OutboxRepoError> {
|
||||
let rows: Vec<OutboxRowRaw> = sqlx::query_as(
|
||||
"WITH due AS ( \
|
||||
SELECT id FROM outbox \
|
||||
WHERE claimed_at IS NULL AND next_attempt_at <= NOW() \
|
||||
ORDER BY next_attempt_at \
|
||||
FOR UPDATE SKIP LOCKED \
|
||||
LIMIT $1 \
|
||||
) \
|
||||
UPDATE outbox SET claimed_at = NOW(), claimed_by = $2 \
|
||||
WHERE id IN (SELECT id FROM due) \
|
||||
RETURNING id, app_id, source_kind, trigger_id, script_id, reply_to, \
|
||||
payload, origin_principal, trigger_depth, \
|
||||
root_execution_id, attempt_count, next_attempt_at, created_at",
|
||||
)
|
||||
.bind(limit)
|
||||
.bind(claimed_by)
|
||||
.fetch_all(&self.pool)
|
||||
.await?;
|
||||
|
||||
Ok(rows.into_iter().filter_map(OutboxRowRaw::hydrate).collect())
|
||||
}
|
||||
|
||||
async fn delete(&self, id: Uuid) -> Result<(), OutboxRepoError> {
|
||||
sqlx::query("DELETE FROM outbox WHERE id = $1")
|
||||
.bind(id)
|
||||
.execute(&self.pool)
|
||||
.await?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn reschedule(
|
||||
&self,
|
||||
id: Uuid,
|
||||
attempt_count: u32,
|
||||
next_attempt_at: DateTime<Utc>,
|
||||
) -> Result<(), OutboxRepoError> {
|
||||
sqlx::query(
|
||||
"UPDATE outbox SET attempt_count = $2, next_attempt_at = $3, \
|
||||
claimed_at = NULL, claimed_by = NULL \
|
||||
WHERE id = $1",
|
||||
)
|
||||
.bind(id)
|
||||
.bind(i32::try_from(attempt_count).unwrap_or(0))
|
||||
.bind(next_attempt_at)
|
||||
.execute(&self.pool)
|
||||
.await?;
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
/// `OutboxWriter` implementation so orchestrator-core (which can't
|
||||
/// depend on manager-core) can enqueue HTTP outbox rows through the
|
||||
/// shared trait.
|
||||
#[async_trait]
|
||||
impl OutboxWriter for PostgresOutboxRepo {
|
||||
async fn enqueue_http(&self, row: NewHttpOutbox) -> Result<Uuid, OutboxWriterError> {
|
||||
self.insert(NewOutboxRow {
|
||||
app_id: row.app_id,
|
||||
source_kind: OutboxSourceKind::Http,
|
||||
trigger_id: Some(TriggerId::from(row.route_id)),
|
||||
script_id: Some(row.script_id),
|
||||
reply_to: row.reply_to,
|
||||
payload: row.payload,
|
||||
origin_principal: row.origin_principal,
|
||||
trigger_depth: row.trigger_depth,
|
||||
root_execution_id: row.root_execution_id,
|
||||
})
|
||||
.await
|
||||
.map_err(|e| OutboxWriterError::Backend(e.to_string()))
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(sqlx::FromRow)]
|
||||
struct OutboxRowRaw {
|
||||
id: Uuid,
|
||||
app_id: Uuid,
|
||||
source_kind: String,
|
||||
trigger_id: Option<Uuid>,
|
||||
script_id: Option<Uuid>,
|
||||
reply_to: Option<Uuid>,
|
||||
payload: serde_json::Value,
|
||||
origin_principal: Option<Uuid>,
|
||||
trigger_depth: i32,
|
||||
root_execution_id: Option<Uuid>,
|
||||
attempt_count: i32,
|
||||
next_attempt_at: DateTime<Utc>,
|
||||
created_at: DateTime<Utc>,
|
||||
}
|
||||
|
||||
impl OutboxRowRaw {
|
||||
fn hydrate(self) -> Option<OutboxRow> {
|
||||
Some(OutboxRow {
|
||||
id: self.id,
|
||||
app_id: self.app_id.into(),
|
||||
source_kind: OutboxSourceKind::from_wire(&self.source_kind)?,
|
||||
trigger_id: self.trigger_id.map(Into::into),
|
||||
script_id: self.script_id.map(Into::into),
|
||||
reply_to: self.reply_to,
|
||||
payload: self.payload,
|
||||
origin_principal: self.origin_principal.map(Into::into),
|
||||
trigger_depth: u32::try_from(self.trigger_depth).unwrap_or(0),
|
||||
root_execution_id: self.root_execution_id.map(Into::into),
|
||||
attempt_count: u32::try_from(self.attempt_count).unwrap_or(0),
|
||||
next_attempt_at: self.next_attempt_at,
|
||||
created_at: self.created_at,
|
||||
})
|
||||
}
|
||||
}
|
||||
62
crates/manager-core/src/principal_resolver.rs
Normal file
62
crates/manager-core/src/principal_resolver.rs
Normal file
@@ -0,0 +1,62 @@
|
||||
//! `PrincipalResolver` — turns a `registered_by_principal` user id from
|
||||
//! a trigger row into the `Principal` the dispatcher passes through to
|
||||
//! the executor. Per design notes §4, a trigger execution runs as the
|
||||
//! user that registered the trigger; the original event's caller is
|
||||
//! recorded elsewhere (on the outbox row, for forensics) and does not
|
||||
//! become the execution principal.
|
||||
|
||||
use async_trait::async_trait;
|
||||
use picloud_shared::{AdminUserId, Principal};
|
||||
|
||||
use crate::admin_user_repo::{AdminUserRepository, AdminUserRepositoryError};
|
||||
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub enum PrincipalResolverError {
|
||||
#[error("user not found: {0}")]
|
||||
NotFound(AdminUserId),
|
||||
#[error("user is inactive: {0}")]
|
||||
Inactive(AdminUserId),
|
||||
#[error("admin user repo error: {0}")]
|
||||
Backend(String),
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
pub trait PrincipalResolver: Send + Sync {
|
||||
async fn resolve(&self, user_id: AdminUserId) -> Result<Principal, PrincipalResolverError>;
|
||||
}
|
||||
|
||||
pub struct AdminPrincipalResolver {
|
||||
users: std::sync::Arc<dyn AdminUserRepository>,
|
||||
}
|
||||
|
||||
impl AdminPrincipalResolver {
|
||||
#[must_use]
|
||||
pub fn new(users: std::sync::Arc<dyn AdminUserRepository>) -> Self {
|
||||
Self { users }
|
||||
}
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl PrincipalResolver for AdminPrincipalResolver {
|
||||
async fn resolve(&self, user_id: AdminUserId) -> Result<Principal, PrincipalResolverError> {
|
||||
let row = self
|
||||
.users
|
||||
.get(user_id)
|
||||
.await
|
||||
.map_err(|e: AdminUserRepositoryError| PrincipalResolverError::Backend(e.to_string()))?
|
||||
.ok_or(PrincipalResolverError::NotFound(user_id))?;
|
||||
if !row.is_active {
|
||||
return Err(PrincipalResolverError::Inactive(user_id));
|
||||
}
|
||||
Ok(Principal {
|
||||
user_id,
|
||||
instance_role: row.instance_role,
|
||||
// Trigger executions are cookie-session-style (no API key
|
||||
// scope restriction). Per-app permissions are evaluated
|
||||
// via `authz::can` against the `app_id` of the resource
|
||||
// the script touches, exactly like an admin invocation.
|
||||
scopes: None,
|
||||
app_binding: None,
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -77,6 +77,12 @@ pub struct CreateRouteRequest {
|
||||
pub path_kind: PathKind,
|
||||
pub path: String,
|
||||
pub method: Option<String>,
|
||||
/// Per-route dispatch mode (v1.1.1). Defaults to `Sync` when
|
||||
/// omitted so older clients aren't broken. `Async` routes return
|
||||
/// `202 Accepted` immediately and run the script in the
|
||||
/// background via the dispatcher.
|
||||
#[serde(default)]
|
||||
pub dispatch_mode: picloud_shared::DispatchMode,
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize)]
|
||||
@@ -211,6 +217,7 @@ async fn create_route<RR: RouteRepository, SR: ScriptRepository>(
|
||||
path_kind: input.path_kind,
|
||||
path: normalized_path,
|
||||
method: input.method,
|
||||
dispatch_mode: input.dispatch_mode,
|
||||
})
|
||||
.await?;
|
||||
refresh_table(&state).await?;
|
||||
@@ -370,6 +377,7 @@ pub fn compile_routes(rows: &[Route]) -> Result<Vec<CompiledRoute>, pattern::Par
|
||||
host: pattern::parse_host(r.host_kind, &r.host, r.host_param_name.as_deref())?,
|
||||
path: pattern::parse_path(r.path_kind, &r.path)?,
|
||||
method: r.method.clone(),
|
||||
dispatch_mode: r.dispatch_mode,
|
||||
})
|
||||
})
|
||||
.collect()
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
//! after every write — see the route_admin module for the binding.
|
||||
|
||||
use async_trait::async_trait;
|
||||
use picloud_shared::{AppId, HostKind, PathKind, Route, ScriptId};
|
||||
use picloud_shared::{AppId, DispatchMode, HostKind, PathKind, Route, ScriptId};
|
||||
use sqlx::PgPool;
|
||||
use uuid::Uuid;
|
||||
|
||||
@@ -20,6 +20,7 @@ pub struct NewRoute {
|
||||
pub path_kind: PathKind,
|
||||
pub path: String,
|
||||
pub method: Option<String>,
|
||||
pub dispatch_mode: DispatchMode,
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
@@ -62,7 +63,7 @@ impl RouteRepository for PostgresRouteRepository {
|
||||
async fn list_all(&self) -> Result<Vec<Route>, ScriptRepositoryError> {
|
||||
let rows = sqlx::query_as::<_, RouteRow>(
|
||||
"SELECT id, app_id, script_id, host_kind, host, host_param_name, \
|
||||
path_kind, path, method, created_at \
|
||||
path_kind, path, method, dispatch_mode, created_at \
|
||||
FROM routes ORDER BY created_at",
|
||||
)
|
||||
.fetch_all(&self.pool)
|
||||
@@ -73,7 +74,7 @@ impl RouteRepository for PostgresRouteRepository {
|
||||
async fn get(&self, route_id: Uuid) -> Result<Option<Route>, ScriptRepositoryError> {
|
||||
let row = sqlx::query_as::<_, RouteRow>(
|
||||
"SELECT id, app_id, script_id, host_kind, host, host_param_name, \
|
||||
path_kind, path, method, created_at \
|
||||
path_kind, path, method, dispatch_mode, created_at \
|
||||
FROM routes WHERE id = $1",
|
||||
)
|
||||
.bind(route_id)
|
||||
@@ -85,7 +86,7 @@ impl RouteRepository for PostgresRouteRepository {
|
||||
async fn list_for_app(&self, app_id: AppId) -> Result<Vec<Route>, ScriptRepositoryError> {
|
||||
let rows = sqlx::query_as::<_, RouteRow>(
|
||||
"SELECT id, app_id, script_id, host_kind, host, host_param_name, \
|
||||
path_kind, path, method, created_at \
|
||||
path_kind, path, method, dispatch_mode, created_at \
|
||||
FROM routes WHERE app_id = $1 ORDER BY created_at",
|
||||
)
|
||||
.bind(app_id.into_inner())
|
||||
@@ -100,7 +101,7 @@ impl RouteRepository for PostgresRouteRepository {
|
||||
) -> Result<Vec<Route>, ScriptRepositoryError> {
|
||||
let rows = sqlx::query_as::<_, RouteRow>(
|
||||
"SELECT id, app_id, script_id, host_kind, host, host_param_name, \
|
||||
path_kind, path, method, created_at \
|
||||
path_kind, path, method, dispatch_mode, created_at \
|
||||
FROM routes WHERE script_id = $1 ORDER BY created_at",
|
||||
)
|
||||
.bind(script_id.into_inner())
|
||||
@@ -113,10 +114,10 @@ impl RouteRepository for PostgresRouteRepository {
|
||||
let res = sqlx::query_as::<_, RouteRow>(
|
||||
"INSERT INTO routes ( \
|
||||
app_id, script_id, host_kind, host, host_param_name, \
|
||||
path_kind, path, method \
|
||||
) VALUES ($1, $2, $3, $4, $5, $6, $7, $8) \
|
||||
path_kind, path, method, dispatch_mode \
|
||||
) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9) \
|
||||
RETURNING id, app_id, script_id, host_kind, host, host_param_name, \
|
||||
path_kind, path, method, created_at",
|
||||
path_kind, path, method, dispatch_mode, created_at",
|
||||
)
|
||||
.bind(input.app_id.into_inner())
|
||||
.bind(input.script_id.into_inner())
|
||||
@@ -126,6 +127,7 @@ impl RouteRepository for PostgresRouteRepository {
|
||||
.bind(path_kind_str(input.path_kind))
|
||||
.bind(&input.path)
|
||||
.bind(input.method.as_deref())
|
||||
.bind(input.dispatch_mode.as_str())
|
||||
.fetch_one(&self.pool)
|
||||
.await;
|
||||
|
||||
@@ -198,6 +200,7 @@ struct RouteRow {
|
||||
path_kind: String,
|
||||
path: String,
|
||||
method: Option<String>,
|
||||
dispatch_mode: String,
|
||||
created_at: chrono::DateTime<chrono::Utc>,
|
||||
}
|
||||
|
||||
@@ -221,6 +224,7 @@ impl From<RouteRow> for Route {
|
||||
},
|
||||
path: r.path,
|
||||
method: r.method,
|
||||
dispatch_mode: DispatchMode::from_wire(&r.dispatch_mode).unwrap_or(DispatchMode::Sync),
|
||||
created_at: r.created_at,
|
||||
}
|
||||
}
|
||||
|
||||
157
crates/manager-core/src/trigger_config.rs
Normal file
157
crates/manager-core/src/trigger_config.rs
Normal file
@@ -0,0 +1,157 @@
|
||||
//! Trigger-framework tunables. Defaults match design notes §3 (retry
|
||||
//! policy) and §4 (retention). Each knob is env-overridable via a
|
||||
//! `PICLOUD_*` variable following the same `tracing::warn` on parse
|
||||
//! error pattern `SandboxCeiling::from_env` uses.
|
||||
|
||||
use std::env;
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "lowercase")]
|
||||
pub enum BackoffShape {
|
||||
Exponential,
|
||||
Linear,
|
||||
Constant,
|
||||
}
|
||||
|
||||
impl BackoffShape {
|
||||
#[must_use]
|
||||
pub const fn as_str(self) -> &'static str {
|
||||
match self {
|
||||
Self::Exponential => "exponential",
|
||||
Self::Linear => "linear",
|
||||
Self::Constant => "constant",
|
||||
}
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
pub fn from_wire(s: &str) -> Option<Self> {
|
||||
match s {
|
||||
"exponential" => Some(Self::Exponential),
|
||||
"linear" => Some(Self::Linear),
|
||||
"constant" => Some(Self::Constant),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy)]
|
||||
pub struct TriggerConfig {
|
||||
/// Maximum `cx.trigger_depth` before the dispatcher refuses
|
||||
/// execution. Above this, the row is dropped + a metric bumped;
|
||||
/// it is NOT dead-lettered (design notes §4: depth-exceeded
|
||||
/// means "you built a loop"). Default 8.
|
||||
pub max_trigger_depth: u32,
|
||||
|
||||
/// Default retry attempts (per-trigger override on the row).
|
||||
pub retry_max_attempts: u32,
|
||||
pub retry_backoff: BackoffShape,
|
||||
pub retry_base_ms: u32,
|
||||
/// ±jitter as a percentage of the computed delay. Applied at
|
||||
/// dispatch time — not per-trigger.
|
||||
pub retry_jitter_pct: u32,
|
||||
|
||||
/// dead-letter retention before GC, in days. Default 30.
|
||||
pub dead_letter_retention_days: u32,
|
||||
/// abandoned-execution retention before GC, in days. Default 7.
|
||||
pub abandoned_retention_days: u32,
|
||||
}
|
||||
|
||||
impl TriggerConfig {
|
||||
#[must_use]
|
||||
pub const fn conservative() -> Self {
|
||||
Self {
|
||||
max_trigger_depth: 8,
|
||||
retry_max_attempts: 3,
|
||||
retry_backoff: BackoffShape::Exponential,
|
||||
retry_base_ms: 1000,
|
||||
retry_jitter_pct: 20,
|
||||
dead_letter_retention_days: 30,
|
||||
abandoned_retention_days: 7,
|
||||
}
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
pub fn from_env() -> Self {
|
||||
let mut c = Self::conservative();
|
||||
load_u32(&mut c.max_trigger_depth, "PICLOUD_MAX_TRIGGER_DEPTH");
|
||||
load_u32(
|
||||
&mut c.retry_max_attempts,
|
||||
"PICLOUD_TRIGGER_RETRY_MAX_ATTEMPTS",
|
||||
);
|
||||
load_backoff(&mut c.retry_backoff, "PICLOUD_TRIGGER_RETRY_BACKOFF");
|
||||
load_u32(&mut c.retry_base_ms, "PICLOUD_TRIGGER_RETRY_BASE_MS");
|
||||
load_u32(&mut c.retry_jitter_pct, "PICLOUD_TRIGGER_RETRY_JITTER_PCT");
|
||||
load_u32(
|
||||
&mut c.dead_letter_retention_days,
|
||||
"PICLOUD_DEAD_LETTER_RETENTION_DAYS",
|
||||
);
|
||||
load_u32(
|
||||
&mut c.abandoned_retention_days,
|
||||
"PICLOUD_ABANDONED_EXECUTIONS_RETENTION_DAYS",
|
||||
);
|
||||
c
|
||||
}
|
||||
}
|
||||
|
||||
impl Default for TriggerConfig {
|
||||
fn default() -> Self {
|
||||
Self::conservative()
|
||||
}
|
||||
}
|
||||
|
||||
fn load_u32(dst: &mut u32, key: &str) {
|
||||
if let Ok(v) = env::var(key) {
|
||||
match v.parse::<u32>() {
|
||||
Ok(n) => *dst = n,
|
||||
Err(e) => {
|
||||
tracing::warn!(env = key, error = %e, "ignoring invalid trigger-config value");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn load_backoff(dst: &mut BackoffShape, key: &str) {
|
||||
if let Ok(v) = env::var(key) {
|
||||
match BackoffShape::from_wire(&v) {
|
||||
Some(b) => *dst = b,
|
||||
None => {
|
||||
tracing::warn!(
|
||||
env = key,
|
||||
value = %v,
|
||||
"ignoring invalid trigger-config backoff shape (use exponential|linear|constant)"
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn conservative_defaults_match_design_notes() {
|
||||
let c = TriggerConfig::conservative();
|
||||
assert_eq!(c.max_trigger_depth, 8);
|
||||
assert_eq!(c.retry_max_attempts, 3);
|
||||
assert_eq!(c.retry_backoff, BackoffShape::Exponential);
|
||||
assert_eq!(c.retry_base_ms, 1000);
|
||||
assert_eq!(c.retry_jitter_pct, 20);
|
||||
assert_eq!(c.dead_letter_retention_days, 30);
|
||||
assert_eq!(c.abandoned_retention_days, 7);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn backoff_round_trips() {
|
||||
for shape in [
|
||||
BackoffShape::Exponential,
|
||||
BackoffShape::Linear,
|
||||
BackoffShape::Constant,
|
||||
] {
|
||||
assert_eq!(BackoffShape::from_wire(shape.as_str()), Some(shape));
|
||||
}
|
||||
assert_eq!(BackoffShape::from_wire("garbage"), None);
|
||||
}
|
||||
}
|
||||
617
crates/manager-core/src/trigger_repo.rs
Normal file
617
crates/manager-core/src/trigger_repo.rs
Normal file
@@ -0,0 +1,617 @@
|
||||
//! `TriggerRepo` — CRUD over the `triggers` parent + per-kind detail
|
||||
//! tables. The admin endpoints (commit 4) sit on top of this; the
|
||||
//! dispatcher (commit 5) reads `list_matching_*` to fan out events to
|
||||
//! handler scripts.
|
||||
|
||||
use async_trait::async_trait;
|
||||
use chrono::{DateTime, Utc};
|
||||
use picloud_shared::{AdminUserId, AppId, KvEventOp, ScriptId, TriggerId};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use sqlx::PgPool;
|
||||
use uuid::Uuid;
|
||||
|
||||
use crate::trigger_config::BackoffShape;
|
||||
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub enum TriggerRepoError {
|
||||
#[error("database error: {0}")]
|
||||
Db(#[from] sqlx::Error),
|
||||
|
||||
#[error("trigger not found: {0}")]
|
||||
NotFound(TriggerId),
|
||||
|
||||
#[error("invalid trigger payload: {0}")]
|
||||
Invalid(String),
|
||||
}
|
||||
|
||||
/// Parent-table row plus the per-kind detail merged in. Serialized
|
||||
/// back to admin clients via the JSON API.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct Trigger {
|
||||
pub id: TriggerId,
|
||||
pub app_id: AppId,
|
||||
pub script_id: ScriptId,
|
||||
pub kind: TriggerKind,
|
||||
pub enabled: bool,
|
||||
pub dispatch_mode: TriggerDispatchMode,
|
||||
pub retry_max_attempts: u32,
|
||||
pub retry_backoff: BackoffShape,
|
||||
pub retry_base_ms: u32,
|
||||
pub registered_by_principal: AdminUserId,
|
||||
pub created_at: DateTime<Utc>,
|
||||
pub updated_at: DateTime<Utc>,
|
||||
pub details: TriggerDetails,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "snake_case")]
|
||||
pub enum TriggerKind {
|
||||
Kv,
|
||||
DeadLetter,
|
||||
}
|
||||
|
||||
impl TriggerKind {
|
||||
#[must_use]
|
||||
pub const fn as_str(self) -> &'static str {
|
||||
match self {
|
||||
Self::Kv => "kv",
|
||||
Self::DeadLetter => "dead_letter",
|
||||
}
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
pub fn from_wire(s: &str) -> Option<Self> {
|
||||
match s {
|
||||
"kv" => Some(Self::Kv),
|
||||
"dead_letter" => Some(Self::DeadLetter),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "snake_case")]
|
||||
pub enum TriggerDispatchMode {
|
||||
Sync,
|
||||
Async,
|
||||
}
|
||||
|
||||
impl TriggerDispatchMode {
|
||||
#[must_use]
|
||||
pub const fn as_str(self) -> &'static str {
|
||||
match self {
|
||||
Self::Sync => "sync",
|
||||
Self::Async => "async",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
#[serde(tag = "kind", rename_all = "snake_case")]
|
||||
pub enum TriggerDetails {
|
||||
Kv {
|
||||
collection_glob: String,
|
||||
ops: Vec<KvEventOp>,
|
||||
},
|
||||
DeadLetter {
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
source_filter: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
trigger_id_filter: Option<TriggerId>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
script_id_filter: Option<ScriptId>,
|
||||
},
|
||||
}
|
||||
|
||||
/// Create payload for a KV trigger. Defaults applied at the admin
|
||||
/// layer (uses `TriggerConfig::from_env` to fill retry settings if
|
||||
/// the request omitted them — keeps the row auditable).
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct CreateKvTrigger {
|
||||
pub script_id: ScriptId,
|
||||
pub collection_glob: String,
|
||||
pub ops: Vec<KvEventOp>,
|
||||
pub dispatch_mode: TriggerDispatchMode,
|
||||
pub retry_max_attempts: u32,
|
||||
pub retry_backoff: BackoffShape,
|
||||
pub retry_base_ms: u32,
|
||||
pub registered_by_principal: AdminUserId,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct CreateDeadLetterTrigger {
|
||||
pub script_id: ScriptId,
|
||||
pub source_filter: Option<String>,
|
||||
pub trigger_id_filter: Option<TriggerId>,
|
||||
pub script_id_filter: Option<ScriptId>,
|
||||
pub registered_by_principal: AdminUserId,
|
||||
}
|
||||
|
||||
/// One match for the dispatcher's "which KV triggers fire on this
|
||||
/// event" lookup. Carries everything the dispatcher needs to construct
|
||||
/// the outbox row.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct KvTriggerMatch {
|
||||
pub trigger_id: TriggerId,
|
||||
pub script_id: ScriptId,
|
||||
pub dispatch_mode: TriggerDispatchMode,
|
||||
pub retry_max_attempts: u32,
|
||||
pub retry_backoff: BackoffShape,
|
||||
pub retry_base_ms: u32,
|
||||
pub registered_by_principal: AdminUserId,
|
||||
}
|
||||
|
||||
/// One match for the dispatcher's "which dead-letter triggers fire
|
||||
/// on this dead-letter row" lookup.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct DeadLetterTriggerMatch {
|
||||
pub trigger_id: TriggerId,
|
||||
pub script_id: ScriptId,
|
||||
pub dispatch_mode: TriggerDispatchMode,
|
||||
pub registered_by_principal: AdminUserId,
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
pub trait TriggerRepo: Send + Sync {
|
||||
async fn create_kv_trigger(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
req: CreateKvTrigger,
|
||||
) -> Result<Trigger, TriggerRepoError>;
|
||||
|
||||
async fn create_dead_letter_trigger(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
req: CreateDeadLetterTrigger,
|
||||
) -> Result<Trigger, TriggerRepoError>;
|
||||
|
||||
async fn list_for_app(&self, app_id: AppId) -> Result<Vec<Trigger>, TriggerRepoError>;
|
||||
|
||||
async fn get(&self, id: TriggerId) -> Result<Option<Trigger>, TriggerRepoError>;
|
||||
|
||||
async fn delete(&self, id: TriggerId) -> Result<bool, TriggerRepoError>;
|
||||
|
||||
/// Dispatcher hot path: find every enabled KV trigger in `app_id`
|
||||
/// whose `collection_glob` matches `collection` and whose `ops`
|
||||
/// covers `op`. Glob matching done in Rust (the column is plain
|
||||
/// TEXT, the matcher applies "*"/"prefix:*" semantics).
|
||||
async fn list_matching_kv(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
collection: &str,
|
||||
op: KvEventOp,
|
||||
) -> Result<Vec<KvTriggerMatch>, TriggerRepoError>;
|
||||
|
||||
/// Dispatcher hot path for dead-letter fan-out. Filters: source
|
||||
/// (or any-source), originating trigger_id (or any), originating
|
||||
/// script_id (or any). Each filter is "match OR is_null".
|
||||
async fn list_matching_dead_letter(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
source: &str,
|
||||
trigger_id: Option<TriggerId>,
|
||||
script_id: Option<ScriptId>,
|
||||
) -> Result<Vec<DeadLetterTriggerMatch>, TriggerRepoError>;
|
||||
}
|
||||
|
||||
// ----------------------------------------------------------------------------
|
||||
// Postgres impl
|
||||
// ----------------------------------------------------------------------------
|
||||
|
||||
pub struct PostgresTriggerRepo {
|
||||
pool: PgPool,
|
||||
}
|
||||
|
||||
impl PostgresTriggerRepo {
|
||||
#[must_use]
|
||||
pub fn new(pool: PgPool) -> Self {
|
||||
Self { pool }
|
||||
}
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl TriggerRepo for PostgresTriggerRepo {
|
||||
async fn create_kv_trigger(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
req: CreateKvTrigger,
|
||||
) -> Result<Trigger, TriggerRepoError> {
|
||||
if req.collection_glob.is_empty() {
|
||||
return Err(TriggerRepoError::Invalid(
|
||||
"collection_glob must not be empty".into(),
|
||||
));
|
||||
}
|
||||
let mut tx = self.pool.begin().await?;
|
||||
let parent: TriggerRow = sqlx::query_as(
|
||||
"INSERT INTO triggers ( \
|
||||
app_id, script_id, kind, enabled, dispatch_mode, \
|
||||
retry_max_attempts, retry_backoff, retry_base_ms, \
|
||||
registered_by_principal \
|
||||
) VALUES ($1, $2, 'kv', TRUE, $3, $4, $5, $6, $7) \
|
||||
RETURNING id, app_id, script_id, kind, enabled, dispatch_mode, \
|
||||
retry_max_attempts, retry_backoff, retry_base_ms, \
|
||||
registered_by_principal, created_at, updated_at",
|
||||
)
|
||||
.bind(app_id.into_inner())
|
||||
.bind(req.script_id.into_inner())
|
||||
.bind(req.dispatch_mode.as_str())
|
||||
.bind(i32::try_from(req.retry_max_attempts).unwrap_or(3))
|
||||
.bind(req.retry_backoff.as_str())
|
||||
.bind(i32::try_from(req.retry_base_ms).unwrap_or(1000))
|
||||
.bind(req.registered_by_principal.into_inner())
|
||||
.fetch_one(&mut *tx)
|
||||
.await?;
|
||||
|
||||
let ops_str: Vec<String> = req.ops.iter().map(|o| o.as_str().to_string()).collect();
|
||||
sqlx::query(
|
||||
"INSERT INTO kv_trigger_details (trigger_id, collection_glob, ops) \
|
||||
VALUES ($1, $2, $3)",
|
||||
)
|
||||
.bind(parent.id)
|
||||
.bind(&req.collection_glob)
|
||||
.bind(&ops_str)
|
||||
.execute(&mut *tx)
|
||||
.await?;
|
||||
|
||||
tx.commit().await?;
|
||||
|
||||
Ok(Trigger {
|
||||
id: parent.id.into(),
|
||||
app_id: parent.app_id.into(),
|
||||
script_id: parent.script_id.into(),
|
||||
kind: TriggerKind::Kv,
|
||||
enabled: parent.enabled,
|
||||
dispatch_mode: dispatch_from_str(&parent.dispatch_mode),
|
||||
retry_max_attempts: u32::try_from(parent.retry_max_attempts).unwrap_or(3),
|
||||
retry_backoff: BackoffShape::from_wire(&parent.retry_backoff)
|
||||
.unwrap_or(BackoffShape::Exponential),
|
||||
retry_base_ms: u32::try_from(parent.retry_base_ms).unwrap_or(1000),
|
||||
registered_by_principal: parent.registered_by_principal.into(),
|
||||
created_at: parent.created_at,
|
||||
updated_at: parent.updated_at,
|
||||
details: TriggerDetails::Kv {
|
||||
collection_glob: req.collection_glob,
|
||||
ops: req.ops,
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
async fn create_dead_letter_trigger(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
req: CreateDeadLetterTrigger,
|
||||
) -> Result<Trigger, TriggerRepoError> {
|
||||
let mut tx = self.pool.begin().await?;
|
||||
// Dead-letter triggers force max_attempts=1 (design notes §4
|
||||
// recursion-stop). Backoff/base_ms irrelevant but the columns
|
||||
// are NOT NULL — store sensible values.
|
||||
let parent: TriggerRow = sqlx::query_as(
|
||||
"INSERT INTO triggers ( \
|
||||
app_id, script_id, kind, enabled, dispatch_mode, \
|
||||
retry_max_attempts, retry_backoff, retry_base_ms, \
|
||||
registered_by_principal \
|
||||
) VALUES ($1, $2, 'dead_letter', TRUE, 'async', 1, 'constant', 0, $3) \
|
||||
RETURNING id, app_id, script_id, kind, enabled, dispatch_mode, \
|
||||
retry_max_attempts, retry_backoff, retry_base_ms, \
|
||||
registered_by_principal, created_at, updated_at",
|
||||
)
|
||||
.bind(app_id.into_inner())
|
||||
.bind(req.script_id.into_inner())
|
||||
.bind(req.registered_by_principal.into_inner())
|
||||
.fetch_one(&mut *tx)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"INSERT INTO dead_letter_trigger_details \
|
||||
(trigger_id, source_filter, trigger_id_filter, script_id_filter) \
|
||||
VALUES ($1, $2, $3, $4)",
|
||||
)
|
||||
.bind(parent.id)
|
||||
.bind(req.source_filter.as_deref())
|
||||
.bind(req.trigger_id_filter.map(TriggerId::into_inner))
|
||||
.bind(req.script_id_filter.map(ScriptId::into_inner))
|
||||
.execute(&mut *tx)
|
||||
.await?;
|
||||
|
||||
tx.commit().await?;
|
||||
|
||||
Ok(Trigger {
|
||||
id: parent.id.into(),
|
||||
app_id: parent.app_id.into(),
|
||||
script_id: parent.script_id.into(),
|
||||
kind: TriggerKind::DeadLetter,
|
||||
enabled: parent.enabled,
|
||||
dispatch_mode: dispatch_from_str(&parent.dispatch_mode),
|
||||
retry_max_attempts: u32::try_from(parent.retry_max_attempts).unwrap_or(1),
|
||||
retry_backoff: BackoffShape::from_wire(&parent.retry_backoff)
|
||||
.unwrap_or(BackoffShape::Constant),
|
||||
retry_base_ms: u32::try_from(parent.retry_base_ms).unwrap_or(0),
|
||||
registered_by_principal: parent.registered_by_principal.into(),
|
||||
created_at: parent.created_at,
|
||||
updated_at: parent.updated_at,
|
||||
details: TriggerDetails::DeadLetter {
|
||||
source_filter: req.source_filter,
|
||||
trigger_id_filter: req.trigger_id_filter,
|
||||
script_id_filter: req.script_id_filter,
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
async fn list_for_app(&self, app_id: AppId) -> Result<Vec<Trigger>, TriggerRepoError> {
|
||||
let parents: Vec<TriggerRow> = sqlx::query_as(
|
||||
"SELECT id, app_id, script_id, kind, enabled, dispatch_mode, \
|
||||
retry_max_attempts, retry_backoff, retry_base_ms, \
|
||||
registered_by_principal, created_at, updated_at \
|
||||
FROM triggers WHERE app_id = $1 ORDER BY created_at DESC",
|
||||
)
|
||||
.bind(app_id.into_inner())
|
||||
.fetch_all(&self.pool)
|
||||
.await?;
|
||||
|
||||
let mut out = Vec::with_capacity(parents.len());
|
||||
for p in parents {
|
||||
out.push(hydrate_one(&self.pool, p).await?);
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
async fn get(&self, id: TriggerId) -> Result<Option<Trigger>, TriggerRepoError> {
|
||||
let parent: Option<TriggerRow> = sqlx::query_as(
|
||||
"SELECT id, app_id, script_id, kind, enabled, dispatch_mode, \
|
||||
retry_max_attempts, retry_backoff, retry_base_ms, \
|
||||
registered_by_principal, created_at, updated_at \
|
||||
FROM triggers WHERE id = $1",
|
||||
)
|
||||
.bind(id.into_inner())
|
||||
.fetch_optional(&self.pool)
|
||||
.await?;
|
||||
match parent {
|
||||
Some(p) => Ok(Some(hydrate_one(&self.pool, p).await?)),
|
||||
None => Ok(None),
|
||||
}
|
||||
}
|
||||
|
||||
async fn delete(&self, id: TriggerId) -> Result<bool, TriggerRepoError> {
|
||||
// ON DELETE CASCADE on the detail tables takes care of them.
|
||||
let res = sqlx::query("DELETE FROM triggers WHERE id = $1")
|
||||
.bind(id.into_inner())
|
||||
.execute(&self.pool)
|
||||
.await?;
|
||||
Ok(res.rows_affected() > 0)
|
||||
}
|
||||
|
||||
async fn list_matching_kv(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
collection: &str,
|
||||
op: KvEventOp,
|
||||
) -> Result<Vec<KvTriggerMatch>, TriggerRepoError> {
|
||||
// Fetch all enabled KV triggers for the app — glob matching
|
||||
// happens in Rust so we don't have to teach the query about
|
||||
// `*` and `prefix:*`. Sets are tiny in practice (one app's
|
||||
// worth of triggers, usually a handful).
|
||||
let rows: Vec<KvMatchRow> = sqlx::query_as(
|
||||
"SELECT t.id, t.script_id, t.dispatch_mode, \
|
||||
t.retry_max_attempts, t.retry_backoff, t.retry_base_ms, \
|
||||
t.registered_by_principal, \
|
||||
d.collection_glob, d.ops \
|
||||
FROM triggers t \
|
||||
JOIN kv_trigger_details d ON d.trigger_id = t.id \
|
||||
WHERE t.app_id = $1 AND t.kind = 'kv' AND t.enabled = TRUE",
|
||||
)
|
||||
.bind(app_id.into_inner())
|
||||
.fetch_all(&self.pool)
|
||||
.await?;
|
||||
|
||||
let op_str = op.as_str();
|
||||
let mut out = Vec::new();
|
||||
for r in rows {
|
||||
if !collection_matches(&r.collection_glob, collection) {
|
||||
continue;
|
||||
}
|
||||
let any_op = r.ops.is_empty();
|
||||
if !any_op && !r.ops.iter().any(|o| o == op_str) {
|
||||
continue;
|
||||
}
|
||||
out.push(KvTriggerMatch {
|
||||
trigger_id: r.id.into(),
|
||||
script_id: r.script_id.into(),
|
||||
dispatch_mode: dispatch_from_str(&r.dispatch_mode),
|
||||
retry_max_attempts: u32::try_from(r.retry_max_attempts).unwrap_or(3),
|
||||
retry_backoff: BackoffShape::from_wire(&r.retry_backoff)
|
||||
.unwrap_or(BackoffShape::Exponential),
|
||||
retry_base_ms: u32::try_from(r.retry_base_ms).unwrap_or(1000),
|
||||
registered_by_principal: r.registered_by_principal.into(),
|
||||
});
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
async fn list_matching_dead_letter(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
source: &str,
|
||||
trigger_id: Option<TriggerId>,
|
||||
script_id: Option<ScriptId>,
|
||||
) -> Result<Vec<DeadLetterTriggerMatch>, TriggerRepoError> {
|
||||
let rows: Vec<DlMatchRow> = sqlx::query_as(
|
||||
"SELECT t.id, t.script_id, t.dispatch_mode, t.registered_by_principal, \
|
||||
d.source_filter, d.trigger_id_filter, d.script_id_filter \
|
||||
FROM triggers t \
|
||||
JOIN dead_letter_trigger_details d ON d.trigger_id = t.id \
|
||||
WHERE t.app_id = $1 AND t.kind = 'dead_letter' AND t.enabled = TRUE \
|
||||
AND (d.source_filter IS NULL OR d.source_filter = $2) \
|
||||
AND (d.trigger_id_filter IS NULL OR d.trigger_id_filter = $3) \
|
||||
AND (d.script_id_filter IS NULL OR d.script_id_filter = $4)",
|
||||
)
|
||||
.bind(app_id.into_inner())
|
||||
.bind(source)
|
||||
.bind(trigger_id.map(TriggerId::into_inner))
|
||||
.bind(script_id.map(ScriptId::into_inner))
|
||||
.fetch_all(&self.pool)
|
||||
.await?;
|
||||
|
||||
Ok(rows
|
||||
.into_iter()
|
||||
.map(|r| DeadLetterTriggerMatch {
|
||||
trigger_id: r.id.into(),
|
||||
script_id: r.script_id.into(),
|
||||
dispatch_mode: dispatch_from_str(&r.dispatch_mode),
|
||||
registered_by_principal: r.registered_by_principal.into(),
|
||||
})
|
||||
.collect())
|
||||
}
|
||||
}
|
||||
|
||||
async fn hydrate_one(pool: &PgPool, parent: TriggerRow) -> Result<Trigger, TriggerRepoError> {
|
||||
let kind = TriggerKind::from_wire(&parent.kind).ok_or_else(|| {
|
||||
TriggerRepoError::Invalid(format!("unknown trigger kind {}", parent.kind))
|
||||
})?;
|
||||
|
||||
let details = match kind {
|
||||
TriggerKind::Kv => {
|
||||
let row: KvDetailRow = sqlx::query_as(
|
||||
"SELECT collection_glob, ops FROM kv_trigger_details WHERE trigger_id = $1",
|
||||
)
|
||||
.bind(parent.id)
|
||||
.fetch_one(pool)
|
||||
.await?;
|
||||
let ops = row
|
||||
.ops
|
||||
.iter()
|
||||
.filter_map(|s| KvEventOp::from_wire(s))
|
||||
.collect();
|
||||
TriggerDetails::Kv {
|
||||
collection_glob: row.collection_glob,
|
||||
ops,
|
||||
}
|
||||
}
|
||||
TriggerKind::DeadLetter => {
|
||||
let row: DlDetailRow = sqlx::query_as(
|
||||
"SELECT source_filter, trigger_id_filter, script_id_filter \
|
||||
FROM dead_letter_trigger_details WHERE trigger_id = $1",
|
||||
)
|
||||
.bind(parent.id)
|
||||
.fetch_one(pool)
|
||||
.await?;
|
||||
TriggerDetails::DeadLetter {
|
||||
source_filter: row.source_filter,
|
||||
trigger_id_filter: row.trigger_id_filter.map(Into::into),
|
||||
script_id_filter: row.script_id_filter.map(Into::into),
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
Ok(Trigger {
|
||||
id: parent.id.into(),
|
||||
app_id: parent.app_id.into(),
|
||||
script_id: parent.script_id.into(),
|
||||
kind,
|
||||
enabled: parent.enabled,
|
||||
dispatch_mode: dispatch_from_str(&parent.dispatch_mode),
|
||||
retry_max_attempts: u32::try_from(parent.retry_max_attempts).unwrap_or(3),
|
||||
retry_backoff: BackoffShape::from_wire(&parent.retry_backoff)
|
||||
.unwrap_or(BackoffShape::Exponential),
|
||||
retry_base_ms: u32::try_from(parent.retry_base_ms).unwrap_or(1000),
|
||||
registered_by_principal: parent.registered_by_principal.into(),
|
||||
created_at: parent.created_at,
|
||||
updated_at: parent.updated_at,
|
||||
details,
|
||||
})
|
||||
}
|
||||
|
||||
fn dispatch_from_str(s: &str) -> TriggerDispatchMode {
|
||||
match s {
|
||||
"sync" => TriggerDispatchMode::Sync,
|
||||
_ => TriggerDispatchMode::Async,
|
||||
}
|
||||
}
|
||||
|
||||
/// Match a `collection_glob` against an actual `collection` name.
|
||||
/// Supported forms (in priority order):
|
||||
/// - `"*"` → matches every collection
|
||||
/// - `"foo*"` → prefix match (anything starting with "foo")
|
||||
/// - `"foo"` → exact match
|
||||
#[must_use]
|
||||
pub fn collection_matches(glob: &str, collection: &str) -> bool {
|
||||
if glob == "*" {
|
||||
return true;
|
||||
}
|
||||
if let Some(prefix) = glob.strip_suffix('*') {
|
||||
return collection.starts_with(prefix);
|
||||
}
|
||||
glob == collection
|
||||
}
|
||||
|
||||
#[derive(sqlx::FromRow)]
|
||||
struct TriggerRow {
|
||||
id: Uuid,
|
||||
app_id: Uuid,
|
||||
script_id: Uuid,
|
||||
kind: String,
|
||||
enabled: bool,
|
||||
dispatch_mode: String,
|
||||
retry_max_attempts: i32,
|
||||
retry_backoff: String,
|
||||
retry_base_ms: i32,
|
||||
registered_by_principal: Uuid,
|
||||
created_at: DateTime<Utc>,
|
||||
updated_at: DateTime<Utc>,
|
||||
}
|
||||
|
||||
#[derive(sqlx::FromRow)]
|
||||
struct KvDetailRow {
|
||||
collection_glob: String,
|
||||
ops: Vec<String>,
|
||||
}
|
||||
|
||||
#[derive(sqlx::FromRow)]
|
||||
#[allow(clippy::struct_field_names)]
|
||||
struct DlDetailRow {
|
||||
source_filter: Option<String>,
|
||||
trigger_id_filter: Option<Uuid>,
|
||||
script_id_filter: Option<Uuid>,
|
||||
}
|
||||
|
||||
#[derive(sqlx::FromRow)]
|
||||
struct KvMatchRow {
|
||||
id: Uuid,
|
||||
script_id: Uuid,
|
||||
dispatch_mode: String,
|
||||
retry_max_attempts: i32,
|
||||
retry_backoff: String,
|
||||
retry_base_ms: i32,
|
||||
registered_by_principal: Uuid,
|
||||
collection_glob: String,
|
||||
ops: Vec<String>,
|
||||
}
|
||||
|
||||
#[derive(sqlx::FromRow)]
|
||||
struct DlMatchRow {
|
||||
id: Uuid,
|
||||
script_id: Uuid,
|
||||
dispatch_mode: String,
|
||||
registered_by_principal: Uuid,
|
||||
#[allow(dead_code)]
|
||||
source_filter: Option<String>,
|
||||
#[allow(dead_code)]
|
||||
trigger_id_filter: Option<Uuid>,
|
||||
#[allow(dead_code)]
|
||||
script_id_filter: Option<Uuid>,
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn collection_matcher_handles_star_prefix_exact() {
|
||||
assert!(collection_matches("*", "widgets"));
|
||||
assert!(collection_matches("*", ""));
|
||||
assert!(collection_matches("users:*", "users:1"));
|
||||
assert!(collection_matches("users:*", "users:"));
|
||||
assert!(!collection_matches("users:*", "orgs:1"));
|
||||
assert!(collection_matches("widgets", "widgets"));
|
||||
assert!(!collection_matches("widgets", "Widgets"));
|
||||
}
|
||||
}
|
||||
748
crates/manager-core/src/triggers_api.rs
Normal file
748
crates/manager-core/src/triggers_api.rs
Normal file
@@ -0,0 +1,748 @@
|
||||
//! `/api/v1/admin/apps/{id}/triggers/*` — trigger CRUD admin endpoints.
|
||||
//!
|
||||
//! Per design notes §2, two kinds ship in v1.1.1: `kv` (with
|
||||
//! collection_glob + ops) and `dead_letter` (with optional source /
|
||||
//! trigger_id / script_id filters). Separate endpoints per kind keep
|
||||
//! validation clean.
|
||||
//!
|
||||
//! Every endpoint is guarded by `Capability::AppManageTriggers(app_id)`
|
||||
//! evaluated after the resource lookup so the capability binds to the
|
||||
//! resource's actual `app_id` (mirrors `apps_api`).
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use axum::extract::{Path, State};
|
||||
use axum::http::StatusCode;
|
||||
use axum::response::{IntoResponse, Json, Response};
|
||||
use axum::routing::{delete, get, post};
|
||||
use axum::{Extension, Router};
|
||||
use picloud_shared::{AppId, KvEventOp, Principal, ScriptId, TriggerId};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use serde_json::json;
|
||||
|
||||
use crate::app_repo::AppRepository;
|
||||
use crate::authz::{require, AuthzDenied, AuthzError, AuthzRepo, Capability};
|
||||
use crate::trigger_config::{BackoffShape, TriggerConfig};
|
||||
use crate::trigger_repo::{
|
||||
CreateDeadLetterTrigger, CreateKvTrigger, Trigger, TriggerDispatchMode, TriggerRepo,
|
||||
TriggerRepoError,
|
||||
};
|
||||
|
||||
#[derive(Clone)]
|
||||
pub struct TriggersState {
|
||||
pub triggers: Arc<dyn TriggerRepo>,
|
||||
pub apps: Arc<dyn AppRepository>,
|
||||
pub authz: Arc<dyn AuthzRepo>,
|
||||
/// Defaults applied to created triggers when the request omits
|
||||
/// retry settings. Kept on the state struct so tests can swap
|
||||
/// in a stricter / looser config without env tinkering.
|
||||
pub config: TriggerConfig,
|
||||
}
|
||||
|
||||
pub fn triggers_router(state: TriggersState) -> Router {
|
||||
Router::new()
|
||||
.route(
|
||||
"/apps/{app_id}/triggers",
|
||||
get(list_triggers).delete(noop_405),
|
||||
)
|
||||
.route("/apps/{app_id}/triggers/kv", post(create_kv_trigger))
|
||||
.route(
|
||||
"/apps/{app_id}/triggers/dead_letter",
|
||||
post(create_dl_trigger),
|
||||
)
|
||||
.route(
|
||||
"/apps/{app_id}/triggers/{trigger_id}",
|
||||
delete(delete_trigger),
|
||||
)
|
||||
.with_state(state)
|
||||
}
|
||||
|
||||
async fn noop_405() -> StatusCode {
|
||||
StatusCode::METHOD_NOT_ALLOWED
|
||||
}
|
||||
|
||||
// ----------------------------------------------------------------------------
|
||||
// DTOs
|
||||
// ----------------------------------------------------------------------------
|
||||
|
||||
#[derive(Debug, Deserialize)]
|
||||
pub struct CreateKvTriggerRequest {
|
||||
pub script_id: ScriptId,
|
||||
pub collection_glob: String,
|
||||
/// Subset of `{insert, update, delete}`. Empty array means "any
|
||||
/// op" (the trigger fires on every mutation in matching
|
||||
/// collections).
|
||||
#[serde(default)]
|
||||
pub ops: Vec<KvEventOp>,
|
||||
#[serde(default = "default_dispatch")]
|
||||
pub dispatch_mode: TriggerDispatchMode,
|
||||
/// Overrides for the platform retry defaults. Omitted fields fall
|
||||
/// back to `TriggerConfig` (env-overridable) at write time.
|
||||
#[serde(default)]
|
||||
pub retry_max_attempts: Option<u32>,
|
||||
#[serde(default)]
|
||||
pub retry_backoff: Option<BackoffShape>,
|
||||
#[serde(default)]
|
||||
pub retry_base_ms: Option<u32>,
|
||||
}
|
||||
|
||||
const fn default_dispatch() -> TriggerDispatchMode {
|
||||
TriggerDispatchMode::Async
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize)]
|
||||
pub struct CreateDeadLetterTriggerRequest {
|
||||
pub script_id: ScriptId,
|
||||
#[serde(default)]
|
||||
pub source_filter: Option<String>,
|
||||
#[serde(default)]
|
||||
pub trigger_id_filter: Option<TriggerId>,
|
||||
#[serde(default)]
|
||||
pub script_id_filter: Option<ScriptId>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Serialize)]
|
||||
pub struct TriggerListResponse {
|
||||
pub triggers: Vec<Trigger>,
|
||||
}
|
||||
|
||||
// ----------------------------------------------------------------------------
|
||||
// Handlers
|
||||
// ----------------------------------------------------------------------------
|
||||
|
||||
async fn list_triggers(
|
||||
State(s): State<TriggersState>,
|
||||
Extension(principal): Extension<Principal>,
|
||||
Path(app_id): Path<AppId>,
|
||||
) -> Result<Json<TriggerListResponse>, TriggersApiError> {
|
||||
ensure_app_exists(&*s.apps, app_id).await?;
|
||||
require(
|
||||
s.authz.as_ref(),
|
||||
&principal,
|
||||
Capability::AppManageTriggers(app_id),
|
||||
)
|
||||
.await?;
|
||||
let triggers = s.triggers.list_for_app(app_id).await?;
|
||||
Ok(Json(TriggerListResponse { triggers }))
|
||||
}
|
||||
|
||||
async fn create_kv_trigger(
|
||||
State(s): State<TriggersState>,
|
||||
Extension(principal): Extension<Principal>,
|
||||
Path(app_id): Path<AppId>,
|
||||
Json(input): Json<CreateKvTriggerRequest>,
|
||||
) -> Result<(StatusCode, Json<Trigger>), TriggersApiError> {
|
||||
ensure_app_exists(&*s.apps, app_id).await?;
|
||||
require(
|
||||
s.authz.as_ref(),
|
||||
&principal,
|
||||
Capability::AppManageTriggers(app_id),
|
||||
)
|
||||
.await?;
|
||||
|
||||
if input.collection_glob.trim().is_empty() {
|
||||
return Err(TriggersApiError::Invalid(
|
||||
"collection_glob must not be empty".into(),
|
||||
));
|
||||
}
|
||||
|
||||
let req = CreateKvTrigger {
|
||||
script_id: input.script_id,
|
||||
collection_glob: input.collection_glob,
|
||||
ops: input.ops,
|
||||
dispatch_mode: input.dispatch_mode,
|
||||
retry_max_attempts: input
|
||||
.retry_max_attempts
|
||||
.unwrap_or(s.config.retry_max_attempts),
|
||||
retry_backoff: input.retry_backoff.unwrap_or(s.config.retry_backoff),
|
||||
retry_base_ms: input.retry_base_ms.unwrap_or(s.config.retry_base_ms),
|
||||
registered_by_principal: principal.user_id,
|
||||
};
|
||||
let created = s.triggers.create_kv_trigger(app_id, req).await?;
|
||||
Ok((StatusCode::CREATED, Json(created)))
|
||||
}
|
||||
|
||||
async fn create_dl_trigger(
|
||||
State(s): State<TriggersState>,
|
||||
Extension(principal): Extension<Principal>,
|
||||
Path(app_id): Path<AppId>,
|
||||
Json(input): Json<CreateDeadLetterTriggerRequest>,
|
||||
) -> Result<(StatusCode, Json<Trigger>), TriggersApiError> {
|
||||
ensure_app_exists(&*s.apps, app_id).await?;
|
||||
require(
|
||||
s.authz.as_ref(),
|
||||
&principal,
|
||||
Capability::AppManageTriggers(app_id),
|
||||
)
|
||||
.await?;
|
||||
let req = CreateDeadLetterTrigger {
|
||||
script_id: input.script_id,
|
||||
source_filter: input.source_filter,
|
||||
trigger_id_filter: input.trigger_id_filter,
|
||||
script_id_filter: input.script_id_filter,
|
||||
registered_by_principal: principal.user_id,
|
||||
};
|
||||
let created = s.triggers.create_dead_letter_trigger(app_id, req).await?;
|
||||
Ok((StatusCode::CREATED, Json(created)))
|
||||
}
|
||||
|
||||
async fn delete_trigger(
|
||||
State(s): State<TriggersState>,
|
||||
Extension(principal): Extension<Principal>,
|
||||
Path((app_id, trigger_id)): Path<(AppId, TriggerId)>,
|
||||
) -> Result<StatusCode, TriggersApiError> {
|
||||
ensure_app_exists(&*s.apps, app_id).await?;
|
||||
// Load the trigger so we can confirm it belongs to the right
|
||||
// app; this prevents a caller from deleting a trigger by id alone
|
||||
// when their capability is bound to a different app.
|
||||
let trigger = s
|
||||
.triggers
|
||||
.get(trigger_id)
|
||||
.await?
|
||||
.ok_or(TriggersApiError::NotFound(trigger_id))?;
|
||||
if trigger.app_id != app_id {
|
||||
return Err(TriggersApiError::NotFound(trigger_id));
|
||||
}
|
||||
require(
|
||||
s.authz.as_ref(),
|
||||
&principal,
|
||||
Capability::AppManageTriggers(app_id),
|
||||
)
|
||||
.await?;
|
||||
if !s.triggers.delete(trigger_id).await? {
|
||||
return Err(TriggersApiError::NotFound(trigger_id));
|
||||
}
|
||||
Ok(StatusCode::NO_CONTENT)
|
||||
}
|
||||
|
||||
async fn ensure_app_exists(
|
||||
apps: &dyn AppRepository,
|
||||
app_id: AppId,
|
||||
) -> Result<(), TriggersApiError> {
|
||||
apps.get_by_id(app_id)
|
||||
.await
|
||||
.map_err(|e| TriggersApiError::Backend(e.to_string()))?
|
||||
.ok_or_else(|| TriggersApiError::AppNotFound(app_id.to_string()))?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
// ----------------------------------------------------------------------------
|
||||
// Errors
|
||||
// ----------------------------------------------------------------------------
|
||||
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub enum TriggersApiError {
|
||||
#[error("app not found: {0}")]
|
||||
AppNotFound(String),
|
||||
|
||||
#[error("trigger not found: {0}")]
|
||||
NotFound(TriggerId),
|
||||
|
||||
#[error("invalid trigger: {0}")]
|
||||
Invalid(String),
|
||||
|
||||
#[error("forbidden")]
|
||||
Forbidden,
|
||||
|
||||
#[error("authorization repo error: {0}")]
|
||||
AuthzRepo(String),
|
||||
|
||||
#[error("trigger backend: {0}")]
|
||||
Backend(String),
|
||||
}
|
||||
|
||||
impl From<AuthzDenied> for TriggersApiError {
|
||||
fn from(d: AuthzDenied) -> Self {
|
||||
match d {
|
||||
AuthzDenied::Denied => Self::Forbidden,
|
||||
AuthzDenied::Repo(e) => Self::AuthzRepo(e.to_string()),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl From<AuthzError> for TriggersApiError {
|
||||
fn from(e: AuthzError) -> Self {
|
||||
Self::AuthzRepo(e.to_string())
|
||||
}
|
||||
}
|
||||
|
||||
impl From<TriggerRepoError> for TriggersApiError {
|
||||
fn from(e: TriggerRepoError) -> Self {
|
||||
match e {
|
||||
TriggerRepoError::NotFound(id) => Self::NotFound(id),
|
||||
TriggerRepoError::Invalid(s) => Self::Invalid(s),
|
||||
TriggerRepoError::Db(e) => Self::Backend(e.to_string()),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl IntoResponse for TriggersApiError {
|
||||
fn into_response(self) -> Response {
|
||||
let (status, body) = match &self {
|
||||
Self::AppNotFound(_) | Self::NotFound(_) => {
|
||||
(StatusCode::NOT_FOUND, json!({ "error": self.to_string() }))
|
||||
}
|
||||
Self::Invalid(_) => (
|
||||
StatusCode::UNPROCESSABLE_ENTITY,
|
||||
json!({ "error": self.to_string() }),
|
||||
),
|
||||
Self::Forbidden => (StatusCode::FORBIDDEN, json!({ "error": self.to_string() })),
|
||||
Self::AuthzRepo(e) => {
|
||||
tracing::error!(error = %e, "triggers authz repo error");
|
||||
(
|
||||
StatusCode::INTERNAL_SERVER_ERROR,
|
||||
json!({ "error": "internal error" }),
|
||||
)
|
||||
}
|
||||
Self::Backend(e) => {
|
||||
tracing::error!(error = %e, "triggers api backend error");
|
||||
(
|
||||
StatusCode::INTERNAL_SERVER_ERROR,
|
||||
json!({ "error": "internal error" }),
|
||||
)
|
||||
}
|
||||
};
|
||||
(status, Json(body)).into_response()
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
//! In-memory tests for the trigger admin path. The Axum routing
|
||||
//! / extractor surface is exercised by integration tests (which
|
||||
//! need a real Postgres for the trigger repo); these tests cover
|
||||
//! the handlers' invariant logic — capability enforcement, app
|
||||
//! validation, default fallback for retry settings.
|
||||
|
||||
use super::*;
|
||||
use crate::app_repo::{AppLookup, AppRepository};
|
||||
use crate::trigger_repo::{
|
||||
DeadLetterTriggerMatch, KvTriggerMatch, Trigger, TriggerDetails, TriggerRepo,
|
||||
TriggerRepoError,
|
||||
};
|
||||
use async_trait::async_trait;
|
||||
use chrono::Utc;
|
||||
use picloud_shared::{AdminUserId, App, AppRole, KvEventOp, ScriptId, TriggerId, UserId};
|
||||
use std::collections::HashMap;
|
||||
use tokio::sync::Mutex;
|
||||
|
||||
#[derive(Default)]
|
||||
struct InMemoryTriggerRepo {
|
||||
inner: Mutex<HashMap<TriggerId, Trigger>>,
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl TriggerRepo for InMemoryTriggerRepo {
|
||||
async fn create_kv_trigger(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
req: CreateKvTrigger,
|
||||
) -> Result<Trigger, TriggerRepoError> {
|
||||
let now = Utc::now();
|
||||
let id = TriggerId::new();
|
||||
let trigger = Trigger {
|
||||
id,
|
||||
app_id,
|
||||
script_id: req.script_id,
|
||||
kind: crate::trigger_repo::TriggerKind::Kv,
|
||||
enabled: true,
|
||||
dispatch_mode: req.dispatch_mode,
|
||||
retry_max_attempts: req.retry_max_attempts,
|
||||
retry_backoff: req.retry_backoff,
|
||||
retry_base_ms: req.retry_base_ms,
|
||||
registered_by_principal: req.registered_by_principal,
|
||||
created_at: now,
|
||||
updated_at: now,
|
||||
details: TriggerDetails::Kv {
|
||||
collection_glob: req.collection_glob,
|
||||
ops: req.ops,
|
||||
},
|
||||
};
|
||||
self.inner.lock().await.insert(id, trigger.clone());
|
||||
Ok(trigger)
|
||||
}
|
||||
async fn create_dead_letter_trigger(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
req: CreateDeadLetterTrigger,
|
||||
) -> Result<Trigger, TriggerRepoError> {
|
||||
let now = Utc::now();
|
||||
let id = TriggerId::new();
|
||||
let trigger = Trigger {
|
||||
id,
|
||||
app_id,
|
||||
script_id: req.script_id,
|
||||
kind: crate::trigger_repo::TriggerKind::DeadLetter,
|
||||
enabled: true,
|
||||
dispatch_mode: TriggerDispatchMode::Async,
|
||||
retry_max_attempts: 1,
|
||||
retry_backoff: BackoffShape::Constant,
|
||||
retry_base_ms: 0,
|
||||
registered_by_principal: req.registered_by_principal,
|
||||
created_at: now,
|
||||
updated_at: now,
|
||||
details: TriggerDetails::DeadLetter {
|
||||
source_filter: req.source_filter,
|
||||
trigger_id_filter: req.trigger_id_filter,
|
||||
script_id_filter: req.script_id_filter,
|
||||
},
|
||||
};
|
||||
self.inner.lock().await.insert(id, trigger.clone());
|
||||
Ok(trigger)
|
||||
}
|
||||
async fn list_for_app(&self, app_id: AppId) -> Result<Vec<Trigger>, TriggerRepoError> {
|
||||
Ok(self
|
||||
.inner
|
||||
.lock()
|
||||
.await
|
||||
.values()
|
||||
.filter(|t| t.app_id == app_id)
|
||||
.cloned()
|
||||
.collect())
|
||||
}
|
||||
async fn get(&self, id: TriggerId) -> Result<Option<Trigger>, TriggerRepoError> {
|
||||
Ok(self.inner.lock().await.get(&id).cloned())
|
||||
}
|
||||
async fn delete(&self, id: TriggerId) -> Result<bool, TriggerRepoError> {
|
||||
Ok(self.inner.lock().await.remove(&id).is_some())
|
||||
}
|
||||
async fn list_matching_kv(
|
||||
&self,
|
||||
_app_id: AppId,
|
||||
_collection: &str,
|
||||
_op: KvEventOp,
|
||||
) -> Result<Vec<KvTriggerMatch>, TriggerRepoError> {
|
||||
Ok(vec![])
|
||||
}
|
||||
async fn list_matching_dead_letter(
|
||||
&self,
|
||||
_app_id: AppId,
|
||||
_source: &str,
|
||||
_trigger_id: Option<TriggerId>,
|
||||
_script_id: Option<ScriptId>,
|
||||
) -> Result<Vec<DeadLetterTriggerMatch>, TriggerRepoError> {
|
||||
Ok(vec![])
|
||||
}
|
||||
}
|
||||
|
||||
struct InMemoryAppRepo {
|
||||
existing: Mutex<HashMap<AppId, App>>,
|
||||
}
|
||||
|
||||
impl InMemoryAppRepo {
|
||||
fn with(app_id: AppId) -> Arc<Self> {
|
||||
let now = Utc::now();
|
||||
let mut existing = HashMap::new();
|
||||
existing.insert(
|
||||
app_id,
|
||||
App {
|
||||
id: app_id,
|
||||
slug: "test".into(),
|
||||
name: "test".into(),
|
||||
description: None,
|
||||
created_at: now,
|
||||
updated_at: now,
|
||||
},
|
||||
);
|
||||
Arc::new(Self {
|
||||
existing: Mutex::new(existing),
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl AppRepository for InMemoryAppRepo {
|
||||
async fn create(
|
||||
&self,
|
||||
_slug: &str,
|
||||
_name: &str,
|
||||
_description: Option<&str>,
|
||||
) -> Result<App, crate::repo::ScriptRepositoryError> {
|
||||
unimplemented!()
|
||||
}
|
||||
async fn create_with_takeover(
|
||||
&self,
|
||||
_slug: &str,
|
||||
_name: &str,
|
||||
_description: Option<&str>,
|
||||
) -> Result<App, crate::repo::ScriptRepositoryError> {
|
||||
unimplemented!()
|
||||
}
|
||||
async fn slug_in_history(
|
||||
&self,
|
||||
_slug: &str,
|
||||
) -> Result<Option<App>, crate::repo::ScriptRepositoryError> {
|
||||
unimplemented!()
|
||||
}
|
||||
async fn list(&self) -> Result<Vec<App>, crate::repo::ScriptRepositoryError> {
|
||||
unimplemented!()
|
||||
}
|
||||
async fn list_for_user(
|
||||
&self,
|
||||
_user_id: AdminUserId,
|
||||
) -> Result<Vec<App>, crate::repo::ScriptRepositoryError> {
|
||||
unimplemented!()
|
||||
}
|
||||
async fn get_by_id(
|
||||
&self,
|
||||
id: AppId,
|
||||
) -> Result<Option<App>, crate::repo::ScriptRepositoryError> {
|
||||
Ok(self.existing.lock().await.get(&id).cloned())
|
||||
}
|
||||
async fn get_by_slug(
|
||||
&self,
|
||||
_slug: &str,
|
||||
) -> Result<Option<App>, crate::repo::ScriptRepositoryError> {
|
||||
unimplemented!()
|
||||
}
|
||||
async fn get_by_slug_or_history(
|
||||
&self,
|
||||
_slug: &str,
|
||||
) -> Result<Option<AppLookup>, crate::repo::ScriptRepositoryError> {
|
||||
unimplemented!()
|
||||
}
|
||||
async fn update(
|
||||
&self,
|
||||
_id: AppId,
|
||||
_name: Option<&str>,
|
||||
_description: Option<Option<&str>>,
|
||||
) -> Result<App, crate::repo::ScriptRepositoryError> {
|
||||
unimplemented!()
|
||||
}
|
||||
async fn rename_slug(
|
||||
&self,
|
||||
_id: AppId,
|
||||
_new_slug: &str,
|
||||
_take_over_history: bool,
|
||||
) -> Result<App, crate::repo::ScriptRepositoryError> {
|
||||
unimplemented!()
|
||||
}
|
||||
async fn delete(&self, _id: AppId) -> Result<(), crate::repo::ScriptRepositoryError> {
|
||||
unimplemented!()
|
||||
}
|
||||
async fn delete_cascade(
|
||||
&self,
|
||||
_id: AppId,
|
||||
) -> Result<(), crate::repo::ScriptRepositoryError> {
|
||||
unimplemented!()
|
||||
}
|
||||
async fn count_scripts_in_app(
|
||||
&self,
|
||||
_id: AppId,
|
||||
) -> Result<i64, crate::repo::ScriptRepositoryError> {
|
||||
unimplemented!()
|
||||
}
|
||||
}
|
||||
|
||||
struct AlwaysAllowAuthzRepo;
|
||||
#[async_trait]
|
||||
impl AuthzRepo for AlwaysAllowAuthzRepo {
|
||||
async fn membership(
|
||||
&self,
|
||||
_user_id: UserId,
|
||||
_app_id: AppId,
|
||||
) -> Result<Option<AppRole>, AuthzError> {
|
||||
Ok(Some(AppRole::AppAdmin))
|
||||
}
|
||||
}
|
||||
|
||||
struct AlwaysDenyAuthzRepo;
|
||||
#[async_trait]
|
||||
impl AuthzRepo for AlwaysDenyAuthzRepo {
|
||||
async fn membership(
|
||||
&self,
|
||||
_user_id: UserId,
|
||||
_app_id: AppId,
|
||||
) -> Result<Option<AppRole>, AuthzError> {
|
||||
Ok(None)
|
||||
}
|
||||
}
|
||||
|
||||
fn member_principal() -> Principal {
|
||||
Principal {
|
||||
user_id: AdminUserId::new(),
|
||||
instance_role: picloud_shared::InstanceRole::Member,
|
||||
scopes: None,
|
||||
app_binding: None,
|
||||
}
|
||||
}
|
||||
|
||||
fn state_with(authz: Arc<dyn AuthzRepo>, app_id: AppId) -> TriggersState {
|
||||
TriggersState {
|
||||
triggers: Arc::new(InMemoryTriggerRepo::default()),
|
||||
apps: InMemoryAppRepo::with(app_id),
|
||||
authz,
|
||||
config: TriggerConfig::conservative(),
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn unknown_app_returns_404() {
|
||||
let state = state_with(Arc::new(AlwaysAllowAuthzRepo), AppId::new());
|
||||
let res = create_kv_trigger(
|
||||
State(state),
|
||||
Extension(member_principal()),
|
||||
Path(AppId::new()), // a different (non-existent) app
|
||||
Json(CreateKvTriggerRequest {
|
||||
script_id: ScriptId::new(),
|
||||
collection_glob: "*".into(),
|
||||
ops: vec![],
|
||||
dispatch_mode: TriggerDispatchMode::Async,
|
||||
retry_max_attempts: None,
|
||||
retry_backoff: None,
|
||||
retry_base_ms: None,
|
||||
}),
|
||||
)
|
||||
.await;
|
||||
let err = res.expect_err("missing app should error");
|
||||
assert!(matches!(err, TriggersApiError::AppNotFound(_)));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn member_without_role_is_forbidden() {
|
||||
let app_id = AppId::new();
|
||||
let state = state_with(Arc::new(AlwaysDenyAuthzRepo), app_id);
|
||||
let res = create_kv_trigger(
|
||||
State(state),
|
||||
Extension(member_principal()),
|
||||
Path(app_id),
|
||||
Json(CreateKvTriggerRequest {
|
||||
script_id: ScriptId::new(),
|
||||
collection_glob: "*".into(),
|
||||
ops: vec![],
|
||||
dispatch_mode: TriggerDispatchMode::Async,
|
||||
retry_max_attempts: None,
|
||||
retry_backoff: None,
|
||||
retry_base_ms: None,
|
||||
}),
|
||||
)
|
||||
.await;
|
||||
let err = res.expect_err("member without role should be forbidden");
|
||||
assert!(matches!(err, TriggersApiError::Forbidden));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn kv_trigger_uses_env_defaults_when_omitted() {
|
||||
let app_id = AppId::new();
|
||||
let mut state = state_with(Arc::new(AlwaysAllowAuthzRepo), app_id);
|
||||
// Tweak the config so we can detect that defaults were used.
|
||||
state.config.retry_max_attempts = 7;
|
||||
state.config.retry_base_ms = 12_345;
|
||||
let (status, Json(trigger)) = create_kv_trigger(
|
||||
State(state),
|
||||
Extension(member_principal()),
|
||||
Path(app_id),
|
||||
Json(CreateKvTriggerRequest {
|
||||
script_id: ScriptId::new(),
|
||||
collection_glob: "widgets".into(),
|
||||
ops: vec![KvEventOp::Insert],
|
||||
dispatch_mode: TriggerDispatchMode::Async,
|
||||
retry_max_attempts: None,
|
||||
retry_backoff: None,
|
||||
retry_base_ms: None,
|
||||
}),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(status, StatusCode::CREATED);
|
||||
assert_eq!(trigger.retry_max_attempts, 7);
|
||||
assert_eq!(trigger.retry_base_ms, 12_345);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn empty_collection_glob_rejected() {
|
||||
let app_id = AppId::new();
|
||||
let state = state_with(Arc::new(AlwaysAllowAuthzRepo), app_id);
|
||||
let res = create_kv_trigger(
|
||||
State(state),
|
||||
Extension(member_principal()),
|
||||
Path(app_id),
|
||||
Json(CreateKvTriggerRequest {
|
||||
script_id: ScriptId::new(),
|
||||
collection_glob: " ".into(),
|
||||
ops: vec![],
|
||||
dispatch_mode: TriggerDispatchMode::Async,
|
||||
retry_max_attempts: None,
|
||||
retry_backoff: None,
|
||||
retry_base_ms: None,
|
||||
}),
|
||||
)
|
||||
.await;
|
||||
let err = res.expect_err("empty glob should reject");
|
||||
assert!(matches!(err, TriggersApiError::Invalid(_)));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn delete_rejects_cross_app_trigger_id() {
|
||||
let app_a = AppId::new();
|
||||
let app_b = AppId::new();
|
||||
let state = state_with(Arc::new(AlwaysAllowAuthzRepo), app_a);
|
||||
// Inject the app_b row into the in-memory apps repo too so
|
||||
// the path-existence check succeeds against app_a.
|
||||
// Insert a trigger that belongs to app_a.
|
||||
let trigger = state
|
||||
.triggers
|
||||
.create_kv_trigger(
|
||||
app_a,
|
||||
CreateKvTrigger {
|
||||
script_id: ScriptId::new(),
|
||||
collection_glob: "*".into(),
|
||||
ops: vec![],
|
||||
dispatch_mode: TriggerDispatchMode::Async,
|
||||
retry_max_attempts: 3,
|
||||
retry_backoff: BackoffShape::Exponential,
|
||||
retry_base_ms: 1000,
|
||||
registered_by_principal: AdminUserId::new(),
|
||||
},
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
let _ = app_b;
|
||||
|
||||
// Attempt to delete via app_b's path — should 404.
|
||||
// First, give the in-memory app repo a record for app_b.
|
||||
// (Otherwise we'd 404 on app-existence before reaching the
|
||||
// cross-app check.)
|
||||
let state = TriggersState {
|
||||
apps: {
|
||||
let now = Utc::now();
|
||||
let mut existing = HashMap::new();
|
||||
existing.insert(
|
||||
app_a,
|
||||
App {
|
||||
id: app_a,
|
||||
slug: "a".into(),
|
||||
name: "a".into(),
|
||||
description: None,
|
||||
created_at: now,
|
||||
updated_at: now,
|
||||
},
|
||||
);
|
||||
existing.insert(
|
||||
app_b,
|
||||
App {
|
||||
id: app_b,
|
||||
slug: "b".into(),
|
||||
name: "b".into(),
|
||||
description: None,
|
||||
created_at: now,
|
||||
updated_at: now,
|
||||
},
|
||||
);
|
||||
Arc::new(InMemoryAppRepo {
|
||||
existing: Mutex::new(existing),
|
||||
})
|
||||
},
|
||||
..state
|
||||
};
|
||||
|
||||
let res = delete_trigger(
|
||||
State(state),
|
||||
Extension(member_principal()),
|
||||
Path((app_b, trigger.id)),
|
||||
)
|
||||
.await;
|
||||
let err = res.expect_err("cross-app delete should 404");
|
||||
assert!(matches!(err, TriggersApiError::NotFound(_)));
|
||||
}
|
||||
}
|
||||
@@ -12,17 +12,20 @@ use axum::{
|
||||
http::{HeaderMap, HeaderName, HeaderValue, StatusCode},
|
||||
response::{IntoResponse, Response},
|
||||
routing::post,
|
||||
Json, Router,
|
||||
Extension, Json, Router,
|
||||
};
|
||||
use chrono::Utc;
|
||||
use picloud_executor_core::{ExecError, ExecRequest, ExecResponse, InvocationType};
|
||||
use picloud_shared::{
|
||||
AppId, ExecutionId, ExecutionLog, ExecutionLogSink, ExecutionStatus, RequestId, ScriptId,
|
||||
AppId, DispatchMode, ExecutionId, ExecutionLog, ExecutionLogSink, ExecutionStatus,
|
||||
HttpDispatchPayload, InboxFailureKind, InboxResult, NewHttpOutbox, OutboxWriter, Principal,
|
||||
RequestId, ScriptId,
|
||||
};
|
||||
use serde_json::Value as Json_;
|
||||
use uuid::Uuid;
|
||||
|
||||
use crate::client::ExecutorClient;
|
||||
use crate::inbox::InboxRegistry;
|
||||
use crate::resolver::{ResolverError, ScriptResolver};
|
||||
use crate::routing::{AppDomainTable, RouteTable};
|
||||
|
||||
@@ -38,6 +41,14 @@ pub struct DataPlaneState<E, R> {
|
||||
/// Routing table for user-defined paths, partitioned per app.
|
||||
/// Shared with the manager (admin router writes; this side reads).
|
||||
pub routes: Arc<RouteTable>,
|
||||
/// NATS-style inbox registry (v1.1.1). Used by sync HTTP via
|
||||
/// outbox to await the dispatcher's delivery on a oneshot
|
||||
/// channel.
|
||||
pub inbox: Arc<InboxRegistry>,
|
||||
/// Writer for the universal trigger outbox (v1.1.1). The sync
|
||||
/// HTTP path inserts a row with `reply_to = inbox_id`; the async
|
||||
/// path inserts with `reply_to = None` and returns 202.
|
||||
pub outbox: Arc<dyn OutboxWriter>,
|
||||
}
|
||||
|
||||
impl<E, R> Clone for DataPlaneState<E, R> {
|
||||
@@ -48,12 +59,19 @@ impl<E, R> Clone for DataPlaneState<E, R> {
|
||||
log_sink: self.log_sink.clone(),
|
||||
app_domains: self.app_domains.clone(),
|
||||
routes: self.routes.clone(),
|
||||
inbox: self.inbox.clone(),
|
||||
outbox: self.outbox.clone(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Build the data-plane router. Handles `POST /execute/:id` — the
|
||||
/// always-available ID-based bypass.
|
||||
///
|
||||
/// Handlers expect an `Extension<Option<Principal>>` to be attached by
|
||||
/// upstream middleware (`manager-core::attach_principal_if_present`);
|
||||
/// requests without that extension panic at extraction time. The
|
||||
/// picloud binary wires this in `build_app`.
|
||||
pub fn data_plane_router<E, R>(state: DataPlaneState<E, R>) -> Router
|
||||
where
|
||||
E: ExecutorClient + 'static,
|
||||
@@ -67,6 +85,10 @@ where
|
||||
/// Build a router that handles ALL paths via the user-defined routing
|
||||
/// table. Intended to be merged into the picloud app router as a
|
||||
/// fallback (after the system routes are mounted).
|
||||
///
|
||||
/// Same middleware expectation as `data_plane_router` — wrap with
|
||||
/// `attach_principal_if_present` so handlers can extract
|
||||
/// `Extension<Option<Principal>>`.
|
||||
pub fn user_routes_router<E, R>(state: DataPlaneState<E, R>) -> Router
|
||||
where
|
||||
E: ExecutorClient + 'static,
|
||||
@@ -84,6 +106,7 @@ where
|
||||
async fn execute_by_id<E, R>(
|
||||
State(state): State<DataPlaneState<E, R>>,
|
||||
Path(id): Path<ScriptId>,
|
||||
Extension(principal): Extension<Option<Principal>>,
|
||||
headers: HeaderMap,
|
||||
body: Bytes,
|
||||
) -> Result<Response, ApiError>
|
||||
@@ -97,7 +120,7 @@ where
|
||||
.await?
|
||||
.ok_or(ApiError::NotFound(id))?;
|
||||
|
||||
let mut req = build_exec_request(id, &script.name, &headers, &body)?;
|
||||
let mut req = build_exec_request(id, &script.name, &headers, &body, script.app_id, principal)?;
|
||||
req.sandbox_overrides = script.sandbox;
|
||||
let request_id = req.request_id;
|
||||
let request_path = req.path.clone();
|
||||
@@ -133,6 +156,7 @@ where
|
||||
|
||||
async fn user_route_handler<E, R>(
|
||||
State(state): State<DataPlaneState<E, R>>,
|
||||
Extension(principal): Extension<Option<Principal>>,
|
||||
request: Request,
|
||||
) -> Result<Response, ApiError>
|
||||
where
|
||||
@@ -190,48 +214,312 @@ where
|
||||
Err(e) => return Err(ApiError::BadRequest(format!("body read failed: {e}"))),
|
||||
};
|
||||
|
||||
let mut req = build_exec_request(
|
||||
let body_json: Json_ = if body_bytes.is_empty() {
|
||||
Json_::Null
|
||||
} else {
|
||||
serde_json::from_slice(&body_bytes)
|
||||
.map_err(|e| ApiError::BadRequest(format!("invalid JSON body: {e}")))?
|
||||
};
|
||||
let header_map: BTreeMap<String, String> = headers
|
||||
.iter()
|
||||
.filter_map(|(k, v)| {
|
||||
v.to_str()
|
||||
.ok()
|
||||
.map(|s| (k.as_str().to_string(), s.to_string()))
|
||||
})
|
||||
.collect();
|
||||
let query = parse_query_string(&query_str);
|
||||
let rest = matched.rest.clone().unwrap_or_default();
|
||||
|
||||
match matched.matched.dispatch_mode {
|
||||
DispatchMode::Async => {
|
||||
handle_async_route(
|
||||
&state,
|
||||
app_id,
|
||||
matched.matched.route_id,
|
||||
matched.matched.script_id,
|
||||
&script.name,
|
||||
&headers,
|
||||
&body_bytes,
|
||||
)?;
|
||||
req.path = path;
|
||||
req.params = matched.params;
|
||||
req.query = parse_query_string(&query_str);
|
||||
req.rest = matched.rest.unwrap_or_default();
|
||||
req.sandbox_overrides = script.sandbox;
|
||||
path,
|
||||
method,
|
||||
header_map,
|
||||
body_json,
|
||||
matched.params,
|
||||
query,
|
||||
rest,
|
||||
script.timeout_seconds,
|
||||
principal,
|
||||
)
|
||||
.await
|
||||
}
|
||||
DispatchMode::Sync => {
|
||||
handle_sync_route(
|
||||
&state,
|
||||
app_id,
|
||||
matched.matched.route_id,
|
||||
matched.matched.script_id,
|
||||
&script.name,
|
||||
path,
|
||||
method,
|
||||
header_map,
|
||||
body_json,
|
||||
matched.params,
|
||||
query,
|
||||
rest,
|
||||
script.timeout_seconds,
|
||||
principal,
|
||||
)
|
||||
.await
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let request_id = req.request_id;
|
||||
let request_path = req.path.clone();
|
||||
let request_headers = req.headers.clone();
|
||||
let request_body = req.body.clone();
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
async fn handle_async_route<E, R>(
|
||||
state: &DataPlaneState<E, R>,
|
||||
app_id: AppId,
|
||||
route_id: Uuid,
|
||||
script_id: ScriptId,
|
||||
script_name: &str,
|
||||
path: String,
|
||||
method: String,
|
||||
headers: BTreeMap<String, String>,
|
||||
body: Json_,
|
||||
params: BTreeMap<String, String>,
|
||||
query: BTreeMap<String, String>,
|
||||
rest: String,
|
||||
timeout_seconds: u32,
|
||||
principal: Option<Principal>,
|
||||
) -> Result<Response, ApiError>
|
||||
where
|
||||
E: ExecutorClient + 'static,
|
||||
R: ScriptResolver + 'static,
|
||||
{
|
||||
let payload = HttpDispatchPayload {
|
||||
script_name: script_name.to_string(),
|
||||
path,
|
||||
method,
|
||||
headers,
|
||||
body,
|
||||
params,
|
||||
query,
|
||||
rest,
|
||||
timeout_seconds,
|
||||
};
|
||||
let payload_value = serde_json::to_value(&payload)
|
||||
.map_err(|e| ApiError::BadRequest(format!("payload serialize: {e}")))?;
|
||||
let execution_id = ExecutionId::new();
|
||||
state
|
||||
.outbox
|
||||
.enqueue_http(NewHttpOutbox {
|
||||
app_id,
|
||||
route_id,
|
||||
script_id,
|
||||
reply_to: None,
|
||||
payload: payload_value,
|
||||
origin_principal: principal.map(|p| p.user_id),
|
||||
trigger_depth: 0,
|
||||
root_execution_id: Some(execution_id),
|
||||
})
|
||||
.await
|
||||
.map_err(|e| ApiError::OutboxWrite(e.to_string()))?;
|
||||
Ok((
|
||||
StatusCode::ACCEPTED,
|
||||
Json(serde_json::json!({
|
||||
"accepted_at": Utc::now().to_rfc3339(),
|
||||
"execution_id": execution_id.to_string(),
|
||||
})),
|
||||
)
|
||||
.into_response())
|
||||
}
|
||||
|
||||
let timeout = Duration::from_secs(u64::from(script.timeout_seconds));
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
async fn handle_sync_route<E, R>(
|
||||
state: &DataPlaneState<E, R>,
|
||||
app_id: AppId,
|
||||
route_id: Uuid,
|
||||
script_id: ScriptId,
|
||||
script_name: &str,
|
||||
path: String,
|
||||
method: String,
|
||||
headers: BTreeMap<String, String>,
|
||||
body: Json_,
|
||||
params: BTreeMap<String, String>,
|
||||
query: BTreeMap<String, String>,
|
||||
rest: String,
|
||||
timeout_seconds: u32,
|
||||
principal: Option<Principal>,
|
||||
) -> Result<Response, ApiError>
|
||||
where
|
||||
E: ExecutorClient + 'static,
|
||||
R: ScriptResolver + 'static,
|
||||
{
|
||||
let payload = HttpDispatchPayload {
|
||||
script_name: script_name.to_string(),
|
||||
path: path.clone(),
|
||||
method,
|
||||
headers: headers.clone(),
|
||||
body: body.clone(),
|
||||
params,
|
||||
query,
|
||||
rest,
|
||||
timeout_seconds,
|
||||
};
|
||||
let payload_value = serde_json::to_value(&payload)
|
||||
.map_err(|e| ApiError::BadRequest(format!("payload serialize: {e}")))?;
|
||||
|
||||
// Register the inbox before writing the outbox row so the
|
||||
// dispatcher can't race-deliver before the orchestrator is
|
||||
// listening.
|
||||
let (inbox_id, rx) = state.inbox.register();
|
||||
|
||||
let execution_id = ExecutionId::new();
|
||||
let outbox_id = state
|
||||
.outbox
|
||||
.enqueue_http(NewHttpOutbox {
|
||||
app_id,
|
||||
route_id,
|
||||
script_id,
|
||||
reply_to: Some(inbox_id),
|
||||
payload: payload_value,
|
||||
origin_principal: principal.map(|p| p.user_id),
|
||||
trigger_depth: 0,
|
||||
root_execution_id: Some(execution_id),
|
||||
})
|
||||
.await
|
||||
.map_err(|e| {
|
||||
// Failed outbox write — abandon the inbox so the dispatcher
|
||||
// can never deliver to a stale entry.
|
||||
state.inbox.cancel(inbox_id);
|
||||
ApiError::OutboxWrite(e.to_string())
|
||||
})?;
|
||||
|
||||
// Wait for the dispatcher's delivery. Outer timeout = script
|
||||
// wall-clock + a small buffer to cover dispatcher latency.
|
||||
let wait_budget = Duration::from_secs(u64::from(timeout_seconds)) + Duration::from_secs(2);
|
||||
let request_id = RequestId::new();
|
||||
let started = Utc::now();
|
||||
let outcome = state.executor.execute(&script.source, req, timeout).await;
|
||||
let result = tokio::time::timeout(wait_budget, rx).await;
|
||||
let finished = Utc::now();
|
||||
|
||||
let log = build_execution_log(
|
||||
script.app_id,
|
||||
matched.matched.script_id,
|
||||
// Tear down the receiver if it's still alive. `inbox.cancel` is a
|
||||
// no-op when the dispatcher already delivered.
|
||||
let _ = state.inbox.cancel(inbox_id);
|
||||
|
||||
let response = match result {
|
||||
Ok(Ok(InboxResult::Success(summary))) => http_response_from_summary(summary),
|
||||
Ok(Ok(InboxResult::Failure { kind, message })) => failure_to_response(kind, &message),
|
||||
Ok(Err(_recv)) => {
|
||||
// Channel was closed without a value — dispatcher dropped
|
||||
// the sender. Treat as platform failure.
|
||||
tracing::warn!(
|
||||
outbox_id = %outbox_id,
|
||||
"inbox channel closed without delivery"
|
||||
);
|
||||
failure_to_response(
|
||||
InboxFailureKind::Platform,
|
||||
"dispatcher closed inbox without delivery",
|
||||
)
|
||||
}
|
||||
Err(_elapsed) => {
|
||||
// Outer timeout — either the script was too slow or the
|
||||
// dispatcher is wedged. Returns 504 by default.
|
||||
failure_to_response(InboxFailureKind::Timeout, "request timed out")
|
||||
}
|
||||
};
|
||||
|
||||
let log = build_inbox_execution_log(
|
||||
app_id,
|
||||
script_id,
|
||||
request_id,
|
||||
request_path,
|
||||
request_headers,
|
||||
request_body,
|
||||
&outcome,
|
||||
path,
|
||||
headers,
|
||||
body,
|
||||
response.status().as_u16(),
|
||||
started,
|
||||
finished,
|
||||
);
|
||||
if let Err(e) = state.log_sink.record(log).await {
|
||||
tracing::warn!(
|
||||
error = %e,
|
||||
script_id = %matched.matched.script_id,
|
||||
%script_id,
|
||||
"failed to persist execution log"
|
||||
);
|
||||
}
|
||||
|
||||
Ok(exec_response_to_http(outcome?))
|
||||
Ok(response)
|
||||
}
|
||||
|
||||
fn http_response_from_summary(summary: picloud_shared::ExecResponseSummary) -> Response {
|
||||
let status =
|
||||
StatusCode::from_u16(summary.status_code).unwrap_or(StatusCode::INTERNAL_SERVER_ERROR);
|
||||
let mut http_headers = HeaderMap::new();
|
||||
for (k, v) in summary.headers {
|
||||
if let (Ok(name), Ok(value)) = (k.parse::<HeaderName>(), v.parse::<HeaderValue>()) {
|
||||
http_headers.insert(name, value);
|
||||
}
|
||||
}
|
||||
http_headers
|
||||
.entry(axum::http::header::CONTENT_TYPE)
|
||||
.or_insert_with(|| HeaderValue::from_static("application/json"));
|
||||
(status, http_headers, Json(summary.body)).into_response()
|
||||
}
|
||||
|
||||
/// Map `InboxFailureKind` onto the design-notes §3 status-code table.
|
||||
fn failure_to_response(kind: InboxFailureKind, message: &str) -> Response {
|
||||
let status = match kind {
|
||||
InboxFailureKind::Validation => StatusCode::UNPROCESSABLE_ENTITY,
|
||||
InboxFailureKind::Runtime => StatusCode::BAD_GATEWAY,
|
||||
InboxFailureKind::Overloaded => StatusCode::SERVICE_UNAVAILABLE,
|
||||
InboxFailureKind::Timeout => StatusCode::GATEWAY_TIMEOUT,
|
||||
InboxFailureKind::OperationBudget => StatusCode::INSUFFICIENT_STORAGE,
|
||||
InboxFailureKind::Platform => StatusCode::INTERNAL_SERVER_ERROR,
|
||||
};
|
||||
let body = Json(serde_json::json!({ "error": message }));
|
||||
if matches!(kind, InboxFailureKind::Overloaded) {
|
||||
return (status, [(axum::http::header::RETRY_AFTER, "1")], body).into_response();
|
||||
}
|
||||
(status, body).into_response()
|
||||
}
|
||||
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
fn build_inbox_execution_log(
|
||||
app_id: AppId,
|
||||
script_id: ScriptId,
|
||||
request_id: RequestId,
|
||||
request_path: String,
|
||||
request_headers: BTreeMap<String, String>,
|
||||
request_body: Json_,
|
||||
response_code: u16,
|
||||
started: chrono::DateTime<Utc>,
|
||||
finished: chrono::DateTime<Utc>,
|
||||
) -> ExecutionLog {
|
||||
let duration_ms = u64::try_from(
|
||||
finished
|
||||
.signed_duration_since(started)
|
||||
.num_milliseconds()
|
||||
.max(0),
|
||||
)
|
||||
.unwrap_or(0);
|
||||
let status = if (200..400).contains(&response_code) {
|
||||
ExecutionStatus::Success
|
||||
} else {
|
||||
ExecutionStatus::Error
|
||||
};
|
||||
ExecutionLog {
|
||||
id: Uuid::new_v4(),
|
||||
app_id,
|
||||
script_id,
|
||||
request_id,
|
||||
request_path,
|
||||
request_headers,
|
||||
request_body,
|
||||
response_code: Some(response_code),
|
||||
response_body: None,
|
||||
script_logs: Json_::Array(vec![]),
|
||||
duration_ms,
|
||||
status,
|
||||
created_at: started,
|
||||
}
|
||||
}
|
||||
|
||||
fn parse_query_string(s: &str) -> BTreeMap<String, String> {
|
||||
@@ -264,6 +552,8 @@ fn build_exec_request(
|
||||
name: &str,
|
||||
headers: &HeaderMap,
|
||||
body: &Bytes,
|
||||
app_id: AppId,
|
||||
principal: Option<Principal>,
|
||||
) -> Result<ExecRequest, ApiError> {
|
||||
let mut hmap = BTreeMap::new();
|
||||
for (k, v) in headers {
|
||||
@@ -279,8 +569,9 @@ fn build_exec_request(
|
||||
.map_err(|e| ApiError::BadRequest(format!("invalid JSON body: {e}")))?
|
||||
};
|
||||
|
||||
let execution_id = ExecutionId::new();
|
||||
Ok(ExecRequest {
|
||||
execution_id: ExecutionId::new(),
|
||||
execution_id,
|
||||
request_id: RequestId::new(),
|
||||
script_id: id,
|
||||
script_name: name.to_string(),
|
||||
@@ -293,6 +584,18 @@ fn build_exec_request(
|
||||
rest: String::new(),
|
||||
// Overwritten by the handler after the script is resolved.
|
||||
sandbox_overrides: picloud_shared::ScriptSandbox::default(),
|
||||
app_id,
|
||||
principal,
|
||||
// Direct invocations are at depth 0 with a self-referential
|
||||
// root. The triggers framework (v1.1.1) increments depth and
|
||||
// preserves the original root for chained executions.
|
||||
trigger_depth: 0,
|
||||
root_execution_id: execution_id,
|
||||
// Direct invocations are never DL handlers — that flag is only
|
||||
// set by the dispatcher when it picks a dead_letter trigger row.
|
||||
is_dead_letter_handler: false,
|
||||
// No originating trigger event for direct ingress.
|
||||
event: None,
|
||||
})
|
||||
}
|
||||
|
||||
@@ -392,14 +695,39 @@ pub enum ApiError {
|
||||
|
||||
#[error("execution error: {0}")]
|
||||
Exec(#[from] ExecError),
|
||||
|
||||
#[error("outbox write failed: {0}")]
|
||||
OutboxWrite(String),
|
||||
}
|
||||
|
||||
impl IntoResponse for ApiError {
|
||||
fn into_response(self) -> Response {
|
||||
// Overloaded is the only variant that needs to attach an HTTP
|
||||
// header (Retry-After), so it short-circuits the (status, body)
|
||||
// reduction below. Axum's tuple builder makes per-arm header
|
||||
// injection awkward otherwise.
|
||||
use ApiError as E;
|
||||
if let E::Exec(ExecError::Overloaded { retry_after_secs }) = &self {
|
||||
let retry = retry_after_secs.to_string();
|
||||
let body = Json(serde_json::json!({ "error": self.to_string() }));
|
||||
return (
|
||||
StatusCode::SERVICE_UNAVAILABLE,
|
||||
[(axum::http::header::RETRY_AFTER, retry)],
|
||||
body,
|
||||
)
|
||||
.into_response();
|
||||
}
|
||||
|
||||
let (status, message) = match &self {
|
||||
E::NotFound(_) => (StatusCode::NOT_FOUND, self.to_string()),
|
||||
E::BadRequest(_) => (StatusCode::BAD_REQUEST, self.to_string()),
|
||||
E::OutboxWrite(e) => {
|
||||
tracing::error!(error = %e, "outbox write failed");
|
||||
(
|
||||
StatusCode::INTERNAL_SERVER_ERROR,
|
||||
"internal error".to_string(),
|
||||
)
|
||||
}
|
||||
E::Resolver(e) => {
|
||||
tracing::error!(error = %e, "resolver failure");
|
||||
(
|
||||
@@ -416,6 +744,7 @@ impl IntoResponse for ApiError {
|
||||
(StatusCode::INSUFFICIENT_STORAGE, e.to_string())
|
||||
}
|
||||
ExecError::Runtime(_) => (StatusCode::BAD_GATEWAY, e.to_string()),
|
||||
ExecError::Overloaded { .. } => unreachable!("handled above"),
|
||||
},
|
||||
};
|
||||
(status, Json(serde_json::json!({ "error": message }))).into_response()
|
||||
|
||||
@@ -4,6 +4,8 @@ use std::time::Duration;
|
||||
use async_trait::async_trait;
|
||||
use picloud_executor_core::{Engine, ExecError, ExecRequest, ExecResponse};
|
||||
|
||||
use crate::gate::{AcquireError, ExecutionGate};
|
||||
|
||||
/// Maximum wall-clock time we'll wait for a single invocation, regardless
|
||||
/// of the per-script `timeout_seconds`. Provides a hard ceiling on
|
||||
/// resource usage independent of misconfigured scripts.
|
||||
@@ -30,14 +32,19 @@ pub trait ExecutorClient: Send + Sync {
|
||||
/// `executor-core::Engine::execute` is synchronous; we offload it to a
|
||||
/// blocking thread so it doesn't park a Tokio worker, and apply the
|
||||
/// wall-clock timeout here.
|
||||
///
|
||||
/// Holds an `ExecutionGate` and acquires a permit before `spawn_blocking`
|
||||
/// so a script storm can't drain the blocking-thread pool. The permit
|
||||
/// drops with the future, returning the slot.
|
||||
pub struct LocalExecutorClient {
|
||||
engine: Arc<Engine>,
|
||||
gate: Arc<ExecutionGate>,
|
||||
}
|
||||
|
||||
impl LocalExecutorClient {
|
||||
#[must_use]
|
||||
pub fn new(engine: Arc<Engine>) -> Self {
|
||||
Self { engine }
|
||||
pub fn new(engine: Arc<Engine>, gate: Arc<ExecutionGate>) -> Self {
|
||||
Self { engine, gate }
|
||||
}
|
||||
}
|
||||
|
||||
@@ -49,6 +56,24 @@ impl ExecutorClient for LocalExecutorClient {
|
||||
req: ExecRequest,
|
||||
timeout: Duration,
|
||||
) -> Result<ExecResponse, ExecError> {
|
||||
// Acquire before spending any wall-clock budget. The permit is
|
||||
// held by this future; on `tokio::time::timeout` firing, the
|
||||
// future drops and the permit returns to the pool — but the
|
||||
// detached `spawn_blocking` thread keeps running until the
|
||||
// Rhai script finishes (or panics). So in-use blocking threads
|
||||
// can briefly exceed the gate's permit count after a timeout.
|
||||
// That is intentional: a new admission can be served while the
|
||||
// already-doomed script winds down, which is preferable to
|
||||
// wedging the slot for the worst-case timeout duration.
|
||||
let _permit =
|
||||
self.gate
|
||||
.try_acquire()
|
||||
.map_err(
|
||||
|AcquireError::Overloaded { retry_after_secs }| ExecError::Overloaded {
|
||||
retry_after_secs,
|
||||
},
|
||||
)?;
|
||||
|
||||
let timeout = timeout.min(HARD_TIMEOUT_CAP);
|
||||
let timeout_secs = u32::try_from(timeout.as_secs()).unwrap_or(u32::MAX);
|
||||
|
||||
|
||||
155
crates/orchestrator-core/src/gate.rs
Normal file
155
crates/orchestrator-core/src/gate.rs
Normal file
@@ -0,0 +1,155 @@
|
||||
//! Global concurrency gate for the data plane.
|
||||
//!
|
||||
//! Wraps a single `tokio::sync::Semaphore` so the executor can refuse
|
||||
//! admission immediately when too many invocations are already in
|
||||
//! flight. Designed for v1.1.0's single-node MVP — one cap across all
|
||||
//! apps and scripts. Per-app or per-script caps come later when a real
|
||||
//! workload surfaces the need.
|
||||
//!
|
||||
//! Policy: **non-blocking, no queue**. If a permit isn't free right
|
||||
//! now, the call returns `AcquireError::Overloaded` and the data-plane
|
||||
//! HTTP layer translates that to a 503 with `Retry-After: 1`. Pushing
|
||||
//! back hard beats letting requests pile up against a finite pool of
|
||||
//! blocking threads (executor work runs under `spawn_blocking`).
|
||||
//!
|
||||
//! Configured via the `PICLOUD_MAX_CONCURRENT_EXECUTIONS` env var.
|
||||
//! Default is 32 — comfortable for a single-node Pi, low enough that
|
||||
//! a script storm doesn't park every blocking thread.
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use thiserror::Error;
|
||||
use tokio::sync::{OwnedSemaphorePermit, Semaphore, TryAcquireError};
|
||||
|
||||
/// Env var consulted by `from_env`.
|
||||
pub const ENV_MAX_CONCURRENT: &str = "PICLOUD_MAX_CONCURRENT_EXECUTIONS";
|
||||
|
||||
/// Default cap when the env var is unset or invalid.
|
||||
pub const DEFAULT_MAX_CONCURRENT: u32 = 32;
|
||||
|
||||
/// `Retry-After` header value (seconds) returned alongside the 503
|
||||
/// when the gate refuses. Fixed for v1.1.0; later versions may compute
|
||||
/// a smarter value from in-flight latency.
|
||||
pub const DEFAULT_RETRY_AFTER_SECS: u32 = 1;
|
||||
|
||||
/// Refused admission. The HTTP layer translates this to 503 with a
|
||||
/// `Retry-After` header.
|
||||
#[derive(Debug, Error)]
|
||||
pub enum AcquireError {
|
||||
#[error("at capacity (retry after {retry_after_secs}s)")]
|
||||
Overloaded { retry_after_secs: u32 },
|
||||
}
|
||||
|
||||
/// Global execution gate. Constructed once at orchestrator startup and
|
||||
/// shared via `Arc`. Holds an inner `Arc<Semaphore>` so permits are
|
||||
/// owned (they release on drop independent of the gate's lifetime).
|
||||
pub struct ExecutionGate {
|
||||
permits: Arc<Semaphore>,
|
||||
max_permits: u32,
|
||||
}
|
||||
|
||||
impl ExecutionGate {
|
||||
/// Construct with an explicit cap. Mostly for tests; production
|
||||
/// uses `from_env`.
|
||||
#[must_use]
|
||||
pub fn new(max_permits: u32) -> Self {
|
||||
Self {
|
||||
permits: Arc::new(Semaphore::new(max_permits as usize)),
|
||||
max_permits,
|
||||
}
|
||||
}
|
||||
|
||||
/// Read `PICLOUD_MAX_CONCURRENT_EXECUTIONS` from the environment.
|
||||
/// Falls back to `DEFAULT_MAX_CONCURRENT` on absence; warns and
|
||||
/// falls back on parse failure or non-positive value. Mirrors the
|
||||
/// `SandboxCeiling::from_env` ergonomics so operators see a
|
||||
/// consistent shape across the env-tunables.
|
||||
#[must_use]
|
||||
pub fn from_env() -> Self {
|
||||
let max = match std::env::var(ENV_MAX_CONCURRENT) {
|
||||
Err(_) => DEFAULT_MAX_CONCURRENT,
|
||||
Ok(v) => match v.parse::<u32>() {
|
||||
Ok(n) if n > 0 => n,
|
||||
Ok(_) => {
|
||||
tracing::warn!(
|
||||
env = ENV_MAX_CONCURRENT,
|
||||
value = %v,
|
||||
"value must be > 0; using default {DEFAULT_MAX_CONCURRENT}"
|
||||
);
|
||||
DEFAULT_MAX_CONCURRENT
|
||||
}
|
||||
Err(e) => {
|
||||
tracing::warn!(
|
||||
env = ENV_MAX_CONCURRENT,
|
||||
value = %v,
|
||||
error = %e,
|
||||
"invalid value; using default {DEFAULT_MAX_CONCURRENT}"
|
||||
);
|
||||
DEFAULT_MAX_CONCURRENT
|
||||
}
|
||||
},
|
||||
};
|
||||
Self::new(max)
|
||||
}
|
||||
|
||||
/// Maximum concurrent permits this gate was configured for. Useful
|
||||
/// for diagnostics / future metrics.
|
||||
#[must_use]
|
||||
pub fn max_permits(&self) -> u32 {
|
||||
self.max_permits
|
||||
}
|
||||
|
||||
/// Non-blocking permit acquisition. Returns the owned permit on
|
||||
/// success (drop releases the slot) or `AcquireError::Overloaded`
|
||||
/// when saturated. Sync because the semaphore's non-blocking try is
|
||||
/// sync — no runtime hop needed.
|
||||
pub fn try_acquire(&self) -> Result<OwnedSemaphorePermit, AcquireError> {
|
||||
self.permits
|
||||
.clone()
|
||||
.try_acquire_owned()
|
||||
.map_err(|err| match err {
|
||||
TryAcquireError::NoPermits | TryAcquireError::Closed => AcquireError::Overloaded {
|
||||
retry_after_secs: DEFAULT_RETRY_AFTER_SECS,
|
||||
},
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn acquire_succeeds_under_capacity() {
|
||||
let gate = ExecutionGate::new(2);
|
||||
let _p1 = gate.try_acquire().expect("first permit available");
|
||||
let _p2 = gate.try_acquire().expect("second permit available");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn acquire_overloaded_when_saturated() {
|
||||
let gate = ExecutionGate::new(1);
|
||||
let _p = gate.try_acquire().expect("first permit available");
|
||||
let AcquireError::Overloaded { retry_after_secs } = gate
|
||||
.try_acquire()
|
||||
.expect_err("second permit must be refused");
|
||||
assert!(retry_after_secs > 0, "retry-after must be positive");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn permit_drop_releases_slot() {
|
||||
let gate = ExecutionGate::new(1);
|
||||
{
|
||||
let _p = gate.try_acquire().expect("first permit available");
|
||||
}
|
||||
let _ = gate
|
||||
.try_acquire()
|
||||
.expect("slot must be returned after permit drops");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn max_permits_exposed() {
|
||||
let gate = ExecutionGate::new(7);
|
||||
assert_eq!(gate.max_permits(), 7);
|
||||
}
|
||||
}
|
||||
139
crates/orchestrator-core/src/inbox.rs
Normal file
139
crates/orchestrator-core/src/inbox.rs
Normal file
@@ -0,0 +1,139 @@
|
||||
//! In-process `InboxRegistry` — the NATS-style request/reply
|
||||
//! implementation for sync HTTP via the trigger outbox (design notes
|
||||
//! §3).
|
||||
//!
|
||||
//! Workflow:
|
||||
//! 1. Orchestrator allocates an `inbox_id`, calls
|
||||
//! `registry.register()` to get a oneshot receiver.
|
||||
//! 2. Orchestrator writes an outbox row with `reply_to = inbox_id`.
|
||||
//! 3. Dispatcher picks the row, runs the script, calls
|
||||
//! `registry.deliver(inbox_id, result)`.
|
||||
//! 4. Orchestrator's `.await` on the receiver fires; it maps the
|
||||
//! `InboxResult` back into an HTTP response.
|
||||
//!
|
||||
//! `Delivered` means the receiver was alive when delivery hit. If the
|
||||
//! orchestrator timed out and dropped the receiver before delivery,
|
||||
//! `Abandoned` comes back — the dispatcher writes an
|
||||
//! `abandoned_executions` row (design notes §3 #9).
|
||||
//!
|
||||
//! Cluster mode (v1.3+) swaps this for a Postgres `LISTEN/NOTIFY`-
|
||||
//! based resolver; the `InboxResolver` trait stays the same.
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::sync::Mutex;
|
||||
|
||||
use async_trait::async_trait;
|
||||
use picloud_shared::{InboxDeliveryOutcome, InboxResolver, InboxResult};
|
||||
use tokio::sync::oneshot;
|
||||
use uuid::Uuid;
|
||||
|
||||
pub struct InboxRegistry {
|
||||
inner: Mutex<HashMap<Uuid, oneshot::Sender<InboxResult>>>,
|
||||
}
|
||||
|
||||
impl InboxRegistry {
|
||||
#[must_use]
|
||||
pub fn new() -> Self {
|
||||
Self {
|
||||
inner: Mutex::new(HashMap::new()),
|
||||
}
|
||||
}
|
||||
|
||||
/// Allocate a new inbox id and register the sender side. The
|
||||
/// caller awaits the returned `Receiver`; the dispatcher delivers
|
||||
/// the outcome via `deliver(id, …)`.
|
||||
#[must_use]
|
||||
pub fn register(&self) -> (Uuid, oneshot::Receiver<InboxResult>) {
|
||||
let id = Uuid::new_v4();
|
||||
let (tx, rx) = oneshot::channel();
|
||||
if let Ok(mut g) = self.inner.lock() {
|
||||
g.insert(id, tx);
|
||||
}
|
||||
(id, rx)
|
||||
}
|
||||
|
||||
/// Cancel a pending inbox (orchestrator timed out and gave up).
|
||||
/// Drops the sender so any future `deliver` returns `Abandoned`.
|
||||
/// Returns `true` if the receiver was still registered.
|
||||
pub fn cancel(&self, id: Uuid) -> bool {
|
||||
self.inner
|
||||
.lock()
|
||||
.map(|mut g| g.remove(&id).is_some())
|
||||
.unwrap_or(false)
|
||||
}
|
||||
}
|
||||
|
||||
impl Default for InboxRegistry {
|
||||
fn default() -> Self {
|
||||
Self::new()
|
||||
}
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl InboxResolver for InboxRegistry {
|
||||
async fn deliver(&self, inbox_id: Uuid, result: InboxResult) -> InboxDeliveryOutcome {
|
||||
let Ok(mut g) = self.inner.lock() else {
|
||||
return InboxDeliveryOutcome::Abandoned;
|
||||
};
|
||||
let Some(tx) = g.remove(&inbox_id) else {
|
||||
return InboxDeliveryOutcome::Abandoned;
|
||||
};
|
||||
// `send` returns Err iff the receiver was dropped — exactly
|
||||
// the abandoned-execution case.
|
||||
if tx.send(result).is_err() {
|
||||
InboxDeliveryOutcome::Abandoned
|
||||
} else {
|
||||
InboxDeliveryOutcome::Delivered
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use picloud_shared::ExecResponseSummary;
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
fn ok_result() -> InboxResult {
|
||||
InboxResult::Success(ExecResponseSummary {
|
||||
status_code: 200,
|
||||
headers: BTreeMap::new(),
|
||||
body: serde_json::json!({ "ok": true }),
|
||||
})
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn register_then_deliver_resolves_receiver() {
|
||||
let reg = InboxRegistry::new();
|
||||
let (id, rx) = reg.register();
|
||||
let outcome = reg.deliver(id, ok_result()).await;
|
||||
assert_eq!(outcome, InboxDeliveryOutcome::Delivered);
|
||||
let received = rx.await.expect("receiver should fire");
|
||||
assert!(matches!(received, InboxResult::Success(_)));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn deliver_to_unknown_id_is_abandoned() {
|
||||
let reg = InboxRegistry::new();
|
||||
let outcome = reg.deliver(Uuid::new_v4(), ok_result()).await;
|
||||
assert_eq!(outcome, InboxDeliveryOutcome::Abandoned);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn dropping_receiver_then_delivering_is_abandoned() {
|
||||
let reg = InboxRegistry::new();
|
||||
let (id, rx) = reg.register();
|
||||
drop(rx);
|
||||
let outcome = reg.deliver(id, ok_result()).await;
|
||||
assert_eq!(outcome, InboxDeliveryOutcome::Abandoned);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn cancel_removes_sender() {
|
||||
let reg = InboxRegistry::new();
|
||||
let (id, _rx) = reg.register();
|
||||
assert!(reg.cancel(id));
|
||||
let outcome = reg.deliver(id, ok_result()).await;
|
||||
assert_eq!(outcome, InboxDeliveryOutcome::Abandoned);
|
||||
}
|
||||
}
|
||||
@@ -10,9 +10,13 @@
|
||||
|
||||
pub mod api;
|
||||
pub mod client;
|
||||
pub mod gate;
|
||||
pub mod inbox;
|
||||
pub mod resolver;
|
||||
pub mod routing;
|
||||
|
||||
pub use api::{data_plane_router, user_routes_router, DataPlaneState};
|
||||
pub use client::{ExecutorClient, LocalExecutorClient, RemoteExecutorClient};
|
||||
pub use gate::{AcquireError, ExecutionGate};
|
||||
pub use inbox::InboxRegistry;
|
||||
pub use resolver::{ResolverError, ScriptResolver};
|
||||
|
||||
@@ -38,6 +38,11 @@ pub struct MatchResult {
|
||||
pub struct Matched {
|
||||
pub route_id: uuid::Uuid,
|
||||
pub script_id: picloud_shared::ScriptId,
|
||||
/// Per-route dispatch mode (v1.1.1). Forwarded to the
|
||||
/// orchestrator's HTTP handler so it can pick the sync or async
|
||||
/// path. Defaults to `Sync` for older routes that predate the
|
||||
/// column.
|
||||
pub dispatch_mode: picloud_shared::DispatchMode,
|
||||
}
|
||||
|
||||
/// A single route ready for matching. `app_id` is carried so the
|
||||
@@ -51,6 +56,7 @@ pub struct CompiledRoute {
|
||||
pub host: HostPattern,
|
||||
pub path: PathPattern,
|
||||
pub method: Option<String>,
|
||||
pub dispatch_mode: picloud_shared::DispatchMode,
|
||||
}
|
||||
|
||||
/// Find the best matching route for the request. Returns `None` if no
|
||||
@@ -180,6 +186,7 @@ fn match_within_bucket(
|
||||
matched: Matched {
|
||||
route_id: route.route_id,
|
||||
script_id: route.script_id,
|
||||
dispatch_mode: route.dispatch_mode,
|
||||
},
|
||||
params: BTreeMap::new(),
|
||||
rest: None,
|
||||
@@ -230,6 +237,7 @@ fn match_within_bucket(
|
||||
matched: Matched {
|
||||
route_id: route.route_id,
|
||||
script_id: route.script_id,
|
||||
dispatch_mode: route.dispatch_mode,
|
||||
},
|
||||
params,
|
||||
rest,
|
||||
@@ -312,6 +320,7 @@ mod tests {
|
||||
host,
|
||||
path: parse_path(path_kind, raw).unwrap(),
|
||||
method: None,
|
||||
dispatch_mode: picloud_shared::DispatchMode::Sync,
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -27,6 +27,7 @@ reqwest = { workspace = true, features = ["json"] }
|
||||
serde.workspace = true
|
||||
serde_json.workspace = true
|
||||
tokio = { workspace = true, features = ["macros", "rt-multi-thread"] }
|
||||
chrono = { workspace = true }
|
||||
clap = { version = "4", features = ["derive"] }
|
||||
toml = "0.8"
|
||||
directories = "5"
|
||||
|
||||
@@ -8,7 +8,10 @@
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
use anyhow::{anyhow, Context, Result};
|
||||
use picloud_shared::{App, AppId, AppRole, ExecutionLog, InstanceRole, Script};
|
||||
use chrono::{DateTime, Utc};
|
||||
use picloud_shared::{
|
||||
AdminUserId, ApiKeyId, App, AppId, AppRole, ExecutionLog, InstanceRole, Scope, Script,
|
||||
};
|
||||
use reqwest::{header, Method, RequestBuilder, StatusCode};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use serde_json::Value;
|
||||
@@ -38,6 +41,7 @@ impl Client {
|
||||
})
|
||||
}
|
||||
|
||||
#[allow(dead_code)] // used by the trailing-slash unit test below.
|
||||
pub fn url(&self) -> &str {
|
||||
&self.url
|
||||
}
|
||||
@@ -97,6 +101,42 @@ impl Client {
|
||||
decode(resp).await
|
||||
}
|
||||
|
||||
/// `GET /api/v1/admin/scripts` — every script the caller can see
|
||||
/// (server filters by membership for `Member`). Lets `pic scripts ls`
|
||||
/// (no `--app`) collapse what used to be an N+1 per-app walk into a
|
||||
/// single request that can't be partially-broken by a concurrent app
|
||||
/// delete.
|
||||
pub async fn scripts_list_all(&self) -> Result<Vec<Script>> {
|
||||
let resp = self
|
||||
.request(Method::GET, "/api/v1/admin/scripts")
|
||||
.send()
|
||||
.await?;
|
||||
decode(resp).await
|
||||
}
|
||||
|
||||
/// `DELETE /api/v1/admin/apps/{id_or_slug}` with optional `?force=true`.
|
||||
/// Server requires `AppAdmin` capability; without `force`, returns
|
||||
/// 409 if the app still has scripts.
|
||||
pub async fn apps_delete(&self, ident: &str, force: bool) -> Result<()> {
|
||||
let path = if force {
|
||||
format!("/api/v1/admin/apps/{ident}?force=true")
|
||||
} else {
|
||||
format!("/api/v1/admin/apps/{ident}")
|
||||
};
|
||||
let resp = self.request(Method::DELETE, &path).send().await?;
|
||||
decode_status(resp).await
|
||||
}
|
||||
|
||||
/// `DELETE /api/v1/admin/scripts/{id}` — requires `AppAdmin` on the
|
||||
/// owning app (stricter than the edit endpoints, by design).
|
||||
pub async fn scripts_delete(&self, id: &str) -> Result<()> {
|
||||
let resp = self
|
||||
.request(Method::DELETE, &format!("/api/v1/admin/scripts/{id}"))
|
||||
.send()
|
||||
.await?;
|
||||
decode_status(resp).await
|
||||
}
|
||||
|
||||
/// `POST /api/v1/admin/scripts`
|
||||
pub async fn scripts_create(&self, body: &CreateScriptBody<'_>) -> Result<Script> {
|
||||
let resp = self
|
||||
@@ -167,6 +207,68 @@ impl Client {
|
||||
.await?;
|
||||
decode(resp).await
|
||||
}
|
||||
|
||||
/// `POST /api/v1/admin/auth/logout` — best-effort: server returns
|
||||
/// 204 whether or not the token matched a live session, so we just
|
||||
/// fire and discard the body. Caller still wipes the local creds.
|
||||
pub async fn auth_logout(&self) -> Result<()> {
|
||||
let resp = self
|
||||
.request(Method::POST, "/api/v1/admin/auth/logout")
|
||||
.send()
|
||||
.await?;
|
||||
decode_status(resp).await
|
||||
}
|
||||
|
||||
/// `GET /api/v1/admin/api-keys` — caller's keys only (server filters
|
||||
/// by user_id, no cross-user enumeration).
|
||||
pub async fn apikeys_list(&self) -> Result<Vec<ApiKeyDto>> {
|
||||
let resp = self
|
||||
.request(Method::GET, "/api/v1/admin/api-keys")
|
||||
.send()
|
||||
.await?;
|
||||
decode(resp).await
|
||||
}
|
||||
|
||||
/// `POST /api/v1/admin/api-keys` — `raw_token` is in the response
|
||||
/// **once** and never appears in `GET /api-keys` afterward.
|
||||
pub async fn apikeys_mint(&self, body: &MintApiKeyBody<'_>) -> Result<MintApiKeyResponseDto> {
|
||||
let resp = self
|
||||
.request(Method::POST, "/api/v1/admin/api-keys")
|
||||
.json(body)
|
||||
.send()
|
||||
.await?;
|
||||
decode(resp).await
|
||||
}
|
||||
|
||||
/// `DELETE /api/v1/admin/api-keys/{id}` — 404 covers both "doesn't
|
||||
/// exist" and "not yours" (server flattens to avoid enumeration).
|
||||
pub async fn apikeys_delete(&self, id: &str) -> Result<()> {
|
||||
let resp = self
|
||||
.request(Method::DELETE, &format!("/api/v1/admin/api-keys/{id}"))
|
||||
.send()
|
||||
.await?;
|
||||
decode_status(resp).await
|
||||
}
|
||||
}
|
||||
|
||||
/// `POST /api/v1/admin/auth/login` — sits outside the `Client` because
|
||||
/// it runs before any token exists. Mirrors the dashboard's login.ts
|
||||
/// wire shape (see `manager-core/src/auth_api.rs:49-60`).
|
||||
pub async fn auth_login(url: &str, username: &str, password: &str) -> Result<LoginResponseDto> {
|
||||
let http = reqwest::Client::builder()
|
||||
.user_agent(concat!("pic/", env!("CARGO_PKG_VERSION")))
|
||||
.build()
|
||||
.context("building HTTP client")?;
|
||||
let body = LoginRequestBody { username, password };
|
||||
let resp = http
|
||||
.post(format!(
|
||||
"{}/api/v1/admin/auth/login",
|
||||
url.trim_end_matches('/')
|
||||
))
|
||||
.json(&body)
|
||||
.send()
|
||||
.await?;
|
||||
decode(resp).await
|
||||
}
|
||||
|
||||
// ---------- DTOs (CLI-local, wire-shape-matched) ----------
|
||||
@@ -216,6 +318,63 @@ struct UpdateScriptBody<'a> {
|
||||
source: &'a str,
|
||||
}
|
||||
|
||||
#[derive(Debug, Serialize)]
|
||||
struct LoginRequestBody<'a> {
|
||||
username: &'a str,
|
||||
password: &'a str,
|
||||
}
|
||||
|
||||
#[allow(dead_code)]
|
||||
#[derive(Debug, Deserialize)]
|
||||
pub struct LoginResponseDto {
|
||||
pub user: LoginUserDto,
|
||||
pub token: String,
|
||||
pub expires_at: DateTime<Utc>,
|
||||
}
|
||||
|
||||
#[allow(dead_code)]
|
||||
#[derive(Debug, Deserialize)]
|
||||
pub struct LoginUserDto {
|
||||
pub id: AdminUserId,
|
||||
pub username: String,
|
||||
pub instance_role: InstanceRole,
|
||||
#[serde(default)]
|
||||
pub email: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Serialize)]
|
||||
pub struct MintApiKeyBody<'a> {
|
||||
pub name: &'a str,
|
||||
pub scopes: &'a [Scope],
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub app_id: Option<AppId>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub expires_at: Option<DateTime<Utc>>,
|
||||
}
|
||||
|
||||
/// Fresh-mint response. The `raw_token` field is the one and only
|
||||
/// chance to capture the bearer string; subsequent `GET /api-keys`
|
||||
/// returns the `ApiKeyDto` portion without it.
|
||||
#[derive(Debug, Deserialize)]
|
||||
pub struct MintApiKeyResponseDto {
|
||||
#[serde(flatten)]
|
||||
pub key: ApiKeyDto,
|
||||
pub raw_token: String,
|
||||
}
|
||||
|
||||
#[allow(dead_code)]
|
||||
#[derive(Debug, Deserialize)]
|
||||
pub struct ApiKeyDto {
|
||||
pub id: ApiKeyId,
|
||||
pub prefix: String,
|
||||
pub name: String,
|
||||
pub scopes: Vec<Scope>,
|
||||
pub app_id: Option<AppId>,
|
||||
pub expires_at: Option<DateTime<Utc>>,
|
||||
pub last_used_at: Option<DateTime<Utc>>,
|
||||
pub created_at: DateTime<Utc>,
|
||||
}
|
||||
|
||||
#[allow(dead_code)]
|
||||
#[derive(Debug)]
|
||||
pub struct ExecuteResponse {
|
||||
@@ -265,6 +424,15 @@ async fn decode<T: for<'de> Deserialize<'de>>(resp: reqwest::Response) -> Result
|
||||
Err(server_error(resp).await)
|
||||
}
|
||||
|
||||
/// Like `decode` but for endpoints whose 2xx response has no body
|
||||
/// (204 No Content) — DELETE handlers, logout.
|
||||
async fn decode_status(resp: reqwest::Response) -> Result<()> {
|
||||
if resp.status().is_success() {
|
||||
return Ok(());
|
||||
}
|
||||
Err(server_error(resp).await)
|
||||
}
|
||||
|
||||
async fn server_error(resp: reqwest::Response) -> anyhow::Error {
|
||||
let status = resp.status();
|
||||
let body = resp.text().await.unwrap_or_default();
|
||||
|
||||
201
crates/picloud-cli/src/cmds/api_keys.rs
Normal file
201
crates/picloud-cli/src/cmds/api_keys.rs
Normal file
@@ -0,0 +1,201 @@
|
||||
//! `pic api-keys` — long-lived bearer-key management.
|
||||
//!
|
||||
//! Server semantics (mirrored from `manager-core/src/api_keys_api.rs`):
|
||||
//! * `raw_token` is returned **once** on mint and never again.
|
||||
//! * `app_id` (optional `--app`) binds the key to one app; instance
|
||||
//! scopes (`instance:*`) are rejected when `--app` is also set.
|
||||
//! * `scopes` is a `text[]` in the wire form (`script:read`, …).
|
||||
|
||||
use anyhow::{anyhow, Result};
|
||||
use chrono::{DateTime, Utc};
|
||||
use picloud_shared::Scope;
|
||||
|
||||
use crate::client::{Client, MintApiKeyBody};
|
||||
use crate::config;
|
||||
use crate::output::{KvBlock, OutputMode, Table};
|
||||
|
||||
pub async fn mint(
|
||||
name: &str,
|
||||
scope_strs: &[String],
|
||||
app_ident: Option<&str>,
|
||||
expires: Option<&str>,
|
||||
mode: OutputMode,
|
||||
) -> Result<()> {
|
||||
let creds = config::resolve()?;
|
||||
let client = Client::from_creds(&creds)?;
|
||||
|
||||
let scopes = parse_scopes(scope_strs)?;
|
||||
let expires_at = expires.map(parse_expires).transpose()?;
|
||||
let app_id = match app_ident {
|
||||
Some(ident) => Some(client.apps_get(ident).await?.app.id),
|
||||
None => None,
|
||||
};
|
||||
|
||||
let body = MintApiKeyBody {
|
||||
name,
|
||||
scopes: &scopes,
|
||||
app_id,
|
||||
expires_at,
|
||||
};
|
||||
let resp = client.apikeys_mint(&body).await?;
|
||||
|
||||
let mut block = KvBlock::new();
|
||||
block
|
||||
.field("id", resp.key.id.to_string())
|
||||
.field("name", resp.key.name.clone())
|
||||
.field("prefix", resp.key.prefix.clone())
|
||||
.field(
|
||||
"scopes",
|
||||
resp.key
|
||||
.scopes
|
||||
.iter()
|
||||
.map(|s| s.as_str())
|
||||
.collect::<Vec<_>>()
|
||||
.join(","),
|
||||
)
|
||||
.field(
|
||||
"app_id",
|
||||
resp.key
|
||||
.app_id
|
||||
.map(|a| a.to_string())
|
||||
.unwrap_or_else(|| "-".into()),
|
||||
)
|
||||
.field(
|
||||
"expires_at",
|
||||
resp.key
|
||||
.expires_at
|
||||
.map(|t| t.to_rfc3339())
|
||||
.unwrap_or_else(|| "-".into()),
|
||||
)
|
||||
.field("token", resp.raw_token.clone());
|
||||
block.print(mode);
|
||||
if matches!(mode, OutputMode::Tsv) {
|
||||
// The token row is human-easy-to-miss in a wall of metadata;
|
||||
// call it out exactly once on the human path. Skip on JSON
|
||||
// since machine consumers don't need the nudge.
|
||||
eprintln!("Save this token — it will not be shown again.");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub async fn ls(mode: OutputMode) -> Result<()> {
|
||||
let creds = config::resolve()?;
|
||||
let client = Client::from_creds(&creds)?;
|
||||
let keys = client.apikeys_list().await?;
|
||||
let mut table = Table::new([
|
||||
"id",
|
||||
"name",
|
||||
"prefix",
|
||||
"scopes",
|
||||
"app_id",
|
||||
"expires_at",
|
||||
"last_used_at",
|
||||
"created_at",
|
||||
]);
|
||||
for k in keys {
|
||||
table.row([
|
||||
k.id.to_string(),
|
||||
k.name,
|
||||
k.prefix,
|
||||
k.scopes
|
||||
.iter()
|
||||
.map(|s| s.as_str())
|
||||
.collect::<Vec<_>>()
|
||||
.join(","),
|
||||
k.app_id
|
||||
.map(|a| a.to_string())
|
||||
.unwrap_or_else(|| "-".into()),
|
||||
k.expires_at
|
||||
.map(|t| t.to_rfc3339())
|
||||
.unwrap_or_else(|| "-".into()),
|
||||
k.last_used_at
|
||||
.map(|t| t.to_rfc3339())
|
||||
.unwrap_or_else(|| "-".into()),
|
||||
k.created_at.to_rfc3339(),
|
||||
]);
|
||||
}
|
||||
table.print(mode);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub async fn rm(id: &str) -> Result<()> {
|
||||
let creds = config::resolve()?;
|
||||
let client = Client::from_creds(&creds)?;
|
||||
client.apikeys_delete(id).await?;
|
||||
println!("Revoked api-key {id}");
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn parse_scopes(raw: &[String]) -> Result<Vec<Scope>> {
|
||||
if raw.is_empty() {
|
||||
return Err(anyhow!(
|
||||
"at least one `--scope` is required (e.g. --scope script:read)"
|
||||
));
|
||||
}
|
||||
raw.iter()
|
||||
.map(|s| Scope::from_wire(s).ok_or_else(|| anyhow!("unknown scope: {s}")))
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// `--expires` accepts either RFC 3339 (`2026-12-31T23:59:59Z`) or a
|
||||
/// shorthand `<N>d` / `<N>h` / `<N>m` (days / hours / minutes from now).
|
||||
/// Shorthand wins for the common "key good for 30 days" case; full
|
||||
/// RFC 3339 keeps the door open for precise cutoffs.
|
||||
fn parse_expires(raw: &str) -> Result<DateTime<Utc>> {
|
||||
if let Some(spec) = raw.strip_suffix('d') {
|
||||
let days: i64 = spec.parse().map_err(|_| anyhow!("bad days: {raw}"))?;
|
||||
return Ok(Utc::now() + chrono::Duration::days(days));
|
||||
}
|
||||
if let Some(spec) = raw.strip_suffix('h') {
|
||||
let hours: i64 = spec.parse().map_err(|_| anyhow!("bad hours: {raw}"))?;
|
||||
return Ok(Utc::now() + chrono::Duration::hours(hours));
|
||||
}
|
||||
if let Some(spec) = raw.strip_suffix('m') {
|
||||
let mins: i64 = spec.parse().map_err(|_| anyhow!("bad minutes: {raw}"))?;
|
||||
return Ok(Utc::now() + chrono::Duration::minutes(mins));
|
||||
}
|
||||
DateTime::parse_from_rfc3339(raw)
|
||||
.map(|d| d.with_timezone(&Utc))
|
||||
.map_err(|e| anyhow!("expected RFC 3339 or `<N>d/h/m`, got {raw:?}: {e}"))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn parse_scopes_accepts_wire_form() {
|
||||
let scopes = parse_scopes(&["script:read".into(), "log:read".into()]).unwrap();
|
||||
assert_eq!(scopes, vec![Scope::ScriptRead, Scope::LogRead]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parse_scopes_rejects_empty() {
|
||||
let err = parse_scopes(&[]).unwrap_err();
|
||||
assert!(format!("{err}").contains("at least one"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parse_scopes_rejects_unknown() {
|
||||
let err = parse_scopes(&["script:nope".into()]).unwrap_err();
|
||||
assert!(format!("{err}").contains("unknown scope"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parse_expires_days_shorthand() {
|
||||
let d = parse_expires("7d").unwrap();
|
||||
let diff = (d - Utc::now()).num_days();
|
||||
assert!((6..=7).contains(&diff), "got {diff}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parse_expires_rfc3339_passes_through() {
|
||||
let d = parse_expires("2030-01-01T00:00:00Z").unwrap();
|
||||
assert_eq!(d.timestamp(), 1893456000);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parse_expires_garbage_errors() {
|
||||
assert!(parse_expires("tomorrow").is_err());
|
||||
}
|
||||
}
|
||||
@@ -1,13 +1,14 @@
|
||||
//! `pic apps ls` and `pic apps create`.
|
||||
//! `pic apps` subcommands: `ls`, `create`, `show`, `delete`.
|
||||
|
||||
use anyhow::Result;
|
||||
use picloud_shared::AppRole;
|
||||
|
||||
use crate::client::{Client, CreateAppBody};
|
||||
use crate::config::load;
|
||||
use crate::output::Table;
|
||||
use crate::config;
|
||||
use crate::output::{KvBlock, OutputMode, Table};
|
||||
|
||||
pub async fn ls() -> Result<()> {
|
||||
let creds = load()?;
|
||||
pub async fn ls(mode: OutputMode) -> Result<()> {
|
||||
let creds = config::resolve()?;
|
||||
let client = Client::from_creds(&creds)?;
|
||||
let apps = client.apps_list().await?;
|
||||
let mut table = Table::new(["slug", "name", "my_role", "created_at"]);
|
||||
@@ -22,12 +23,12 @@ pub async fn ls() -> Result<()> {
|
||||
app.created_at.to_rfc3339(),
|
||||
]);
|
||||
}
|
||||
table.print();
|
||||
table.print(mode);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub async fn create(slug: &str, name: Option<&str>, description: Option<&str>) -> Result<()> {
|
||||
let creds = load()?;
|
||||
let creds = config::resolve()?;
|
||||
let client = Client::from_creds(&creds)?;
|
||||
let body = CreateAppBody {
|
||||
slug,
|
||||
@@ -38,3 +39,46 @@ pub async fn create(slug: &str, name: Option<&str>, description: Option<&str>) -
|
||||
println!("Created app {}", app.slug);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// `pic apps show <slug>` — single-app inspect using the lookup
|
||||
/// endpoint, which carries `my_role` for the caller (the `ls` endpoint
|
||||
/// doesn't).
|
||||
pub async fn show(ident: &str, mode: OutputMode) -> Result<()> {
|
||||
let creds = config::resolve()?;
|
||||
let client = Client::from_creds(&creds)?;
|
||||
let lookup = client.apps_get(ident).await?;
|
||||
let mut block = KvBlock::new();
|
||||
block
|
||||
.field("id", lookup.app.id.to_string())
|
||||
.field("slug", lookup.app.slug.clone())
|
||||
.field("name", lookup.app.name.clone())
|
||||
.field(
|
||||
"description",
|
||||
lookup.app.description.clone().unwrap_or_else(|| "-".into()),
|
||||
)
|
||||
.field("my_role", role_label(lookup.my_role.as_ref()))
|
||||
.field("created_at", lookup.app.created_at.to_rfc3339())
|
||||
.field("updated_at", lookup.app.updated_at.to_rfc3339());
|
||||
block.print(mode);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// `pic apps delete <slug> [--force]`. Without `--force` the server
|
||||
/// returns 409 if the app still owns scripts — surface that as a
|
||||
/// useful error rather than swallowing.
|
||||
pub async fn delete(ident: &str, force: bool) -> Result<()> {
|
||||
let creds = config::resolve()?;
|
||||
let client = Client::from_creds(&creds)?;
|
||||
client.apps_delete(ident, force).await?;
|
||||
println!("Deleted app {ident}");
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn role_label(role: Option<&AppRole>) -> String {
|
||||
// Use the wire form so the CLI label matches what the dashboard
|
||||
// shows and what the membership APIs accept.
|
||||
match role {
|
||||
Some(r) => r.as_str().to_string(),
|
||||
None => "-".into(),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,46 +1,118 @@
|
||||
//! `pic login` — interactively (or via PICLOUD_URL/PICLOUD_TOKEN env
|
||||
//! shortcut for non-interactive contexts like CI and integration tests)
|
||||
//! capture the URL + bearer token, validate against `/auth/me`, save.
|
||||
//! `pic login` — primary auth entry point.
|
||||
//!
|
||||
//! Two flows:
|
||||
//! * **username + password** (default, interactive): POST
|
||||
//! `/api/v1/admin/auth/login` with the credentials and persist the
|
||||
//! returned session token. Mirrors the dashboard's login form.
|
||||
//! * **paste-a-token** (`--token <T>`, or `PICLOUD_TOKEN` env): skip
|
||||
//! the credential exchange and persist a bearer string directly.
|
||||
//! Used by CI and by anyone using a long-lived API key minted via
|
||||
//! `pic api-keys mint`. Validated against `/auth/me` before save.
|
||||
//!
|
||||
//! `--url <U>` (or `PICLOUD_URL`) overrides the URL prompt non-interactively.
|
||||
|
||||
use std::io::{self, BufRead, Write};
|
||||
|
||||
use anyhow::Result;
|
||||
use anyhow::{Context, Result};
|
||||
use picloud_shared::InstanceRole;
|
||||
|
||||
use crate::client::Client;
|
||||
use crate::client::{self, Client};
|
||||
use crate::config::{save, Credentials};
|
||||
|
||||
const DEFAULT_URL: &str = "http://localhost:8000";
|
||||
|
||||
pub async fn run() -> Result<()> {
|
||||
let (url, token) = collect_credentials()?;
|
||||
let client = Client::new(&url, &token)?;
|
||||
let me = client.auth_me().await?;
|
||||
pub async fn run(url_arg: Option<&str>, token_arg: Option<&str>) -> Result<()> {
|
||||
let url = resolve_url(url_arg)?;
|
||||
let token_from_env = std::env::var("PICLOUD_TOKEN")
|
||||
.ok()
|
||||
.filter(|s| !s.is_empty());
|
||||
let bearer_token = token_arg.map(str::to_string).or(token_from_env);
|
||||
|
||||
let (token, username, role) = match bearer_token {
|
||||
Some(t) => login_with_bearer(&url, &t).await?,
|
||||
None => login_with_password(&url).await?,
|
||||
};
|
||||
|
||||
let creds = Credentials {
|
||||
url: client.url().to_string(),
|
||||
url: url.clone(),
|
||||
token,
|
||||
username: me.username.clone(),
|
||||
username: username.clone(),
|
||||
};
|
||||
save(&creds)?;
|
||||
println!(
|
||||
"Logged in as {} ({}) at {}",
|
||||
me.username,
|
||||
instance_role_label(&me.instance_role),
|
||||
creds.url
|
||||
"Logged in as {username} ({}) at {url}",
|
||||
instance_role_label(&role)
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn collect_credentials() -> Result<(String, String)> {
|
||||
// Non-interactive shortcut: both vars set → use as-is. Used by the
|
||||
// integration test and any CI flow that wants to skip the prompts.
|
||||
if let (Ok(url), Ok(tok)) = (std::env::var("PICLOUD_URL"), std::env::var("PICLOUD_TOKEN")) {
|
||||
if !url.is_empty() && !tok.is_empty() {
|
||||
return Ok((url, tok));
|
||||
async fn login_with_password(url: &str) -> Result<(String, String, InstanceRole)> {
|
||||
let username = prompt_line("Username: ")?;
|
||||
if username.is_empty() {
|
||||
anyhow::bail!("username is required");
|
||||
}
|
||||
let password = read_password()?;
|
||||
let resp = client::auth_login(url, &username, &password).await?;
|
||||
Ok((resp.token, resp.user.username, resp.user.instance_role))
|
||||
}
|
||||
|
||||
/// Read a password without echoing it where possible. Falls back to a
|
||||
/// plain stdin read when no controlling terminal is attached — CI
|
||||
/// systems and `cargo test`'s piped stdin both land here, and dying
|
||||
/// outright would block scripted use entirely. The fallback is louder
|
||||
/// (visible characters), but it's that or no functioning login.
|
||||
fn read_password() -> Result<String> {
|
||||
match rpassword::prompt_password("Password: ") {
|
||||
Ok(p) => Ok(p),
|
||||
Err(_) => {
|
||||
eprint!("Password: ");
|
||||
io::stderr().flush()?;
|
||||
let mut buf = String::new();
|
||||
io::stdin()
|
||||
.lock()
|
||||
.read_line(&mut buf)
|
||||
.context("reading password from stdin")?;
|
||||
Ok(buf.trim_end_matches(['\r', '\n']).to_string())
|
||||
}
|
||||
}
|
||||
let url = prompt_with_default("PiCloud URL", DEFAULT_URL)?;
|
||||
let token = rpassword::prompt_password("API token: ")?;
|
||||
Ok((url, token))
|
||||
}
|
||||
|
||||
/// Bearer-token path: validate against `/auth/me` so a typo doesn't get
|
||||
/// persisted, then trust the username the server reports rather than
|
||||
/// whatever the user typed (which they didn't type at all in this mode).
|
||||
async fn login_with_bearer(url: &str, token: &str) -> Result<(String, String, InstanceRole)> {
|
||||
let client = Client::new(url, token)?;
|
||||
let me = client.auth_me().await?;
|
||||
Ok((token.to_string(), me.username, me.instance_role))
|
||||
}
|
||||
|
||||
fn instance_role_label(role: &InstanceRole) -> &'static str {
|
||||
match role {
|
||||
InstanceRole::Owner => "owner",
|
||||
InstanceRole::Admin => "admin",
|
||||
InstanceRole::Member => "member",
|
||||
}
|
||||
}
|
||||
|
||||
fn resolve_url(url_arg: Option<&str>) -> Result<String> {
|
||||
if let Some(u) = url_arg {
|
||||
return Ok(u.trim_end_matches('/').to_string());
|
||||
}
|
||||
if let Ok(env_url) = std::env::var("PICLOUD_URL") {
|
||||
if !env_url.is_empty() {
|
||||
return Ok(env_url.trim_end_matches('/').to_string());
|
||||
}
|
||||
}
|
||||
let typed = prompt_with_default("PiCloud URL", DEFAULT_URL)?;
|
||||
Ok(typed.trim_end_matches('/').to_string())
|
||||
}
|
||||
|
||||
fn prompt_line(label: &str) -> Result<String> {
|
||||
print!("{label}");
|
||||
io::stdout().flush()?;
|
||||
let mut buf = String::new();
|
||||
io::stdin().lock().read_line(&mut buf)?;
|
||||
Ok(buf.trim().to_string())
|
||||
}
|
||||
|
||||
fn prompt_with_default(label: &str, default: &str) -> Result<String> {
|
||||
@@ -55,12 +127,3 @@ fn prompt_with_default(label: &str, default: &str) -> Result<String> {
|
||||
trimmed.to_string()
|
||||
})
|
||||
}
|
||||
|
||||
fn instance_role_label(role: &picloud_shared::InstanceRole) -> &'static str {
|
||||
use picloud_shared::InstanceRole as R;
|
||||
match role {
|
||||
R::Owner => "owner",
|
||||
R::Admin => "admin",
|
||||
R::Member => "member",
|
||||
}
|
||||
}
|
||||
|
||||
29
crates/picloud-cli/src/cmds/logout.rs
Normal file
29
crates/picloud-cli/src/cmds/logout.rs
Normal file
@@ -0,0 +1,29 @@
|
||||
//! `pic logout` — revoke the saved session server-side, then wipe the
|
||||
//! local credentials file.
|
||||
//!
|
||||
//! Idempotent: if the file doesn't exist or the server already forgot
|
||||
//! the session, we still succeed. The point is leaving the user in a
|
||||
//! clean "no token" state, not enforcing that a session existed.
|
||||
|
||||
use anyhow::Result;
|
||||
|
||||
use crate::client::Client;
|
||||
use crate::config;
|
||||
|
||||
pub async fn run() -> Result<()> {
|
||||
// Load before delete so we have a token to POST /logout with; if
|
||||
// there's no creds file there's also nothing to revoke server-side.
|
||||
let creds = config::load().ok();
|
||||
|
||||
if let Some(creds) = creds {
|
||||
let client = Client::from_creds(&creds)?;
|
||||
// Best-effort: a 4xx (token already invalid) or network error
|
||||
// shouldn't block the local wipe. The whole point of logout is
|
||||
// leaving no credentials on disk.
|
||||
let _ = client.auth_logout().await;
|
||||
}
|
||||
|
||||
config::delete()?;
|
||||
println!("Logged out");
|
||||
Ok(())
|
||||
}
|
||||
@@ -1,27 +1,48 @@
|
||||
//! `pic logs <script-id>` — print recent execution log rows.
|
||||
//!
|
||||
//! In TSV mode emits a header + truncated-summary rows (`pic logs` was
|
||||
//! previously headerless — inconsistent with `apps ls` / `scripts ls`).
|
||||
//! In JSON mode emits the raw `ExecutionLog` array (no truncation),
|
||||
//! letting `jq` consumers see request/response bodies in full.
|
||||
|
||||
use anyhow::Result;
|
||||
use picloud_shared::ExecutionStatus;
|
||||
use picloud_shared::{ExecutionLog, ExecutionStatus};
|
||||
|
||||
use crate::client::Client;
|
||||
use crate::config::load;
|
||||
use crate::config;
|
||||
use crate::output::{OutputMode, Table};
|
||||
|
||||
pub async fn run(script_id: &str, limit: u32) -> Result<()> {
|
||||
let creds = load()?;
|
||||
pub async fn run(script_id: &str, limit: u32, mode: OutputMode) -> Result<()> {
|
||||
let creds = config::resolve()?;
|
||||
let client = Client::from_creds(&creds)?;
|
||||
let entries = client.logs_list(script_id, limit).await?;
|
||||
for e in entries {
|
||||
let summary = summarize(&e.response_body, &e.script_logs);
|
||||
println!(
|
||||
"{}\t{}\t{}",
|
||||
e.created_at.to_rfc3339(),
|
||||
status_label(&e.status),
|
||||
truncate(&summary, 120),
|
||||
);
|
||||
match mode {
|
||||
OutputMode::Tsv => render_tsv(&entries),
|
||||
OutputMode::Json => render_json(&entries),
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn render_tsv(entries: &[ExecutionLog]) {
|
||||
let mut table = Table::new(["created_at", "status", "summary"]);
|
||||
for e in entries {
|
||||
let summary = summarize(&e.response_body, &e.script_logs);
|
||||
table.row([
|
||||
e.created_at.to_rfc3339(),
|
||||
status_label(&e.status).to_string(),
|
||||
truncate(&summary, 120),
|
||||
]);
|
||||
}
|
||||
table.print(OutputMode::Tsv);
|
||||
}
|
||||
|
||||
fn render_json(entries: &[ExecutionLog]) {
|
||||
// Pretty for human jq-piping; consumers that want compact can pipe
|
||||
// through `jq -c`.
|
||||
let s = serde_json::to_string_pretty(entries).unwrap_or_else(|_| "[]".to_string());
|
||||
println!("{s}");
|
||||
}
|
||||
|
||||
fn status_label(s: &ExecutionStatus) -> &'static str {
|
||||
match s {
|
||||
ExecutionStatus::Success => "success",
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
pub mod api_keys;
|
||||
pub mod apps;
|
||||
pub mod login;
|
||||
pub mod logout;
|
||||
pub mod logs;
|
||||
pub mod scripts;
|
||||
pub mod whoami;
|
||||
|
||||
@@ -1,17 +1,19 @@
|
||||
//! `pic scripts ls | deploy | invoke`.
|
||||
//! `pic scripts ls | deploy | invoke | delete`.
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::io::{self, Read, Write};
|
||||
use std::path::Path;
|
||||
|
||||
use anyhow::{anyhow, Context, Result};
|
||||
use picloud_shared::AppId;
|
||||
use serde_json::Value;
|
||||
|
||||
use crate::client::{Client, CreateScriptBody};
|
||||
use crate::config::load;
|
||||
use crate::output::Table;
|
||||
use crate::config;
|
||||
use crate::output::{OutputMode, Table};
|
||||
|
||||
pub async fn ls(app: Option<&str>) -> Result<()> {
|
||||
let creds = load()?;
|
||||
pub async fn ls(app: Option<&str>, mode: OutputMode) -> Result<()> {
|
||||
let creds = config::resolve()?;
|
||||
let client = Client::from_creds(&creds)?;
|
||||
|
||||
let mut table = Table::new(["id", "app_slug", "name", "version", "updated_at"]);
|
||||
@@ -29,24 +31,29 @@ pub async fn ls(app: Option<&str>) -> Result<()> {
|
||||
]);
|
||||
}
|
||||
} else {
|
||||
// No filter → walk every accessible app. One request per app is
|
||||
// fine at MVP scale (handful of apps); a bulk endpoint can come
|
||||
// later if the count grows.
|
||||
let apps = client.apps_list().await?;
|
||||
for a in apps {
|
||||
let scripts = client.scripts_list_by_app(&a.slug).await?;
|
||||
// No filter → use the single `GET /admin/scripts` call. Server
|
||||
// filters by membership for `Member`; for `Admin`/`Owner` it
|
||||
// returns every script. Two requests total (apps + scripts) run
|
||||
// in parallel; the per-app walk we used to do here aborted on
|
||||
// the first 404 when another caller deleted an app mid-listing,
|
||||
// and was the entire reason a 5× retry existed in the tests.
|
||||
let (apps, scripts) = tokio::try_join!(client.apps_list(), client.scripts_list_all())?;
|
||||
let slug_by_id: HashMap<AppId, String> = apps.into_iter().map(|a| (a.id, a.slug)).collect();
|
||||
for s in scripts {
|
||||
let app_slug = slug_by_id
|
||||
.get(&s.app_id)
|
||||
.cloned()
|
||||
.unwrap_or_else(|| "-".to_string());
|
||||
table.row([
|
||||
s.id.to_string(),
|
||||
a.slug.clone(),
|
||||
app_slug,
|
||||
s.name,
|
||||
s.version.to_string(),
|
||||
s.updated_at.to_rfc3339(),
|
||||
]);
|
||||
}
|
||||
}
|
||||
}
|
||||
table.print();
|
||||
table.print(mode);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
@@ -56,7 +63,7 @@ pub async fn deploy(
|
||||
name_override: Option<&str>,
|
||||
description: Option<&str>,
|
||||
) -> Result<()> {
|
||||
let creds = load()?;
|
||||
let creds = config::resolve()?;
|
||||
let client = Client::from_creds(&creds)?;
|
||||
|
||||
let source =
|
||||
@@ -99,7 +106,7 @@ pub async fn deploy(
|
||||
}
|
||||
|
||||
pub async fn invoke(id: &str, body_arg: Option<&str>, headers: &[(String, String)]) -> Result<()> {
|
||||
let creds = load()?;
|
||||
let creds = config::resolve()?;
|
||||
let client = Client::from_creds(&creds)?;
|
||||
|
||||
let body = parse_body_arg(body_arg)?;
|
||||
@@ -115,6 +122,18 @@ pub async fn invoke(id: &str, body_arg: Option<&str>, headers: &[(String, String
|
||||
}
|
||||
}
|
||||
|
||||
/// `pic scripts delete <id>`. Requires `AppAdmin` on the owning app
|
||||
/// server-side, which is stricter than the edit endpoints — Editor
|
||||
/// can deploy/update but not destroy. Surfaces that as a 403 with the
|
||||
/// usual role hint.
|
||||
pub async fn delete(id: &str) -> Result<()> {
|
||||
let creds = config::resolve()?;
|
||||
let client = Client::from_creds(&creds)?;
|
||||
client.scripts_delete(id).await?;
|
||||
println!("Deleted script {id}");
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn parse_body_arg(arg: Option<&str>) -> Result<Value> {
|
||||
match arg {
|
||||
None => Ok(Value::Object(serde_json::Map::new())),
|
||||
|
||||
@@ -1,22 +1,34 @@
|
||||
//! `pic whoami` — re-validates the saved token by hitting `/auth/me`
|
||||
//! every time. Cached username in the credentials file is for
|
||||
//! display-only contexts; this command is the source of truth.
|
||||
//!
|
||||
//! TSV output uses `KvBlock` (aligned `key: value` rows), JSON output
|
||||
//! is a flat object — both downstream-friendly without the user having
|
||||
//! to parse a headerless tab-line.
|
||||
|
||||
use anyhow::Result;
|
||||
use picloud_shared::InstanceRole;
|
||||
|
||||
use crate::client::Client;
|
||||
use crate::config::load;
|
||||
use crate::config;
|
||||
use crate::output::{KvBlock, OutputMode};
|
||||
|
||||
pub async fn run() -> Result<()> {
|
||||
let creds = load()?;
|
||||
pub async fn run(mode: OutputMode) -> Result<()> {
|
||||
let creds = config::resolve()?;
|
||||
let client = Client::from_creds(&creds)?;
|
||||
let me = client.auth_me().await?;
|
||||
let role = match me.instance_role {
|
||||
picloud_shared::InstanceRole::Owner => "owner",
|
||||
picloud_shared::InstanceRole::Admin => "admin",
|
||||
picloud_shared::InstanceRole::Member => "member",
|
||||
InstanceRole::Owner => "owner",
|
||||
InstanceRole::Admin => "admin",
|
||||
InstanceRole::Member => "member",
|
||||
};
|
||||
let email = me.email.as_deref().unwrap_or("-");
|
||||
println!("{}\t{role}\t{email}\t{}", me.username, creds.url);
|
||||
let mut block = KvBlock::new();
|
||||
block
|
||||
.field("username", me.username)
|
||||
.field("role", role)
|
||||
.field("email", email)
|
||||
.field("url", creds.url.clone());
|
||||
block.print(mode);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
@@ -43,6 +43,41 @@ pub fn load() -> Result<Credentials> {
|
||||
toml::from_str(&body).with_context(|| format!("failed to parse {}", path.display()))
|
||||
}
|
||||
|
||||
/// Resolution order used by every non-login command:
|
||||
/// 1. If both `PICLOUD_URL` and `PICLOUD_TOKEN` are set (and non-empty),
|
||||
/// use them directly. Matches gcloud/aws/kubectl semantics — env
|
||||
/// wins so CI never accidentally reads a developer's stale file.
|
||||
/// 2. Otherwise fall back to the on-disk credentials file.
|
||||
///
|
||||
/// Username is best-effort: env mode has no way to know the real one
|
||||
/// (no round-trip to `/auth/me`), so it shows as `"-"` in `whoami`
|
||||
/// output. Callers that need the canonical username re-fetch via
|
||||
/// `Client::auth_me`.
|
||||
pub fn resolve() -> Result<Credentials> {
|
||||
if let (Ok(url), Ok(token)) = (std::env::var("PICLOUD_URL"), std::env::var("PICLOUD_TOKEN")) {
|
||||
if !url.is_empty() && !token.is_empty() {
|
||||
return Ok(Credentials {
|
||||
url,
|
||||
token,
|
||||
username: "-".to_string(),
|
||||
});
|
||||
}
|
||||
}
|
||||
load()
|
||||
}
|
||||
|
||||
/// Delete the on-disk credentials file. Idempotent — silently succeeds
|
||||
/// if the file is already gone (the user already logged out, or never
|
||||
/// logged in to begin with).
|
||||
pub fn delete() -> Result<()> {
|
||||
let path = credentials_path()?;
|
||||
match fs::remove_file(&path) {
|
||||
Ok(()) => Ok(()),
|
||||
Err(err) if err.kind() == std::io::ErrorKind::NotFound => Ok(()),
|
||||
Err(err) => Err(err).with_context(|| format!("removing {}", path.display())),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn save(creds: &Credentials) -> Result<()> {
|
||||
let path = credentials_path()?;
|
||||
if let Some(parent) = path.parent() {
|
||||
|
||||
@@ -14,17 +14,31 @@ mod cmds;
|
||||
mod config;
|
||||
mod output;
|
||||
|
||||
use crate::output::OutputMode;
|
||||
|
||||
#[derive(Parser)]
|
||||
#[command(name = "pic", version, about = "PiCloud command-line client")]
|
||||
struct Cli {
|
||||
/// Output format for `ls` / `show` / `whoami` / `logs` commands.
|
||||
/// TSV stays pipe-friendly; JSON is `jq`-ready.
|
||||
#[arg(long, value_enum, global = true, default_value_t = OutputMode::Tsv)]
|
||||
output: OutputMode,
|
||||
|
||||
#[command(subcommand)]
|
||||
cmd: Cmd,
|
||||
}
|
||||
|
||||
#[derive(Subcommand)]
|
||||
enum Cmd {
|
||||
/// Save URL + bearer token to `~/.picloud/credentials`.
|
||||
Login,
|
||||
/// Authenticate with the server. Default flow prompts for username
|
||||
/// + password and saves the returned session token; `--token` skips
|
||||
/// the password exchange and persists a bearer string directly (use
|
||||
/// this for long-lived API keys minted via `pic api-keys mint`).
|
||||
Login(LoginArgs),
|
||||
|
||||
/// Revoke the saved session server-side and delete the local
|
||||
/// credentials file. Idempotent.
|
||||
Logout,
|
||||
|
||||
/// Print the principal the saved token resolves to.
|
||||
Whoami,
|
||||
@@ -41,8 +55,35 @@ enum Cmd {
|
||||
cmd: ScriptsCmd,
|
||||
},
|
||||
|
||||
/// Long-lived bearer API key management.
|
||||
#[command(name = "api-keys")]
|
||||
ApiKeys {
|
||||
#[command(subcommand)]
|
||||
cmd: ApiKeysCmd,
|
||||
},
|
||||
|
||||
/// Tail recent execution logs for a script.
|
||||
Logs(LogsArgs),
|
||||
|
||||
/// Top-level alias for `pic scripts invoke <id>`.
|
||||
Invoke(InvokeArgs),
|
||||
|
||||
/// Top-level alias for `pic scripts deploy <file> --app <slug>`.
|
||||
Deploy(DeployArgs),
|
||||
}
|
||||
|
||||
#[derive(Args)]
|
||||
struct LoginArgs {
|
||||
/// Override the URL prompt non-interactively. Also reads
|
||||
/// `PICLOUD_URL`.
|
||||
#[arg(long)]
|
||||
url: Option<String>,
|
||||
|
||||
/// Skip the username + password exchange and persist this bearer
|
||||
/// directly (validated against `/auth/me` first). Also reads
|
||||
/// `PICLOUD_TOKEN`.
|
||||
#[arg(long)]
|
||||
token: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Subcommand)]
|
||||
@@ -58,12 +99,23 @@ enum AppsCmd {
|
||||
#[arg(long)]
|
||||
description: Option<String>,
|
||||
},
|
||||
|
||||
/// Show a single app, including the caller's role in it.
|
||||
Show { ident: String },
|
||||
|
||||
/// Delete an app. Without `--force`, the server rejects if the app
|
||||
/// still owns scripts.
|
||||
Delete {
|
||||
ident: String,
|
||||
#[arg(long)]
|
||||
force: bool,
|
||||
},
|
||||
}
|
||||
|
||||
#[derive(Subcommand)]
|
||||
enum ScriptsCmd {
|
||||
/// List scripts. With `--app`, scoped to one app; without,
|
||||
/// iterates over every app the caller can see.
|
||||
/// List scripts. With `--app`, scoped to one app; without, one
|
||||
/// `GET /admin/scripts` for everything the caller can see.
|
||||
Ls {
|
||||
#[arg(long)]
|
||||
app: Option<String>,
|
||||
@@ -71,7 +123,18 @@ enum ScriptsCmd {
|
||||
|
||||
/// Upload a `.rhai` file. Patches the existing script with the
|
||||
/// matching name in `--app` if one exists, otherwise creates it.
|
||||
Deploy {
|
||||
Deploy(DeployArgs),
|
||||
|
||||
/// POST to `/api/v1/execute/{id}`. Body via `--body @path`,
|
||||
/// `--body @-` for stdin, or inline JSON.
|
||||
Invoke(InvokeArgs),
|
||||
|
||||
/// Delete a script. Requires AppAdmin on the owning app.
|
||||
Delete { id: String },
|
||||
}
|
||||
|
||||
#[derive(Args)]
|
||||
struct DeployArgs {
|
||||
file: PathBuf,
|
||||
#[arg(long)]
|
||||
app: String,
|
||||
@@ -79,17 +142,40 @@ enum ScriptsCmd {
|
||||
name: Option<String>,
|
||||
#[arg(long)]
|
||||
description: Option<String>,
|
||||
},
|
||||
}
|
||||
|
||||
/// POST to `/api/v1/execute/{id}`. Body via `--body @path`,
|
||||
/// `--body @-` for stdin, or inline JSON.
|
||||
Invoke {
|
||||
#[derive(Args)]
|
||||
struct InvokeArgs {
|
||||
id: String,
|
||||
#[arg(long)]
|
||||
body: Option<String>,
|
||||
#[arg(short = 'H', long = "header", value_parser = client::parse_kv_header)]
|
||||
headers: Vec<(String, String)>,
|
||||
}
|
||||
|
||||
#[derive(Subcommand)]
|
||||
enum ApiKeysCmd {
|
||||
/// Mint a new long-lived bearer key. Token printed exactly once.
|
||||
Mint {
|
||||
name: String,
|
||||
/// Repeat for multiple scopes: `--scope script:read --scope log:read`.
|
||||
#[arg(long = "scope", required = true)]
|
||||
scopes: Vec<String>,
|
||||
/// Bind the key to a single app (slug or id). Rejects
|
||||
/// `instance:*` scopes when set.
|
||||
#[arg(long)]
|
||||
app: Option<String>,
|
||||
/// Absolute RFC 3339 (`2026-12-31T23:59:59Z`) or shorthand
|
||||
/// `<N>d`/`<N>h`/`<N>m`.
|
||||
#[arg(long)]
|
||||
expires: Option<String>,
|
||||
},
|
||||
|
||||
/// List the caller's keys (no `raw_token` after mint).
|
||||
Ls,
|
||||
|
||||
/// Revoke a key by id.
|
||||
Rm { id: String },
|
||||
}
|
||||
|
||||
#[derive(Args)]
|
||||
@@ -102,10 +188,12 @@ struct LogsArgs {
|
||||
#[tokio::main(flavor = "current_thread")]
|
||||
async fn main() -> ExitCode {
|
||||
let cli = Cli::parse();
|
||||
let mode = cli.output;
|
||||
let result = match cli.cmd {
|
||||
Cmd::Login => cmds::login::run().await,
|
||||
Cmd::Whoami => cmds::whoami::run().await,
|
||||
Cmd::Apps { cmd: AppsCmd::Ls } => cmds::apps::ls().await,
|
||||
Cmd::Login(args) => cmds::login::run(args.url.as_deref(), args.token.as_deref()).await,
|
||||
Cmd::Logout => cmds::logout::run().await,
|
||||
Cmd::Whoami => cmds::whoami::run(mode).await,
|
||||
Cmd::Apps { cmd: AppsCmd::Ls } => cmds::apps::ls(mode).await,
|
||||
Cmd::Apps {
|
||||
cmd:
|
||||
AppsCmd::Create {
|
||||
@@ -114,22 +202,60 @@ async fn main() -> ExitCode {
|
||||
description,
|
||||
},
|
||||
} => cmds::apps::create(&slug, name.as_deref(), description.as_deref()).await,
|
||||
Cmd::Apps {
|
||||
cmd: AppsCmd::Show { ident },
|
||||
} => cmds::apps::show(&ident, mode).await,
|
||||
Cmd::Apps {
|
||||
cmd: AppsCmd::Delete { ident, force },
|
||||
} => cmds::apps::delete(&ident, force).await,
|
||||
Cmd::Scripts {
|
||||
cmd: ScriptsCmd::Ls { app },
|
||||
} => cmds::scripts::ls(app.as_deref()).await,
|
||||
} => cmds::scripts::ls(app.as_deref(), mode).await,
|
||||
Cmd::Scripts {
|
||||
cmd: ScriptsCmd::Deploy(args),
|
||||
} => {
|
||||
cmds::scripts::deploy(
|
||||
&args.file,
|
||||
&args.app,
|
||||
args.name.as_deref(),
|
||||
args.description.as_deref(),
|
||||
)
|
||||
.await
|
||||
}
|
||||
Cmd::Scripts {
|
||||
cmd: ScriptsCmd::Invoke(args),
|
||||
} => cmds::scripts::invoke(&args.id, args.body.as_deref(), &args.headers).await,
|
||||
Cmd::Scripts {
|
||||
cmd: ScriptsCmd::Delete { id },
|
||||
} => cmds::scripts::delete(&id).await,
|
||||
Cmd::ApiKeys {
|
||||
cmd:
|
||||
ScriptsCmd::Deploy {
|
||||
file,
|
||||
app,
|
||||
ApiKeysCmd::Mint {
|
||||
name,
|
||||
description,
|
||||
scopes,
|
||||
app,
|
||||
expires,
|
||||
},
|
||||
} => cmds::scripts::deploy(&file, &app, name.as_deref(), description.as_deref()).await,
|
||||
Cmd::Scripts {
|
||||
cmd: ScriptsCmd::Invoke { id, body, headers },
|
||||
} => cmds::scripts::invoke(&id, body.as_deref(), &headers).await,
|
||||
Cmd::Logs(LogsArgs { script_id, limit }) => cmds::logs::run(&script_id, limit).await,
|
||||
} => cmds::api_keys::mint(&name, &scopes, app.as_deref(), expires.as_deref(), mode).await,
|
||||
Cmd::ApiKeys {
|
||||
cmd: ApiKeysCmd::Ls,
|
||||
} => cmds::api_keys::ls(mode).await,
|
||||
Cmd::ApiKeys {
|
||||
cmd: ApiKeysCmd::Rm { id },
|
||||
} => cmds::api_keys::rm(&id).await,
|
||||
Cmd::Logs(LogsArgs { script_id, limit }) => cmds::logs::run(&script_id, limit, mode).await,
|
||||
Cmd::Invoke(args) => {
|
||||
cmds::scripts::invoke(&args.id, args.body.as_deref(), &args.headers).await
|
||||
}
|
||||
Cmd::Deploy(args) => {
|
||||
cmds::scripts::deploy(
|
||||
&args.file,
|
||||
&args.app,
|
||||
args.name.as_deref(),
|
||||
args.description.as_deref(),
|
||||
)
|
||||
.await
|
||||
}
|
||||
};
|
||||
|
||||
match result {
|
||||
|
||||
@@ -1,11 +1,34 @@
|
||||
//! Tab-separated table writer + error formatting.
|
||||
//! Output rendering for the CLI.
|
||||
//!
|
||||
//! Aligned columns are nice for humans but `\t`-separated stays
|
||||
//! pipe-friendly: `pic apps ls | awk -F'\t' '{print $1}'` works without
|
||||
//! parsing box-drawing.
|
||||
//! Two formats:
|
||||
//! * **TSV** (default): aligned columns separated by `\t`. Stays
|
||||
//! pipe-friendly — `pic apps ls | awk -F'\t' '{print $1}'` works
|
||||
//! without parsing box-drawing.
|
||||
//! * **JSON**: array of `{column: value, …}` objects (for tables) or
|
||||
//! a flat object (for single-row `show`/`whoami`). Designed to be
|
||||
//! `jq`-friendly without escaping the table column names.
|
||||
//!
|
||||
//! Mode is set globally by the top-level `--output` flag and threaded
|
||||
//! through every command. Single-row commands (`whoami`, `apps show`)
|
||||
//! use `KvBlock`; everything plural uses `Table`.
|
||||
|
||||
use std::io::{self, Write};
|
||||
|
||||
use clap::ValueEnum;
|
||||
use serde_json::{Map, Value};
|
||||
|
||||
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, ValueEnum)]
|
||||
#[clap(rename_all = "lowercase")]
|
||||
pub enum OutputMode {
|
||||
#[default]
|
||||
Tsv,
|
||||
Json,
|
||||
}
|
||||
|
||||
// ----------------------------------------------------------------------------
|
||||
// Table — list views (`apps ls`, `scripts ls`, `logs`)
|
||||
// ----------------------------------------------------------------------------
|
||||
|
||||
pub struct Table {
|
||||
headers: Vec<String>,
|
||||
rows: Vec<Vec<String>>,
|
||||
@@ -32,7 +55,7 @@ impl Table {
|
||||
self
|
||||
}
|
||||
|
||||
pub fn render(&self) -> String {
|
||||
pub fn render_tsv(&self) -> String {
|
||||
let mut widths: Vec<usize> = self.headers.iter().map(String::len).collect();
|
||||
for row in &self.rows {
|
||||
for (i, cell) in row.iter().enumerate() {
|
||||
@@ -52,8 +75,36 @@ impl Table {
|
||||
out
|
||||
}
|
||||
|
||||
pub fn print(&self) {
|
||||
let s = self.render();
|
||||
/// JSON form: `[{header: cell, …}, …]`. Cells go in as strings even
|
||||
/// when they happen to look like numbers — the CLI doesn't carry
|
||||
/// type information all the way through (e.g., `version` is already
|
||||
/// `to_string`'d at the call site). Consumers that need typed
|
||||
/// numbers should parse `jq -r '.[].version|tonumber'`.
|
||||
pub fn render_json(&self) -> String {
|
||||
let arr: Vec<Value> = self
|
||||
.rows
|
||||
.iter()
|
||||
.map(|row| {
|
||||
let mut obj = Map::new();
|
||||
for (i, header) in self.headers.iter().enumerate() {
|
||||
let cell = row.get(i).cloned().unwrap_or_default();
|
||||
obj.insert(header.clone(), Value::String(cell));
|
||||
}
|
||||
Value::Object(obj)
|
||||
})
|
||||
.collect();
|
||||
serde_json::to_string_pretty(&Value::Array(arr)).unwrap_or_else(|_| "[]".to_string())
|
||||
}
|
||||
|
||||
pub fn print(&self, mode: OutputMode) {
|
||||
let s = match mode {
|
||||
OutputMode::Tsv => self.render_tsv(),
|
||||
OutputMode::Json => {
|
||||
let mut s = self.render_json();
|
||||
s.push('\n');
|
||||
s
|
||||
}
|
||||
};
|
||||
// Best-effort write — broken pipe from `| head` etc. shouldn't
|
||||
// surface as an error.
|
||||
let _ = io::stdout().write_all(s.as_bytes());
|
||||
@@ -78,26 +129,124 @@ fn write_row(out: &mut String, row: &[String], widths: &[usize]) {
|
||||
out.push('\n');
|
||||
}
|
||||
|
||||
// ----------------------------------------------------------------------------
|
||||
// KvBlock — single-row views (`whoami`, `apps show`)
|
||||
// ----------------------------------------------------------------------------
|
||||
|
||||
/// One row's worth of fields, rendered as aligned `key: value` lines in
|
||||
/// TSV mode (one line per field — easier on the eye than a 1-row table)
|
||||
/// or a flat JSON object.
|
||||
pub struct KvBlock {
|
||||
fields: Vec<(String, String)>,
|
||||
}
|
||||
|
||||
impl KvBlock {
|
||||
pub fn new() -> Self {
|
||||
Self { fields: Vec::new() }
|
||||
}
|
||||
|
||||
pub fn field(&mut self, key: impl Into<String>, value: impl Into<String>) -> &mut Self {
|
||||
self.fields.push((key.into(), value.into()));
|
||||
self
|
||||
}
|
||||
|
||||
pub fn render_tsv(&self) -> String {
|
||||
let key_width = self.fields.iter().map(|(k, _)| k.len()).max().unwrap_or(0);
|
||||
let mut out = String::new();
|
||||
for (k, v) in &self.fields {
|
||||
out.push_str(k);
|
||||
for _ in k.len()..key_width {
|
||||
out.push(' ');
|
||||
}
|
||||
out.push('\t');
|
||||
out.push_str(v);
|
||||
out.push('\n');
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
pub fn render_json(&self) -> String {
|
||||
let mut obj = Map::new();
|
||||
for (k, v) in &self.fields {
|
||||
obj.insert(k.clone(), Value::String(v.clone()));
|
||||
}
|
||||
serde_json::to_string_pretty(&Value::Object(obj)).unwrap_or_else(|_| "{}".to_string())
|
||||
}
|
||||
|
||||
pub fn print(&self, mode: OutputMode) {
|
||||
let s = match mode {
|
||||
OutputMode::Tsv => self.render_tsv(),
|
||||
OutputMode::Json => {
|
||||
let mut s = self.render_json();
|
||||
s.push('\n');
|
||||
s
|
||||
}
|
||||
};
|
||||
let _ = io::stdout().write_all(s.as_bytes());
|
||||
}
|
||||
}
|
||||
|
||||
// ----------------------------------------------------------------------------
|
||||
// Errors
|
||||
// ----------------------------------------------------------------------------
|
||||
|
||||
pub fn print_error(err: &anyhow::Error) {
|
||||
let mut stderr = io::stderr();
|
||||
let _ = writeln!(stderr, "error: {err:#}");
|
||||
}
|
||||
|
||||
// ----------------------------------------------------------------------------
|
||||
// Tests
|
||||
// ----------------------------------------------------------------------------
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn table_aligns_columns() {
|
||||
fn table_aligns_columns_tsv() {
|
||||
let mut t = Table::new(["slug", "name"]);
|
||||
t.row(["a", "Alpha"]).row(["bravo", "B"]);
|
||||
let out = t.render();
|
||||
let out = t.render_tsv();
|
||||
assert_eq!(out, "slug \tname\na \tAlpha\nbravo\tB\n");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn table_empty_rows() {
|
||||
fn table_empty_rows_tsv() {
|
||||
let t = Table::new(["a", "b"]);
|
||||
assert_eq!(t.render(), "a\tb\n");
|
||||
assert_eq!(t.render_tsv(), "a\tb\n");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn table_render_json_is_array_of_objects() {
|
||||
let mut t = Table::new(["slug", "name"]);
|
||||
t.row(["a", "Alpha"]).row(["bravo", "B"]);
|
||||
let raw = t.render_json();
|
||||
let v: Value = serde_json::from_str(&raw).expect("valid JSON");
|
||||
let arr = v.as_array().expect("array");
|
||||
assert_eq!(arr.len(), 2);
|
||||
assert_eq!(arr[0]["slug"], "a");
|
||||
assert_eq!(arr[0]["name"], "Alpha");
|
||||
assert_eq!(arr[1]["slug"], "bravo");
|
||||
assert_eq!(arr[1]["name"], "B");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn kv_block_tsv_aligns_keys() {
|
||||
let mut b = KvBlock::new();
|
||||
b.field("username", "admin").field("role", "owner");
|
||||
let out = b.render_tsv();
|
||||
// username (8 chars) defines the key width.
|
||||
assert_eq!(out, "username\tadmin\nrole \towner\n");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn kv_block_json_is_flat_object() {
|
||||
let mut b = KvBlock::new();
|
||||
b.field("username", "admin").field("role", "owner");
|
||||
let raw = b.render_json();
|
||||
let v: Value = serde_json::from_str(&raw).expect("valid JSON");
|
||||
assert_eq!(v["username"], "admin");
|
||||
assert_eq!(v["role"], "owner");
|
||||
}
|
||||
}
|
||||
|
||||
170
crates/picloud-cli/tests/api_keys.rs
Normal file
170
crates/picloud-cli/tests/api_keys.rs
Normal file
@@ -0,0 +1,170 @@
|
||||
//! `pic api-keys` — mint / ls / rm journeys.
|
||||
//!
|
||||
//! Server semantics asserted here:
|
||||
//! * `mint` emits the `raw_token` *exactly once* and never on `ls`.
|
||||
//! * A minted key is a valid bearer for `/auth/me`.
|
||||
//! * After `rm`, the same token is rejected (401).
|
||||
|
||||
use predicates::prelude::*;
|
||||
use serde_json::Value;
|
||||
|
||||
use crate::common;
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn mint_prints_raw_token_once_and_ls_omits_it() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
let name = format!("pic-cli-mint-{}", common::unique_slug("k"));
|
||||
|
||||
let out = common::pic_as(&env)
|
||||
.args([
|
||||
"--output",
|
||||
"json",
|
||||
"api-keys",
|
||||
"mint",
|
||||
&name,
|
||||
"--scope",
|
||||
"script:read",
|
||||
])
|
||||
.output()
|
||||
.expect("api-keys mint");
|
||||
assert!(out.status.success(), "mint failed: {out:?}");
|
||||
let body: Value = serde_json::from_slice(&out.stdout).expect("JSON");
|
||||
let token = body["token"]
|
||||
.as_str()
|
||||
.expect("mint should expose `token`")
|
||||
.to_string();
|
||||
let key_id = body["id"]
|
||||
.as_str()
|
||||
.expect("mint should expose `id`")
|
||||
.to_string();
|
||||
assert!(
|
||||
token.starts_with("pic_"),
|
||||
"tokens are pic_-prefixed: {token}"
|
||||
);
|
||||
|
||||
// `ls` must NEVER carry the raw token. The key row should appear,
|
||||
// identified by name, but `token` is mint-only.
|
||||
let ls = common::pic_as(&env)
|
||||
.args(["--output", "json", "api-keys", "ls"])
|
||||
.output()
|
||||
.expect("api-keys ls");
|
||||
assert!(ls.status.success(), "ls failed: {ls:?}");
|
||||
let ls_body: Value = serde_json::from_slice(&ls.stdout).expect("JSON");
|
||||
let arr = ls_body.as_array().expect("array");
|
||||
let row = arr
|
||||
.iter()
|
||||
.find(|r| r.get("id").and_then(Value::as_str) == Some(key_id.as_str()))
|
||||
.expect("our key in ls");
|
||||
assert!(
|
||||
row.get("token").is_none(),
|
||||
"ls must not expose raw_token: {row}"
|
||||
);
|
||||
|
||||
// Cleanup so we don't leak keys across runs.
|
||||
common::pic_as(&env)
|
||||
.args(["api-keys", "rm", &key_id])
|
||||
.assert()
|
||||
.success();
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn minted_key_works_as_bearer() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
let name = format!("pic-cli-bearer-{}", common::unique_slug("k"));
|
||||
|
||||
let mint = common::pic_as(&env)
|
||||
.args([
|
||||
"--output",
|
||||
"json",
|
||||
"api-keys",
|
||||
"mint",
|
||||
&name,
|
||||
"--scope",
|
||||
"script:read",
|
||||
])
|
||||
.output()
|
||||
.expect("mint");
|
||||
assert!(mint.status.success());
|
||||
let body: Value = serde_json::from_slice(&mint.stdout).unwrap();
|
||||
let token = body["token"].as_str().unwrap().to_string();
|
||||
let id = body["id"].as_str().unwrap().to_string();
|
||||
|
||||
// Drive whoami with the minted token — proves the bearer string we
|
||||
// captured really is what the server stamped.
|
||||
let key_env = common::custom_env(&fx.url, &token);
|
||||
common::seed_credentials(&key_env, &fx.admin_username);
|
||||
common::pic_as(&key_env)
|
||||
.args(["whoami"])
|
||||
.assert()
|
||||
.success()
|
||||
.stdout(predicate::str::contains(fx.admin_username.as_str()));
|
||||
|
||||
common::pic_as(&env)
|
||||
.args(["api-keys", "rm", &id])
|
||||
.assert()
|
||||
.success();
|
||||
}
|
||||
|
||||
/// After `rm`, the bearer token is dead server-side: a follow-up
|
||||
/// `whoami` driven by it must 401, not 500.
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn rm_revokes_the_token() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
let name = format!("pic-cli-rm-{}", common::unique_slug("k"));
|
||||
|
||||
let mint = common::pic_as(&env)
|
||||
.args([
|
||||
"--output",
|
||||
"json",
|
||||
"api-keys",
|
||||
"mint",
|
||||
&name,
|
||||
"--scope",
|
||||
"script:read",
|
||||
])
|
||||
.output()
|
||||
.expect("mint");
|
||||
let body: Value = serde_json::from_slice(&mint.stdout).unwrap();
|
||||
let token = body["token"].as_str().unwrap().to_string();
|
||||
let id = body["id"].as_str().unwrap().to_string();
|
||||
|
||||
common::pic_as(&env)
|
||||
.args(["api-keys", "rm", &id])
|
||||
.assert()
|
||||
.success()
|
||||
.stdout(predicate::str::contains(format!("Revoked api-key {id}")));
|
||||
|
||||
let dead = common::custom_env(&fx.url, &token);
|
||||
common::pic_as(&dead)
|
||||
.args(["whoami"])
|
||||
.assert()
|
||||
.failure()
|
||||
.stderr(predicate::str::contains("HTTP 401"));
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn mint_with_unknown_scope_is_rejected_client_side() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
|
||||
common::pic_as(&env)
|
||||
.args(["api-keys", "mint", "doomed", "--scope", "script:nope"])
|
||||
.assert()
|
||||
.failure()
|
||||
.stderr(predicate::str::contains("unknown scope"));
|
||||
}
|
||||
268
crates/picloud-cli/tests/apps.rs
Normal file
268
crates/picloud-cli/tests/apps.rs
Normal file
@@ -0,0 +1,268 @@
|
||||
//! `pic apps create` / `pic apps ls` edge cases. The integration smoke
|
||||
//! test covers the happy path; this module covers conflict, validation,
|
||||
//! and the persistence of the optional `--name` / `--description` flags
|
||||
//! (which `apps ls` doesn't surface).
|
||||
|
||||
use predicates::prelude::*;
|
||||
use serde_json::Value;
|
||||
|
||||
use crate::common;
|
||||
use crate::common::cleanup::AppGuard;
|
||||
use crate::common::member;
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn create_with_name_and_description_persists() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
let slug = common::unique_slug("apps-named");
|
||||
|
||||
common::pic_as(&env)
|
||||
.args([
|
||||
"apps",
|
||||
"create",
|
||||
&slug,
|
||||
"--name",
|
||||
"Pretty Name",
|
||||
"--description",
|
||||
"test description",
|
||||
])
|
||||
.assert()
|
||||
.success();
|
||||
let _guard = AppGuard::new(&env.url, &env.token, &slug);
|
||||
|
||||
// `apps ls` only shows slug+name+role+created_at, so verify the
|
||||
// persisted shape via the admin GET endpoint.
|
||||
let client = reqwest::blocking::Client::new();
|
||||
let resp = client
|
||||
.get(format!("{}/api/v1/admin/apps/{}", env.url, slug))
|
||||
.bearer_auth(&env.token)
|
||||
.send()
|
||||
.expect("GET app");
|
||||
assert!(resp.status().is_success(), "GET app failed: {resp:?}");
|
||||
let body: Value = resp.json().expect("app json");
|
||||
assert_eq!(body["slug"].as_str(), Some(slug.as_str()));
|
||||
assert_eq!(body["name"].as_str(), Some("Pretty Name"));
|
||||
assert_eq!(body["description"].as_str(), Some("test description"));
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn create_duplicate_slug_conflicts() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
let slug = common::unique_slug("apps-dup");
|
||||
|
||||
common::pic_as(&env)
|
||||
.args(["apps", "create", &slug])
|
||||
.assert()
|
||||
.success();
|
||||
let _guard = AppGuard::new(&env.url, &env.token, &slug);
|
||||
|
||||
common::pic_as(&env)
|
||||
.args(["apps", "create", &slug])
|
||||
.assert()
|
||||
.failure()
|
||||
.stderr(predicate::str::contains("409").or(predicate::str::contains("conflict")));
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn create_invalid_slug_rejected() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
|
||||
// Server slug regex is `^[a-z0-9][a-z0-9-]{0,62}$` — uppercase
|
||||
// breaks the rule on the very first char. The server returns 422
|
||||
// (`InvalidSlug` → `UNPROCESSABLE_ENTITY`), not 400 — the previous
|
||||
// `"HTTP 4"` predicate would have silently matched any other 4xx
|
||||
// (a regressed 401 from broken auth, for example).
|
||||
common::pic_as(&env)
|
||||
.args(["apps", "create", "NotALowerSlug"])
|
||||
.assert()
|
||||
.failure()
|
||||
.stderr(predicate::str::contains("HTTP 422"));
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn ls_includes_created_app_with_expected_columns() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
let slug = common::unique_slug("apps-ls");
|
||||
|
||||
common::pic_as(&env)
|
||||
.args(["apps", "create", &slug])
|
||||
.assert()
|
||||
.success();
|
||||
let _guard = AppGuard::new(&env.url, &env.token, &slug);
|
||||
|
||||
let out = common::pic_as(&env)
|
||||
.args(["apps", "ls"])
|
||||
.output()
|
||||
.expect("apps ls");
|
||||
assert!(out.status.success(), "apps ls failed: {out:?}");
|
||||
let stdout = String::from_utf8(out.stdout).expect("utf8 stdout");
|
||||
let mut lines = stdout.lines();
|
||||
let header = lines.next().expect("header row");
|
||||
assert_eq!(
|
||||
common::cells(header),
|
||||
vec!["slug", "name", "my_role", "created_at"]
|
||||
);
|
||||
|
||||
// The slug must appear in some data row and its row's my_role column
|
||||
// is dashed (the ls endpoint doesn't compute it per-app).
|
||||
let row = lines
|
||||
.map(common::cells)
|
||||
.find(|c| c.first().copied() == Some(slug.as_str()))
|
||||
.unwrap_or_else(|| panic!("slug {slug} not in apps ls output: {stdout}"));
|
||||
assert_eq!(row.len(), 4, "row should have 4 cells: {row:?}");
|
||||
assert_eq!(row[2], "-", "my_role column should be dashed: {row:?}");
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn delete_removes_app_from_ls() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
let slug = common::unique_slug("apps-del");
|
||||
|
||||
common::pic_as(&env)
|
||||
.args(["apps", "create", &slug])
|
||||
.assert()
|
||||
.success();
|
||||
|
||||
common::pic_as(&env)
|
||||
.args(["apps", "delete", &slug])
|
||||
.assert()
|
||||
.success()
|
||||
.stdout(predicate::str::contains(format!("Deleted app {slug}")));
|
||||
|
||||
let out = common::pic_as(&env)
|
||||
.args(["apps", "ls"])
|
||||
.output()
|
||||
.expect("apps ls");
|
||||
assert!(out.status.success());
|
||||
let stdout = String::from_utf8(out.stdout).unwrap();
|
||||
assert!(
|
||||
!stdout.lines().any(|l| l.starts_with(&slug)),
|
||||
"deleted slug should not appear in ls: {stdout}"
|
||||
);
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn delete_with_scripts_errors_without_force() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
let slug = common::unique_slug("apps-del-busy");
|
||||
common::pic_as(&env)
|
||||
.args(["apps", "create", &slug])
|
||||
.assert()
|
||||
.success();
|
||||
// AppGuard is the safety net: if the no-force delete fails (as
|
||||
// expected) the app stays around; AppGuard force-deletes on drop.
|
||||
let _guard = AppGuard::new(&env.url, &env.token, &slug);
|
||||
|
||||
let fixture = common::fixture_path("hello.rhai");
|
||||
common::pic_as(&env)
|
||||
.args([
|
||||
"scripts",
|
||||
"deploy",
|
||||
fixture.to_str().unwrap(),
|
||||
"--app",
|
||||
&slug,
|
||||
])
|
||||
.assert()
|
||||
.success();
|
||||
|
||||
common::pic_as(&env)
|
||||
.args(["apps", "delete", &slug])
|
||||
.assert()
|
||||
.failure()
|
||||
// Server `HasScripts` → 409 with a "scripts present" message.
|
||||
.stderr(predicate::str::contains("HTTP 409"));
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn delete_with_scripts_succeeds_with_force() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
let slug = common::unique_slug("apps-del-force");
|
||||
common::pic_as(&env)
|
||||
.args(["apps", "create", &slug])
|
||||
.assert()
|
||||
.success();
|
||||
|
||||
let fixture = common::fixture_path("hello.rhai");
|
||||
common::pic_as(&env)
|
||||
.args([
|
||||
"scripts",
|
||||
"deploy",
|
||||
fixture.to_str().unwrap(),
|
||||
"--app",
|
||||
&slug,
|
||||
])
|
||||
.assert()
|
||||
.success();
|
||||
|
||||
common::pic_as(&env)
|
||||
.args(["apps", "delete", &slug, "--force"])
|
||||
.assert()
|
||||
.success()
|
||||
.stdout(predicate::str::contains(format!("Deleted app {slug}")));
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn show_prints_my_role_for_member() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let admin_env = common::admin_env(fx);
|
||||
let slug = common::unique_slug("apps-show");
|
||||
common::pic_as(&admin_env)
|
||||
.args(["apps", "create", &slug])
|
||||
.assert()
|
||||
.success();
|
||||
let _g = AppGuard::new(&admin_env.url, &admin_env.token, &slug);
|
||||
|
||||
let m = member::member_user(fx, &common::unique_username("show"));
|
||||
member::grant_membership(fx, &slug, &m.id, "viewer");
|
||||
|
||||
let member_env = common::custom_env(&fx.url, &m.token);
|
||||
common::seed_credentials(&member_env, &m.username);
|
||||
|
||||
let out = common::pic_as(&member_env)
|
||||
.args(["apps", "show", &slug])
|
||||
.output()
|
||||
.expect("apps show");
|
||||
assert!(out.status.success(), "apps show failed: {out:?}");
|
||||
let stdout = String::from_utf8(out.stdout).unwrap();
|
||||
// KvBlock output: `my_role` row carries the wire form (`viewer`).
|
||||
assert!(
|
||||
stdout
|
||||
.lines()
|
||||
.any(|l| l.starts_with("my_role") && l.trim_end().ends_with("viewer")),
|
||||
"show should surface my_role=viewer, got: {stdout}"
|
||||
);
|
||||
assert!(
|
||||
stdout.lines().any(|l| l.starts_with("slug")),
|
||||
"show should include slug row: {stdout}"
|
||||
);
|
||||
}
|
||||
288
crates/picloud-cli/tests/auth.rs
Normal file
288
crates/picloud-cli/tests/auth.rs
Normal file
@@ -0,0 +1,288 @@
|
||||
//! Login + whoami journeys beyond the happy path: bad tokens, missing
|
||||
//! credentials file, stale on-disk creds, and the role-label rendered
|
||||
//! by `pic login`.
|
||||
|
||||
use predicates::prelude::*;
|
||||
|
||||
use crate::common;
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn login_persists_credentials_with_correct_perms() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
|
||||
common::pic_as(&env).args(["login"]).assert().success();
|
||||
|
||||
let creds_path = env.config_dir.path().join("credentials");
|
||||
let body = std::fs::read_to_string(&creds_path).expect("credentials file");
|
||||
assert!(
|
||||
body.contains(&format!("url = \"{}\"", env.url)),
|
||||
"creds missing url line: {body}",
|
||||
);
|
||||
assert!(
|
||||
body.contains(&format!("token = \"{}\"", env.token)),
|
||||
"creds missing token line: {body}",
|
||||
);
|
||||
assert!(
|
||||
body.contains(&format!("username = \"{}\"", fx.admin_username)),
|
||||
"creds missing username line: {body}",
|
||||
);
|
||||
|
||||
#[cfg(unix)]
|
||||
{
|
||||
use std::os::unix::fs::PermissionsExt;
|
||||
let mode = std::fs::metadata(&creds_path).unwrap().permissions().mode() & 0o777;
|
||||
assert_eq!(mode, 0o600, "credentials file must be 0600, got {mode:o}");
|
||||
}
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn login_rejects_bad_token() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::custom_env(&fx.url, "pic_garbage_token");
|
||||
|
||||
common::pic_as(&env)
|
||||
.args(["login"])
|
||||
.assert()
|
||||
.failure()
|
||||
.stderr(predicate::str::contains("401").or(predicate::str::contains("token rejected")));
|
||||
|
||||
let creds_path = env.config_dir.path().join("credentials");
|
||||
assert!(
|
||||
!creds_path.exists(),
|
||||
"failed login must not persist credentials"
|
||||
);
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn whoami_without_credentials_errors() {
|
||||
let Some(_fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
// Build a TestEnv directly so the config dir stays empty —
|
||||
// `admin_env` would seed a credentials file, masking the bug
|
||||
// this test is supposed to catch.
|
||||
let env = common::TestEnv {
|
||||
url: String::new(),
|
||||
token: String::new(),
|
||||
config_dir: tempfile::TempDir::new().unwrap(),
|
||||
home: tempfile::TempDir::new().unwrap(),
|
||||
};
|
||||
|
||||
common::pic_no_env(&env)
|
||||
.args(["whoami"])
|
||||
.assert()
|
||||
.failure()
|
||||
.stderr(predicate::str::contains("pic login"));
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn whoami_with_stale_token_errors() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
|
||||
let body = format!(
|
||||
"url = \"{}\"\ntoken = \"pic_stale_token\"\nusername = \"ghost\"\n",
|
||||
env.url
|
||||
);
|
||||
std::fs::write(env.config_dir.path().join("credentials"), body).unwrap();
|
||||
|
||||
common::pic_no_env(&env)
|
||||
.args(["whoami"])
|
||||
.assert()
|
||||
.failure()
|
||||
.stderr(predicate::str::contains("401").or(predicate::str::contains("token rejected")));
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn login_prints_member_role_label() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let username = common::unique_username("auth");
|
||||
let m = common::member::member_user(fx, &username);
|
||||
let env = common::custom_env(&fx.url, &m.token);
|
||||
|
||||
common::pic_as(&env)
|
||||
.args(["login"])
|
||||
.assert()
|
||||
.success()
|
||||
.stdout(predicate::str::contains(format!(
|
||||
"Logged in as {} (member)",
|
||||
m.username
|
||||
)));
|
||||
}
|
||||
|
||||
/// Drive the real username+password flow end-to-end. `pic_no_env`
|
||||
/// strips `PICLOUD_TOKEN` so login can't short-circuit through the
|
||||
/// bearer path; stdin feeds `username\npassword\n` (the URL is supplied
|
||||
/// via `--url` to avoid the third prompt).
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn login_with_username_and_password_persists() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let username = common::unique_username("lpw");
|
||||
let m = common::member::member_user(fx, &username);
|
||||
let env = common::custom_env(&fx.url, ""); // empty token — file gets written by login
|
||||
|
||||
let stdin_payload = format!("{}\n{}\n", m.username, common::member::MEMBER_PASSWORD);
|
||||
common::pic_no_env(&env)
|
||||
.args(["login", "--url", &fx.url])
|
||||
.write_stdin(stdin_payload)
|
||||
.assert()
|
||||
.success()
|
||||
.stdout(predicate::str::contains(format!(
|
||||
"Logged in as {} (member)",
|
||||
m.username
|
||||
)));
|
||||
|
||||
let creds_path = env.config_dir.path().join("credentials");
|
||||
let body = std::fs::read_to_string(&creds_path).expect("credentials file");
|
||||
assert!(
|
||||
body.contains(&format!("username = \"{}\"", m.username)),
|
||||
"creds should carry the canonical username: {body}",
|
||||
);
|
||||
// The token persisted must be a real session token, not whatever
|
||||
// the user typed — a regression where we accidentally saved the
|
||||
// password as the token would fail this check.
|
||||
assert!(
|
||||
!body.contains(common::member::MEMBER_PASSWORD),
|
||||
"password leaked into credentials file: {body}",
|
||||
);
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn login_with_wrong_password_errors() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let username = common::unique_username("lpwbad");
|
||||
let m = common::member::member_user(fx, &username);
|
||||
let env = common::custom_env(&fx.url, "");
|
||||
|
||||
let stdin_payload = format!("{}\nwrong-password\n", m.username);
|
||||
common::pic_no_env(&env)
|
||||
.args(["login", "--url", &fx.url])
|
||||
.write_stdin(stdin_payload)
|
||||
.assert()
|
||||
.failure()
|
||||
.stderr(predicate::str::contains("HTTP 401"));
|
||||
|
||||
let creds_path = env.config_dir.path().join("credentials");
|
||||
assert!(
|
||||
!creds_path.exists(),
|
||||
"failed login must not persist credentials"
|
||||
);
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn logout_clears_local_credentials() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
// Use a member's token so we don't yank the admin session out from
|
||||
// under parallel tests. The local-file cleanup is the same.
|
||||
let username = common::unique_username("lout");
|
||||
let m = common::member::member_user(fx, &username);
|
||||
let env = common::custom_env(&fx.url, &m.token);
|
||||
common::seed_credentials(&env, &m.username);
|
||||
|
||||
let creds_path = env.config_dir.path().join("credentials");
|
||||
assert!(creds_path.exists(), "precondition: creds file seeded");
|
||||
|
||||
common::pic_no_env(&env)
|
||||
.args(["logout"])
|
||||
.assert()
|
||||
.success()
|
||||
.stdout(predicate::str::contains("Logged out"));
|
||||
assert!(
|
||||
!creds_path.exists(),
|
||||
"credentials file should be removed after logout"
|
||||
);
|
||||
}
|
||||
|
||||
/// `pic logout` is meant to be idempotent: running it with no
|
||||
/// credentials file present is not an error.
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn logout_is_idempotent_when_already_logged_out() {
|
||||
let Some(_fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::TestEnv {
|
||||
url: String::new(),
|
||||
token: String::new(),
|
||||
config_dir: tempfile::TempDir::new().unwrap(),
|
||||
home: tempfile::TempDir::new().unwrap(),
|
||||
};
|
||||
common::pic_no_env(&env)
|
||||
.args(["logout"])
|
||||
.assert()
|
||||
.success()
|
||||
.stdout(predicate::str::contains("Logged out"));
|
||||
}
|
||||
|
||||
/// Server-side session invalidation: after `pic logout`, a subsequent
|
||||
/// `pic whoami` driven by the same (now-stale) token must 401.
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn logout_invalidates_server_session() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let username = common::unique_username("lout2");
|
||||
let m = common::member::member_user(fx, &username);
|
||||
let env = common::custom_env(&fx.url, &m.token);
|
||||
common::seed_credentials(&env, &m.username);
|
||||
|
||||
common::pic_no_env(&env).args(["logout"]).assert().success();
|
||||
|
||||
// Replay the member's old token explicitly — pic_no_env reads the
|
||||
// (now-deleted) file, so we go back to env-driven mode with the
|
||||
// stale bearer.
|
||||
let stale = common::custom_env(&fx.url, &m.token);
|
||||
common::pic_as(&stale)
|
||||
.args(["whoami"])
|
||||
.assert()
|
||||
.failure()
|
||||
.stderr(predicate::str::contains("HTTP 401"));
|
||||
}
|
||||
|
||||
/// Env vars must override the on-disk credentials file globally. Write
|
||||
/// garbage into the file, set env to the real admin creds, and prove
|
||||
/// every read-side command (here `whoami`) goes via env.
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn env_vars_override_credentials_file() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::custom_env(&fx.url, &fx.admin_token);
|
||||
// Garbage in the file: would 401 if used.
|
||||
let body = format!(
|
||||
"url = \"{}\"\ntoken = \"pic_stale_garbage_token\"\nusername = \"ghost\"\n",
|
||||
env.url
|
||||
);
|
||||
std::fs::write(env.config_dir.path().join("credentials"), body).unwrap();
|
||||
|
||||
common::pic_as(&env)
|
||||
.args(["whoami"])
|
||||
.assert()
|
||||
.success()
|
||||
.stdout(predicate::str::contains(fx.admin_username.as_str()));
|
||||
}
|
||||
@@ -12,4 +12,12 @@
|
||||
//! cargo test -p picloud-cli --test cli -- --include-ignored
|
||||
|
||||
mod common;
|
||||
mod integration;
|
||||
|
||||
mod api_keys;
|
||||
mod apps;
|
||||
mod auth;
|
||||
mod invoke;
|
||||
mod logs;
|
||||
mod output;
|
||||
mod roles;
|
||||
mod scripts;
|
||||
|
||||
@@ -30,7 +30,9 @@ pub fn member_user(fx: &Fixture, username: &str) -> MemberUser {
|
||||
.json(&json!({
|
||||
"username": username,
|
||||
"password": MEMBER_PASSWORD,
|
||||
"instance_role": "Member",
|
||||
// InstanceRole / AppRole serialize via `rename_all =
|
||||
// "snake_case"` — wire forms are always lowercase.
|
||||
"instance_role": "member",
|
||||
}))
|
||||
.send()
|
||||
.expect("create member user");
|
||||
|
||||
@@ -89,20 +89,27 @@ pub struct TestEnv {
|
||||
pub home: TempDir,
|
||||
}
|
||||
|
||||
/// Per-test env pre-loaded with the admin token. Mirrors what the seed
|
||||
/// test built inline.
|
||||
/// Per-test env pre-loaded with the admin token, and a credentials
|
||||
/// file already on disk so non-login commands ("pic apps create", …)
|
||||
/// can run without first calling `pic login`. As of the env-var
|
||||
/// consistency fix, `PICLOUD_URL`/`PICLOUD_TOKEN` (set by `pic_as`)
|
||||
/// also work for *every* command, not just `login` — `config::resolve`
|
||||
/// reads them first and falls back to the on-disk file.
|
||||
pub fn admin_env(fx: &Fixture) -> TestEnv {
|
||||
TestEnv {
|
||||
let env = TestEnv {
|
||||
url: fx.url.clone(),
|
||||
token: fx.admin_token.clone(),
|
||||
config_dir: TempDir::new().expect("config tempdir"),
|
||||
home: TempDir::new().expect("home tempdir"),
|
||||
}
|
||||
};
|
||||
seed_credentials(&env, &fx.admin_username);
|
||||
env
|
||||
}
|
||||
|
||||
/// Per-test env pre-loaded with a specific (URL, token) pair. Used by
|
||||
/// tests that want a non-admin token, a bogus token, or an unreachable
|
||||
/// URL.
|
||||
/// URL. Does **not** seed a credentials file — call `seed_credentials`
|
||||
/// explicitly when the test needs to run non-login commands.
|
||||
pub fn custom_env(url: &str, token: &str) -> TestEnv {
|
||||
TestEnv {
|
||||
url: url.to_string(),
|
||||
@@ -112,6 +119,19 @@ pub fn custom_env(url: &str, token: &str) -> TestEnv {
|
||||
}
|
||||
}
|
||||
|
||||
/// Write a valid credentials TOML into `env.config_dir` so subsequent
|
||||
/// `pic_as(&env)` invocations can issue non-login subcommands. Mirrors
|
||||
/// the file shape `pic login` produces (url/token/username). Tests that
|
||||
/// exercise the "no credentials" / "stale token" error paths construct
|
||||
/// `TestEnv` directly to keep the config dir empty.
|
||||
pub fn seed_credentials(env: &TestEnv, username: &str) {
|
||||
let body = format!(
|
||||
"url = \"{}\"\ntoken = \"{}\"\nusername = \"{}\"\n",
|
||||
env.url, env.token, username,
|
||||
);
|
||||
std::fs::write(env.config_dir.path().join("credentials"), body).expect("seed credentials file");
|
||||
}
|
||||
|
||||
/// `pic` invocation with the env wired up — credentials dir, HOME, and
|
||||
/// the `PICLOUD_URL`/`PICLOUD_TOKEN` shortcut env vars.
|
||||
pub fn pic_as(env: &TestEnv) -> AssertCommand {
|
||||
@@ -183,6 +203,53 @@ pub fn fixture_path(name: &str) -> PathBuf {
|
||||
.join(name)
|
||||
}
|
||||
|
||||
/// Create a fresh app and deploy a `tests/fixtures/<fixture_name>` into
|
||||
/// it. Returns the new script id plus the `AppGuard` that cleans the
|
||||
/// app (and its scripts via `force=true`) on Drop. Used by invoke /
|
||||
/// logs / output journeys that all need "deploy something, then drive
|
||||
/// `pic` against it".
|
||||
pub fn deploy_fixture(
|
||||
env: &TestEnv,
|
||||
app_label: &str,
|
||||
fixture_name: &str,
|
||||
) -> (String, cleanup::AppGuard) {
|
||||
let slug = unique_slug(app_label);
|
||||
pic_as(env)
|
||||
.args(["apps", "create", &slug])
|
||||
.assert()
|
||||
.success();
|
||||
let guard = cleanup::AppGuard::new(&env.url, &env.token, &slug);
|
||||
|
||||
let fixture = fixture_path(fixture_name);
|
||||
pic_as(env)
|
||||
.args([
|
||||
"scripts",
|
||||
"deploy",
|
||||
fixture.to_str().unwrap(),
|
||||
"--app",
|
||||
&slug,
|
||||
])
|
||||
.assert()
|
||||
.success();
|
||||
|
||||
let out = pic_as(env)
|
||||
.args(["scripts", "ls", "--app", &slug])
|
||||
.output()
|
||||
.expect("scripts ls");
|
||||
let id = parse_first_id(std::str::from_utf8(&out.stdout).unwrap())
|
||||
.expect("scripts ls should produce one row");
|
||||
(id, guard)
|
||||
}
|
||||
|
||||
/// Split a row from `pic apps ls` / `pic scripts ls` into trimmed
|
||||
/// cells. The output writer space-pads each cell to its column's max
|
||||
/// width before the tab, so raw `split('\t')` leaves trailing spaces;
|
||||
/// this helper hides that detail from tests that only care about the
|
||||
/// logical values.
|
||||
pub fn cells(row: &str) -> Vec<&str> {
|
||||
row.split('\t').map(str::trim).collect()
|
||||
}
|
||||
|
||||
/// First data row's first tab-delimited cell, used to extract IDs from
|
||||
/// `pic scripts ls` output. The header is expected to start with "id".
|
||||
pub fn parse_first_id(table: &str) -> Option<String> {
|
||||
|
||||
7
crates/picloud-cli/tests/fixtures/boom.rhai
vendored
Normal file
7
crates/picloud-cli/tests/fixtures/boom.rhai
vendored
Normal file
@@ -0,0 +1,7 @@
|
||||
// Returns a structured 500. The execution is still `Success` in the
|
||||
// log because the script ran cleanly — for an `Error`-status log entry
|
||||
// use throw.rhai instead.
|
||||
#{
|
||||
statusCode: 500,
|
||||
body: #{ ok: false, why: "intentional" },
|
||||
}
|
||||
6
crates/picloud-cli/tests/fixtures/echo.rhai
vendored
Normal file
6
crates/picloud-cli/tests/fixtures/echo.rhai
vendored
Normal file
@@ -0,0 +1,6 @@
|
||||
// Echoes the request body and headers back so invoke tests can verify
|
||||
// that `--body` (inline / @file / @-) and `-H` flow through end-to-end.
|
||||
#{
|
||||
body: ctx.request.body,
|
||||
headers: ctx.request.headers,
|
||||
}
|
||||
5
crates/picloud-cli/tests/fixtures/loud.rhai
vendored
Normal file
5
crates/picloud-cli/tests/fixtures/loud.rhai
vendored
Normal file
@@ -0,0 +1,5 @@
|
||||
// Logs a long line so the logs-truncation test has something to chew on.
|
||||
// `pic logs` truncates the summary cell to 120 characters; this line is
|
||||
// 240 chars after the prefix so the truncation is unambiguous.
|
||||
log::info("xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx");
|
||||
#{ ok: true }
|
||||
4
crates/picloud-cli/tests/fixtures/throw.rhai
vendored
Normal file
4
crates/picloud-cli/tests/fixtures/throw.rhai
vendored
Normal file
@@ -0,0 +1,4 @@
|
||||
// Throws a Rhai runtime error. The orchestrator records this as
|
||||
// `ExecutionStatus::Error` in the execution log (a structured 5xx
|
||||
// response is recorded as `Success`).
|
||||
throw "boom";
|
||||
@@ -1,131 +0,0 @@
|
||||
//! End-to-end smoke test: login → whoami → apps create → apps ls →
|
||||
//! scripts deploy (create + update) → scripts ls → scripts invoke →
|
||||
//! logs. The original seed test, refactored to run against the shared
|
||||
//! fixture so subsequent journey modules don't each pay for a server
|
||||
//! spawn.
|
||||
|
||||
#![allow(clippy::too_many_lines)]
|
||||
|
||||
use predicates::prelude::*;
|
||||
use serde_json::Value;
|
||||
|
||||
use crate::common;
|
||||
use crate::common::cleanup::AppGuard;
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn end_to_end_login_deploy_invoke_logs() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
let slug = common::unique_slug("e2e");
|
||||
let username = &fx.admin_username;
|
||||
|
||||
// 1) login
|
||||
common::pic_as(&env)
|
||||
.args(["login"])
|
||||
.assert()
|
||||
.success()
|
||||
.stdout(predicate::str::contains(format!("Logged in as {username}")));
|
||||
|
||||
let creds_path = env.config_dir.path().join("credentials");
|
||||
assert!(
|
||||
creds_path.exists(),
|
||||
"credentials file should exist after login"
|
||||
);
|
||||
let body = std::fs::read_to_string(&creds_path).unwrap();
|
||||
assert!(body.contains(&env.url), "creds should contain url: {body}");
|
||||
assert!(
|
||||
body.contains(username.as_str()),
|
||||
"creds should contain username: {body}"
|
||||
);
|
||||
|
||||
// 2) whoami
|
||||
common::pic_as(&env)
|
||||
.args(["whoami"])
|
||||
.assert()
|
||||
.success()
|
||||
.stdout(predicate::str::contains(username.clone()));
|
||||
|
||||
// 3) apps create
|
||||
common::pic_as(&env)
|
||||
.args(["apps", "create", &slug])
|
||||
.assert()
|
||||
.success()
|
||||
.stdout(predicate::str::contains(format!("Created app {slug}")));
|
||||
|
||||
// Cleanup no matter what subsequent assertions do.
|
||||
let _guard = AppGuard::new(&env.url, &env.token, &slug);
|
||||
|
||||
// 4) apps ls
|
||||
common::pic_as(&env)
|
||||
.args(["apps", "ls"])
|
||||
.assert()
|
||||
.success()
|
||||
.stdout(predicate::str::contains(slug.as_str()));
|
||||
|
||||
// 5) scripts deploy (create then update)
|
||||
let fixture = common::fixture_path("hello.rhai");
|
||||
common::pic_as(&env)
|
||||
.args([
|
||||
"scripts",
|
||||
"deploy",
|
||||
fixture.to_str().unwrap(),
|
||||
"--app",
|
||||
&slug,
|
||||
])
|
||||
.assert()
|
||||
.success()
|
||||
.stdout(predicate::str::contains("Created hello v1"));
|
||||
|
||||
common::pic_as(&env)
|
||||
.args([
|
||||
"scripts",
|
||||
"deploy",
|
||||
fixture.to_str().unwrap(),
|
||||
"--app",
|
||||
&slug,
|
||||
])
|
||||
.assert()
|
||||
.success()
|
||||
.stdout(predicate::str::contains("Updated hello v2"));
|
||||
|
||||
// 6) scripts ls and capture the id
|
||||
let ls_out = common::pic_as(&env)
|
||||
.args(["scripts", "ls", "--app", &slug])
|
||||
.output()
|
||||
.expect("scripts ls");
|
||||
assert!(ls_out.status.success(), "scripts ls failed: {ls_out:?}");
|
||||
let id = common::parse_first_id(std::str::from_utf8(&ls_out.stdout).unwrap())
|
||||
.expect("scripts ls should print at least one row");
|
||||
|
||||
// 7) invoke
|
||||
let invoke_out = common::pic_as(&env)
|
||||
.args(["scripts", "invoke", &id])
|
||||
.output()
|
||||
.expect("scripts invoke");
|
||||
assert!(
|
||||
invoke_out.status.success(),
|
||||
"invoke failed: {}",
|
||||
String::from_utf8_lossy(&invoke_out.stderr)
|
||||
);
|
||||
let parsed: Value =
|
||||
serde_json::from_slice(&invoke_out.stdout).expect("invoke stdout should be JSON");
|
||||
assert_eq!(
|
||||
parsed["ok"], true,
|
||||
"expected hello.rhai response, got {parsed}"
|
||||
);
|
||||
|
||||
// 8) logs (the invoke above should have produced exactly one row)
|
||||
let logs_out = common::pic_as(&env)
|
||||
.args(["logs", &id])
|
||||
.output()
|
||||
.expect("pic logs");
|
||||
assert!(logs_out.status.success(), "logs failed: {logs_out:?}");
|
||||
let stdout = String::from_utf8_lossy(&logs_out.stdout);
|
||||
assert!(
|
||||
stdout.lines().any(|l| !l.trim().is_empty()),
|
||||
"logs should have at least one row, got: {stdout}"
|
||||
);
|
||||
}
|
||||
171
crates/picloud-cli/tests/invoke.rs
Normal file
171
crates/picloud-cli/tests/invoke.rs
Normal file
@@ -0,0 +1,171 @@
|
||||
//! `pic scripts invoke` — body sources (inline, `@file`, `@-`), header
|
||||
//! propagation, exit-code semantics for non-2xx responses, and 404
|
||||
//! handling for unknown ids.
|
||||
|
||||
use predicates::prelude::*;
|
||||
use serde_json::Value;
|
||||
|
||||
use crate::common;
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn invoke_with_inline_json_body_echoes() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
let (id, _guard) = common::deploy_fixture(&env, "invoke-inline", "echo.rhai");
|
||||
|
||||
let out = common::pic_as(&env)
|
||||
.args(["scripts", "invoke", &id, "--body", r#"{"x":1}"#])
|
||||
.output()
|
||||
.expect("invoke");
|
||||
assert!(out.status.success(), "invoke failed: {out:?}");
|
||||
let parsed: Value = serde_json::from_slice(&out.stdout).expect("stdout JSON");
|
||||
assert_eq!(parsed["body"]["x"], 1, "echoed body: {parsed}");
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn invoke_with_file_body() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
let (id, _guard) = common::deploy_fixture(&env, "invoke-file", "echo.rhai");
|
||||
|
||||
let tmp = tempfile::NamedTempFile::new().expect("tempfile");
|
||||
std::fs::write(tmp.path(), r#"{"src":"file"}"#).unwrap();
|
||||
let body_arg = format!("@{}", tmp.path().display());
|
||||
|
||||
let out = common::pic_as(&env)
|
||||
.args(["scripts", "invoke", &id, "--body", &body_arg])
|
||||
.output()
|
||||
.expect("invoke");
|
||||
assert!(out.status.success(), "invoke failed: {out:?}");
|
||||
let parsed: Value = serde_json::from_slice(&out.stdout).expect("stdout JSON");
|
||||
assert_eq!(parsed["body"]["src"], "file", "echoed body: {parsed}");
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn invoke_with_stdin_body() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
let (id, _guard) = common::deploy_fixture(&env, "invoke-stdin", "echo.rhai");
|
||||
|
||||
let assert = common::pic_as(&env)
|
||||
.args(["scripts", "invoke", &id, "--body", "@-"])
|
||||
.write_stdin(r#"{"src":"stdin"}"#)
|
||||
.assert()
|
||||
.success();
|
||||
let out = assert.get_output();
|
||||
let parsed: Value = serde_json::from_slice(&out.stdout).expect("stdout JSON");
|
||||
assert_eq!(parsed["body"]["src"], "stdin", "echoed body: {parsed}");
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn invoke_propagates_headers() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
let (id, _guard) = common::deploy_fixture(&env, "invoke-hdr", "echo.rhai");
|
||||
|
||||
let out = common::pic_as(&env)
|
||||
.args([
|
||||
"scripts",
|
||||
"invoke",
|
||||
&id,
|
||||
"-H",
|
||||
"X-Foo: bar",
|
||||
"-H",
|
||||
"X-Baz=qux",
|
||||
])
|
||||
.output()
|
||||
.expect("invoke");
|
||||
assert!(out.status.success(), "invoke failed: {out:?}");
|
||||
let parsed: Value = serde_json::from_slice(&out.stdout).expect("stdout JSON");
|
||||
// HTTP normalises header names to lowercase.
|
||||
assert_eq!(parsed["headers"]["x-foo"], "bar", "echoed: {parsed}");
|
||||
assert_eq!(parsed["headers"]["x-baz"], "qux", "echoed: {parsed}");
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn invoke_unknown_script_id_errors() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
|
||||
// Any well-formed UUID that doesn't exist server-side. The
|
||||
// orchestrator's `/execute/{id}` handler returns 404 specifically
|
||||
// for unknown ids — tighten the predicate so a regressed 401
|
||||
// wouldn't sneak through.
|
||||
let bogus = "00000000-0000-0000-0000-000000000000";
|
||||
common::pic_as(&env)
|
||||
.args(["scripts", "invoke", bogus])
|
||||
.assert()
|
||||
.failure()
|
||||
.stderr(predicate::str::contains("HTTP 404"));
|
||||
}
|
||||
|
||||
/// `pic invoke <id>` (top-level alias) and `pic scripts invoke <id>`
|
||||
/// must hit the same handler and produce identical-shape stdout.
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn top_level_invoke_alias_works() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
let (id, _guard) = common::deploy_fixture(&env, "inv-alias", "hello.rhai");
|
||||
|
||||
let nested = common::pic_as(&env)
|
||||
.args(["scripts", "invoke", &id])
|
||||
.output()
|
||||
.expect("scripts invoke");
|
||||
assert!(nested.status.success());
|
||||
let nested_body: Value = serde_json::from_slice(&nested.stdout).unwrap();
|
||||
|
||||
let aliased = common::pic_as(&env)
|
||||
.args(["invoke", &id])
|
||||
.output()
|
||||
.expect("invoke (top-level)");
|
||||
assert!(aliased.status.success());
|
||||
let aliased_body: Value = serde_json::from_slice(&aliased.stdout).unwrap();
|
||||
|
||||
assert_eq!(
|
||||
nested_body, aliased_body,
|
||||
"top-level alias should produce identical body to scripts invoke"
|
||||
);
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn invoke_non_2xx_exits_nonzero_but_prints_body() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
let (id, _guard) = common::deploy_fixture(&env, "invoke-500", "boom.rhai");
|
||||
|
||||
let out = common::pic_as(&env)
|
||||
.args(["scripts", "invoke", &id])
|
||||
.output()
|
||||
.expect("invoke");
|
||||
assert!(!out.status.success(), "expected non-zero exit: {out:?}");
|
||||
let stderr = String::from_utf8_lossy(&out.stderr);
|
||||
assert!(
|
||||
stderr.contains("<- HTTP 500"),
|
||||
"stderr should report HTTP 500: {stderr}"
|
||||
);
|
||||
let parsed: Value = serde_json::from_slice(&out.stdout)
|
||||
.unwrap_or_else(|e| panic!("stdout was not JSON ({e}): {:?}", out.stdout));
|
||||
assert_eq!(parsed["ok"], false, "boom body: {parsed}");
|
||||
assert_eq!(parsed["why"], "intentional", "boom body: {parsed}");
|
||||
}
|
||||
179
crates/picloud-cli/tests/logs.rs
Normal file
179
crates/picloud-cli/tests/logs.rs
Normal file
@@ -0,0 +1,179 @@
|
||||
//! `pic logs <script-id>` — emptiness, status labels, `--limit`
|
||||
//! clamping, error path for unknown ids, and the 120-char truncate
|
||||
//! applied to the summary column.
|
||||
|
||||
use predicates::prelude::*;
|
||||
|
||||
use crate::common;
|
||||
|
||||
/// Pick out the data rows from `pic logs` TSV output — the header line
|
||||
/// (`created_at\tstatus\tsummary`) is now always present, so the old
|
||||
/// "no non-empty lines means no logs" check needs to skip it.
|
||||
fn data_rows(stdout: &str) -> Vec<&str> {
|
||||
stdout
|
||||
.lines()
|
||||
.filter(|l| !l.trim().is_empty())
|
||||
.filter(|l| !l.starts_with("created_at"))
|
||||
.collect()
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn logs_for_fresh_script_is_empty() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
let (id, _guard) = common::deploy_fixture(&env, "logs-empty", "hello.rhai");
|
||||
|
||||
let out = common::pic_as(&env)
|
||||
.args(["logs", &id])
|
||||
.output()
|
||||
.expect("logs");
|
||||
assert!(out.status.success(), "logs failed: {out:?}");
|
||||
let stdout = String::from_utf8(out.stdout).unwrap();
|
||||
assert!(
|
||||
data_rows(&stdout).is_empty(),
|
||||
"expected no log rows (header is allowed), got: {stdout}"
|
||||
);
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn logs_after_invoke_records_success_row() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
let (id, _guard) = common::deploy_fixture(&env, "logs-ok", "hello.rhai");
|
||||
|
||||
common::pic_as(&env)
|
||||
.args(["scripts", "invoke", &id])
|
||||
.assert()
|
||||
.success();
|
||||
|
||||
let out = common::pic_as(&env)
|
||||
.args(["logs", &id])
|
||||
.output()
|
||||
.expect("logs");
|
||||
assert!(out.status.success(), "logs failed: {out:?}");
|
||||
let stdout = String::from_utf8(out.stdout).unwrap();
|
||||
let rows = data_rows(&stdout);
|
||||
assert_eq!(rows.len(), 1, "expected 1 data row, got: {stdout}");
|
||||
let cols: Vec<&str> = rows[0].split('\t').map(str::trim).collect();
|
||||
assert_eq!(
|
||||
cols.len(),
|
||||
3,
|
||||
"row should be 3 tab-delimited cells: {rows:?}"
|
||||
);
|
||||
assert_eq!(cols[1], "success");
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn logs_records_error_for_throwing_script() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
let (id, _guard) = common::deploy_fixture(&env, "logs-err", "throw.rhai");
|
||||
|
||||
// The invoke is expected to fail — we only care that the execution
|
||||
// gets recorded with `Error` status.
|
||||
let _ = common::pic_as(&env)
|
||||
.args(["scripts", "invoke", &id])
|
||||
.output();
|
||||
|
||||
let out = common::pic_as(&env)
|
||||
.args(["logs", &id])
|
||||
.output()
|
||||
.expect("logs");
|
||||
assert!(out.status.success(), "logs failed: {out:?}");
|
||||
let stdout = String::from_utf8(out.stdout).unwrap();
|
||||
let row = data_rows(&stdout)
|
||||
.into_iter()
|
||||
.next()
|
||||
.expect("at least one data row");
|
||||
let cols: Vec<&str> = row.split('\t').map(str::trim).collect();
|
||||
assert_eq!(cols[1], "error", "expected error status, got row: {row}");
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn logs_respects_limit_flag() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
let (id, _guard) = common::deploy_fixture(&env, "logs-limit", "hello.rhai");
|
||||
|
||||
for _ in 0..3 {
|
||||
common::pic_as(&env)
|
||||
.args(["scripts", "invoke", &id])
|
||||
.assert()
|
||||
.success();
|
||||
}
|
||||
|
||||
let out = common::pic_as(&env)
|
||||
.args(["logs", &id, "--limit", "1"])
|
||||
.output()
|
||||
.expect("logs");
|
||||
assert!(out.status.success(), "logs failed: {out:?}");
|
||||
let stdout = String::from_utf8(out.stdout).unwrap();
|
||||
let rows = data_rows(&stdout).len();
|
||||
assert_eq!(rows, 1, "expected --limit 1, got rows: {stdout}");
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn logs_for_unknown_id_errors() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
|
||||
let bogus = "00000000-0000-0000-0000-000000000000";
|
||||
common::pic_as(&env)
|
||||
.args(["logs", bogus])
|
||||
.assert()
|
||||
.failure()
|
||||
// 404 specifically — same `NotFound(ScriptId)` path the get/edit
|
||||
// endpoints use.
|
||||
.stderr(predicate::str::contains("HTTP 404"));
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn logs_truncates_long_summary() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
let (id, _guard) = common::deploy_fixture(&env, "logs-loud", "loud.rhai");
|
||||
|
||||
common::pic_as(&env)
|
||||
.args(["scripts", "invoke", &id])
|
||||
.assert()
|
||||
.success();
|
||||
|
||||
let out = common::pic_as(&env)
|
||||
.args(["logs", &id])
|
||||
.output()
|
||||
.expect("logs");
|
||||
assert!(out.status.success(), "logs failed: {out:?}");
|
||||
let stdout = String::from_utf8(out.stdout).unwrap();
|
||||
let row = data_rows(&stdout)
|
||||
.into_iter()
|
||||
.next()
|
||||
.expect("at least one data row");
|
||||
let summary = row.split('\t').nth(2).expect("summary column");
|
||||
assert!(
|
||||
summary.ends_with('…'),
|
||||
"summary should be truncated with `…`, got: {summary}"
|
||||
);
|
||||
let chars = summary.chars().count();
|
||||
assert!(
|
||||
chars <= 121,
|
||||
"summary should be ≤120 chars + the truncation marker, got {chars}: {summary}"
|
||||
);
|
||||
}
|
||||
289
crates/picloud-cli/tests/output.rs
Normal file
289
crates/picloud-cli/tests/output.rs
Normal file
@@ -0,0 +1,289 @@
|
||||
//! Output-shape invariants — the contracts downstream `jq`/`awk`
|
||||
//! pipelines depend on: column headers, stdout-vs-stderr separation,
|
||||
//! and RFC3339 timestamps.
|
||||
|
||||
use serde_json::Value;
|
||||
|
||||
use crate::common;
|
||||
use crate::common::cleanup::AppGuard;
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn apps_ls_header_columns() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
|
||||
let out = common::pic_as(&env)
|
||||
.args(["apps", "ls"])
|
||||
.output()
|
||||
.expect("apps ls");
|
||||
assert!(out.status.success());
|
||||
let stdout = String::from_utf8(out.stdout).unwrap();
|
||||
let header = stdout.lines().next().expect("header row");
|
||||
assert_eq!(
|
||||
common::cells(header),
|
||||
vec!["slug", "name", "my_role", "created_at"]
|
||||
);
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn scripts_ls_header_columns() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
let slug = common::unique_slug("out-ls");
|
||||
common::pic_as(&env)
|
||||
.args(["apps", "create", &slug])
|
||||
.assert()
|
||||
.success();
|
||||
let _guard = AppGuard::new(&env.url, &env.token, &slug);
|
||||
|
||||
let out = common::pic_as(&env)
|
||||
.args(["scripts", "ls", "--app", &slug])
|
||||
.output()
|
||||
.expect("scripts ls");
|
||||
assert!(out.status.success());
|
||||
let stdout = String::from_utf8(out.stdout).unwrap();
|
||||
let header = stdout.lines().next().expect("header row");
|
||||
assert_eq!(
|
||||
common::cells(header),
|
||||
vec!["id", "app_slug", "name", "version", "updated_at"]
|
||||
);
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn invoke_separates_stdout_and_stderr() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
let (id, _guard) = common::deploy_fixture(&env, "out-inv", "hello.rhai");
|
||||
|
||||
let out = common::pic_as(&env)
|
||||
.args(["scripts", "invoke", &id])
|
||||
.output()
|
||||
.expect("invoke");
|
||||
assert!(out.status.success());
|
||||
|
||||
let stderr = String::from_utf8(out.stderr).unwrap();
|
||||
assert!(
|
||||
stderr.starts_with("<- HTTP 200"),
|
||||
"stderr should announce HTTP status: {stderr:?}"
|
||||
);
|
||||
|
||||
let parsed: Value = serde_json::from_slice(&out.stdout)
|
||||
.expect("stdout should be JSON only, with no status prefix");
|
||||
assert_eq!(parsed["ok"], true, "body: {parsed}");
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn error_goes_to_stderr_not_stdout() {
|
||||
let Some(_fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
// Use a pristine env (no credentials file) so `whoami` is guaranteed
|
||||
// to fail at the `config::load` step — `admin_env` would pre-seed
|
||||
// creds and the command would succeed.
|
||||
let env = common::TestEnv {
|
||||
url: String::new(),
|
||||
token: String::new(),
|
||||
config_dir: tempfile::TempDir::new().unwrap(),
|
||||
home: tempfile::TempDir::new().unwrap(),
|
||||
};
|
||||
|
||||
let out = common::pic_no_env(&env)
|
||||
.args(["whoami"])
|
||||
.output()
|
||||
.expect("whoami");
|
||||
assert!(!out.status.success(), "expected failure, got: {out:?}");
|
||||
assert!(
|
||||
out.stdout.is_empty(),
|
||||
"stdout should be empty on error, got: {:?}",
|
||||
String::from_utf8_lossy(&out.stdout),
|
||||
);
|
||||
let stderr = String::from_utf8(out.stderr).unwrap();
|
||||
assert!(
|
||||
stderr.contains("error:"),
|
||||
"stderr should be prefixed with `error:`: {stderr}"
|
||||
);
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn apps_ls_created_at_is_rfc3339() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
let slug = common::unique_slug("out-date");
|
||||
common::pic_as(&env)
|
||||
.args(["apps", "create", &slug])
|
||||
.assert()
|
||||
.success();
|
||||
let _guard = AppGuard::new(&env.url, &env.token, &slug);
|
||||
|
||||
let out = common::pic_as(&env)
|
||||
.args(["apps", "ls"])
|
||||
.output()
|
||||
.expect("apps ls");
|
||||
let stdout = String::from_utf8(out.stdout).unwrap();
|
||||
let row = stdout
|
||||
.lines()
|
||||
.map(common::cells)
|
||||
.find(|c| c.first().copied() == Some(slug.as_str()))
|
||||
.unwrap_or_else(|| panic!("slug {slug} missing in: {stdout}"));
|
||||
let created_at = row.get(3).expect("created_at cell");
|
||||
|
||||
// Accept the RFC3339 shape without pulling in chrono — `YYYY-MM-DDTHH:MM:SS`
|
||||
// with optional fraction + timezone is enough of a contract for the test.
|
||||
assert!(
|
||||
created_at.len() >= 20
|
||||
&& created_at.as_bytes()[4] == b'-'
|
||||
&& created_at.as_bytes()[7] == b'-'
|
||||
&& created_at.as_bytes()[10] == b'T'
|
||||
&& created_at.as_bytes()[13] == b':'
|
||||
&& created_at.as_bytes()[16] == b':',
|
||||
"created_at not RFC3339-shaped: {created_at}"
|
||||
);
|
||||
}
|
||||
|
||||
/// `--output json` is the global pipeline-friendly format. Validates
|
||||
/// `apps ls` returns a real JSON array (not a TSV-with-quotes hack).
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn apps_ls_json_output_is_valid_array() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
let slug = common::unique_slug("out-json-apps");
|
||||
common::pic_as(&env)
|
||||
.args(["apps", "create", &slug])
|
||||
.assert()
|
||||
.success();
|
||||
let _guard = AppGuard::new(&env.url, &env.token, &slug);
|
||||
|
||||
let out = common::pic_as(&env)
|
||||
.args(["--output", "json", "apps", "ls"])
|
||||
.output()
|
||||
.expect("apps ls --output json");
|
||||
assert!(out.status.success(), "apps ls failed: {out:?}");
|
||||
let v: Value = serde_json::from_slice(&out.stdout).expect("stdout should be JSON");
|
||||
let arr = v.as_array().expect("apps ls JSON should be an array");
|
||||
assert!(
|
||||
arr.iter()
|
||||
.any(|row| row.get("slug").and_then(Value::as_str) == Some(slug.as_str())),
|
||||
"json should include created slug: {v}"
|
||||
);
|
||||
// The header row must NOT bleed into JSON output — the rendered
|
||||
// objects use header *keys*, not data cells.
|
||||
assert!(
|
||||
arr.iter().all(|row| row.get("slug").is_some()),
|
||||
"every row should have a `slug` key: {v}"
|
||||
);
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn scripts_ls_json_output_has_app_slug() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
let slug = common::unique_slug("out-json-scr");
|
||||
common::pic_as(&env)
|
||||
.args(["apps", "create", &slug])
|
||||
.assert()
|
||||
.success();
|
||||
let _guard = AppGuard::new(&env.url, &env.token, &slug);
|
||||
let fixture = common::fixture_path("hello.rhai");
|
||||
common::pic_as(&env)
|
||||
.args([
|
||||
"scripts",
|
||||
"deploy",
|
||||
fixture.to_str().unwrap(),
|
||||
"--app",
|
||||
&slug,
|
||||
])
|
||||
.assert()
|
||||
.success();
|
||||
|
||||
let out = common::pic_as(&env)
|
||||
.args(["--output", "json", "scripts", "ls", "--app", &slug])
|
||||
.output()
|
||||
.expect("scripts ls --output json");
|
||||
assert!(out.status.success());
|
||||
let v: Value = serde_json::from_slice(&out.stdout).expect("stdout JSON");
|
||||
let arr = v.as_array().expect("array");
|
||||
let row = arr
|
||||
.iter()
|
||||
.find(|r| r.get("name").and_then(Value::as_str) == Some("hello"))
|
||||
.expect("hello row");
|
||||
assert_eq!(row["app_slug"].as_str(), Some(slug.as_str()));
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn logs_json_output_is_array_of_objects() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
let (id, _guard) = common::deploy_fixture(&env, "out-json-log", "hello.rhai");
|
||||
common::pic_as(&env)
|
||||
.args(["scripts", "invoke", &id])
|
||||
.assert()
|
||||
.success();
|
||||
|
||||
let out = common::pic_as(&env)
|
||||
.args(["--output", "json", "logs", &id])
|
||||
.output()
|
||||
.expect("logs --output json");
|
||||
assert!(out.status.success());
|
||||
let v: Value = serde_json::from_slice(&out.stdout).expect("stdout JSON");
|
||||
let arr = v.as_array().expect("array");
|
||||
assert!(!arr.is_empty(), "expected at least one log");
|
||||
// Schema: each row carries the raw `ExecutionLog`, not the
|
||||
// truncated summary the TSV form uses.
|
||||
assert!(
|
||||
arr[0].get("status").is_some(),
|
||||
"log row missing status: {arr:?}"
|
||||
);
|
||||
}
|
||||
|
||||
/// TSV `whoami` used to be a single tab-separated line with no labels;
|
||||
/// downstream tools couldn't tell which column was the role. Now it's
|
||||
/// a key/value block with stable labels.
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn whoami_tsv_has_labeled_rows() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
|
||||
let out = common::pic_as(&env)
|
||||
.args(["whoami"])
|
||||
.output()
|
||||
.expect("whoami");
|
||||
assert!(out.status.success());
|
||||
let stdout = String::from_utf8(out.stdout).unwrap();
|
||||
let labels: Vec<&str> = stdout
|
||||
.lines()
|
||||
.filter_map(|l| l.split('\t').next())
|
||||
.map(str::trim)
|
||||
.collect();
|
||||
assert!(
|
||||
labels.contains(&"username"),
|
||||
"missing username row: {stdout}"
|
||||
);
|
||||
assert!(labels.contains(&"role"), "missing role row: {stdout}");
|
||||
assert!(labels.contains(&"email"), "missing email row: {stdout}");
|
||||
assert!(labels.contains(&"url"), "missing url row: {stdout}");
|
||||
}
|
||||
146
crates/picloud-cli/tests/roles.rs
Normal file
146
crates/picloud-cli/tests/roles.rs
Normal file
@@ -0,0 +1,146 @@
|
||||
//! RBAC mirror of the dashboard's role-shadowing specs. A Member user
|
||||
//! is minted via the admin API, granted (or denied) membership on an
|
||||
//! app, then `pic` is driven against the member's bearer token to
|
||||
//! confirm the server's capability gates surface as expected exit
|
||||
//! codes / error messages.
|
||||
|
||||
use predicates::prelude::*;
|
||||
|
||||
use crate::common;
|
||||
use crate::common::cleanup::AppGuard;
|
||||
use crate::common::member;
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn member_apps_ls_only_shows_their_apps() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let admin_env = common::admin_env(fx);
|
||||
|
||||
let slug_visible = common::unique_slug("roles-visible");
|
||||
let slug_hidden = common::unique_slug("roles-hidden");
|
||||
common::pic_as(&admin_env)
|
||||
.args(["apps", "create", &slug_visible])
|
||||
.assert()
|
||||
.success();
|
||||
let _g1 = AppGuard::new(&admin_env.url, &admin_env.token, &slug_visible);
|
||||
common::pic_as(&admin_env)
|
||||
.args(["apps", "create", &slug_hidden])
|
||||
.assert()
|
||||
.success();
|
||||
let _g2 = AppGuard::new(&admin_env.url, &admin_env.token, &slug_hidden);
|
||||
|
||||
let m = member::member_user(fx, &common::unique_username("rls"));
|
||||
member::grant_membership(fx, &slug_visible, &m.id, "viewer");
|
||||
let member_env = common::custom_env(&fx.url, &m.token);
|
||||
common::seed_credentials(&member_env, &m.username);
|
||||
|
||||
let out = common::pic_as(&member_env)
|
||||
.args(["apps", "ls"])
|
||||
.output()
|
||||
.expect("apps ls");
|
||||
assert!(out.status.success(), "apps ls failed: {out:?}");
|
||||
let stdout = String::from_utf8(out.stdout).unwrap();
|
||||
assert!(
|
||||
stdout.contains(&slug_visible),
|
||||
"member should see {slug_visible}, got: {stdout}"
|
||||
);
|
||||
assert!(
|
||||
!stdout.contains(&slug_hidden),
|
||||
"member should NOT see {slug_hidden}, got: {stdout}"
|
||||
);
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn viewer_cannot_deploy_but_editor_can() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let admin_env = common::admin_env(fx);
|
||||
let slug = common::unique_slug("roles-write");
|
||||
common::pic_as(&admin_env)
|
||||
.args(["apps", "create", &slug])
|
||||
.assert()
|
||||
.success();
|
||||
let _g = AppGuard::new(&admin_env.url, &admin_env.token, &slug);
|
||||
|
||||
let m = member::member_user(fx, &common::unique_username("vw"));
|
||||
member::grant_membership(fx, &slug, &m.id, "viewer");
|
||||
let member_env = common::custom_env(&fx.url, &m.token);
|
||||
common::seed_credentials(&member_env, &m.username);
|
||||
|
||||
let fixture = common::fixture_path("hello.rhai");
|
||||
common::pic_as(&member_env)
|
||||
.args([
|
||||
"scripts",
|
||||
"deploy",
|
||||
fixture.to_str().unwrap(),
|
||||
"--app",
|
||||
&slug,
|
||||
])
|
||||
.assert()
|
||||
.failure()
|
||||
// `Forbidden` → 403. A regressed predicate of `"HTTP 4"` would
|
||||
// have masked an auth break (401) as an authz issue.
|
||||
.stderr(predicate::str::contains("HTTP 403"));
|
||||
|
||||
// Promote to Editor and retry — the same command should now succeed.
|
||||
member::update_membership(fx, &slug, &m.id, "editor");
|
||||
common::pic_as(&member_env)
|
||||
.args([
|
||||
"scripts",
|
||||
"deploy",
|
||||
fixture.to_str().unwrap(),
|
||||
"--app",
|
||||
&slug,
|
||||
])
|
||||
.assert()
|
||||
.success()
|
||||
.stdout(predicate::str::contains("Created hello v1"));
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn member_can_invoke_any_script_with_id() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
// `/api/v1/execute/{id}` is the unguarded data-plane ingress — even
|
||||
// a member with no app membership can hit it as long as they hold
|
||||
// a valid token (the orchestrator doesn't gate it).
|
||||
let admin_env = common::admin_env(fx);
|
||||
let (id, _guard) = common::deploy_fixture(&admin_env, "roles-inv", "hello.rhai");
|
||||
|
||||
let m = member::member_user(fx, &common::unique_username("inv"));
|
||||
let member_env = common::custom_env(&fx.url, &m.token);
|
||||
common::seed_credentials(&member_env, &m.username);
|
||||
|
||||
common::pic_as(&member_env)
|
||||
.args(["scripts", "invoke", &id])
|
||||
.assert()
|
||||
.success();
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn non_member_cannot_read_logs() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let admin_env = common::admin_env(fx);
|
||||
let (id, _guard) = common::deploy_fixture(&admin_env, "roles-log", "hello.rhai");
|
||||
|
||||
let m = member::member_user(fx, &common::unique_username("rl"));
|
||||
let member_env = common::custom_env(&fx.url, &m.token);
|
||||
common::seed_credentials(&member_env, &m.username);
|
||||
|
||||
common::pic_as(&member_env)
|
||||
.args(["logs", &id])
|
||||
.assert()
|
||||
.failure()
|
||||
// Non-member → 403 from the authz layer, not 404 — the script
|
||||
// exists; the caller just can't see it.
|
||||
.stderr(predicate::str::contains("HTTP 403"));
|
||||
}
|
||||
240
crates/picloud-cli/tests/scripts.rs
Normal file
240
crates/picloud-cli/tests/scripts.rs
Normal file
@@ -0,0 +1,240 @@
|
||||
//! `pic scripts deploy` / `pic scripts ls` edge cases beyond the
|
||||
//! smoke test: unknown app, name override, version bumping, missing
|
||||
//! file, and the no-`--app` walk across every accessible app.
|
||||
|
||||
use predicates::prelude::*;
|
||||
|
||||
use crate::common;
|
||||
use crate::common::cleanup::AppGuard;
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn deploy_against_unknown_app_errors() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
let fixture = common::fixture_path("hello.rhai");
|
||||
let bogus_slug = common::unique_slug("nope");
|
||||
|
||||
common::pic_as(&env)
|
||||
.args([
|
||||
"scripts",
|
||||
"deploy",
|
||||
fixture.to_str().unwrap(),
|
||||
"--app",
|
||||
&bogus_slug,
|
||||
])
|
||||
.assert()
|
||||
.failure()
|
||||
// Specifically 404 — `apps_get` short-circuits before the deploy
|
||||
// request even starts. Loose `"HTTP 4"` would have matched a
|
||||
// regressed 401 from broken auth.
|
||||
.stderr(predicate::str::contains("HTTP 404"));
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn deploy_with_name_override() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
let slug = common::unique_slug("scripts-named");
|
||||
common::pic_as(&env)
|
||||
.args(["apps", "create", &slug])
|
||||
.assert()
|
||||
.success();
|
||||
let _guard = AppGuard::new(&env.url, &env.token, &slug);
|
||||
|
||||
let fixture = common::fixture_path("hello.rhai");
|
||||
common::pic_as(&env)
|
||||
.args([
|
||||
"scripts",
|
||||
"deploy",
|
||||
fixture.to_str().unwrap(),
|
||||
"--app",
|
||||
&slug,
|
||||
"--name",
|
||||
"custom-name",
|
||||
])
|
||||
.assert()
|
||||
.success()
|
||||
.stdout(predicate::str::contains("Created custom-name v1"));
|
||||
|
||||
common::pic_as(&env)
|
||||
.args([
|
||||
"scripts",
|
||||
"deploy",
|
||||
fixture.to_str().unwrap(),
|
||||
"--app",
|
||||
&slug,
|
||||
"--name",
|
||||
"custom-name",
|
||||
])
|
||||
.assert()
|
||||
.success()
|
||||
.stdout(predicate::str::contains("Updated custom-name v2"));
|
||||
|
||||
let out = common::pic_as(&env)
|
||||
.args(["scripts", "ls", "--app", &slug])
|
||||
.output()
|
||||
.expect("scripts ls");
|
||||
assert!(out.status.success());
|
||||
let stdout = String::from_utf8(out.stdout).unwrap();
|
||||
assert!(
|
||||
stdout
|
||||
.lines()
|
||||
.map(common::cells)
|
||||
.any(|c| c.get(2).copied() == Some("custom-name") && c.get(3).copied() == Some("2")),
|
||||
"expected custom-name v2 row, got: {stdout}",
|
||||
);
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn deploy_bumps_version_each_redeploy() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
let slug = common::unique_slug("scripts-bump");
|
||||
common::pic_as(&env)
|
||||
.args(["apps", "create", &slug])
|
||||
.assert()
|
||||
.success();
|
||||
let _guard = AppGuard::new(&env.url, &env.token, &slug);
|
||||
|
||||
let fixture = common::fixture_path("hello.rhai");
|
||||
for expected in ["Created hello v1", "Updated hello v2", "Updated hello v3"] {
|
||||
common::pic_as(&env)
|
||||
.args([
|
||||
"scripts",
|
||||
"deploy",
|
||||
fixture.to_str().unwrap(),
|
||||
"--app",
|
||||
&slug,
|
||||
])
|
||||
.assert()
|
||||
.success()
|
||||
.stdout(predicate::str::contains(expected));
|
||||
}
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn deploy_missing_file_errors() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
let slug = common::unique_slug("scripts-missing");
|
||||
common::pic_as(&env)
|
||||
.args(["apps", "create", &slug])
|
||||
.assert()
|
||||
.success();
|
||||
let _guard = AppGuard::new(&env.url, &env.token, &slug);
|
||||
|
||||
let missing = std::env::temp_dir().join(common::unique_slug("ghost") + ".rhai");
|
||||
common::pic_as(&env)
|
||||
.args([
|
||||
"scripts",
|
||||
"deploy",
|
||||
missing.to_str().unwrap(),
|
||||
"--app",
|
||||
&slug,
|
||||
])
|
||||
.assert()
|
||||
.failure()
|
||||
.stderr(predicate::str::contains("reading"));
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn ls_without_app_walks_every_accessible_app() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
let slug_a = common::unique_slug("scripts-walk-a");
|
||||
let slug_b = common::unique_slug("scripts-walk-b");
|
||||
|
||||
common::pic_as(&env)
|
||||
.args(["apps", "create", &slug_a])
|
||||
.assert()
|
||||
.success();
|
||||
let _guard_a = AppGuard::new(&env.url, &env.token, &slug_a);
|
||||
common::pic_as(&env)
|
||||
.args(["apps", "create", &slug_b])
|
||||
.assert()
|
||||
.success();
|
||||
let _guard_b = AppGuard::new(&env.url, &env.token, &slug_b);
|
||||
|
||||
let fixture = common::fixture_path("hello.rhai");
|
||||
for slug in [&slug_a, &slug_b] {
|
||||
common::pic_as(&env)
|
||||
.args([
|
||||
"scripts",
|
||||
"deploy",
|
||||
fixture.to_str().unwrap(),
|
||||
"--app",
|
||||
slug,
|
||||
])
|
||||
.assert()
|
||||
.success();
|
||||
}
|
||||
|
||||
// `pic scripts ls` (no `--app`) issues a single `GET /admin/scripts`
|
||||
// against the server now — there's nothing per-app to race against
|
||||
// a concurrent AppGuard drop. The previous implementation walked
|
||||
// `apps_list` followed by per-app `scripts_list_by_app` calls and
|
||||
// aborted on the first 404, which forced this test to retry 5× to
|
||||
// paper over the bug. Both the walk and the retry are gone.
|
||||
let out = common::pic_as(&env)
|
||||
.args(["scripts", "ls"])
|
||||
.output()
|
||||
.expect("scripts ls");
|
||||
assert!(out.status.success(), "scripts ls failed: {out:?}");
|
||||
let stdout = String::from_utf8(out.stdout).unwrap();
|
||||
|
||||
let slugs: std::collections::HashSet<&str> = stdout
|
||||
.lines()
|
||||
.map(common::cells)
|
||||
.filter_map(|c| c.get(1).copied())
|
||||
.collect();
|
||||
assert!(
|
||||
slugs.contains(slug_a.as_str()),
|
||||
"missing app A in: {stdout}"
|
||||
);
|
||||
assert!(
|
||||
slugs.contains(slug_b.as_str()),
|
||||
"missing app B in: {stdout}"
|
||||
);
|
||||
}
|
||||
|
||||
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
|
||||
#[test]
|
||||
fn delete_removes_script_from_ls() {
|
||||
let Some(fx) = common::fixture_or_skip() else {
|
||||
return;
|
||||
};
|
||||
let env = common::admin_env(fx);
|
||||
let (id, _guard) = common::deploy_fixture(&env, "scripts-del", "hello.rhai");
|
||||
|
||||
common::pic_as(&env)
|
||||
.args(["scripts", "delete", &id])
|
||||
.assert()
|
||||
.success()
|
||||
.stdout(predicate::str::contains(format!("Deleted script {id}")));
|
||||
|
||||
let out = common::pic_as(&env)
|
||||
.args(["scripts", "ls"])
|
||||
.output()
|
||||
.expect("scripts ls");
|
||||
assert!(out.status.success());
|
||||
let stdout = String::from_utf8(out.stdout).unwrap();
|
||||
assert!(
|
||||
!stdout.contains(&id),
|
||||
"deleted script id should not appear in ls: {stdout}"
|
||||
);
|
||||
}
|
||||
@@ -11,21 +11,28 @@ use axum::{routing::get, Json, Router};
|
||||
use picloud_executor_core::{Engine, Limits};
|
||||
use picloud_manager_core::{
|
||||
admin_router, admins_router, api_keys_router, app_members_router, apps_api, apps_router,
|
||||
auth_router, compile_routes, migrations, require_authenticated, route_admin_router,
|
||||
AdminSessionRepository, AdminState, AdminUserRepository, AdminsState, ApiKeyRepository,
|
||||
ApiKeysState, AppDomainRepository, AppMembersRepository, AppMembersState, AppRepository,
|
||||
AppsState, AuthState, AuthzRepo, PostgresAdminSessionRepository, PostgresAdminUserRepository,
|
||||
PostgresApiKeyRepository, PostgresAppDomainRepository, PostgresAppMembersRepository,
|
||||
PostgresAppRepository, PostgresExecutionLogRepository, PostgresExecutionLogSink,
|
||||
PostgresRouteRepository, PostgresScriptRepository, RepoResolver, RouteAdminState,
|
||||
RouteRepository, SandboxCeiling,
|
||||
attach_principal_if_present, auth_router, compile_routes, dead_letters_router, migrations,
|
||||
require_authenticated, route_admin_router, triggers_router, AbandonedRepo,
|
||||
AdminPrincipalResolver, AdminSessionRepository, AdminState, AdminUserRepository, AdminsState,
|
||||
ApiKeyRepository, ApiKeysState, AppDomainRepository, AppMembersRepository, AppMembersState,
|
||||
AppRepository, AppsState, AuthState, AuthzRepo, DeadLetterRepo, DeadLettersState, Dispatcher,
|
||||
KvServiceImpl, OutboxEventEmitter, OutboxRepo, PostgresAbandonedRepo,
|
||||
PostgresAdminSessionRepository, PostgresAdminUserRepository, PostgresApiKeyRepository,
|
||||
PostgresAppDomainRepository, PostgresAppMembersRepository, PostgresAppRepository,
|
||||
PostgresDeadLetterRepo, PostgresDeadLetterService, PostgresExecutionLogRepository,
|
||||
PostgresExecutionLogSink, PostgresKvRepo, PostgresOutboxRepo, PostgresRouteRepository,
|
||||
PostgresScriptRepository, PostgresTriggerRepo, PrincipalResolver, RepoResolver,
|
||||
RouteAdminState, RouteRepository, SandboxCeiling, ScriptRepository, TriggerConfig, TriggerRepo,
|
||||
TriggersState,
|
||||
};
|
||||
use picloud_orchestrator_core::routing::{AppDomainTable, RouteTable};
|
||||
use picloud_orchestrator_core::{
|
||||
data_plane_router, user_routes_router, DataPlaneState, LocalExecutorClient,
|
||||
data_plane_router, user_routes_router, DataPlaneState, ExecutionGate, InboxRegistry,
|
||||
LocalExecutorClient,
|
||||
};
|
||||
use picloud_shared::{
|
||||
ExecutionLogSink, ScriptValidator, API_VERSION, PRODUCT_VERSION, SDK_VERSION, WIRE_VERSION,
|
||||
DeadLetterService, ExecutionLogSink, InboxResolver, KvService, OutboxWriter, ScriptValidator,
|
||||
ServiceEventEmitter, Services, API_VERSION, PRODUCT_VERSION, SDK_VERSION, WIRE_VERSION,
|
||||
};
|
||||
use sqlx::postgres::PgPoolOptions;
|
||||
use sqlx::PgPool;
|
||||
@@ -82,8 +89,6 @@ fn read_session_ttl() -> Duration {
|
||||
/// `/version`) stays open — it's the public ingress for user scripts.
|
||||
#[allow(clippy::too_many_lines)]
|
||||
pub async fn build_app(pool: PgPool, auth: AuthDeps) -> anyhow::Result<Router> {
|
||||
let engine = Arc::new(Engine::new(Limits::default()));
|
||||
|
||||
let script_repo = Arc::new(PostgresScriptRepository::new(pool.clone()));
|
||||
let log_repo = Arc::new(PostgresExecutionLogRepository::new(pool.clone()));
|
||||
let log_sink: Arc<dyn ExecutionLogSink> = Arc::new(PostgresExecutionLogSink::new(pool.clone()));
|
||||
@@ -95,10 +100,43 @@ pub async fn build_app(pool: PgPool, auth: AuthDeps) -> anyhow::Result<Router> {
|
||||
// (CRUD over the table) and `AuthzRepo` (single-row membership lookup
|
||||
// for capability checks). Construct it once and clone the Arc into
|
||||
// both trait views — same allocation, two vtables.
|
||||
let members_concrete = Arc::new(PostgresAppMembersRepository::new(pool));
|
||||
let members_concrete = Arc::new(PostgresAppMembersRepository::new(pool.clone()));
|
||||
let members: Arc<dyn AppMembersRepository> = members_concrete.clone();
|
||||
let authz: Arc<dyn AuthzRepo> = members_concrete;
|
||||
|
||||
// Triggers framework storage. The outbox event emitter routes
|
||||
// KV mutations into the outbox; the dispatcher fans them out.
|
||||
let trigger_repo: Arc<dyn TriggerRepo> = Arc::new(PostgresTriggerRepo::new(pool.clone()));
|
||||
// PostgresOutboxRepo implements both `OutboxRepo` (the dispatcher
|
||||
// surface) and `OutboxWriter` (the orchestrator surface). Construct
|
||||
// the concrete Arc once, clone it into each trait view — same
|
||||
// allocation, two vtables (mirrors how `members_concrete` above is
|
||||
// used as both `AppMembersRepository` and `AuthzRepo`).
|
||||
let outbox_concrete = Arc::new(PostgresOutboxRepo::new(pool.clone()));
|
||||
let outbox_repo: Arc<dyn OutboxRepo> = outbox_concrete.clone();
|
||||
let outbox_writer: Arc<dyn OutboxWriter> = outbox_concrete;
|
||||
let dl_repo: Arc<dyn DeadLetterRepo> = Arc::new(PostgresDeadLetterRepo::new(pool.clone()));
|
||||
let abandoned_repo: Arc<dyn AbandonedRepo> = Arc::new(PostgresAbandonedRepo::new(pool.clone()));
|
||||
let trigger_config = TriggerConfig::from_env();
|
||||
|
||||
// SDK services bundle. v1.1.1 ships the KV store + the
|
||||
// outbox-backed event emitter + the dead-letter service (replay /
|
||||
// resolve).
|
||||
let kv_repo = Arc::new(PostgresKvRepo::new(pool));
|
||||
let events: Arc<dyn ServiceEventEmitter> = Arc::new(OutboxEventEmitter::new(
|
||||
trigger_repo.clone(),
|
||||
outbox_repo.clone(),
|
||||
));
|
||||
let kv: Arc<dyn KvService> =
|
||||
Arc::new(KvServiceImpl::new(kv_repo, authz.clone(), events.clone()));
|
||||
let dl_service: Arc<dyn DeadLetterService> = Arc::new(PostgresDeadLetterService::new(
|
||||
dl_repo.clone(),
|
||||
outbox_repo.clone(),
|
||||
authz.clone(),
|
||||
));
|
||||
let services = Services::new(kv, dl_service.clone(), events);
|
||||
let engine = Arc::new(Engine::new(Limits::default(), services));
|
||||
|
||||
// Compile the routes table once at startup; admin writes refresh it.
|
||||
let route_table = Arc::new(RouteTable::new());
|
||||
let initial = route_repo.list_all().await?;
|
||||
@@ -126,7 +164,37 @@ pub async fn build_app(pool: PgPool, auth: AuthDeps) -> anyhow::Result<Router> {
|
||||
let resolver = Arc::new(RepoResolver::new(PostgresScriptRepoHandle(
|
||||
script_repo.clone(),
|
||||
)));
|
||||
let executor = Arc::new(LocalExecutorClient::new(engine.clone()));
|
||||
// Single global gate — overflow is rejected with 503 + Retry-After.
|
||||
// See `ExecutionGate` docs and `PICLOUD_MAX_CONCURRENT_EXECUTIONS`.
|
||||
let gate = Arc::new(ExecutionGate::from_env());
|
||||
let executor = Arc::new(LocalExecutorClient::new(engine.clone(), gate.clone()));
|
||||
|
||||
// Dispatcher — single tokio task that polls the outbox and routes
|
||||
// due rows to the executor. Shares the `ExecutionGate` with sync
|
||||
// HTTP per design notes §2 (one cap for everything).
|
||||
let dispatcher_script_repo: Arc<dyn ScriptRepository> =
|
||||
Arc::new(PostgresScriptRepoHandle(script_repo.clone()));
|
||||
let principals: Arc<dyn PrincipalResolver> =
|
||||
Arc::new(AdminPrincipalResolver::new(auth.users.clone()));
|
||||
// The InboxRegistry is constructed once and shared between the
|
||||
// orchestrator (registers receivers, awaits) and the dispatcher
|
||||
// (delivers results). Two Arc views on the same allocation.
|
||||
let inbox_registry = Arc::new(InboxRegistry::new());
|
||||
let inbox_resolver: Arc<dyn InboxResolver> = inbox_registry.clone();
|
||||
Dispatcher {
|
||||
outbox: outbox_repo.clone(),
|
||||
triggers: trigger_repo.clone(),
|
||||
scripts: dispatcher_script_repo,
|
||||
dead_letters: dl_repo.clone(),
|
||||
abandoned: abandoned_repo.clone(),
|
||||
principals,
|
||||
executor: executor.clone(),
|
||||
gate,
|
||||
inbox: inbox_resolver,
|
||||
config: trigger_config,
|
||||
instance_id: format!("picloud-{}", std::process::id()),
|
||||
}
|
||||
.spawn();
|
||||
|
||||
let admin = AdminState {
|
||||
repo: Arc::new(PostgresScriptRepoHandle(script_repo.clone())),
|
||||
@@ -149,6 +217,30 @@ pub async fn build_app(pool: PgPool, auth: AuthDeps) -> anyhow::Result<Router> {
|
||||
log_sink,
|
||||
app_domains: app_domain_table.clone(),
|
||||
routes: route_table,
|
||||
inbox: inbox_registry,
|
||||
outbox: outbox_writer,
|
||||
};
|
||||
// Weekly retention sweepers for dead_letters + abandoned_executions.
|
||||
// Defaults: 30 days / 7 days (design notes §3 #9 + §4 retention).
|
||||
picloud_manager_core::spawn_dead_letter_gc(
|
||||
dl_repo.clone(),
|
||||
trigger_config.dead_letter_retention_days,
|
||||
);
|
||||
picloud_manager_core::spawn_abandoned_gc(
|
||||
abandoned_repo.clone(),
|
||||
trigger_config.abandoned_retention_days,
|
||||
);
|
||||
let triggers_state = TriggersState {
|
||||
triggers: trigger_repo,
|
||||
apps: apps_repo.clone(),
|
||||
authz: authz.clone(),
|
||||
config: trigger_config,
|
||||
};
|
||||
let dead_letters_state = DeadLettersState {
|
||||
repo: dl_repo,
|
||||
service: dl_service,
|
||||
apps: apps_repo.clone(),
|
||||
authz: authz.clone(),
|
||||
};
|
||||
let apps_state = AppsState {
|
||||
apps: apps_repo,
|
||||
@@ -191,6 +283,8 @@ pub async fn build_app(pool: PgPool, auth: AuthDeps) -> anyhow::Result<Router> {
|
||||
.merge(apps_router(apps_state))
|
||||
.merge(app_members_router(app_members_state))
|
||||
.merge(api_keys_router(api_keys_state))
|
||||
.merge(triggers_router(triggers_state))
|
||||
.merge(dead_letters_router(dead_letters_state))
|
||||
.layer(from_fn_with_state(
|
||||
auth_state.clone(),
|
||||
require_authenticated,
|
||||
@@ -200,16 +294,31 @@ pub async fn build_app(pool: PgPool, auth: AuthDeps) -> anyhow::Result<Router> {
|
||||
// facade above; the bare module path is retained so it's discoverable.
|
||||
let _ = apps_api::AppsState::clone;
|
||||
|
||||
// Opportunistic principal extraction on every data-plane request.
|
||||
// Always inserts `Extension<Option<Principal>>`: Some for authed
|
||||
// ingress (bearer / cookie), None otherwise. Handlers depend on
|
||||
// this layer being applied — scoped to the data-plane routers so
|
||||
// the admin path (which uses `require_authenticated`) doesn't
|
||||
// double-resolve the same token.
|
||||
let data_plane_routed = data_plane_router(data_plane.clone()).layer(from_fn_with_state(
|
||||
auth_state.clone(),
|
||||
attach_principal_if_present,
|
||||
));
|
||||
let user_routes = user_routes_router(data_plane).layer(from_fn_with_state(
|
||||
auth_state.clone(),
|
||||
attach_principal_if_present,
|
||||
));
|
||||
|
||||
let api_v1 = Router::new()
|
||||
.nest("/admin", auth_router(auth_state))
|
||||
.nest("/admin", guarded_admin)
|
||||
.merge(data_plane_router(data_plane.clone()));
|
||||
.merge(data_plane_routed);
|
||||
|
||||
Ok(Router::new()
|
||||
.route("/healthz", get(healthz))
|
||||
.route("/version", get(version))
|
||||
.nest(&format!("/api/v{API_VERSION}"), api_v1)
|
||||
.merge(user_routes_router(data_plane))
|
||||
.merge(user_routes)
|
||||
.layer(TraceLayer::new_for_http()))
|
||||
}
|
||||
|
||||
|
||||
118
crates/shared/src/dead_letters.rs
Normal file
118
crates/shared/src/dead_letters.rs
Normal file
@@ -0,0 +1,118 @@
|
||||
//! `DeadLetterService` — Rhai SDK contract for replaying and resolving
|
||||
//! dead letters. Surface kept intentionally narrow for v1.1.1 (no
|
||||
//! `list` — deferred to v1.2 per `docs/v1.1.x-design-notes.md` §4).
|
||||
//!
|
||||
//! Both methods are gated by `Capability::AppDeadLetterManage(AppId)`
|
||||
//! evaluated inside the impl. Public-HTTP scripts running with
|
||||
//! `cx.principal = None` will fail the check, which matches the
|
||||
//! design's expectation (managing dead letters is an admin act).
|
||||
|
||||
use async_trait::async_trait;
|
||||
use serde::{Deserialize, Serialize};
|
||||
use thiserror::Error;
|
||||
use uuid::Uuid;
|
||||
|
||||
use crate::SdkCallCx;
|
||||
|
||||
/// Opaque identifier for a `dead_letters` row.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
|
||||
#[serde(transparent)]
|
||||
pub struct DeadLetterId(pub Uuid);
|
||||
|
||||
impl DeadLetterId {
|
||||
#[must_use]
|
||||
pub fn new() -> Self {
|
||||
Self(Uuid::new_v4())
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
pub fn into_inner(self) -> Uuid {
|
||||
self.0
|
||||
}
|
||||
}
|
||||
|
||||
impl Default for DeadLetterId {
|
||||
fn default() -> Self {
|
||||
Self::new()
|
||||
}
|
||||
}
|
||||
|
||||
impl From<Uuid> for DeadLetterId {
|
||||
fn from(u: Uuid) -> Self {
|
||||
Self(u)
|
||||
}
|
||||
}
|
||||
|
||||
impl From<DeadLetterId> for Uuid {
|
||||
fn from(id: DeadLetterId) -> Self {
|
||||
id.0
|
||||
}
|
||||
}
|
||||
|
||||
impl std::fmt::Display for DeadLetterId {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
self.0.fmt(f)
|
||||
}
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
pub trait DeadLetterService: Send + Sync {
|
||||
/// Re-enqueue the original event into the outbox. The dead-letter
|
||||
/// row is marked `resolution = 'replayed'` regardless of whether
|
||||
/// the retry ultimately succeeds.
|
||||
async fn replay(&self, cx: &SdkCallCx, id: DeadLetterId) -> Result<(), DeadLetterError>;
|
||||
|
||||
/// Mark the row resolved with the given reason (typically
|
||||
/// `"ignored"` from the dashboard or `"handled_by_script"` from
|
||||
/// inside a `dead_letter` trigger handler).
|
||||
async fn resolve(
|
||||
&self,
|
||||
cx: &SdkCallCx,
|
||||
id: DeadLetterId,
|
||||
reason: &str,
|
||||
) -> Result<(), DeadLetterError>;
|
||||
}
|
||||
|
||||
#[derive(Debug, Error)]
|
||||
pub enum DeadLetterError {
|
||||
#[error("dead-letter row not found")]
|
||||
NotFound,
|
||||
|
||||
#[error("forbidden")]
|
||||
Forbidden,
|
||||
|
||||
#[error("invalid resolution reason: {0}")]
|
||||
InvalidResolution(String),
|
||||
|
||||
#[error("dead-letter backend error: {0}")]
|
||||
Backend(String),
|
||||
}
|
||||
|
||||
/// Stub used to bootstrap the `Services` bundle before the real
|
||||
/// Postgres-backed implementation lands. Behaves like
|
||||
/// `NoopEventEmitter` — every call returns `Backend("...")` so scripts
|
||||
/// see a clear "not yet implemented" error rather than silently
|
||||
/// no-op'ing. Replaced by `PostgresDeadLetterService` in the v1.1.1
|
||||
/// dead-letter PR.
|
||||
#[derive(Debug, Default, Clone, Copy)]
|
||||
pub struct NoopDeadLetterService;
|
||||
|
||||
#[async_trait]
|
||||
impl DeadLetterService for NoopDeadLetterService {
|
||||
async fn replay(&self, _cx: &SdkCallCx, _id: DeadLetterId) -> Result<(), DeadLetterError> {
|
||||
Err(DeadLetterError::Backend(
|
||||
"dead_letters::replay is not yet wired in".into(),
|
||||
))
|
||||
}
|
||||
|
||||
async fn resolve(
|
||||
&self,
|
||||
_cx: &SdkCallCx,
|
||||
_id: DeadLetterId,
|
||||
_reason: &str,
|
||||
) -> Result<(), DeadLetterError> {
|
||||
Err(DeadLetterError::Backend(
|
||||
"dead_letters::resolve is not yet wired in".into(),
|
||||
))
|
||||
}
|
||||
}
|
||||
119
crates/shared/src/events.rs
Normal file
119
crates/shared/src/events.rs
Normal file
@@ -0,0 +1,119 @@
|
||||
//! `ServiceEventEmitter` — the contract every stateful SDK service uses
|
||||
//! to publish events into the (future) triggers framework.
|
||||
//!
|
||||
//! v1.1.0 ships only the trait shape and a `NoopEventEmitter` that
|
||||
//! drops every event. The real outbox-backed implementation lands with
|
||||
//! the triggers PR in v1.1.1; locking the trait now means services
|
||||
//! written in subsequent v1.1.x PRs (KV, docs, files, …) don't have to
|
||||
//! re-thread their plumbing when the dispatcher arrives.
|
||||
//!
|
||||
//! Design rationale (full discussion: `docs/sdk-shape.md`):
|
||||
//! * Async — outbox writes hit Postgres.
|
||||
//! * Cx is passed in so the emitter can attribute the event to the
|
||||
//! `app_id` / `principal` / `execution_id` that produced it.
|
||||
//! * Events carry their semantic identity (`source` + `op`) plus
|
||||
//! optional locator (`collection` + `key`) and optional payloads
|
||||
//! (`payload` for the new value, `old_payload` for the previous on
|
||||
//! updates). The dispatcher matches on (source, op, collection)
|
||||
//! filters to decide which scripts to fan out to.
|
||||
|
||||
use async_trait::async_trait;
|
||||
use thiserror::Error;
|
||||
|
||||
use crate::SdkCallCx;
|
||||
|
||||
/// Trait every stateful service depends on to emit events. The host
|
||||
/// binary constructs one instance and clones the Arc into each service.
|
||||
#[async_trait]
|
||||
pub trait ServiceEventEmitter: Send + Sync {
|
||||
/// Publish a single event. Implementations are expected to be
|
||||
/// fire-and-forget from the caller's perspective: the outbox impl
|
||||
/// will return `Ok(())` once the event is durably persisted, the
|
||||
/// dispatcher reads it out-of-band.
|
||||
async fn emit(&self, cx: &SdkCallCx, event: ServiceEvent) -> Result<(), EmitError>;
|
||||
}
|
||||
|
||||
/// One service event. `source` and `op` are `&'static str` because they
|
||||
/// come from a fixed enumeration baked into each service (`"kv"` +
|
||||
/// `"insert"`/`"update"`/`"delete"`, etc.) — never from user data.
|
||||
/// `collection`/`key`/payloads come from user data and are owned.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct ServiceEvent {
|
||||
/// Service namespace. Matches the Rhai module name: `"kv"`,
|
||||
/// `"docs"`, `"files"`, etc.
|
||||
pub source: &'static str,
|
||||
|
||||
/// Operation verb. Each service defines its own vocabulary;
|
||||
/// dispatcher filters match on the literal string.
|
||||
pub op: &'static str,
|
||||
|
||||
/// Affected collection, when the service is collection-scoped
|
||||
/// (`kv`, `docs`, `files`). `None` for collection-less events.
|
||||
pub collection: Option<String>,
|
||||
|
||||
/// Affected key/id within the collection, when applicable.
|
||||
pub key: Option<String>,
|
||||
|
||||
/// New value after the operation, when carrying it is cheap and
|
||||
/// useful. `None` for deletes.
|
||||
pub payload: Option<serde_json::Value>,
|
||||
|
||||
/// Previous value before the operation, populated on `update` /
|
||||
/// `delete` so triggers can diff. `None` on `insert`.
|
||||
pub old_payload: Option<serde_json::Value>,
|
||||
}
|
||||
|
||||
/// Errors an emitter can surface upward. The noop impl never returns
|
||||
/// these; the v1.1.1 outbox impl uses `Unavailable` for pool/connection
|
||||
/// failures and `Rejected` for malformed payloads (e.g. event JSON too
|
||||
/// large for the outbox row).
|
||||
#[derive(Debug, Error)]
|
||||
pub enum EmitError {
|
||||
#[error("event sink unavailable: {0}")]
|
||||
Unavailable(String),
|
||||
#[error("event sink rejected event: {0}")]
|
||||
Rejected(String),
|
||||
}
|
||||
|
||||
/// Default emitter for v1.1.0. Accepts every event, persists nothing,
|
||||
/// always returns `Ok(())`. Wired in the picloud binary; the v1.1.1
|
||||
/// triggers PR swaps this for a Postgres outbox writer.
|
||||
#[derive(Debug, Default, Clone, Copy)]
|
||||
pub struct NoopEventEmitter;
|
||||
|
||||
#[async_trait]
|
||||
impl ServiceEventEmitter for NoopEventEmitter {
|
||||
async fn emit(&self, _cx: &SdkCallCx, _event: ServiceEvent) -> Result<(), EmitError> {
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
// Compile-time check that ServiceEventEmitter is dyn-safe — every
|
||||
// service holds it as `Arc<dyn ServiceEventEmitter>` and would
|
||||
// silently break the workspace if a non-object-safe method snuck
|
||||
// in. Behavioural tests for the noop impl come for free once a
|
||||
// service exercises it (v1.1.1+); avoid pulling tokio into
|
||||
// `picloud-shared` just for a one-line `emit().await` check.
|
||||
#[allow(dead_code)]
|
||||
fn assert_dyn_compatible(_e: &dyn ServiceEventEmitter) {}
|
||||
|
||||
#[test]
|
||||
fn service_event_construction_is_explicit() {
|
||||
// Pin the field layout so a re-ordering in a future PR causes a
|
||||
// compile failure here rather than silently misattributing
|
||||
// events. Hash-derive isn't appropriate (serde_json::Value isn't
|
||||
// Hash), so structural construction is the assertion.
|
||||
let _ = ServiceEvent {
|
||||
source: "kv",
|
||||
op: "insert",
|
||||
collection: Some("widgets".into()),
|
||||
key: Some("k1".into()),
|
||||
payload: Some(serde_json::json!({"v": 1})),
|
||||
old_payload: None,
|
||||
};
|
||||
}
|
||||
}
|
||||
16
crates/shared/src/exec_summary.rs
Normal file
16
crates/shared/src/exec_summary.rs
Normal file
@@ -0,0 +1,16 @@
|
||||
//! `ExecResponseSummary` — a flattened, crate-portable view of an
|
||||
//! `ExecResponse` for use by `InboxResult`. Lives in
|
||||
//! `picloud-shared` because the dispatcher (manager-core) and the
|
||||
//! orchestrator-core inbox registry both need to read it, and
|
||||
//! `executor-core::ExecResponse` is owned by a leaf crate.
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct ExecResponseSummary {
|
||||
pub status_code: u16,
|
||||
pub headers: BTreeMap<String, String>,
|
||||
pub body: serde_json::Value,
|
||||
}
|
||||
@@ -53,3 +53,4 @@ id_type!(RequestId);
|
||||
id_type!(AdminUserId);
|
||||
id_type!(AppId);
|
||||
id_type!(ApiKeyId);
|
||||
id_type!(TriggerId);
|
||||
|
||||
86
crates/shared/src/inbox.rs
Normal file
86
crates/shared/src/inbox.rs
Normal file
@@ -0,0 +1,86 @@
|
||||
//! `InboxResolver` — abstraction the dispatcher uses to deliver sync
|
||||
//! HTTP results back to the orchestrator that's awaiting them on a
|
||||
//! oneshot channel. Lives in `picloud-shared` because the dispatcher
|
||||
//! (manager-core) and the registry impl (orchestrator-core) live in
|
||||
//! different crates and need a shared trait surface.
|
||||
//!
|
||||
//! v1.1.1 ships an in-process implementation in `orchestrator-core`
|
||||
//! that keeps a `HashMap<inbox_id, oneshot::Sender<...>>`. Cluster
|
||||
//! mode (v1.3+) swaps this for a Postgres `LISTEN/NOTIFY`-based
|
||||
//! resolver without touching the dispatcher code (design notes §3
|
||||
//! implementation table).
|
||||
//!
|
||||
//! Until commit 6 wires up the real registry, `NoopInboxResolver`
|
||||
//! (`Abandoned` for every attempt) keeps the dispatcher able to run.
|
||||
|
||||
use async_trait::async_trait;
|
||||
use uuid::Uuid;
|
||||
|
||||
use crate::ExecResponseSummary;
|
||||
|
||||
/// Result of trying to hand back a sync-HTTP outcome.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum InboxDeliveryOutcome {
|
||||
/// Receiver still attached; result was delivered. Dispatcher
|
||||
/// deletes the outbox row.
|
||||
Delivered,
|
||||
/// Receiver was dropped (orchestrator timed out). Dispatcher
|
||||
/// writes an `abandoned_executions` row.
|
||||
Abandoned,
|
||||
}
|
||||
|
||||
/// Outcome shape the dispatcher delivers to the inbox. Carries enough
|
||||
/// to reconstruct an HTTP response — full body via JSON, optional
|
||||
/// error string when the executor reported a failure.
|
||||
#[derive(Debug, Clone)]
|
||||
pub enum InboxResult {
|
||||
/// Successful execution. `response` is the `ExecResponse` summary
|
||||
/// (status code + body + headers + logs).
|
||||
Success(ExecResponseSummary),
|
||||
/// Failure modes — script threw, op-budget, timeout, etc. The
|
||||
/// orchestrator maps these to the design-notes §3 status codes
|
||||
/// (422/502/503/504/507/500) when responding to the HTTP caller.
|
||||
Failure {
|
||||
kind: InboxFailureKind,
|
||||
message: String,
|
||||
},
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum InboxFailureKind {
|
||||
/// Script's Rhai code threw or hit a runtime error → 502.
|
||||
Runtime,
|
||||
/// Wall-clock exceeded → 504.
|
||||
Timeout,
|
||||
/// Operation budget exceeded → 507.
|
||||
OperationBudget,
|
||||
/// Gate refused admission → 503.
|
||||
Overloaded,
|
||||
/// Script parse failure / bad-request → 422.
|
||||
Validation,
|
||||
/// Platform problem (executor crashed, dispatcher crashed, etc.) → 500.
|
||||
Platform,
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
pub trait InboxResolver: Send + Sync {
|
||||
/// Attempt to deliver `result` to the receiver registered under
|
||||
/// `inbox_id`. Returns `Delivered` if the channel was alive,
|
||||
/// `Abandoned` if the receiver was already dropped (the
|
||||
/// orchestrator's timeout fired before the dispatcher got here).
|
||||
async fn deliver(&self, inbox_id: Uuid, result: InboxResult) -> InboxDeliveryOutcome;
|
||||
}
|
||||
|
||||
/// Bootstrap impl used before the real registry is wired in. Every
|
||||
/// delivery is treated as abandoned — the dispatcher records an
|
||||
/// abandoned-execution row and moves on. Replaced in `build_app` with
|
||||
/// the in-process `InboxRegistry` from orchestrator-core.
|
||||
#[derive(Debug, Default, Clone, Copy)]
|
||||
pub struct NoopInboxResolver;
|
||||
|
||||
#[async_trait]
|
||||
impl InboxResolver for NoopInboxResolver {
|
||||
async fn deliver(&self, _inbox_id: Uuid, _result: InboxResult) -> InboxDeliveryOutcome {
|
||||
InboxDeliveryOutcome::Abandoned
|
||||
}
|
||||
}
|
||||
140
crates/shared/src/kv.rs
Normal file
140
crates/shared/src/kv.rs
Normal file
@@ -0,0 +1,140 @@
|
||||
//! `KvService` — the v1.1.1 key-value store contract.
|
||||
//!
|
||||
//! Lives in `picloud-shared` (not `executor-core`) so the Rhai bridge,
|
||||
//! the manager-core Postgres impl, and any future in-memory test impl
|
||||
//! can all depend on the same trait without dragging
|
||||
//! `executor-core` into `manager-core`'s dep graph.
|
||||
//!
|
||||
//! Implementations MUST derive every storage `app_id` from `cx.app_id`
|
||||
//! — never from a script-passed argument. That is the cross-app
|
||||
//! isolation boundary; see `docs/sdk-shape.md`.
|
||||
|
||||
use async_trait::async_trait;
|
||||
use thiserror::Error;
|
||||
|
||||
use crate::SdkCallCx;
|
||||
|
||||
/// `KvService` is collection-scoped. Scripts get a handle via
|
||||
/// `kv::collection(name)` and call `get`/`set`/`has`/`delete`/`list`
|
||||
/// on it. The trait surface accepts the collection by name so the
|
||||
/// Postgres impl can avoid an extra round-trip to materialize the
|
||||
/// collection (collections are namespaces, not first-class rows).
|
||||
#[async_trait]
|
||||
pub trait KvService: Send + Sync {
|
||||
async fn get(
|
||||
&self,
|
||||
cx: &SdkCallCx,
|
||||
collection: &str,
|
||||
key: &str,
|
||||
) -> Result<Option<serde_json::Value>, KvError>;
|
||||
|
||||
async fn set(
|
||||
&self,
|
||||
cx: &SdkCallCx,
|
||||
collection: &str,
|
||||
key: &str,
|
||||
value: serde_json::Value,
|
||||
) -> Result<(), KvError>;
|
||||
|
||||
async fn delete(&self, cx: &SdkCallCx, collection: &str, key: &str) -> Result<bool, KvError>;
|
||||
|
||||
async fn has(&self, cx: &SdkCallCx, collection: &str, key: &str) -> Result<bool, KvError>;
|
||||
|
||||
/// Cursor-style pagination. `cursor` is opaque to the caller;
|
||||
/// implementations encode the resume key inside. `None` cursor
|
||||
/// starts from the beginning. Implementations cap `limit` at a
|
||||
/// reasonable ceiling internally (script can't request an unbounded
|
||||
/// page).
|
||||
async fn list(
|
||||
&self,
|
||||
cx: &SdkCallCx,
|
||||
collection: &str,
|
||||
cursor: Option<&str>,
|
||||
limit: u32,
|
||||
) -> Result<KvListPage, KvError>;
|
||||
}
|
||||
|
||||
/// One page of keys from `KvService::list`. `next_cursor` is `Some`
|
||||
/// when more pages exist, `None` when exhausted. The cursor encoding
|
||||
/// is implementation-defined (the Postgres impl base64-encodes the
|
||||
/// last key).
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct KvListPage {
|
||||
pub keys: Vec<String>,
|
||||
pub next_cursor: Option<String>,
|
||||
}
|
||||
|
||||
/// Stub used by the test harness so executor-core integration tests
|
||||
/// (which don't touch KV) can construct a `Services` bundle without
|
||||
/// spinning up Postgres. Every call returns
|
||||
/// `KvError::Backend("...")` so accidental KV use surfaces clearly.
|
||||
#[derive(Debug, Default, Clone, Copy)]
|
||||
pub struct NoopKvService;
|
||||
|
||||
#[async_trait]
|
||||
impl KvService for NoopKvService {
|
||||
async fn get(
|
||||
&self,
|
||||
_cx: &SdkCallCx,
|
||||
_collection: &str,
|
||||
_key: &str,
|
||||
) -> Result<Option<serde_json::Value>, KvError> {
|
||||
Err(KvError::Backend("kv is not wired in".into()))
|
||||
}
|
||||
|
||||
async fn set(
|
||||
&self,
|
||||
_cx: &SdkCallCx,
|
||||
_collection: &str,
|
||||
_key: &str,
|
||||
_value: serde_json::Value,
|
||||
) -> Result<(), KvError> {
|
||||
Err(KvError::Backend("kv is not wired in".into()))
|
||||
}
|
||||
|
||||
async fn delete(
|
||||
&self,
|
||||
_cx: &SdkCallCx,
|
||||
_collection: &str,
|
||||
_key: &str,
|
||||
) -> Result<bool, KvError> {
|
||||
Err(KvError::Backend("kv is not wired in".into()))
|
||||
}
|
||||
|
||||
async fn has(&self, _cx: &SdkCallCx, _collection: &str, _key: &str) -> Result<bool, KvError> {
|
||||
Err(KvError::Backend("kv is not wired in".into()))
|
||||
}
|
||||
|
||||
async fn list(
|
||||
&self,
|
||||
_cx: &SdkCallCx,
|
||||
_collection: &str,
|
||||
_cursor: Option<&str>,
|
||||
_limit: u32,
|
||||
) -> Result<KvListPage, KvError> {
|
||||
Err(KvError::Backend("kv is not wired in".into()))
|
||||
}
|
||||
}
|
||||
|
||||
/// Failure modes surfaced to the Rhai bridge. The bridge converts each
|
||||
/// to a Rhai runtime error string; the discriminants exist so internal
|
||||
/// callers (admin endpoints, tests, GC) can react more precisely.
|
||||
#[derive(Debug, Error)]
|
||||
pub enum KvError {
|
||||
/// Empty collection name; rejected at the SDK boundary per
|
||||
/// `docs/sdk-shape.md`.
|
||||
#[error("collection name must not be empty")]
|
||||
InvalidCollection,
|
||||
|
||||
/// Caller principal lacked the required capability. Only raised
|
||||
/// when `cx.principal.is_some()` — scripts running with
|
||||
/// `principal: None` (public HTTP) operate under script-as-gate
|
||||
/// semantics and skip the capability check.
|
||||
#[error("forbidden")]
|
||||
Forbidden,
|
||||
|
||||
/// Anything else — Postgres unavailable, serialization failure,
|
||||
/// etc. The string is safe to surface to a script.
|
||||
#[error("kv backend error: {0}")]
|
||||
Backend(String),
|
||||
}
|
||||
@@ -6,24 +6,44 @@
|
||||
|
||||
pub mod app;
|
||||
pub mod auth;
|
||||
pub mod dead_letters;
|
||||
pub mod error;
|
||||
pub mod events;
|
||||
pub mod exec_summary;
|
||||
pub mod execution_log;
|
||||
pub mod ids;
|
||||
pub mod inbox;
|
||||
pub mod kv;
|
||||
pub mod log_sink;
|
||||
pub mod outbox_writer;
|
||||
pub mod route;
|
||||
pub mod sandbox;
|
||||
pub mod script;
|
||||
pub mod sdk_cx;
|
||||
pub mod services;
|
||||
pub mod trigger_event;
|
||||
pub mod validator;
|
||||
pub mod version;
|
||||
|
||||
pub use app::{App, AppDomain, DomainShape};
|
||||
pub use auth::{AppRole, InstanceRole, Principal, Scope, UserId};
|
||||
pub use dead_letters::{DeadLetterError, DeadLetterId, DeadLetterService, NoopDeadLetterService};
|
||||
pub use error::Error;
|
||||
pub use events::{EmitError, NoopEventEmitter, ServiceEvent, ServiceEventEmitter};
|
||||
pub use exec_summary::ExecResponseSummary;
|
||||
pub use execution_log::{ExecutionLog, ExecutionStatus};
|
||||
pub use ids::{AdminUserId, ApiKeyId, AppId, ExecutionId, RequestId, ScriptId};
|
||||
pub use ids::{AdminUserId, ApiKeyId, AppId, ExecutionId, RequestId, ScriptId, TriggerId};
|
||||
pub use inbox::{
|
||||
InboxDeliveryOutcome, InboxFailureKind, InboxResolver, InboxResult, NoopInboxResolver,
|
||||
};
|
||||
pub use kv::{KvError, KvListPage, KvService, NoopKvService};
|
||||
pub use log_sink::{ExecutionLogSink, LogSinkError};
|
||||
pub use route::{HostKind, PathKind, Route};
|
||||
pub use outbox_writer::{HttpDispatchPayload, NewHttpOutbox, OutboxWriter, OutboxWriterError};
|
||||
pub use route::{DispatchMode, HostKind, PathKind, Route};
|
||||
pub use sandbox::ScriptSandbox;
|
||||
pub use script::Script;
|
||||
pub use sdk_cx::SdkCallCx;
|
||||
pub use services::Services;
|
||||
pub use trigger_event::{DeadLetterEventDetail, KvEventOp, TriggerEvent};
|
||||
pub use validator::{ScriptValidator, ValidationError};
|
||||
pub use version::{API_VERSION, PRODUCT_VERSION, SDK_VERSION, WIRE_VERSION};
|
||||
|
||||
72
crates/shared/src/outbox_writer.rs
Normal file
72
crates/shared/src/outbox_writer.rs
Normal file
@@ -0,0 +1,72 @@
|
||||
//! `OutboxWriter` — minimal trait the orchestrator-core sync-HTTP path
|
||||
//! uses to enqueue rows into the universal trigger outbox. The
|
||||
//! manager-core `PostgresOutboxRepo` implements this in addition to
|
||||
//! its richer `OutboxRepo` surface; defining it here lets
|
||||
//! orchestrator-core depend on the trait without pulling in
|
||||
//! manager-core (which would invert the dependency arrow).
|
||||
|
||||
use async_trait::async_trait;
|
||||
use serde::{Deserialize, Serialize};
|
||||
use thiserror::Error;
|
||||
use uuid::Uuid;
|
||||
|
||||
use crate::{AdminUserId, AppId, ExecutionId, ScriptId};
|
||||
|
||||
/// What the orchestrator hands to the outbox when it ingests an HTTP
|
||||
/// request. Carries enough for the dispatcher to reconstruct the
|
||||
/// `ExecRequest` end-to-end.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct NewHttpOutbox {
|
||||
pub app_id: AppId,
|
||||
/// `routes.id` of the matched route. Discriminated against
|
||||
/// `triggers.id` by `source_kind = 'http'` on the outbox row.
|
||||
pub route_id: Uuid,
|
||||
/// Pre-resolved script so the dispatcher doesn't re-look it up.
|
||||
pub script_id: ScriptId,
|
||||
/// `Some(inbox_id)` for sync HTTP (the orchestrator awaits a
|
||||
/// channel keyed on this id). `None` for `dispatch_mode = async`
|
||||
/// — dispatcher fires-and-forgets, no reply path.
|
||||
pub reply_to: Option<Uuid>,
|
||||
/// Serialized `HttpDispatchPayload` (defined below) — everything
|
||||
/// the dispatcher needs to reconstruct an `ExecRequest`.
|
||||
pub payload: serde_json::Value,
|
||||
/// The principal that ingressed the HTTP request (Some when
|
||||
/// authenticated, None for public). Forensic only; the script
|
||||
/// executes as the route's app principal model, not this.
|
||||
pub origin_principal: Option<AdminUserId>,
|
||||
/// `0` for direct HTTP ingress; the dispatcher will increment
|
||||
/// for any further fan-out triggered by the script.
|
||||
pub trigger_depth: u32,
|
||||
pub root_execution_id: Option<ExecutionId>,
|
||||
}
|
||||
|
||||
/// The shape the orchestrator serializes into `NewHttpOutbox.payload`
|
||||
/// (the JSONB column). Mirrored on the dispatcher side so it can
|
||||
/// rebuild an `ExecRequest`.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct HttpDispatchPayload {
|
||||
pub script_name: String,
|
||||
pub path: String,
|
||||
pub method: String,
|
||||
pub headers: std::collections::BTreeMap<String, String>,
|
||||
pub body: serde_json::Value,
|
||||
pub params: std::collections::BTreeMap<String, String>,
|
||||
pub query: std::collections::BTreeMap<String, String>,
|
||||
pub rest: String,
|
||||
pub timeout_seconds: u32,
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
pub trait OutboxWriter: Send + Sync {
|
||||
/// Insert a sync- or async-HTTP outbox row. Returns the row's id
|
||||
/// — the orchestrator stores it locally for forensics and to
|
||||
/// correlate `abandoned_executions` rows when the dispatcher's
|
||||
/// inbox delivery fails.
|
||||
async fn enqueue_http(&self, row: NewHttpOutbox) -> Result<Uuid, OutboxWriterError>;
|
||||
}
|
||||
|
||||
#[derive(Debug, Error)]
|
||||
pub enum OutboxWriterError {
|
||||
#[error("outbox write failed: {0}")]
|
||||
Backend(String),
|
||||
}
|
||||
@@ -37,6 +37,38 @@ pub enum PathKind {
|
||||
Param,
|
||||
}
|
||||
|
||||
/// Per-route dispatch mode (v1.1.1). `Sync` = orchestrator awaits the
|
||||
/// executor and returns the response in the same HTTP request. `Async`
|
||||
/// = orchestrator writes the request to the trigger outbox, returns
|
||||
/// `202 Accepted` immediately, and the dispatcher runs the script in
|
||||
/// the background (with retries + dead-letter).
|
||||
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq, Default)]
|
||||
#[serde(rename_all = "lowercase")]
|
||||
pub enum DispatchMode {
|
||||
#[default]
|
||||
Sync,
|
||||
Async,
|
||||
}
|
||||
|
||||
impl DispatchMode {
|
||||
#[must_use]
|
||||
pub const fn as_str(self) -> &'static str {
|
||||
match self {
|
||||
Self::Sync => "sync",
|
||||
Self::Async => "async",
|
||||
}
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
pub fn from_wire(s: &str) -> Option<Self> {
|
||||
match s {
|
||||
"sync" => Some(Self::Sync),
|
||||
"async" => Some(Self::Async),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct Route {
|
||||
pub id: Uuid,
|
||||
@@ -60,5 +92,12 @@ pub struct Route {
|
||||
/// `None` = any method.
|
||||
pub method: Option<String>,
|
||||
|
||||
/// v1.1.1: per-route dispatch mode. `Sync` (default) → orchestrator
|
||||
/// awaits the executor inline. `Async` → orchestrator writes to
|
||||
/// the outbox + returns `202 Accepted`; dispatcher fires the
|
||||
/// script in the background with retries.
|
||||
#[serde(default)]
|
||||
pub dispatch_mode: DispatchMode,
|
||||
|
||||
pub created_at: DateTime<Utc>,
|
||||
}
|
||||
|
||||
69
crates/shared/src/sdk_cx.rs
Normal file
69
crates/shared/src/sdk_cx.rs
Normal file
@@ -0,0 +1,69 @@
|
||||
//! `SdkCallCx` — per-call context every stateful SDK service receives.
|
||||
//!
|
||||
//! Service trait methods (added by subsequent v1.1.x PRs starting with
|
||||
//! KV) all take `&SdkCallCx` so they can:
|
||||
//! * scope by `app_id` for cross-app isolation,
|
||||
//! * audit `principal` when authenticated,
|
||||
//! * carry `execution_id` / `request_id` into emitted events,
|
||||
//! * bound trigger chains via `trigger_depth` / `root_execution_id`.
|
||||
//!
|
||||
//! The struct lives in `picloud-shared` (not `executor-core`) because
|
||||
//! future service impls live in `manager-core` and the trait that hands
|
||||
//! the cx in is shared by both sides. Pure value type — no handles, no
|
||||
//! DB pool references, no allocations beyond what's in `Principal`.
|
||||
|
||||
use crate::{AppId, ExecutionId, Principal, RequestId, TriggerEvent};
|
||||
|
||||
/// Per-invocation context for every stateful SDK service call.
|
||||
///
|
||||
/// Constructed once at the start of an invocation by `executor-core`
|
||||
/// from the incoming `ExecRequest`, then handed (by reference) to every
|
||||
/// service trait method the script triggers during execution. Services
|
||||
/// MUST derive `app_id` from this struct — never from script-passed
|
||||
/// arguments — to preserve cross-app isolation.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct SdkCallCx {
|
||||
/// Owning application for this invocation. Source of truth for
|
||||
/// every `(app_id, …)` storage lookup the script makes.
|
||||
pub app_id: AppId,
|
||||
|
||||
/// Caller identity, when authenticated. `None` for unauthenticated
|
||||
/// data-plane HTTP requests (the common case for public endpoints);
|
||||
/// `Some` when the call came in via the dashboard, an API key, or a
|
||||
/// future authed surface.
|
||||
pub principal: Option<Principal>,
|
||||
|
||||
/// Unique id for THIS execution. Matches `ExecRequest.execution_id`.
|
||||
pub execution_id: ExecutionId,
|
||||
|
||||
/// Unique id for the ingress request that started the chain. The
|
||||
/// same `request_id` is shared across every execution triggered by
|
||||
/// the same request (direct + trigger fan-out).
|
||||
pub request_id: RequestId,
|
||||
|
||||
/// `0` for direct invocations (HTTP request, manual run). Each
|
||||
/// indirect invocation through the triggers framework (v1.1.1)
|
||||
/// increments this; the dispatcher rejects beyond a configured
|
||||
/// ceiling to prevent runaway feedback loops.
|
||||
pub trigger_depth: u32,
|
||||
|
||||
/// `== execution_id` when `trigger_depth == 0`; otherwise the
|
||||
/// `execution_id` of the original ingress execution. Lets the audit
|
||||
/// log group every fan-out execution under the originating event.
|
||||
pub root_execution_id: ExecutionId,
|
||||
|
||||
/// `true` only when this invocation is a `dead_letter` trigger
|
||||
/// handler. Set by the dispatcher when it picks an outbox row
|
||||
/// whose trigger has `kind = 'dead_letter'`. The retry / dead-
|
||||
/// letter machinery short-circuits when this is set: handlers
|
||||
/// execute once, with no retry, and a failed run can NEVER be
|
||||
/// dead-lettered itself (design notes §4 recursion-stop rule).
|
||||
/// `false` for every other invocation, including the script
|
||||
/// being used as a non-DL trigger handler.
|
||||
pub is_dead_letter_handler: bool,
|
||||
|
||||
/// The event that fired this script, when it's a triggered
|
||||
/// invocation. `None` for direct ingress (HTTP request, manual
|
||||
/// run). Surfaced to scripts as `ctx.event`.
|
||||
pub event: Option<TriggerEvent>,
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user