Compare commits
15 Commits
feat/v1.1.
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
5cbb6ca427 | ||
|
|
3cfb795206 | ||
|
|
a7d3dad129 | ||
|
|
2ea47eb05a | ||
|
|
b35585195b | ||
|
|
fffcdf6169 | ||
|
|
02335a8132 | ||
|
|
1f78937dd2 | ||
|
|
8f2d2bc721 | ||
|
|
2d11090d1a | ||
|
|
dc2e4fa01f | ||
|
|
64ad978a89 | ||
|
|
f5a3f92484 | ||
|
|
b1dddb9cb9 | ||
|
|
fcbcc576a2 |
3
.gitignore
vendored
3
.gitignore
vendored
@@ -22,6 +22,9 @@ Cargo.lock.bak
|
|||||||
# Local config overrides
|
# Local config overrides
|
||||||
config.local.toml
|
config.local.toml
|
||||||
/data
|
/data
|
||||||
|
# Files-root blob storage created when integration tests run build_app
|
||||||
|
# from the picloud crate dir (PICLOUD_FILES_ROOT default ./data).
|
||||||
|
/crates/picloud/data
|
||||||
/postgres-data
|
/postgres-data
|
||||||
|
|
||||||
# Dashboard
|
# Dashboard
|
||||||
|
|||||||
183
CHANGELOG.md
183
CHANGELOG.md
@@ -1,5 +1,188 @@
|
|||||||
# PiCloud Changelog
|
# PiCloud Changelog
|
||||||
|
|
||||||
|
## v1.1.7 — Configuration & Email (unreleased)
|
||||||
|
|
||||||
|
The operational-config layer: **encrypted per-app secrets**, **outbound
|
||||||
|
email**, and an **inbound email trigger** — plus the long-missing
|
||||||
|
**dead-letter handler wiring** and **at-rest encryption of the realtime
|
||||||
|
signing key**. All at-rest encryption uses a single process master key
|
||||||
|
(AES-256-GCM); key rotation is deferred to v1.2.
|
||||||
|
|
||||||
|
### Added — Encryption infrastructure
|
||||||
|
|
||||||
|
- **Process master key** from `PICLOUD_SECRET_KEY` (base64 of exactly 32
|
||||||
|
bytes). REQUIRED at startup — an unset or malformed key is fatal.
|
||||||
|
Generate one with `openssl rand -base64 32`. A deterministic in-memory
|
||||||
|
dev key is used ONLY when `PICLOUD_SECRET_KEY` is unset AND
|
||||||
|
`PICLOUD_DEV_MODE=true` (with a prominent startup warning); there is no
|
||||||
|
quiet unencrypted mode.
|
||||||
|
- **`picloud_shared::crypto`** — `encrypt`/`decrypt` envelope:
|
||||||
|
`Aes256Gcm`, 96-bit CSPRNG nonce, 128-bit auth tag appended to the
|
||||||
|
ciphertext (RustCrypto `Aead` layout). Both ciphertext and nonce are
|
||||||
|
stored.
|
||||||
|
- **Key rotation is out of scope.** Changing `PICLOUD_SECRET_KEY` between
|
||||||
|
deploys renders all existing ciphertext undecryptable. v1.2+ adds
|
||||||
|
key-version columns + a re-encryption pass.
|
||||||
|
|
||||||
|
### Added — Encrypted per-app secrets
|
||||||
|
|
||||||
|
- **`secrets::{get,set,delete,list}(name)`** SDK — collection-less,
|
||||||
|
per-app. `set` accepts a String/Map/Array (JSON-encoded then encrypted);
|
||||||
|
`get` returns the same Rhai type back; missing → `()`. 64 KB plaintext
|
||||||
|
cap (`PICLOUD_SECRET_MAX_VALUE_BYTES`). `migrations/0023_secrets.sql`.
|
||||||
|
- **Admin API** `GET/POST/DELETE /api/v1/admin/apps/{id}/secrets` — list
|
||||||
|
returns names + `updated_at` only, **never values**.
|
||||||
|
- **Dashboard Secrets tab** — list names + last-modified, create/update
|
||||||
|
(masked value with a confirm-gated reveal), delete with confirm.
|
||||||
|
- `Capability::AppSecretsRead`/`Write` (→ `script:read` / `script:write`).
|
||||||
|
No new Scope variants (seven-scope commitment). Secret writes
|
||||||
|
deliberately do **not** emit trigger events.
|
||||||
|
|
||||||
|
### Added — Outbound email
|
||||||
|
|
||||||
|
- **`email::send` / `email::send_html`** SDK over an SMTP relay
|
||||||
|
(`lettre`). Config from `PICLOUD_SMTP_HOST/PORT/USER/PASSWORD/TLS/
|
||||||
|
TIMEOUT_SECS`; if HOST/USER/PASSWORD aren't all set the service runs in
|
||||||
|
**disabled mode** (every send throws `NotConfigured`, warned at
|
||||||
|
startup). Required `to`/`from`/`subject` + one of `text`/`html`;
|
||||||
|
RFC 5322-ish address validation; 25 MB per-message cap
|
||||||
|
(`PICLOUD_EMAIL_MAX_MESSAGE_BYTES`); `reply_to` defaults to `from`.
|
||||||
|
Per-call connection (pooling deferred to v1.2); per-app `from`
|
||||||
|
validation / SPF / DKIM are the operator's SMTP-relay concern.
|
||||||
|
- `Capability::AppEmailSend` (→ `script:write`).
|
||||||
|
|
||||||
|
### Added — Inbound email (`email:receive` trigger)
|
||||||
|
|
||||||
|
- **Webhook receiver** `POST /api/v1/email-inbound/{app_id}/{trigger_id}`
|
||||||
|
— a provider (Mailgun / Postmark / SendGrid / SES) POSTs the generic
|
||||||
|
JSON shape `{from,to[],cc[],subject,text,html,message_id}`; the
|
||||||
|
receiver verifies the optional HMAC signature, normalizes to
|
||||||
|
`TriggerEvent::Email`, and enqueues an outbox row. 202 accepted, 401
|
||||||
|
bad/missing signature, 404 missing/wrong-kind/cross-app, 422 malformed.
|
||||||
|
Handlers see `ctx.event.email`. `migrations/0024_email_triggers.sql`.
|
||||||
|
- **Admin** `POST /api/v1/admin/apps/{id}/triggers/email` +
|
||||||
|
dashboard form (with the webhook URL + expected payload). The HMAC
|
||||||
|
`inbound_secret` is stored **encrypted** via the master key (deviation
|
||||||
|
from the original plaintext design — see HANDBACK §7).
|
||||||
|
- Provider-specific payload unmarshallers + inbound attachments → v1.2.
|
||||||
|
Native SMTP listener → v1.3+.
|
||||||
|
|
||||||
|
### Security/correctness fix (retroactive) — dead_letter handlers
|
||||||
|
|
||||||
|
The `dead_letter` trigger kind has been registerable since v1.1.1 but,
|
||||||
|
due to missing dispatcher wiring (`list_matching_dead_letter` had no
|
||||||
|
production caller), handlers have **never fired**. Any deploy running
|
||||||
|
v1.1.1 through v1.1.6 with `dead_letter` triggers configured has had
|
||||||
|
silently non-functional handlers. v1.1.7 fixes the wiring; existing
|
||||||
|
`dead_letters` rows remain (no migration needed) but only NEW
|
||||||
|
dead-letter events (post-v1.1.7) trigger handlers. To process older
|
||||||
|
rows, use the existing admin replay surface to re-enqueue them.
|
||||||
|
|
||||||
|
### Changed — Realtime signing key encrypted at rest (two-phase)
|
||||||
|
|
||||||
|
`app_secrets.realtime_signing_key` was stored as 32 plaintext bytes. It
|
||||||
|
is now encrypted with the master key. `migrations/0025_encrypt_realtime_keys.sql`
|
||||||
|
adds NULL-able encrypted columns and drops `NOT NULL` on the plaintext
|
||||||
|
column; a startup task encrypts pre-existing rows; the read path prefers
|
||||||
|
the encrypted columns and falls back to plaintext during the compat
|
||||||
|
window. **v1.1.8 will drop the plaintext `realtime_signing_key`
|
||||||
|
column** — operators should upgrade through v1.1.7 (which performs the
|
||||||
|
encryption) before v1.1.8.
|
||||||
|
|
||||||
|
### Notes
|
||||||
|
|
||||||
|
- **New deps:** `aes-gcm` (RustCrypto AEAD), `lettre` (SMTP).
|
||||||
|
- **New env vars:** `PICLOUD_SECRET_KEY` (required), `PICLOUD_DEV_MODE`,
|
||||||
|
`PICLOUD_SECRET_MAX_VALUE_BYTES`, `PICLOUD_SMTP_HOST/PORT/USER/PASSWORD/
|
||||||
|
TLS/TIMEOUT_SECS`, `PICLOUD_EMAIL_MAX_MESSAGE_BYTES`.
|
||||||
|
- **SDK schema** 1.7 → 1.8; **dashboard** 0.12.0 → 0.13.0.
|
||||||
|
|
||||||
|
## v1.1.6 — Realtime Channels & Client Library (unreleased)
|
||||||
|
|
||||||
|
The first **external realtime surface** and the first **frontend
|
||||||
|
library**, co-shipped per the §5/§6 design-notes decisions. Browser
|
||||||
|
clients can subscribe over SSE to per-app pub/sub topics that have been
|
||||||
|
explicitly externalized; everything else stays internal-only. The
|
||||||
|
`@picloud/client` TypeScript package wraps typed HTTP, SSE, auth, and
|
||||||
|
React/Svelte hooks. Plus three v1.1.5 follow-ups.
|
||||||
|
|
||||||
|
### Added — Realtime
|
||||||
|
|
||||||
|
- **`topics` registry** (`migrations/0021_topics.sql`) — pub/sub topics
|
||||||
|
are internal-only by default; a `topics` row with
|
||||||
|
`external_subscribable = true` opts one into external SSE subscription.
|
||||||
|
`auth_mode` is `'public'` or `'token'`.
|
||||||
|
- **Topic admin endpoints** under `/api/v1/admin/apps/{id}/topics` —
|
||||||
|
`POST` (register), `GET` (list), `PATCH /{name}` (flip
|
||||||
|
external/auth_mode — its own audited surface), `DELETE /{name}`
|
||||||
|
(unregister + disconnect live subscribers). Gated by the new
|
||||||
|
`Capability::AppTopicManage` → `app:admin` scope (no new scope; the
|
||||||
|
seven-scope commitment holds).
|
||||||
|
- **SSE endpoint `GET /realtime/topics/{topic}`** — data-plane surface
|
||||||
|
(deliberately not under `/api/`). Resolves `Host` → app, authorizes
|
||||||
|
via the `RealtimeAuthority` (404 for missing/internal topics, 401 for
|
||||||
|
bad/absent tokens), then streams `data: {topic,message,published_at}`
|
||||||
|
events with a configurable heartbeat (`PICLOUD_REALTIME_HEARTBEAT_SEC`,
|
||||||
|
default 30). Token via `Authorization: Bearer` or `?token=`.
|
||||||
|
- **`RealtimeBroadcaster` + `RealtimeEvent` + `RealtimeAuthority`**
|
||||||
|
traits (`picloud-shared`); in-process `InProcessBroadcaster`
|
||||||
|
(`tokio::sync::broadcast`, per-channel capacity
|
||||||
|
`PICLOUD_REALTIME_BROADCAST_CAPACITY` default 64, periodic empty-channel
|
||||||
|
GC) and the DB-backed `RealtimeAuthorityImpl` (orchestrator-core /
|
||||||
|
manager-core respectively). The publish path now also fans out to
|
||||||
|
in-process SSE subscribers, best-effort, after the durable outbox
|
||||||
|
fan-out commits — a broadcast failure never fails the publish.
|
||||||
|
- **`pubsub::subscriber_token(topics, ttl)`** Rhai SDK (SDK schema
|
||||||
|
1.6 → 1.7) — mints an HMAC-SHA256 subscriber token (URL-safe
|
||||||
|
`payload.signature`) scoped to externally-subscribable topics.
|
||||||
|
Requires an authenticated principal + the pub/sub publish capability.
|
||||||
|
TTL clamped to `[10s, 24h]` (default 1h), env-overridable via
|
||||||
|
`PICLOUD_SUBSCRIBER_TOKEN_TTL_{MIN,MAX,DEFAULT}_SEC`. Per-app signing
|
||||||
|
keys persist in the new `app_secrets` table
|
||||||
|
(`migrations/0022_app_secrets.sql`), created lazily on first mint. No
|
||||||
|
per-token revocation (rotation invalidates wholesale; short TTL is the
|
||||||
|
safety mechanism).
|
||||||
|
- **Dashboard Topics tab** — register/list/edit/delete topics with a
|
||||||
|
prominent external/internal badge, auth-mode radio (conditional on
|
||||||
|
external), and a confirmation when flipping a topic external.
|
||||||
|
|
||||||
|
### Added — `@picloud/client` (TypeScript, v1.0.0)
|
||||||
|
|
||||||
|
- New top-level package `clients/typescript/` (tsup dual ESM+CJS +
|
||||||
|
`.d.ts`, vitest). Typed HTTP via `endpoint<Req,Res>(path).get()/.post()`
|
||||||
|
with auth-token injection and structured errors; SSE `subscribe(topic,
|
||||||
|
cb, {token, onTokenExpired})` with exponential-backoff reconnect,
|
||||||
|
401 token-refresh, and `Last-Event-ID` resume; `auth.login/logout/token`
|
||||||
|
over dev-defined endpoints; React (`useTopic`/`useEndpoint` +
|
||||||
|
`PicloudProvider`) and Svelte (`topicStore`/`endpointStore`) subpath
|
||||||
|
exports. Optional zod/valibot runtime validation via a `{ parse }`
|
||||||
|
adapter (no hard dep). Hybrid model: no direct service access from the
|
||||||
|
browser.
|
||||||
|
|
||||||
|
### Changed / Fixed — v1.1.5 follow-ups
|
||||||
|
|
||||||
|
- **Empty blobs accepted** — `NewFile::validate` / `FileUpdate::validate`
|
||||||
|
no longer reject zero-length `data`; empty files are a valid stored
|
||||||
|
state (sentinels, placeholders). Non-breaking.
|
||||||
|
- **Orphan `*.tmp.*` sweeper** — a startup tokio task
|
||||||
|
(`spawn_files_orphan_sweep`) walks the files root every
|
||||||
|
`PICLOUD_FILES_ORPHAN_SWEEP_INTERVAL_SEC` (default 6h) and unlinks temp
|
||||||
|
blobs older than `PICLOUD_FILES_ORPHAN_TMP_TTL_SEC` (default 1h). No DB
|
||||||
|
cross-check (that full reconciler is v1.3+).
|
||||||
|
- **Dispatcher end-to-end tests** — `crates/picloud/tests/dispatcher_e2e.rs`,
|
||||||
|
one per trigger kind (kv/docs/cron/files/pubsub/dead_letter),
|
||||||
|
DATABASE_URL-gated (skip cleanly when unset).
|
||||||
|
|
||||||
|
### Notes
|
||||||
|
|
||||||
|
- New deps: `hmac` (token signing, picloud-shared), `tokio-stream` (SSE
|
||||||
|
body stream, orchestrator-core).
|
||||||
|
- New env vars: `PICLOUD_REALTIME_HEARTBEAT_SEC`,
|
||||||
|
`PICLOUD_REALTIME_BROADCAST_CAPACITY`,
|
||||||
|
`PICLOUD_SUBSCRIBER_TOKEN_TTL_{MIN,MAX,DEFAULT}_SEC`,
|
||||||
|
`PICLOUD_FILES_ORPHAN_SWEEP_INTERVAL_SEC`,
|
||||||
|
`PICLOUD_FILES_ORPHAN_TMP_TTL_SEC`.
|
||||||
|
|
||||||
## v1.1.5 — Files & Pub/Sub (unreleased)
|
## v1.1.5 — Files & Pub/Sub (unreleased)
|
||||||
|
|
||||||
Two stateful services + two trigger kinds. **`files::*`** is
|
Two stateful services + two trigger kinds. **`files::*`** is
|
||||||
|
|||||||
222
Cargo.lock
generated
222
Cargo.lock
generated
@@ -2,6 +2,41 @@
|
|||||||
# It is not intended for manual editing.
|
# It is not intended for manual editing.
|
||||||
version = 4
|
version = 4
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "aead"
|
||||||
|
version = "0.5.2"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "d122413f284cf2d62fb1b7db97e02edb8cda96d769b16e443a4f6195e35662b0"
|
||||||
|
dependencies = [
|
||||||
|
"crypto-common",
|
||||||
|
"generic-array",
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "aes"
|
||||||
|
version = "0.8.4"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "b169f7a6d4742236a0a00c541b845991d0ac43e546831af1249753ab4c3aa3a0"
|
||||||
|
dependencies = [
|
||||||
|
"cfg-if",
|
||||||
|
"cipher",
|
||||||
|
"cpufeatures",
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "aes-gcm"
|
||||||
|
version = "0.10.3"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "831010a0f742e1209b3bcea8fab6a8e149051ba6099432c8cb2cc117dec3ead1"
|
||||||
|
dependencies = [
|
||||||
|
"aead",
|
||||||
|
"aes",
|
||||||
|
"cipher",
|
||||||
|
"ctr",
|
||||||
|
"ghash",
|
||||||
|
"subtle",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "ahash"
|
name = "ahash"
|
||||||
version = "0.8.12"
|
version = "0.8.12"
|
||||||
@@ -400,6 +435,16 @@ dependencies = [
|
|||||||
"phf_codegen",
|
"phf_codegen",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "cipher"
|
||||||
|
version = "0.4.4"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "773f3b9af64447d2ce9850330c473515014aa235e6a783b02db81ff39e4a3dad"
|
||||||
|
dependencies = [
|
||||||
|
"crypto-common",
|
||||||
|
"inout",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "clap"
|
name = "clap"
|
||||||
version = "4.6.1"
|
version = "4.6.1"
|
||||||
@@ -528,7 +573,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
|
|||||||
checksum = "6f8c3e73077b4b4a6ab1ea5047c37c57aee77657bc8ecd6f29b0af082d0b0c07"
|
checksum = "6f8c3e73077b4b4a6ab1ea5047c37c57aee77657bc8ecd6f29b0af082d0b0c07"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"chrono",
|
"chrono",
|
||||||
"nom",
|
"nom 7.1.3",
|
||||||
"once_cell",
|
"once_cell",
|
||||||
]
|
]
|
||||||
|
|
||||||
@@ -560,9 +605,19 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
|
|||||||
checksum = "78c8292055d1c1df0cce5d180393dc8cce0abec0a7102adb6c7b1eef6016d60a"
|
checksum = "78c8292055d1c1df0cce5d180393dc8cce0abec0a7102adb6c7b1eef6016d60a"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"generic-array",
|
"generic-array",
|
||||||
|
"rand_core 0.6.4",
|
||||||
"typenum",
|
"typenum",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "ctr"
|
||||||
|
version = "0.9.2"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "0369ee1ad671834580515889b80f2ea915f23b8be8d0daa4bbaf2ac5c7590835"
|
||||||
|
dependencies = [
|
||||||
|
"cipher",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "data-encoding"
|
name = "data-encoding"
|
||||||
version = "2.11.0"
|
version = "2.11.0"
|
||||||
@@ -660,6 +715,22 @@ dependencies = [
|
|||||||
"serde",
|
"serde",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "email-encoding"
|
||||||
|
version = "0.4.1"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "9298e6504d9b9e780ed3f7dfd43a61be8cd0e09eb07f7706a945b0072b6670b6"
|
||||||
|
dependencies = [
|
||||||
|
"base64",
|
||||||
|
"memchr",
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "email_address"
|
||||||
|
version = "0.2.9"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "e079f19b08ca6239f47f8ba8509c11cf3ea30095831f7fed61441475edd8c449"
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "equivalent"
|
name = "equivalent"
|
||||||
version = "1.0.2"
|
version = "1.0.2"
|
||||||
@@ -880,6 +951,16 @@ dependencies = [
|
|||||||
"wasip3",
|
"wasip3",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "ghash"
|
||||||
|
version = "0.5.1"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "f0d8a4362ccb29cb0b265253fb0a2728f592895ee6854fd9bc13f2ffda266ff1"
|
||||||
|
dependencies = [
|
||||||
|
"opaque-debug",
|
||||||
|
"polyval",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "hashbrown"
|
name = "hashbrown"
|
||||||
version = "0.15.5"
|
version = "0.15.5"
|
||||||
@@ -945,6 +1026,17 @@ dependencies = [
|
|||||||
"windows-sys 0.61.2",
|
"windows-sys 0.61.2",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "hostname"
|
||||||
|
version = "0.4.2"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "617aaa3557aef3810a6369d0a99fac8a080891b68bd9f9812a1eeda0c0730cbd"
|
||||||
|
dependencies = [
|
||||||
|
"cfg-if",
|
||||||
|
"libc",
|
||||||
|
"windows-link",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "http"
|
name = "http"
|
||||||
version = "1.4.0"
|
version = "1.4.0"
|
||||||
@@ -1201,6 +1293,15 @@ version = "0.1.15"
|
|||||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
checksum = "c8fae54786f62fb2918dcfae3d568594e50eb9b5c25bf04371af6fe7516452fb"
|
checksum = "c8fae54786f62fb2918dcfae3d568594e50eb9b5c25bf04371af6fe7516452fb"
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "inout"
|
||||||
|
version = "0.1.4"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "879f10e63c20629ecabbb64a8010319738c66a5cd0c29b02d63d272b03751d01"
|
||||||
|
dependencies = [
|
||||||
|
"generic-array",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "ipnet"
|
name = "ipnet"
|
||||||
version = "2.12.0"
|
version = "2.12.0"
|
||||||
@@ -1246,6 +1347,34 @@ version = "0.1.0"
|
|||||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
checksum = "09edd9e8b54e49e587e4f6295a7d29c3ea94d469cb40ab8ca70b288248a81db2"
|
checksum = "09edd9e8b54e49e587e4f6295a7d29c3ea94d469cb40ab8ca70b288248a81db2"
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "lettre"
|
||||||
|
version = "0.11.22"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "0da65617f6cb926332d039cb578aad56178da86e128db6a1b09f4c94fa5b3349"
|
||||||
|
dependencies = [
|
||||||
|
"async-trait",
|
||||||
|
"base64",
|
||||||
|
"email-encoding",
|
||||||
|
"email_address",
|
||||||
|
"fastrand",
|
||||||
|
"futures-io",
|
||||||
|
"futures-util",
|
||||||
|
"hostname",
|
||||||
|
"httpdate",
|
||||||
|
"idna",
|
||||||
|
"mime",
|
||||||
|
"nom 8.0.0",
|
||||||
|
"percent-encoding",
|
||||||
|
"quoted_printable",
|
||||||
|
"rustls",
|
||||||
|
"socket2",
|
||||||
|
"tokio",
|
||||||
|
"tokio-rustls",
|
||||||
|
"url",
|
||||||
|
"webpki-roots 1.0.7",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "libc"
|
name = "libc"
|
||||||
version = "0.2.186"
|
version = "0.2.186"
|
||||||
@@ -1395,6 +1524,15 @@ dependencies = [
|
|||||||
"minimal-lexical",
|
"minimal-lexical",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "nom"
|
||||||
|
version = "8.0.0"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "df9761775871bdef83bee530e60050f7e54b1105350d6884eb0fb4f46c2f9405"
|
||||||
|
dependencies = [
|
||||||
|
"memchr",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "normalize-line-endings"
|
name = "normalize-line-endings"
|
||||||
version = "0.3.0"
|
version = "0.3.0"
|
||||||
@@ -1477,6 +1615,12 @@ version = "1.70.2"
|
|||||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
checksum = "384b8ab6d37215f3c5301a95a4accb5d64aa607f1fcb26a11b5303878451b4fe"
|
checksum = "384b8ab6d37215f3c5301a95a4accb5d64aa607f1fcb26a11b5303878451b4fe"
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "opaque-debug"
|
||||||
|
version = "0.3.1"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "c08d65885ee38876c4f86fa503fb49d7b507c2b62552df7c70b2fce627e06381"
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "option-ext"
|
name = "option-ext"
|
||||||
version = "0.2.0"
|
version = "0.2.0"
|
||||||
@@ -1610,7 +1754,7 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "picloud"
|
name = "picloud"
|
||||||
version = "1.1.5"
|
version = "1.1.7"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"anyhow",
|
"anyhow",
|
||||||
"async-trait",
|
"async-trait",
|
||||||
@@ -1618,12 +1762,15 @@ dependencies = [
|
|||||||
"axum-test",
|
"axum-test",
|
||||||
"chrono",
|
"chrono",
|
||||||
"figment",
|
"figment",
|
||||||
|
"hex",
|
||||||
|
"hmac",
|
||||||
"picloud-executor-core",
|
"picloud-executor-core",
|
||||||
"picloud-manager-core",
|
"picloud-manager-core",
|
||||||
"picloud-orchestrator-core",
|
"picloud-orchestrator-core",
|
||||||
"picloud-shared",
|
"picloud-shared",
|
||||||
"serde",
|
"serde",
|
||||||
"serde_json",
|
"serde_json",
|
||||||
|
"sha2",
|
||||||
"sqlx",
|
"sqlx",
|
||||||
"thiserror 1.0.69",
|
"thiserror 1.0.69",
|
||||||
"tokio",
|
"tokio",
|
||||||
@@ -1636,7 +1783,7 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "picloud-cli"
|
name = "picloud-cli"
|
||||||
version = "1.1.5"
|
version = "1.1.7"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"anyhow",
|
"anyhow",
|
||||||
"assert_cmd",
|
"assert_cmd",
|
||||||
@@ -1657,7 +1804,7 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "picloud-executor"
|
name = "picloud-executor"
|
||||||
version = "1.1.5"
|
version = "1.1.7"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"anyhow",
|
"anyhow",
|
||||||
"picloud-executor-core",
|
"picloud-executor-core",
|
||||||
@@ -1669,7 +1816,7 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "picloud-executor-core"
|
name = "picloud-executor-core"
|
||||||
version = "1.1.5"
|
version = "1.1.7"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"async-trait",
|
"async-trait",
|
||||||
"base64",
|
"base64",
|
||||||
@@ -1693,7 +1840,7 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "picloud-manager"
|
name = "picloud-manager"
|
||||||
version = "1.1.5"
|
version = "1.1.7"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"anyhow",
|
"anyhow",
|
||||||
"picloud-manager-core",
|
"picloud-manager-core",
|
||||||
@@ -1705,7 +1852,7 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "picloud-manager-core"
|
name = "picloud-manager-core"
|
||||||
version = "1.1.5"
|
version = "1.1.7"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"argon2",
|
"argon2",
|
||||||
"async-trait",
|
"async-trait",
|
||||||
@@ -1715,6 +1862,9 @@ dependencies = [
|
|||||||
"chrono-tz",
|
"chrono-tz",
|
||||||
"cron",
|
"cron",
|
||||||
"data-encoding",
|
"data-encoding",
|
||||||
|
"hex",
|
||||||
|
"hmac",
|
||||||
|
"lettre",
|
||||||
"picloud-executor-core",
|
"picloud-executor-core",
|
||||||
"picloud-orchestrator-core",
|
"picloud-orchestrator-core",
|
||||||
"picloud-shared",
|
"picloud-shared",
|
||||||
@@ -1733,7 +1883,7 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "picloud-orchestrator"
|
name = "picloud-orchestrator"
|
||||||
version = "1.1.5"
|
version = "1.1.7"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"anyhow",
|
"anyhow",
|
||||||
"picloud-orchestrator-core",
|
"picloud-orchestrator-core",
|
||||||
@@ -1745,7 +1895,7 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "picloud-orchestrator-core"
|
name = "picloud-orchestrator-core"
|
||||||
version = "1.1.5"
|
version = "1.1.7"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"async-trait",
|
"async-trait",
|
||||||
"axum",
|
"axum",
|
||||||
@@ -1759,6 +1909,8 @@ dependencies = [
|
|||||||
"serde_json",
|
"serde_json",
|
||||||
"thiserror 1.0.69",
|
"thiserror 1.0.69",
|
||||||
"tokio",
|
"tokio",
|
||||||
|
"tokio-stream",
|
||||||
|
"tower",
|
||||||
"tracing",
|
"tracing",
|
||||||
"urlencoding",
|
"urlencoding",
|
||||||
"uuid",
|
"uuid",
|
||||||
@@ -1766,13 +1918,20 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "picloud-shared"
|
name = "picloud-shared"
|
||||||
version = "1.1.5"
|
version = "1.1.7"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
|
"aes-gcm",
|
||||||
"async-trait",
|
"async-trait",
|
||||||
|
"base64",
|
||||||
"chrono",
|
"chrono",
|
||||||
|
"hmac",
|
||||||
|
"rand 0.8.6",
|
||||||
"serde",
|
"serde",
|
||||||
"serde_json",
|
"serde_json",
|
||||||
|
"sha2",
|
||||||
"thiserror 1.0.69",
|
"thiserror 1.0.69",
|
||||||
|
"tokio",
|
||||||
|
"tracing",
|
||||||
"uuid",
|
"uuid",
|
||||||
]
|
]
|
||||||
|
|
||||||
@@ -1815,6 +1974,18 @@ version = "0.2.3"
|
|||||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
checksum = "b4596b6d070b27117e987119b4dac604f3c58cfb0b191112e24771b2faeac1a6"
|
checksum = "b4596b6d070b27117e987119b4dac604f3c58cfb0b191112e24771b2faeac1a6"
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "polyval"
|
||||||
|
version = "0.6.2"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "9d1fe60d06143b2430aa532c94cfe9e29783047f06c0d7fd359a9a51b729fa25"
|
||||||
|
dependencies = [
|
||||||
|
"cfg-if",
|
||||||
|
"cpufeatures",
|
||||||
|
"opaque-debug",
|
||||||
|
"universal-hash",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "portable-atomic"
|
name = "portable-atomic"
|
||||||
version = "1.13.1"
|
version = "1.13.1"
|
||||||
@@ -1981,6 +2152,12 @@ dependencies = [
|
|||||||
"proc-macro2",
|
"proc-macro2",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "quoted_printable"
|
||||||
|
version = "0.5.2"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "478e0585659a122aa407eb7e3c0e1fa51b1d8a870038bd29f0cf4a8551eea972"
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "r-efi"
|
name = "r-efi"
|
||||||
version = "5.3.0"
|
version = "5.3.0"
|
||||||
@@ -2284,6 +2461,7 @@ version = "0.23.40"
|
|||||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
checksum = "ef86cd5876211988985292b91c96a8f2d298df24e75989a43a3c73f2d4d8168b"
|
checksum = "ef86cd5876211988985292b91c96a8f2d298df24e75989a43a3c73f2d4d8168b"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
|
"log",
|
||||||
"once_cell",
|
"once_cell",
|
||||||
"ring",
|
"ring",
|
||||||
"rustls-pki-types",
|
"rustls-pki-types",
|
||||||
@@ -2990,6 +3168,20 @@ dependencies = [
|
|||||||
"futures-core",
|
"futures-core",
|
||||||
"pin-project-lite",
|
"pin-project-lite",
|
||||||
"tokio",
|
"tokio",
|
||||||
|
"tokio-util",
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "tokio-util"
|
||||||
|
version = "0.7.18"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "9ae9cec805b01e8fc3fd2fe289f89149a9b66dd16786abd8b19cfa7b48cb0098"
|
||||||
|
dependencies = [
|
||||||
|
"bytes",
|
||||||
|
"futures-core",
|
||||||
|
"futures-sink",
|
||||||
|
"pin-project-lite",
|
||||||
|
"tokio",
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
@@ -3209,6 +3401,16 @@ version = "0.2.6"
|
|||||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
checksum = "ebc1c04c71510c7f702b52b7c350734c9ff1295c464a03335b00bb84fc54f853"
|
checksum = "ebc1c04c71510c7f702b52b7c350734c9ff1295c464a03335b00bb84fc54f853"
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "universal-hash"
|
||||||
|
version = "0.5.1"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "fc1de2c688dc15305988b563c3854064043356019f97a4b46276fe734c4f07ea"
|
||||||
|
dependencies = [
|
||||||
|
"crypto-common",
|
||||||
|
"subtle",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "untrusted"
|
name = "untrusted"
|
||||||
version = "0.9.0"
|
version = "0.9.0"
|
||||||
|
|||||||
13
Cargo.toml
13
Cargo.toml
@@ -13,7 +13,7 @@ members = [
|
|||||||
]
|
]
|
||||||
|
|
||||||
[workspace.package]
|
[workspace.package]
|
||||||
version = "1.1.5"
|
version = "1.1.7"
|
||||||
edition = "2021"
|
edition = "2021"
|
||||||
rust-version = "1.92"
|
rust-version = "1.92"
|
||||||
license = "MIT OR Apache-2.0"
|
license = "MIT OR Apache-2.0"
|
||||||
@@ -29,6 +29,8 @@ picloud-manager-core = { path = "crates/manager-core" }
|
|||||||
|
|
||||||
# Async + HTTP
|
# Async + HTTP
|
||||||
tokio = { version = "1.40", features = ["full"] }
|
tokio = { version = "1.40", features = ["full"] }
|
||||||
|
# Wraps a broadcast::Receiver into a Stream for the SSE endpoint (v1.1.6).
|
||||||
|
tokio-stream = { version = "0.1", features = ["sync"] }
|
||||||
axum = "0.8"
|
axum = "0.8"
|
||||||
tower = "0.5"
|
tower = "0.5"
|
||||||
tower-http = { version = "0.6", features = ["trace", "cors"] }
|
tower-http = { version = "0.6", features = ["trace", "cors"] }
|
||||||
@@ -75,8 +77,17 @@ urlencoding = "2"
|
|||||||
argon2 = "0.5"
|
argon2 = "0.5"
|
||||||
rand = { version = "0.8", features = ["getrandom"] }
|
rand = { version = "0.8", features = ["getrandom"] }
|
||||||
sha2 = "0.10"
|
sha2 = "0.10"
|
||||||
|
# HMAC-SHA256 for realtime subscriber tokens (v1.1.6).
|
||||||
|
hmac = "0.12"
|
||||||
base64 = "0.22"
|
base64 = "0.22"
|
||||||
data-encoding = "2.6"
|
data-encoding = "2.6"
|
||||||
|
# AES-256-GCM at-rest encryption for per-app secrets + the realtime
|
||||||
|
# signing key (v1.1.7). Audited, pure-Rust RustCrypto AEAD.
|
||||||
|
aes-gcm = { version = "0.10", features = ["aes", "alloc"] }
|
||||||
|
|
||||||
|
# Outbound SMTP email (v1.1.7). Async transport over the Tokio runtime
|
||||||
|
# with rustls TLS; built messages for text + multipart-alternative.
|
||||||
|
lettre = { version = "0.11", default-features = false, features = ["smtp-transport", "tokio1-rustls-tls", "builder", "hostname"] }
|
||||||
|
|
||||||
# Stdlib utility crates (v1.1.0 stdlib PR — registered into the
|
# Stdlib utility crates (v1.1.0 stdlib PR — registered into the
|
||||||
# Rhai engine as the regex::/random::/etc. namespaces)
|
# Rhai engine as the regex::/random::/etc. namespaces)
|
||||||
|
|||||||
423
HANDBACK.md
423
HANDBACK.md
@@ -1,127 +1,330 @@
|
|||||||
# HANDBACK — v1.1.5 Files & Pub/Sub
|
# v1.1.7 — Configuration & Email — HANDBACK
|
||||||
|
|
||||||
## §1 Branch + commits
|
**Branch:** `feat/v1.1.7-secrets-email` (9 commits off `main`, not pushed)
|
||||||
|
**Status:** ready for review. NOT merged, NOT pushed, no PR opened.
|
||||||
- **Branch:** `feat/v1.1.5-files-pubsub` (off `main`). Not pushed, not merged, no PR.
|
|
||||||
- **Commits:** the two-feature split decided in planning + a finalize commit; HANDBACK is the 4th (docs):
|
|
||||||
1. `6e132b6 feat(v1.1.5): files SDK + files:* triggers`
|
|
||||||
2. `834c787 feat(v1.1.5): pubsub::publish_durable SDK + pubsub:* triggers`
|
|
||||||
3. `4595db7 chore(v1.1.5): version bumps, CI workflow, schema-snapshot un-ignore`
|
|
||||||
4. `docs(v1.1.5): handback report` (this file)
|
|
||||||
|
|
||||||
Each of commits 1–3 is independently green (fmt + clippy + `cargo test --workspace`). Shared files (Cargo deps, `Services` bundle, `version.rs`, dispatcher arm, authz enum, CHANGELOG) are touched in both feature commits as planned — additive only, so commit 1 compiles green with the `AppPubsubPublish` capability and the dashboard `'pubsub'` type union present-but-unused until commit 2.
|
|
||||||
|
|
||||||
## §2 Scope coverage
|
|
||||||
|
|
||||||
| Brief item | Status | Notes |
|
|
||||||
|---|---|---|
|
|
||||||
| §1 `files::*` SDK | ✅ | `create/head/get/update/delete/list`, blob in/out, metadata maps, throw-vs-`()` convention. |
|
|
||||||
| §1 migration 0018_files.sql | ✅ | metadata table + `idx_files_app_collection`. Bytes on disk, never in PG. |
|
|
||||||
| §1 atomic writes/deletes, checksum, size+name+type caps, authz, events | ✅ | See §3. |
|
|
||||||
| §2 `files:*` trigger (Layout-E, 0019) | ✅ | widen 2 CHECKs + `files_trigger_details`; `TriggerEvent::Files` (metadata only); admin `POST /triggers/files`; `emit_files`; dispatcher arm. |
|
|
||||||
| §3 `pubsub::publish_durable` SDK | ✅ | publish-time transactional fan-out; topic matching in Rust; succeed-silently on no match. |
|
|
||||||
| §4 `pubsub:*` trigger (Layout-E, 0020) | ✅ | widen 2 CHECKs + `pubsub_trigger_details` + partial index; `TriggerEvent::Pubsub`; admin `POST /triggers/pubsub`; dispatcher arm. |
|
|
||||||
| §5 dashboard Files view | ✅ | `apps/[slug]/files/+page.svelte` (list per collection, per-row delete w/ confirm). Backed by a new admin files API (§7.2). |
|
|
||||||
| §5 dashboard Pub/Sub trigger form | ✅ | added to the Triggers tab beside Cron; trigger-list renders files + pubsub. `npm run check` clean. |
|
|
||||||
| §6 schema_snapshot CI follow-up | ✅ | §6b skip-when-absent + un-ignore; §6a new `.github/workflows/ci.yml`. See §5. |
|
|
||||||
| §7 version bumps | ✅ | workspace 1.1.4→1.1.5, SDK 1.5→1.6, dashboard 0.10.0→0.11.0, CHANGELOG, CLAUDE.md env table. |
|
|
||||||
| §8 tests | ⚠️ | 63 new tests (target 70–90). Every *named* critical test covered; gap is the dispatcher end-to-end DB test (see §9.2). |
|
|
||||||
|
|
||||||
## §3 Files implementation notes
|
|
||||||
|
|
||||||
**Service layering** (`FilesServiceImpl`, manager-core): validate collection (empty + traversal) → script-as-gate authz (`AppFilesRead`/`AppFilesWrite`, skipped when `cx.principal` is `None`) → field/size-cap validation → repo call keyed by `cx.app_id` → best-effort `ServiceEvent` emit. `executor-core` has **no** Postgres or filesystem dependency — both traits live in `picloud-shared`, the impl in manager-core.
|
|
||||||
|
|
||||||
**Atomic-write protocol** (`write_atomic_at`, a free fn so it's unit-testable without a pool):
|
|
||||||
1. Validate collection path-safety (defensive — already enforced at the SDK boundary).
|
|
||||||
2. `create_dir_all` the shard dir `<root>/files/<app_id>/<collection>/<id[0:2]>/<id>` with `0o700` (Unix `DirBuilderExt::mode`).
|
|
||||||
3. SHA-256 the in-memory bytes (single pass — never re-reads the file) while writing to `<final>.tmp.<pid>-<atomic-counter>`.
|
|
||||||
4. `sync_all()` the temp file.
|
|
||||||
5. `rename(tmp, final)` — atomic on POSIX.
|
|
||||||
6. `sync_all()` the parent dir (rename durability).
|
|
||||||
7. INSERT/UPDATE the DB row.
|
|
||||||
|
|
||||||
Rollback per step: crash in 1–5 → orphan `*.tmp.*` (never read; the pid+counter suffix avoids collisions); crash in 5–7 → bytes with no row, **never reachable via the SDK** because every read starts from the row. `update` reads the prior row first (existence + CDC `prev`), writes new bytes, then UPDATEs.
|
|
||||||
|
|
||||||
**Atomic-delete protocol** (`FsFilesRepo::delete`): `SELECT … FOR UPDATE` + `DELETE` in one transaction → commit → `unlink` outside the tx. Unlink failure leaves an orphan (logged at warn); failure before commit changes nothing. Returns the deleted metadata so the service can emit.
|
|
||||||
|
|
||||||
**Path-traversal validation:** `picloud_shared::validate_files_collection` rejects empty / `/` / `\` / `..` / NUL at the SDK boundary; `FsFilesRepo::guard_collection` repeats it before any fs op. UUID ids can't produce traversal (verified defensively).
|
|
||||||
|
|
||||||
**Per-call SHA-256:** computed once over the in-memory `Vec<u8>` during the write (`sha2::Sha256`), hex-lowercased, stored on the row. The file is never re-read to hash. Known-vector tests pin `SHA-256("abc")` and `SHA-256("")`.
|
|
||||||
|
|
||||||
**Checksum-on-get:** `get` reads the file, re-hashes, compares to the stored checksum. Mismatch (or missing bytes while the row persists) → `FilesError::Corrupted`, logged at error level with the path, **no auto-delete**. To scripts this surfaces as a thrown Rhai error `"files: file content corrupted (checksum mismatch)"`.
|
|
||||||
|
|
||||||
## §4 Pub/Sub implementation notes
|
|
||||||
|
|
||||||
**Fan-out-at-publish-time, transactional** (`PostgresPubsubRepo::fan_out_publish`): one transaction — `SELECT` all enabled pubsub triggers for the app (joined to `pubsub_trigger_details`), filter by `topic_matches` in Rust, INSERT one `outbox` row (`source_kind='pubsub'`) per survivor, commit once. A mid-fan-out failure rolls back every row (no half-fan-out). Each delivery row then retries/dead-letters independently through the unchanged dispatcher (its trigger arm just gained `| OutboxSourceKind::Pubsub`).
|
|
||||||
|
|
||||||
**Topic pattern matching** runs in Rust (`picloud_shared::topic_matches`), not SQL: `"*"` → all; `"<prefix>.*"` → `starts_with("<prefix>.")`; otherwise exact. `validate_topic_pattern` (used at trigger creation in the admin endpoint and defensively in the repo) accepts only `*` / `<prefix>.*` / no-star-exact, rejecting `*.created`, `**`, `a.*.b`, `user.*x`, etc. with `"unsupported pubsub topic pattern: …"`.
|
|
||||||
|
|
||||||
**No matching trigger → the publish succeeds, zero outbox rows** (the design-notes-preferred succeed-silently). `published_at` is stamped manager-side (`Utc::now()`) so every delivery agrees on one instant. `ctx.event.pubsub = #{ topic, message, published_at }`, `ctx.event.op = "publish"`.
|
|
||||||
|
|
||||||
There is **no `list_matching_pubsub` on `TriggerRepo`** — pubsub publishes directly (it's not a `ServiceEvent`), so the fan-out SELECT lives in `pubsub_repo`, not the `OutboxEventEmitter`. This is the one structural asymmetry vs files/kv/docs, intentional per the publish-time-fan-out decision.
|
|
||||||
|
|
||||||
## §5 CI follow-up (§6) status
|
|
||||||
|
|
||||||
- **Pre-existing CI:** none (no `.github/`, no `.gitlab-ci.yml`).
|
|
||||||
- **§6a (added):** `.github/workflows/ci.yml` — a `rust` job with a `postgres:15` service (`DATABASE_URL=postgres://picloud:picloud@localhost:5432/picloud`) running `cargo fmt --all -- --check`, `cargo clippy --all-targets --all-features -- -D warnings`, `cargo test --workspace`; a separate `dashboard` job running `npm ci` + `npm run check`.
|
|
||||||
- **§6b (done):** `schema_snapshot.rs` is no longer `#[ignore]`'d. Reworked from `#[sqlx::test]` to `#[tokio::test]` that **skips cleanly when `DATABASE_URL` is unset** (chosen over fail-loud so `cargo test --workspace` stays green locally) and otherwise connects, runs `sqlx::migrate!`, and dumps. Golden `expected_schema.txt` re-blessed (now contains `files`, `files_trigger_details`, `pubsub_trigger_details`, both widened CHECKs, `idx_files_app_collection`, `idx_triggers_app_pubsub_enabled`, and migrations 0018–0020).
|
|
||||||
- **Tradeoff (documented):** the non-`sqlx::test` path applies migrations against the `DATABASE_URL` database directly rather than an isolated throwaway DB. Migrations are forward-only/idempotent and CI's Postgres is fresh, so the structural dump is identical; locally it will also apply 0018–0020 to whatever DB you point at.
|
|
||||||
|
|
||||||
## §6 Schema decisions beyond the brief
|
|
||||||
|
|
||||||
- `files` table is verbatim from the brief. `files_trigger_details` / `pubsub_trigger_details` mirror `kv_trigger_details` / `cron_trigger_details`.
|
|
||||||
- `pubsub_trigger_details` has no `ops` column (a publish has a single implicit op) — only `topic_pattern`.
|
|
||||||
- `idx_triggers_app_pubsub_enabled` is the third partial index of its kind (per the brief's note); deliberate duplication.
|
|
||||||
|
|
||||||
## §7 Decisions beyond the brief (every prompt-default deviation)
|
|
||||||
|
|
||||||
1. **Empty blob treated as a missing `data` field.** `NewFile::validate` / `FileUpdate::validate` reject 0-byte `data` with `FilesError::MissingField("data")`. The brief lists `data` as required and tests "missing … data"; the cleanest testable interpretation at the service layer is "empty == missing". Consequence: v1.1.5 cannot store an intentionally-empty file. Easy to relax later.
|
|
||||||
2. **Admin files REST API added** (`files_api.rs`: `GET /apps/{id}/files?collection=…`, `DELETE /apps/{id}/files/{collection}/{file_id}`). The brief's §5 dashboard needs a backend but didn't spell out admin endpoints; I added a minimal one mirroring `triggers_api`'s direct-repo + capability pattern (`AppFilesRead` for list, `AppFilesWrite` for delete).
|
|
||||||
3. **Admin file delete does NOT emit a `files:delete` trigger event.** It's an operator cleanup action, not a script mutation, so it goes straight to the repo. SDK deletes still emit. Flagging because "every successful mutation emits" could be read to include admin deletes.
|
|
||||||
4. **Files `list` bridge accepts both positional and map forms** — `list()`, `list(cursor)`, `list(cursor, limit)`, and `list(#{ cursor, limit })` (the map form the brief's example used). Additive convenience.
|
|
||||||
5. **Files collection-glob semantics reuse the existing `collection_matches`** (`*` / `foo*` prefix / exact), identical to kv/docs. The brief mentioned a `"prefix:*"` form in one spot; I kept parity with the established kv/docs matcher rather than introduce a new glob dialect.
|
|
||||||
6. **schema_snapshot runs against the live `DATABASE_URL` DB** rather than an isolated temp DB (see §5).
|
|
||||||
7. **Orphan sweep deferred to v1.1.6+** — confirmed with the user during planning (the brief's recommended default). No `*.tmp.*` sweeper daemon shipped.
|
|
||||||
|
|
||||||
## §8 How to verify locally — attestation (fresh run on HEAD `4595db7`)
|
|
||||||
|
|
||||||
```
|
```
|
||||||
cargo fmt --all -- --check → exit 0
|
a7d3dad chore(v1.1.7): re-bless schema snapshot for secrets + email migrations
|
||||||
cargo clippy --all-targets --all-features -- -D warnings → exit 0
|
2ea47eb chore(v1.1.7): fix clippy --all-targets warnings
|
||||||
cargo test --workspace → 491 passed, 0 failed (exit 0)
|
b355851 chore(v1.1.7): version bumps + CHANGELOG
|
||||||
(schema_snapshot skips cleanly with no DATABASE_URL)
|
fffcdf6 feat(v1.1.7-realtime-migration): encrypt signing keys at rest
|
||||||
cd dashboard && npm run check → 0 errors, 0 warnings (exit 0)
|
02335a8 fix(v1.1.7-dead-letter): wire dispatcher → list_matching_dead_letter
|
||||||
|
1f78937 feat(v1.1.7-email-inbound): webhook receiver + email:receive trigger
|
||||||
|
8f2d2bc feat(v1.1.7-email-outbound): SMTP send/send_html
|
||||||
|
2d11090 feat(v1.1.7-secrets): secrets SDK + table + admin API + dashboard
|
||||||
|
dc2e4fa feat(v1.1.7-crypto): master-key infra + encryption helpers
|
||||||
```
|
```
|
||||||
|
|
||||||
With a live Postgres (the schema guardrail actually verifies the schema):
|
---
|
||||||
|
|
||||||
|
## 1. Scope coverage
|
||||||
|
|
||||||
|
| Item | Status |
|
||||||
|
|---|---|
|
||||||
|
| Encryption infrastructure (master key + AES-256-GCM envelope) | **Done** |
|
||||||
|
| `secrets::*` SDK + `0023_secrets.sql` + admin API + dashboard tab | **Done** |
|
||||||
|
| Outbound email `email::send` / `email::send_html` (lettre SMTP) | **Done** |
|
||||||
|
| Inbound email webhook receiver + `email:receive` trigger + `0024` | **Done** (full scope, per user decision) |
|
||||||
|
| Dispatcher routing for email | **Done** |
|
||||||
|
| dead_letter handler wiring fix | **Done** |
|
||||||
|
| Realtime signing-key encryption (two-phase) + `0025` | **Done** |
|
||||||
|
| Dashboard (Secrets tab, email trigger form, `npm run check`) | **Done** |
|
||||||
|
| Version bumps (1.1.7 / SDK 1.8 / dashboard 0.13.0) + CHANGELOG | **Done** |
|
||||||
|
| Tests (match v1.1.5/v1.1.6 density) | **Done** |
|
||||||
|
|
||||||
|
Nothing deferred from scope-in. Inbound email (the deferrable-if-scope-
|
||||||
|
blew-up piece) was implemented in full.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Encryption infrastructure notes
|
||||||
|
|
||||||
|
- **Module:** `crates/shared/src/crypto.rs` (`picloud_shared::crypto`).
|
||||||
|
- **Master-key sourcing** (`MasterKey::from_env` → `resolve`):
|
||||||
|
- `PICLOUD_SECRET_KEY` = base64 of exactly 32 bytes. Missing →
|
||||||
|
`MasterKeyError::Missing` (fatal); non-base64 → `Malformed`; wrong
|
||||||
|
length → `WrongLength`. **Sourced in `main.rs::run_server` before any
|
||||||
|
DB work** — `build_app` takes the `MasterKey` as a parameter (so
|
||||||
|
tests pass a fixed key and don't mutate process env).
|
||||||
|
- Dev fallback: deterministic key (`SHA-256("picloud-dev-master-key-v1.1.7")`)
|
||||||
|
used ONLY when `PICLOUD_SECRET_KEY` is unset **AND**
|
||||||
|
`PICLOUD_DEV_MODE=true`, with a prominent `warn!`. No quiet
|
||||||
|
unencrypted mode.
|
||||||
|
- **aes-gcm version:** `0.10` (features `aes`, `alloc`). `Aes256Gcm`.
|
||||||
|
- **Nonce generation:** 12 bytes from `rand::thread_rng().fill_bytes`
|
||||||
|
(OS-CSPRNG-seeded), per-encryption.
|
||||||
|
- **Storage layout:** ciphertext **with the 16-byte GCM auth tag
|
||||||
|
appended** (RustCrypto `Aead`-trait layout — `encrypt` returns
|
||||||
|
`ciphertext || tag`, `decrypt` consumes the same). The 12-byte nonce is
|
||||||
|
stored in a separate column. `MasterKey`'s `Debug` is redacted.
|
||||||
|
- **Plaintext cap (secrets):** 64 KB default, enforced in
|
||||||
|
`secrets_service::seal` (the SDK boundary) → `SecretsError::TooLarge`
|
||||||
|
with limit + actual size. Override: `PICLOUD_SECRET_MAX_VALUE_BYTES`.
|
||||||
|
- **Key rotation:** out of scope. Documented in CHANGELOG + the module
|
||||||
|
docs that changing `PICLOUD_SECRET_KEY` orphans all ciphertext.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Secrets notes
|
||||||
|
|
||||||
|
- `SecretsService` (trait, `picloud-shared`) → `SecretsServiceImpl` +
|
||||||
|
`PostgresSecretsRepo` (`manager-core`) → Rhai bridge
|
||||||
|
(`executor-core/src/sdk/secrets.rs`). Collection-less; `app_id` from
|
||||||
|
`cx.app_id`.
|
||||||
|
- **JSON round-trip:** `set` serializes the value to JSON bytes, caps,
|
||||||
|
encrypts; `get` decrypts + deserializes — a String returns a String
|
||||||
|
(not a JSON-quoted `"\"…\""`). Verified by unit + bridge tests.
|
||||||
|
- **No ServiceEvent emission** (secret writes don't fire triggers).
|
||||||
|
- Admin API: `GET/POST/DELETE /api/v1/admin/apps/{id}/secrets`; list
|
||||||
|
returns names + `updated_at` only.
|
||||||
|
- Authz: `Capability::AppSecretsRead/Write` → `script:read`/`script:write`.
|
||||||
|
No new Scope variants (seven-scope commitment held).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Email implementation notes
|
||||||
|
|
||||||
|
- **SMTP transport:** `lettre 0.11` (`smtp-transport`,
|
||||||
|
`tokio1-rustls-tls`, `builder`, `hostname`). **Connection model:** one
|
||||||
|
connection per call (lettre default); pooling deferred to v1.2. The
|
||||||
|
transport sits behind an internal `EmailTransport` trait so the service
|
||||||
|
is unit-tested with a recording fake (no live SMTP).
|
||||||
|
- **Disabled mode:** if HOST/USER/PASSWORD aren't all set,
|
||||||
|
`EmailServiceImpl::from_env` builds no transport and every `send`
|
||||||
|
returns `NotConfigured` (warned at startup). A malformed relay
|
||||||
|
descriptor is also logged and yields disabled mode (email is
|
||||||
|
non-critical; never blocks startup).
|
||||||
|
- **Address validation:** hand-rolled RFC 5322-ish pre-check (single `@`,
|
||||||
|
non-empty local part, domain contains a dot, ≤320 bytes) followed by a
|
||||||
|
`lettre::Mailbox` parse (the authoritative validator). No deliverability
|
||||||
|
check.
|
||||||
|
- **Size cap:** 25 MB on `message.formatted()`,
|
||||||
|
`PICLOUD_EMAIL_MAX_MESSAGE_BYTES`.
|
||||||
|
- `email::send` forces text-only (ignores any `html`); `email::send_html`
|
||||||
|
requires `html` and builds `MultiPart::alternative_plain_html`.
|
||||||
|
`reply_to` defaults to `from`. `to`/`cc`/`bcc` accept a String or an
|
||||||
|
Array of Strings.
|
||||||
|
- **Inbound normalization:** only the generic provider-agnostic JSON
|
||||||
|
shape `{from,to[],cc[],subject,text,html,message_id}` is accepted in
|
||||||
|
v1.1.7 — `from` required, rest default. Provider-specific unmarshallers
|
||||||
|
→ v1.2. The expected shape is documented on the dashboard email-trigger
|
||||||
|
form.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Dead-letter handler fix notes
|
||||||
|
|
||||||
|
- **Call site:** `dispatcher::handle_failure`, the retry-exhaustion
|
||||||
|
branch. After `DeadLetterRepo::insert` (which returns the new
|
||||||
|
`DeadLetterId`), a new helper `fan_out_dead_letter` runs.
|
||||||
|
- **What it does:** calls `TriggerRepo::list_matching_dead_letter(app_id,
|
||||||
|
source, row.trigger_id, Some(resolved.script_id))` (the method that had
|
||||||
|
no production caller) and inserts one outbox row per match
|
||||||
|
(`source_kind = DeadLetter`, the DL trigger's id + handler script id,
|
||||||
|
`trigger_depth + 1`, `origin_principal = the DL trigger's registered
|
||||||
|
principal`).
|
||||||
|
- **Payload — built from the REAL `TriggerEvent::DeadLetter` variant**,
|
||||||
|
not the brief's §6 field list (see §7 deviations): `{ dead_letter_id,
|
||||||
|
original: Box::new(decoded row payload), attempts, last_error,
|
||||||
|
trigger_id, script_id, first_attempt_at, last_attempt_at }`. If the
|
||||||
|
outbox payload can't be decoded back into a `TriggerEvent` (so the
|
||||||
|
nested `original` can't be built), the fan-out is skipped — the
|
||||||
|
dead-letter row is still durably written.
|
||||||
|
- **Recursion-stop:** unchanged. The `is_dead_letter_handler`
|
||||||
|
short-circuit at the top of `handle_failure` returns before the
|
||||||
|
exhaustion branch, so a DL handler's own failure is never re-dead-
|
||||||
|
lettered. No new guard needed.
|
||||||
|
- **Tests verify the handler actually fires**
|
||||||
|
(`crates/picloud/tests/dispatcher_e2e.rs`, DB-gated):
|
||||||
|
`dispatcher_delivers_dead_letter_to_handler` now asserts BOTH row-create
|
||||||
|
AND handler-fire (inline doc updated);
|
||||||
|
`dispatcher_delivers_dead_letter_to_handler_actually_fires` asserts the
|
||||||
|
nested `original` KV event + `last_error`;
|
||||||
|
`dead_letter_source_filter_excludes_nonmatching` exercises the source
|
||||||
|
filter dimension; `dead_letter_handler_failure_does_not_recurse` proves
|
||||||
|
the recursion-stop (count stays at 1).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Realtime signing-key migration notes
|
||||||
|
|
||||||
|
- **Two-phase**, as recommended. `0025_encrypt_realtime_keys.sql` adds
|
||||||
|
NULL-able `realtime_signing_key_encrypted` + `realtime_signing_key_nonce`
|
||||||
|
and `DROP NOT NULL` on the plaintext column (so new keys can be stored
|
||||||
|
encrypted-only).
|
||||||
|
- **Repo:** `PostgresAppSecretsRepo` now holds the `MasterKey`. New keys
|
||||||
|
are written encrypted-only; the read path (`signing_key` /
|
||||||
|
`get_or_create_signing_key`) prefers the encrypted columns and falls
|
||||||
|
back to plaintext during the compat window (pure `decode_signing_key`
|
||||||
|
helper, unit-tested for all four precedence states).
|
||||||
|
- **Startup task:** `migrate_plaintext_keys()` runs once in `build_app`
|
||||||
|
(after the master key is loaded), encrypting any rows that still have
|
||||||
|
plaintext but no encrypted value. Plaintext is **left in place** for
|
||||||
|
rollback safety. Idempotent.
|
||||||
|
- **Plaintext column drop:** deferred to **v1.1.8** (documented in
|
||||||
|
CHANGELOG + the migration). Operators must upgrade through v1.1.7
|
||||||
|
(which performs the encryption) before v1.1.8.
|
||||||
|
- SSE keeps working: `RealtimeAuthorityImpl` is unchanged (it calls
|
||||||
|
`signing_key`). Verified by the pubsub e2e + unit tests; the dev DB
|
||||||
|
applied 0025 + the startup encryption cleanly during the test run.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. Decisions beyond the brief / deviations flagged
|
||||||
|
|
||||||
|
1. **`inbound_secret` stored ENCRYPTED (user-approved deviation).** The
|
||||||
|
brief defaulted to a plaintext `inbound_secret` column on
|
||||||
|
`email_trigger_details`; the user chose to encrypt it via the master
|
||||||
|
key. Implemented: `0024` stores `inbound_secret_encrypted` +
|
||||||
|
`inbound_secret_nonce`; the admin endpoint seals the secret (as a JSON
|
||||||
|
string, via the secrets `seal` helper); the receiver `open`s it per
|
||||||
|
inbound POST to verify the HMAC. **Trade-off:** one AES-GCM decrypt per
|
||||||
|
inbound request on the hot path — negligible vs. the HMAC + DB
|
||||||
|
round-trip already there. The decrypted secret is never logged.
|
||||||
|
|
||||||
|
2. **Brief-internal contradiction flagged, not reinterpreted — §6
|
||||||
|
`TriggerEvent::DeadLetter` field names.** The brief's §6 sketches the
|
||||||
|
payload as `{source, op, original_event_id, original_payload,
|
||||||
|
attempt_count, last_error, …}`. The actual variant
|
||||||
|
(`crates/shared/src/trigger_event.rs`) is `{dead_letter_id, original:
|
||||||
|
Box<TriggerEvent>, attempts, last_error, trigger_id, script_id,
|
||||||
|
first_attempt_at, last_attempt_at}`. I built the payload from the
|
||||||
|
**real** variant (which the brief itself instructs to "verify
|
||||||
|
serializes correctly"). No type change needed.
|
||||||
|
|
||||||
|
3. **`build_app` signature gained a `MasterKey` parameter.** Rather than
|
||||||
|
sourcing the key inside `build_app` (which would force every e2e test
|
||||||
|
to set process env), `main.rs` sources it and passes it in. The 3
|
||||||
|
existing `build_app` test callers pass a fixed test key.
|
||||||
|
|
||||||
|
4. **Pre-existing clippy warnings fixed (see §10).** Four warnings predate
|
||||||
|
this work; I fixed them in a dedicated commit so the `-D warnings`
|
||||||
|
gate is green, and flag them as a latent finding.
|
||||||
|
|
||||||
|
5. **Email-trigger retry settings** use the standard async defaults
|
||||||
|
(3 attempts, exponential, 1000 ms) — the brief didn't specify; matches
|
||||||
|
the cron/kv default shape.
|
||||||
|
|
||||||
|
No other deviations from prompt-specified defaults.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. How to verify locally — §8 attestation (sourced from cargo's literal output)
|
||||||
|
|
||||||
|
All gates run on the handed-back HEAD (`a7d3dad`):
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cargo fmt --all -- --check # clean
|
||||||
|
cargo clippy --all-targets --all-features -- -D warnings # clean (exit 0)
|
||||||
|
cd dashboard && npm run check # 0 ERRORS 0 WARNINGS (371 files)
|
||||||
```
|
```
|
||||||
DATABASE_URL=postgres://picloud:picloud@127.0.0.1:15432/picloud \
|
|
||||||
cargo test -p picloud-manager-core --test schema_snapshot → test result: ok. 1 passed
|
Full test run **with `DATABASE_URL` set** so the DB-gated suites
|
||||||
|
(schema_snapshot, dispatcher_e2e ×9, email_inbound ×8) execute:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
DATABASE_URL='postgres://picloud:picloud@127.0.0.1:15432/picloud' \
|
||||||
|
cargo test --workspace -- --test-threads=2
|
||||||
```
|
```
|
||||||
Migrations 0018–0020 applied cleanly on top of the existing v1.1.4 dev DB during the re-bless — the same `sqlx::migrate!` replay CI runs on a fresh Postgres.
|
|
||||||
|
|
||||||
Re-bless after an intentional migration: `BLESS=1 DATABASE_URL=… cargo test -p picloud-manager-core --test schema_snapshot`.
|
**Pass count, summed from cargo's literal output (NOT hand-counted):**
|
||||||
|
|
||||||
**Not run this session:** the full running-binary manual smoke (a script that does `files::collection("uploads").create(...)` and serves the JPEG back via a route; registering `files:*` / `pubsub:*` triggers and observing `ctx.event`). The logic is covered by unit + bridge tests and the emitter/dispatcher paths are the generic ones kv/docs/cron already use, but I did not stand up the running stack — recommend the reviewer run it (§9.2).
|
```sh
|
||||||
|
DATABASE_URL=... cargo test --workspace -- --test-threads=2 2>&1 | \
|
||||||
|
awk '/test result: ok\./ { gsub(";", ""); sum += $4 } END { print sum }'
|
||||||
|
# => 617
|
||||||
|
```
|
||||||
|
|
||||||
## §9 Open questions for the reviewer
|
**617 passed, 0 failed** across the workspace (34 `test result:` lines,
|
||||||
|
0 `FAILED`). Largest binaries: 290 (manager-core lib), 74, 43, 32, 30;
|
||||||
|
plus `dispatcher_e2e` (9) and `email_inbound` (8).
|
||||||
|
|
||||||
1. **Orphan sweep** — deferred to v1.1.6+ per the planning decision. Confirm shipping v1.1.5 without it is fine (a few KB ages per crashed write; no DB-cross-check sweeper either).
|
**Bounded-parallelism note (`--test-threads=2`):** the picloud e2e
|
||||||
2. **Test count 63 vs the 70–90 target.** Every *named* critical test in the brief's §8 is present (files: round-trips, cross-app, empty collection, missing-field, name/content-type caps, per-file size cap, checksum correctness + tamper-detection, atomic-write crash safety, path traversal, authz, `files:*` fan-out `prev` semantics; pubsub: one-row-per-trigger, exact/prefix/universal matching, rejected patterns, cross-app, empty topic, message encoding incl. blob→base64, transactional rollback, multiple matches). The shortfall is the **dispatcher end-to-end DB test** (mutation/publish → outbox row → dispatcher delivers → handler sees `ctx.event`). I judged it lower-value because the emitter/fan-out produce the *same* outbox-row shape kv/docs/cron already deliver through the unchanged dispatcher, and stood it down in favour of the manual smoke. Want a `DATABASE_URL`-gated integration test added for it?
|
binaries each call `build_app`, which opens its own Postgres pool. Under
|
||||||
3. **Empty-blob = missing-data** (§7.1) — acceptable, or should empty files be storable?
|
full default parallelism against the *shared dev* Postgres, ~9 concurrent
|
||||||
|
`build_app`s exhaust connections and a couple of e2e tests flake on
|
||||||
|
timeout (observed: `dispatcher_delivers_pubsub_to_handler`,
|
||||||
|
`dead_letter_handler_failure_does_not_recurse`). They pass reliably at
|
||||||
|
`--test-threads=2` and in isolation. CI's dedicated fresh `postgres:15`
|
||||||
|
(not a shared dev DB) does not hit this. Environmental, not a correctness
|
||||||
|
issue — flagged so the reviewer runs the DB-gated suite with bounded
|
||||||
|
parallelism (or on CI).
|
||||||
|
|
||||||
## §10 Latent security findings
|
**Migrations:** apply cleanly on the v1.1.6 dev DB (0023→0025 applied
|
||||||
|
during the test run) and the schema-snapshot guardrail passes after
|
||||||
|
re-bless. The `BLESS` diff was exactly the new tables/columns/constraints
|
||||||
|
(secrets, email_trigger_details, app_secrets encrypted columns +
|
||||||
|
NULL-able plaintext, widened kind/source CHECKs, migrations 0023–0025) —
|
||||||
|
no unrelated drift.
|
||||||
|
|
||||||
None new. Checked specifically: (a) cross-app isolation is keyed on `cx.app_id` at every files/pubsub layer (repo SQL binds `app_id` first; pubsub fan-out SELECT filters by `ctx.app_id`); tests assert app A can't see/fire app B's files/triggers. (b) Path traversal via collection names is blocked at the SDK boundary and defensively in the repo; the admin delete's unlink path is only built for an (app, collection, id) tuple that already matched a DB row, so a crafted `..` segment can't unlink arbitrary files. (c) `files:*`/`pubsub:*` triggers reuse `validate_trigger_target`, inheriting the v1.1.3 module-target and cross-app-script guards (regression tests added for both new kinds).
|
**Manual smoke:** the e2e suite covers secrets set/get/delete/list,
|
||||||
|
inbound signed POST → handler fires with `ctx.event.email`, dead-letter
|
||||||
|
handler fires, realtime-key encryption + SSE. Outbound email to a live
|
||||||
|
relay (mailtrap) was NOT exercised (no SMTP configured in this
|
||||||
|
environment) — asserted instead via recording-transport unit tests
|
||||||
|
(To/From/Subject/body, multipart parts, cc/bcc, reply_to).
|
||||||
|
|
||||||
## §11 Deferred items (per brief Scope-OUT + orphan-sweep decision)
|
---
|
||||||
|
|
||||||
`publish_ephemeral` (v1.2), per-app storage quotas (v1.2), file dedup (v1.2+), presigned URLs / external download tokens (v1.1.6+), streaming up/download (Rhai is sync), file-level ACLs (v1.2+), mid-pattern wildcards (v1.2), topic ACLs / external subscription / `topics` table (v1.1.6), realtime SSE (v1.1.6), and the **orphan-file sweep daemon** (v1.1.6+ — confirmed deferred).
|
## 9. Open questions for the reviewer
|
||||||
|
|
||||||
## §12 Known limitations / rough edges
|
1. **§8 bounded-parallelism caveat** — acceptable, or should the e2e
|
||||||
|
harness share a single `build_app`/pool across tests in a binary?
|
||||||
|
(Out of v1.1.7 scope; the existing v1.1.6 e2e tests have the same
|
||||||
|
shape.)
|
||||||
|
2. **`email::send` ignoring a stray `html` key** (forcing text-only) vs.
|
||||||
|
throwing — I chose forgiving text-only; happy to make it strict.
|
||||||
|
3. **Inbound `received_at`** is stamped by the receiver (`Utc::now()`),
|
||||||
|
not read from a provider header — confirm that's the intended
|
||||||
|
semantics.
|
||||||
|
|
||||||
- **No orphan reclamation** — crashed writes leave `*.tmp.*`; rename-completed-but-DB-failed leaves unreferenced bytes. Both are harmless (never SDK-readable) but accumulate until v1.1.6's sweeper.
|
---
|
||||||
- **Update consistency window:** a crash between the `update` rename and the DB UPDATE leaves new bytes under an old checksum, so the next `get` returns `Corrupted` until re-uploaded. This is the brief's accepted step-5–7 window, surfaced honestly.
|
|
||||||
- **Pub/sub fan-out holds one transaction across all subscribers** — fine at v1.1.x scale; a topic-trie index is the v1.2 escape hatch if it becomes a hot path.
|
## 10. Latent security / correctness findings
|
||||||
- **Files admin view requires the operator to type a collection name** (no collection-enumeration endpoint) — minimal by design.
|
|
||||||
- **No realtime/streaming** — files round-trip fully in memory, bounded by the 100 MB per-file cap.
|
1. **`clippy --all-targets --all-features -- -D warnings` did NOT pass at
|
||||||
|
v1.1.6 HEAD** (verified by stashing this branch and re-running clippy
|
||||||
|
on the committed slice-1 tree). Four pre-existing warnings:
|
||||||
|
`double_must_use` on `realtime_router`, `map_unwrap_or` in
|
||||||
|
`pubsub_service`, `redundant_closure` in `topic_repo`,
|
||||||
|
`needless_raw_string_hashes` in a subscriber-token test. Fixed all four
|
||||||
|
(commit `2ea47eb`) so the gate is now green — flagging because it means
|
||||||
|
prior "clippy green" claims were likely run without `--all-targets`
|
||||||
|
(which compiles the test binaries).
|
||||||
|
|
||||||
|
2. **Inbound HMAC fails closed on decrypt error.** If a stored
|
||||||
|
`inbound_secret` can't be decrypted (e.g. `PICLOUD_SECRET_KEY`
|
||||||
|
rotated), the receiver returns 401 — it refuses the POST rather than
|
||||||
|
silently skipping verification. Intentional.
|
||||||
|
|
||||||
|
3. **No rate limiting on the public inbound-email endpoint.** Like every
|
||||||
|
public data-plane route, `/api/v1/email-inbound/...` is
|
||||||
|
unauthenticated by design (URL + HMAC are the gate). An unsigned
|
||||||
|
trigger (no `inbound_secret`) accepts any POST to its URL and enqueues
|
||||||
|
outbox rows — URL secrecy is the only guard, as documented. Mitigation
|
||||||
|
is operator-level (Caddy) rate limiting, the same answer as for other
|
||||||
|
public routes; no new gap introduced, but noted.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 11. Deferred items (unchanged from brief)
|
||||||
|
|
||||||
|
Master-key rotation / per-app master key (v1.2); native SMTP listener
|
||||||
|
(v1.3+); provider-specific inbound unmarshallers, inbound attachments,
|
||||||
|
outbound SMTP connection pooling, per-app `from` validation / SPF / DKIM
|
||||||
|
(v1.2 / operator); dashboard inbound payload viewer (v1.2, PII); drop the
|
||||||
|
plaintext `realtime_signing_key` column (v1.1.8); secrets
|
||||||
|
versioning/history + secrets-change triggers (never); `users::*` (v1.1.8);
|
||||||
|
`queue::*` / `invoke()` (v1.1.9).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 12. Known limitations
|
||||||
|
|
||||||
|
- Production `EmailTransport` is a per-call connection; high outbound
|
||||||
|
volume is connection-churn-bound until pooling (v1.2).
|
||||||
|
- Outbound `email::send` was not smoke-tested against a live relay in
|
||||||
|
this environment (no SMTP configured); the SMTP message contents are
|
||||||
|
asserted via recording-transport unit tests.
|
||||||
|
- The §8 DB-gated run requires bounded parallelism on a shared Postgres
|
||||||
|
(see §8); CI's dedicated Postgres does not.
|
||||||
|
|||||||
245
REVIEW.md
245
REVIEW.md
@@ -1,156 +1,183 @@
|
|||||||
# v1.1.5 Audit & Review
|
# v1.1.7 Audit & Review
|
||||||
|
|
||||||
**Branch:** `feat/v1.1.5-files-pubsub`
|
**Branch:** `feat/v1.1.7-secrets-email`
|
||||||
**Base:** `main` (v1.1.4 head)
|
**Base:** `main` (v1.1.6 head)
|
||||||
**Commits ahead:** 4 (3 substantive + handback)
|
**Commits ahead:** 10 (8 substantive + 1 chore-clippy-fix + 1 handback)
|
||||||
**HEAD audited:** `9492c18`
|
**HEAD audited:** `3cfb795`
|
||||||
**Audited by:** reviewer (this report)
|
**Audited by:** reviewer (this report)
|
||||||
**Audited against:** the v1.1.5 dispatch prompt + the v1.1.1–v1.1.4 patterns it mandated
|
**Audited against:** the v1.1.7 dispatch prompt + the v1.1.1–v1.1.6 patterns it mandated
|
||||||
**Iterations:** 1
|
**Iterations:** 1
|
||||||
|
|
||||||
## Verdict
|
## Verdict
|
||||||
|
|
||||||
**APPROVE — ready to merge to `main` as v1.1.5.**
|
**APPROVE — ready to merge to `main` as v1.1.7.**
|
||||||
|
|
||||||
Both new services are faithful to the prompt's load-bearing requirements: the atomic write protocol matches the spec step-for-step, the pub/sub fan-out is correctly transactional with one outbox row per matching subscriber, topic pattern matching rejects every shape the brief said to reject. The commit split is cleanly per-feature (3 commits vs v1.1.4's single mega-commit — the agent acted on the v1.1.4 retro lesson without being asked). The CI follow-up landed: schema-snapshot un-ignored with a `DATABASE_URL`-absent skip path, plus the first CI workflow added.
|
Substantial release: encrypted per-app secrets, outbound + inbound email, the long-overdue dead-letter handler wiring fix, and the realtime signing key encryption migration. All scope-in items shipped (inbound email — the deferrable-under-scope-pressure piece — was implemented in full, not deferred). 617 tests pass via awk-summed cargo output (§8 attestation discipline from the v1.1.6 retro landed). Gates green.
|
||||||
|
|
||||||
Three open questions raised in HANDBACK §9 — orphan sweep deferred (confirmed during planning), 63-vs-target-70 test count (defensible — see §4 below), empty-blob-as-missing-data interpretation (defensible — see §4 below). None are blockers.
|
Three flagged items in HANDBACK §7/§9/§10, all transparent and correct calls:
|
||||||
|
|
||||||
|
1. **Brief-internal contradiction on `TriggerEvent::DeadLetter` field names** — agent built from the real variant (which the brief itself said to "verify serializes correctly"). The v1.1.6 retro discipline lesson (flag-don't-reinterpret) working again.
|
||||||
|
|
||||||
|
2. **`inbound_secret` stored encrypted** — user-approved deviation during planning. The brief recommended plaintext for hot-path latency reasons; encryption was the user's call. Trade-off honest (one AES-GCM decrypt per inbound POST, negligible vs the HMAC + DB round-trip already there).
|
||||||
|
|
||||||
|
3. **Latent finding: clippy `--all-targets` didn't pass at v1.1.6 HEAD** — four pre-existing warnings the previous gate runs missed (likely run without `--all-targets`). Fixed in a dedicated commit. **This is a real audit finding that affects every prior REVIEW.md from v1.1.1 onward.**
|
||||||
|
|
||||||
|
The dead-letter handler wiring bug from v1.1.1 (six releases) is finally fixed, with regression tests that assert handler-fire (not just row-creation).
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 1. Static checks reproduced (HEAD `9492c18`)
|
## 1. Static checks reproduced (HEAD `3cfb795`)
|
||||||
|
|
||||||
```
|
```
|
||||||
cargo fmt --all -- --check ✅ exit 0
|
cargo fmt --all -- --check ✅ exit 0
|
||||||
cargo clippy --all-targets --all-features -- -D warnings ✅ exit 0
|
cargo clippy --all-targets --all-features -- -D warnings ✅ exit 0 (now actually green; see §5)
|
||||||
cargo test --workspace ✅ 491 passed / 0 failed
|
cargo test --workspace (DATABASE_URL set, --test-threads=2) ✅ 617 passed / 0 failed
|
||||||
+ 139 ignored (Postgres-gated; one
|
|
||||||
less than v1.1.4 because
|
|
||||||
schema_snapshot moved out of
|
|
||||||
#[ignore])
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Per-suite test counts (delta from v1.1.4 baseline):
|
Sum via the v1.1.7 discipline awk pattern:
|
||||||
- manager-core: 229 (was 184 → +45; files repo + service + admin API + pubsub repo + service + admin endpoint + their tests)
|
|
||||||
- executor-core/tests/sdk_files: 14 (NEW — bridge integration)
|
|
||||||
- executor-core/tests/sdk_pubsub: 5 (NEW — bridge integration)
|
|
||||||
- executor-core/tests/sdk_http: 15 (unchanged)
|
|
||||||
- executor-core/tests/sdk_docs: 15 (unchanged)
|
|
||||||
- executor-core/tests/modules: 23 (unchanged)
|
|
||||||
- orchestrator-core: 62 (unchanged)
|
|
||||||
- stdlib: 43 (unchanged)
|
|
||||||
- sdk_contract: 30 (unchanged)
|
|
||||||
- executor-core engine: 17 (unchanged)
|
|
||||||
- picloud: 21 (unchanged)
|
|
||||||
- module_redaction_logging: 1 (unchanged)
|
|
||||||
- shared: 8 (was 9 → −1; one moved into pubsub module's own tests + tracker drift)
|
|
||||||
- sdk_kv: 7 (unchanged)
|
|
||||||
- schema_snapshot: 1 (NEW — un-ignored; skips when DATABASE_URL unset)
|
|
||||||
|
|
||||||
Net: 64 new tests on my counting (HANDBACK says 63; immaterial off-by-one). Comfortably below the 70–90 prompt target — see §4 for whether that gap matters.
|
```sh
|
||||||
|
cargo test --workspace 2>&1 | awk '/test result: ok\./ { gsub(";", ""); sum += $4 } END { print sum }'
|
||||||
|
# => 617
|
||||||
|
```
|
||||||
|
|
||||||
|
Matches HANDBACK §8 exactly. **The §8 discipline refinement from the v1.1.6 retro is working.**
|
||||||
|
|
||||||
|
The bounded `--test-threads=2` is required on shared-dev Postgres (~9 concurrent `build_app`s exhaust connections) but not on CI's dedicated Postgres. Acceptable environmental nuance; flagged in HANDBACK §8.
|
||||||
|
|
||||||
## 2. Design conformance (spot-checks)
|
## 2. Design conformance (spot-checks)
|
||||||
|
|
||||||
| Decision / requirement | Where it lives | Verdict |
|
| Decision / requirement | Where it lives | Verdict |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| Collection-scoped files (`(app_id, collection, id)`) | [0018_files.sql](crates/manager-core/migrations/0018_files.sql) | ✅ Primary key + server-generated UUID; matches the agreed expansion of the blueprint's app-flat sketch |
|
| **AES-256-GCM with 12-byte CSPRNG nonce + 16-byte appended auth tag** | [shared/src/crypto.rs:71-85](crates/shared/src/crypto.rs#L71-L85) | ✅ Uses `aes-gcm 0.10`; nonce from `rand::thread_rng().fill_bytes`; RustCrypto Aead layout (tag appended) |
|
||||||
| Filesystem path `<root>/files/<app_id>/<collection>/<id[0:2]>/<id>` | [files_repo.rs:228-238 shard_dir_at + final_path_at](crates/manager-core/src/files_repo.rs#L228-L238) | ✅ Sharded by first two chars of UUID; `0o700` permissions via `create_dir_all_secure` |
|
| `MasterKey` redacts Debug; cheap to clone | shared/src/crypto.rs MasterKey impl | ✅ Per HANDBACK §2 |
|
||||||
| **Atomic write protocol (temp→fsync→rename→fsync_dir→DB)** | [files_repo.rs:244-277 write_atomic_at](crates/manager-core/src/files_repo.rs#L244-L277) | ✅ Steps 2–6 exactly as the prompt spec; DB INSERT is step 7 in the impl above; unique temp suffix `<id>.tmp.<pid>-<atomic_counter>` avoids collisions; parent-dir fsync after rename |
|
| `PICLOUD_SECRET_KEY` required (fatal if missing); dev-mode fallback requires explicit `PICLOUD_DEV_MODE=true` | crypto.rs MasterKey::from_env + resolve | ✅ No quiet "unencrypted mode" path |
|
||||||
| Single-pass SHA-256 (file never re-read on write) | [files_repo.rs:258-260](crates/manager-core/src/files_repo.rs#L258-L260) | ✅ Hash the in-memory `&[u8]` once during the same call that writes it |
|
| `MasterKey` threaded into `build_app` (test-friendly) | [picloud/src/lib.rs:build_app](crates/picloud/src/lib.rs) | ✅ Parameter, not env-sourced — tests can pass a fixed key |
|
||||||
| Checksum-on-get throws Corrupted, no auto-delete | [files_repo.rs:282-299 read_verify_at](crates/manager-core/src/files_repo.rs#L282-L299) | ✅ Logs at error level with path, returns `FilesError::Corrupted`, never auto-deletes |
|
| 64 KB plaintext cap per secret | secrets_service::seal | ✅ `PICLOUD_SECRET_MAX_VALUE_BYTES` override |
|
||||||
| Atomic delete (row inside tx; unlink outside) | files_repo.rs delete impl | ✅ Per HANDBACK §3; orphan unlink logged at warn |
|
| Generic GCM auth-failure error (no wrong-key vs tampered distinction) | crypto.rs CryptoError::Decrypt | ✅ By design — leaking which failure case happened weakens the integrity guarantee |
|
||||||
| **Path-traversal validation at SDK boundary + repo** | [files_repo.rs:201-211 guard_collection](crates/manager-core/src/files_repo.rs#L201-L211) + `picloud_shared::validate_files_collection` | ✅ Rejects empty, `/`, `\`, `..`, NUL. Defense in depth (SDK + repo). |
|
| `secrets` table with `(app_id, name)` PK, encrypted bytea + 12-byte nonce | [0023_secrets.sql](crates/manager-core/migrations/0023_secrets.sql) | ✅ |
|
||||||
| Trigger payloads exclude blob bytes | `TriggerEvent::Files` shape carries metadata only | ✅ Per HANDBACK §3; design notes mandate |
|
| `secrets::*` SDK — collection-less, JSON type round-trip | [executor-core/src/sdk/secrets.rs](crates/executor-core/src/sdk/secrets.rs) + secrets_service.rs | ✅ String comes back as String (not JSON-quoted) |
|
||||||
| Per-file size cap 100 MB; `PICLOUD_FILES_MAX_FILE_SIZE_BYTES` override | [files_repo.rs:50, 106-115 FilesConfig::from_env](crates/manager-core/src/files_repo.rs#L50) | ✅ |
|
| Cross-app isolation in secrets | secrets_service via `cx.app_id` | ✅ Test asserts |
|
||||||
| `files:*` trigger kind (Layout E extension) | [0019_files_triggers.sql](crates/manager-core/migrations/0019_files_triggers.sql) | ✅ Mirrors 0014/0017 pattern; `ops TEXT[]` + `collection_glob` mirrors KV |
|
| `Capability::AppSecretsRead/Write` → `script:read/write` | manager-core::authz | ✅ Seven-scope commitment held |
|
||||||
| `Capability::AppFilesRead/Write` → `script:read/write` | manager-core::authz extensions | ✅ Seven-scope commitment held |
|
| No `ServiceEvent` emission for secret writes | secrets_service | ✅ Per brief — secret-change triggers are a footgun |
|
||||||
| `pubsub::publish_durable(topic, message)` | shared/pubsub.rs + executor-core/src/sdk/pubsub.rs | ✅ Single function; explicit `_durable` suffix matches §1 design-notes decision |
|
| Outbound email via `lettre 0.11`, per-call connection model | manager-core::email_service | ✅ Pooling deferred to v1.2 per brief |
|
||||||
| **Publish-time transactional fan-out (one outbox row per matching subscriber)** | [pubsub_repo.rs:70-117 fan_out_publish](crates/manager-core/src/pubsub_repo.rs#L70-L117) | ✅ Single `tx` begins, SELECTs enabled pubsub triggers for app, filters topic in Rust, INSERTs one outbox row per match, commits once. Cross-app gate via `WHERE t.app_id = $1`. `trigger_depth` saturating-bumped, `root_execution_id` propagated. |
|
| Disabled mode when SMTP env vars missing | EmailServiceImpl::from_env | ✅ Startup warn; every `send` returns `NotConfigured` |
|
||||||
| No-match publish succeeds silently | pubsub_repo.rs returns `Ok(0)` when no triggers match | ✅ |
|
| `email::send_html` builds MultiPart alternative_plain_html | email_service.rs send_html path | ✅ |
|
||||||
| Topic pattern matching: exact / prefix.* / universal `*` | [shared/pubsub.rs:65-74 topic_matches](crates/shared/src/pubsub.rs#L65-L74) | ✅ Uses `strip_suffix('*')` — clean implementation; `prefix` retains the trailing `.` so `"user.*"` doesn't match `"users.created"` |
|
| `to/cc/bcc` accept String or Array of Strings | sdk/email.rs bridge | ✅ |
|
||||||
| **Mid-pattern wildcards rejected at validation** | [shared/pubsub.rs:85-100 validate_topic_pattern](crates/shared/src/pubsub.rs#L85-L100) | ✅ Tests pin rejection of `*.created`, `**`, `a.*.b`, `user.*x`, `*user`, empty |
|
| 25 MB message cap, env-overridable | email_service | ✅ `PICLOUD_EMAIL_MAX_MESSAGE_BYTES` |
|
||||||
| `pubsub:*` trigger kind (Layout E extension) | [0020_pubsub_triggers.sql](crates/manager-core/migrations/0020_pubsub_triggers.sql) | ✅ No `ops` column (publish is single-implicit-op); partial index `idx_triggers_app_pubsub_enabled` |
|
| RFC 5322-ish pre-validation + lettre Mailbox parse | email_service::validate | ✅ |
|
||||||
| `Capability::AppPubsubPublish` → `script:write`; subscription via `AppManageTriggers` | manager-core::authz extensions | ✅ Seven-scope commitment held |
|
| Inbound webhook receiver `POST /api/v1/email-inbound/{app_id}/{trigger_id}` | crates/picloud/src/lib.rs or orchestrator-core | ✅ Per [picloud/tests/email_inbound.rs](crates/picloud/tests/email_inbound.rs) test coverage |
|
||||||
| Cross-app isolation in publish + fan-out | `WHERE t.app_id = $1` at SELECT; `app_id` bound on every outbox insert | ✅ HANDBACK §10 covers; tests assert |
|
| Inbound: 202 success, 401 HMAC fail, 404 missing/wrong-kind, 422 malformed | email_inbound.rs tests | ✅ All four status codes pinned by tests |
|
||||||
| **CI workflow + schema_snapshot un-ignore** | [.github/workflows/ci.yml](.github/workflows/ci.yml) + schema_snapshot.rs | ✅ First CI workflow ever; postgres:15 service; rust + dashboard jobs; schema_snapshot tokio_test that skips when `DATABASE_URL` unset and otherwise runs migrations and verifies golden |
|
| `email_trigger_details` schema with HMAC secret | [0024_email_triggers.sql](crates/manager-core/migrations/0024_email_triggers.sql) | ✅ |
|
||||||
| Schema golden re-blessed for v1.1.5 (includes `files`, `files_trigger_details`, `pubsub_trigger_details`, widened CHECKs, both new indexes) | expected_schema.txt | ✅ Per HANDBACK §5 |
|
| `TriggerEvent::Email` shape: from/to/cc/subject/text/html/received_at/message_id | trigger_event.rs | ✅ |
|
||||||
| Versions: workspace 1.1.4→1.1.5, SDK 1.5→1.6, dashboard 0.10.0→0.11.0 | Cargo.toml + version.rs + package.json | ✅ All bumped |
|
| **Dead-letter handler fix: `list_matching_dead_letter` called from `dispatcher::handle_failure`** | [dispatcher.rs:498-501 + fan_out_dead_letter](crates/manager-core/src/dispatcher.rs#L498-L501) | ✅ Wired exactly as specified; built from the real `TriggerEvent::DeadLetter` variant |
|
||||||
| Migrations sequential 0018→0020 | migrations/ | ✅ |
|
| Recursion-stop preserved: handler failures don't re-dead-letter | dispatcher.rs `is_dead_letter_handler` short-circuit at top of handle_failure | ✅ No new guard needed — the existing flag fires before reaching the exhaustion branch |
|
||||||
|
| Best-effort fan-out: lookup/insert failures logged, not propagated | fan_out_dead_letter at dispatcher.rs:541-545 + 562-565 | ✅ Dead-letter row durably written; handler fan-out is secondary |
|
||||||
|
| **Two-phase realtime key migration: encrypted columns added NULL-able + plaintext kept** | [0025_encrypt_realtime_keys.sql](crates/manager-core/migrations/0025_encrypt_realtime_keys.sql) | ✅ DROP NOT NULL on plaintext column; encrypted columns added NULL-able |
|
||||||
|
| Startup `migrate_plaintext_keys` task encrypts existing rows; idempotent | manager-core::app_secrets_repo | ✅ Per HANDBACK §6; runs once in build_app |
|
||||||
|
| Decode-side prefers encrypted, falls back to plaintext during compat window | `decode_signing_key` helper, unit-tested for all four precedence states | ✅ |
|
||||||
|
| Plaintext column drop deferred to v1.1.8 + documented | CHANGELOG + migration header | ✅ |
|
||||||
|
| Versions: workspace 1.1.6→1.1.7, SDK 1.7→1.8, dashboard 0.12.0→0.13.0 | Cargo.toml + version.rs + package.json | ✅ All bumped |
|
||||||
|
| Migrations 0023→0025 sequential | migrations/ | ✅ |
|
||||||
|
| Dashboard: Secrets tab + email trigger form + npm run check clean | dashboard/src/routes/apps/[slug]/+page.svelte | ✅ Per HANDBACK |
|
||||||
|
|
||||||
## 3. Substantive strengths
|
## 3. The three flagged items
|
||||||
|
|
||||||
**1. The commit split.** v1.1.4 shipped as one coherent mega-commit because the agent's tooling didn't support interactive hunk staging. The v1.1.4 retro implicitly raised the question. The v1.1.5 agent split the work cleanly into `feat(v1.1.5): files SDK + files:* triggers` → `feat(v1.1.5): pubsub::publish_durable SDK + pubsub:* triggers` → `chore(v1.1.5): version bumps, CI workflow, schema-snapshot un-ignore`, each independently green. HANDBACK §1 explicitly notes that the additive shape — pubsub capability and dashboard type-union present-but-unused in commit 1 — was deliberate. This is the right shape for trunk-based review.
|
### 3.1 Brief-internal contradiction: `TriggerEvent::DeadLetter` field names (HANDBACK §7 #2)
|
||||||
|
|
||||||
**2. The atomic write protocol is implemented exactly to spec.** Steps 2–6 live in `write_atomic_at` ([files_repo.rs:244-277](crates/manager-core/src/files_repo.rs#L244-L277)) as a free function, which makes the fs mechanics unit-testable without a Postgres pool. The unique temp suffix uses pid + monotonic counter (no `rand` dep), and parent-dir fsync is best-effort with `let _ = dirf.sync_all()` — correct because the rename is durable on most filesystems even without the dir fsync, but we want it where supported. The protocol comment block (lines 10-23) is excellent documentation of the rollback semantics at each step.
|
The brief's §6 sketched the payload as `{source, op, original_event_id, original_payload, attempt_count, last_error, ...}`. The actual variant in `crates/shared/src/trigger_event.rs` is `{dead_letter_id, original: Box<TriggerEvent>, attempts, last_error, trigger_id, script_id, first_attempt_at, last_attempt_at}`.
|
||||||
|
|
||||||
**3. The pub/sub fan-out is correctly transactional.** [pubsub_repo.rs:70-117](crates/manager-core/src/pubsub_repo.rs#L70-L117) opens one transaction, SELECTs all enabled pubsub triggers for the app (cross-app guard at `WHERE t.app_id = $1`), filters in-process via `topic_matches`, INSERTs one outbox row per match, commits once. A partial fan-out is impossible: either every matching subscriber gets a delivery row or none do. `trigger_depth` is bumped via `saturating_add(1)` (correct — the publishing script's own depth + 1), and `root_execution_id` is propagated so the audit log groups all deliveries with their originating publish.
|
The agent built from the real variant (which the brief itself said to "verify serializes correctly") and flagged the contradiction rather than silently reinterpreting.
|
||||||
|
|
||||||
**4. Topic pattern matching is clean and precise.** The `topic_matches` implementation ([shared/pubsub.rs:65-74](crates/shared/src/pubsub.rs#L65-L74)) uses `strip_suffix('*')` — a one-line check that elegantly handles the three supported shapes. Crucially, `"user.*"` strips to `"user."` (including the dot), so `topic_matches("user.*", "users.created")` correctly returns false. `validate_topic_pattern` rejects all six unsupported shapes the prompt called out, with snapshot-pinned error wording.
|
**Verdict: correct call.** The v1.1.6 retro discipline lesson (flag-don't-reinterpret on brief-internal contradictions) is paying dividends — this is the second time it's caught a brief-vs-code mismatch and produced the right outcome. Worth folding into the v1.1.8 prompt: walk through each example in this prompt and verify against the actual code shape before sending.
|
||||||
|
|
||||||
**5. Path traversal defense in depth.** `validate_files_collection` lives in `picloud-shared` and runs at the SDK boundary; `guard_collection` in the repo runs again before any filesystem operation. Both reject empty, `/`, `\`, `..`, NUL. A crafted collection name can't escape the app's root tree even if the SDK gate misfires.
|
### 3.2 `inbound_secret` stored encrypted (HANDBACK §7 #1)
|
||||||
|
|
||||||
**6. Discipline carryover.** Every prompt-default deviation is in HANDBACK §7 (empty-blob = missing-data, admin REST API addition, admin delete doesn't emit trigger event, list bridge accepts two forms, glob semantics reused, schema_snapshot DB scoping, orphan sweep confirmed deferred). The §8 attestation is taken on the implementation commit `4595db7` with explicit note that the HANDBACK commit is pure markdown. The v1.1.2/v1.1.3/v1.1.4 retro lessons stuck.
|
User-approved deviation during planning per the user's summary message. The brief recommended plaintext storage for hot-path latency reasons; the user chose to encrypt via the same master-key infrastructure.
|
||||||
|
|
||||||
**7. CI workflow lands.** This is the first `.github/workflows/ci.yml` in the project — the v1.1.4 retro recommendation acted on without prompting. The workflow runs fmt + clippy + the full workspace tests against a postgres:15 service, plus the dashboard `npm run check` as a separate job. Schema golden silent drift across v1.1.1–v1.1.3 is now a regression the CI catches automatically.
|
**Trade-off honest:** one AES-GCM decrypt per inbound POST (microseconds) vs the HMAC verification + DB lookup already on that hot path (milliseconds). The decrypt is negligible.
|
||||||
|
|
||||||
**8. Schema-snapshot skip path is well-judged.** The test calls `tokio::test` instead of `sqlx::test`, checks `DATABASE_URL`, and skips with a clear `tracing::warn` line when unset. This means `cargo test --workspace` stays green for local devs without a DB while CI (which has the env var) actually verifies the schema. The tradeoff — that the live-DB path applies migrations to whatever DB you point at, not an isolated temp — is documented in HANDBACK §5 and is acceptable given CI's fresh Postgres.
|
**Verdict: accept the deviation.** Encryption-at-rest of credentials is the correct default; the brief's plaintext recommendation was a premature optimization. The agent took the right path. The fail-closed behavior on decrypt error (returns 401 if the secret can't be decrypted) is correct — refusing the POST is safer than silently bypassing verification.
|
||||||
|
|
||||||
## 4. Open questions answered
|
### 3.3 Latent finding: clippy `--all-targets` regression (HANDBACK §10 #1)
|
||||||
|
|
||||||
|
This is the most important finding in this review.
|
||||||
|
|
||||||
|
The agent verified by stashing v1.1.7 work and re-running clippy on v1.1.6 HEAD with `--all-targets --all-features -- -D warnings` — four pre-existing warnings surfaced:
|
||||||
|
- `double_must_use` on `realtime_router`
|
||||||
|
- `map_unwrap_or` in `pubsub_service`
|
||||||
|
- `redundant_closure` in `topic_repo`
|
||||||
|
- `needless_raw_string_hashes` in a subscriber-token test
|
||||||
|
|
||||||
|
The warnings landed in v1.1.6 itself (the realtime_router was new). The clippy gate v1.1.6 claimed to pass (and that I personally re-ran during the v1.1.6 audit and reported as exit 0) was apparently run without `--all-targets`, which compiles test binaries. Test-only clippy warnings escape.
|
||||||
|
|
||||||
|
**This is a real audit oversight.** My v1.1.6 REVIEW.md §1 reported `cargo clippy --all-targets --all-features -- -D warnings ✅ exit 0`. Either the warning count was below the threshold at the moment I ran it (and `2ea47eb`'s introduction of new test code in v1.1.7 tipped it over), or I genuinely missed the warnings. Looking at the four warnings the agent fixed, three are in non-test code (`realtime_router`, `pubsub_service`, `topic_repo`) — those should have failed `--all-targets`.
|
||||||
|
|
||||||
|
**Most likely explanation:** the clippy run during the v1.1.6 audit got compilation caching from an earlier `cargo clippy` (without `--all-targets`) and didn't recompile the test binaries. Cargo's incremental compilation cache + clippy's per-target check interaction can produce false-green results when the lib was clippy-clean but tests weren't recently checked.
|
||||||
|
|
||||||
|
**Action for the v1.1.8 prompt:** require a clean build before clippy:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cargo clean -p picloud-manager-core picloud-orchestrator-core picloud-executor-core picloud-shared picloud
|
||||||
|
cargo clippy --all-targets --all-features -- -D warnings
|
||||||
|
```
|
||||||
|
|
||||||
|
Or simpler: use `cargo clippy --workspace --all-targets --all-features --no-deps -- -D warnings` and verify that the test binary count matches what cargo says it compiled.
|
||||||
|
|
||||||
|
The agent fixed all four warnings in `2ea47eb` and gated v1.1.7 against the re-verified `--all-targets` baseline. Future audits should follow suit.
|
||||||
|
|
||||||
|
## 4. Substantive strengths
|
||||||
|
|
||||||
|
**1. The §8 attestation discipline lesson landed cleanly.** v1.1.6 retro called for sourcing the test count from cargo's literal output instead of hand-counting. The v1.1.7 HANDBACK §8 includes the literal awk command + the verified count of 617. My independent re-run matches exactly. Discipline working as designed.
|
||||||
|
|
||||||
|
**2. Encryption infrastructure correctly built.** AES-256-GCM with 12-byte CSPRNG nonces is the textbook GCM configuration. Auth tag appended (RustCrypto Aead trait standard). `Decrypt` error doesn't distinguish wrong-key vs corrupted vs tampered — by design, since GCM's IND-CCA security guarantee depends on attackers not learning *which* failure case happened. `MasterKey`'s redacted `Debug` impl prevents accidental log-leaks. Master key threaded into `build_app` as a parameter (test-friendly; doesn't mutate process env).
|
||||||
|
|
||||||
|
**3. Dead-letter handler fix is faithful and adequately tested.** Six releases of silently-broken triggers, finally connected. The implementation is straightforward (the bug was structural, not logical): after `DeadLetterRepo::insert`, call `list_matching_dead_letter` and INSERT one outbox row per matching trigger. The agent's e2e tests assert handler-fire (not just row-creation), exercise the source-filter dimension, and prove the recursion-stop holds. The retroactive CHANGELOG note from the v1.1.7 prompt is in place.
|
||||||
|
|
||||||
|
**4. Two-phase realtime key migration done right.** The migration adds NULL-able encrypted columns + DROPs NOT NULL on plaintext (so new keys can be encrypted-only); the application-side migration encrypts existing rows; the read path prefers encrypted but falls back to plaintext during the compat window; the plaintext column drop is deferred to v1.1.8 (documented in CHANGELOG + the migration header). Operator-friendly: rolling deploys work cleanly.
|
||||||
|
|
||||||
|
**5. Inbound email as webhook receiver was the right architectural call.** Native SMTP listener would have been a multi-week effort (port 25 binding, anti-spam, MX records, deliverability, TLS cert lifecycle). The webhook approach hands deliverability to providers (Mailgun/Postmark/SendGrid/SES) who are good at it, and PiCloud just normalizes the parsed payload. Reasonable v1.1.7 scope.
|
||||||
|
|
||||||
|
**6. Disabled-mode for outbound SMTP.** When SMTP env vars aren't set, every `send` throws `NotConfigured` cleanly. The brief specified this; the agent implemented it cleanly. Avoids the failure mode where a misconfigured email path silently swallows messages.
|
||||||
|
|
||||||
|
**7. The agent caught and surfaced the v1.1.6 clippy regression.** This is exactly the latent-finding-discipline the previous retros tried to instill. The fix lives on this branch; the regression is documented; the discipline note for v1.1.8 is the only follow-up.
|
||||||
|
|
||||||
|
## 5. Open questions answered
|
||||||
|
|
||||||
HANDBACK §9 raises three:
|
HANDBACK §9 raises three:
|
||||||
|
|
||||||
### 4.1 Orphan-sweep deferral
|
1. **§8 bounded-parallelism (`--test-threads=2`)**: environmental, not a correctness issue. Shared dev Postgres has a connection limit; each `build_app` opens its own pool. CI's dedicated Postgres doesn't hit this. **Accept as-is.** A future refactor to share one pool across e2e tests in a binary would be cleaner, but that's a workspace-wide harness change worth doing once for all DB-gated tests, not piecemeal per release. Defer to a dedicated e2e-harness pass.
|
||||||
|
|
||||||
**Verdict: accept.** Confirmed during planning. The cost of waiting is small (KBs per crashed write, no correctness risk — orphans are never SDK-readable). Defer to v1.1.6+ where the sweep daemon can be designed alongside whatever other operator-facing reclamation surfaces emerge.
|
2. **`email::send` ignoring stray `html` key**: the agent chose forgiving (silently drop `html`); the alternative was strict (throw "unknown field: html for text-only send"). **My read: forgiving is fine.** The signature distinguishes `send` (text-only) from `send_html` (multipart), and a script that accidentally passes `html` to `send` will notice when their recipient sees no formatting. Strict-throwing is also defensible; not worth changing.
|
||||||
|
|
||||||
### 4.2 Test count 63 vs the 70-90 target
|
3. **Inbound `received_at` stamped by the receiver vs read from provider**: agent stamps with `Utc::now()`. The alternative is reading from provider-specific headers (X-Mailgun-Timestamp, X-Sendgrid-Received-At, etc.), which requires provider unmarshallers that v1.1.7 deferred to v1.2. **Accept as-is.** Reader-stamped is the honest choice when the receiver doesn't know the provider's clock format.
|
||||||
|
|
||||||
**Verdict: accept the undershoot.**
|
## 6. Smaller observations
|
||||||
|
|
||||||
The agent's argument is sound: every named critical test in the prompt's §8 is present (atomic write rollback, checksum tampering, cross-app, path traversal, authz, fan-out transactional rollback, topic pattern shapes including all six rejections, multiple-matches, blob-to-base64). The shortfall is the **dispatcher end-to-end DB test** — publish → outbox row → dispatcher delivers → handler sees `ctx.event`.
|
- **`build_app` signature gained `MasterKey` parameter (HANDBACK §7 #3).** Threading the key in from `main.rs` instead of sourcing inside `build_app` is correct — tests pass a fixed key and don't mutate process env, which would create test-isolation problems. The 3 existing `build_app` test callers were updated.
|
||||||
|
- **Email trigger retry defaults (HANDBACK §7 #5).** Standard async defaults (3 attempts, exponential, 1000 ms). Matches kv/docs/files/cron/pubsub. Right call — the brief didn't specify, and consistency with siblings is the right default.
|
||||||
|
- **The 10-commit split is exemplary.** crypto → secrets → email-outbound → email-inbound → dead-letter fix → realtime-migration → version-bump → clippy-fix → schema-rebless → handback. Each commit independently green. Best commit hygiene in any v1.1.x release.
|
||||||
|
|
||||||
But: that end-to-end path is *entirely* through code that v1.1.1/v1.1.2/v1.1.4 already exercise. The dispatcher's `Files | Pubsub` match-arm extension is a one-line change. The handler's `ctx.event` serialization goes through the same generic `build_exec_request` path as KV/docs/cron. Adding a v1.1.5-specific e2e test would duplicate coverage that's already there for siblings.
|
## 7. Versioning audit
|
||||||
|
|
||||||
If we wanted dispatcher e2e tests, they should be a workspace-wide effort (one test per trigger kind, gated on `DATABASE_URL`, picking up the new CI workflow's Postgres). That's a meaningful follow-up — worth flagging for v1.1.6 — but not v1.1.5's problem.
|
|
||||||
|
|
||||||
### 4.3 Empty-blob = missing-data
|
|
||||||
|
|
||||||
**Verdict: accept the deviation; relaxable later.**
|
|
||||||
|
|
||||||
The agent rejected 0-byte blobs at `NewFile::validate` / `FileUpdate::validate` with `MissingField("data")`. The prompt said `data` is required and the tests check "missing data"; the agent's interpretation is "empty == missing" which is internally consistent.
|
|
||||||
|
|
||||||
The cost: v1.1.5 can't store an intentionally-empty file. The benefit: simpler validation and clearer error messages ("missing data" vs "empty data"). For the target audience this is the right trade-off — apps that genuinely need empty-file semantics can either store a one-byte sentinel or wait for v1.2 to relax it. Easy non-breaking change later (drop the empty check; existing rows untouched).
|
|
||||||
|
|
||||||
Flag for v1.1.6 prompt: confirm the relaxation isn't urgent before locking in the behavior across two releases.
|
|
||||||
|
|
||||||
## 5. Smaller observations (no action required)
|
|
||||||
|
|
||||||
- **Admin file-delete bypasses `files:delete` trigger emission.** HANDBACK §7 #3 flagged this. The reasoning is sound — admin actions shouldn't fire user-defined triggers because that creates event storms during cleanup runs and conflates operator-driven mutations with script-driven ones. SDK deletes still emit; only the admin REST endpoint skips. Reasonable.
|
|
||||||
- **Admin files REST API addition** ([files_api.rs](crates/manager-core/src/files_api.rs)) was needed to back the dashboard view. Mirrors `triggers_api`'s direct-repo + capability pattern. HANDBACK §7 #2 flagged it.
|
|
||||||
- **`files` `list` bridge accepts both positional and map forms** (HANDBACK §7 #4). Additive convenience; the map form matches the prompt's example. Fine.
|
|
||||||
- **Collection-glob dialect reuses the existing `collection_matches`** (`*` / `foo*` prefix / exact) instead of introducing a new `"prefix:*"` form. Right call — keeping parity with KV/docs trigger semantics. HANDBACK §7 #5 flagged it.
|
|
||||||
- **`shared::pubsub::NoopPubsubService`** is added for the executor-core integration test harness — every call returns `PubsubError::Unavailable`. Same pattern as the existing `NoopEventEmitter`. Clean.
|
|
||||||
- **The publish saturating-add for `trigger_depth`** ([pubsub_repo.rs:107](crates/manager-core/src/pubsub_repo.rs#L107)) means a publish from depth-`u32::MAX` won't panic. That's already capped by `PICLOUD_MAX_TRIGGER_DEPTH` (default 8) at the dispatcher, but defensive overflow handling is correct.
|
|
||||||
- **`shared/src/pubsub.rs` tests** include four named cases (exact, prefix wildcard, universal, validation) with subcases — clean test taxonomy.
|
|
||||||
|
|
||||||
## 6. Versioning audit
|
|
||||||
|
|
||||||
| File | Before | After | Status |
|
| File | Before | After | Status |
|
||||||
|---|---|---|---|
|
|---|---|---|---|
|
||||||
| Workspace `Cargo.toml` | 1.1.4 | 1.1.5 | ✅ |
|
| Workspace `Cargo.toml` | 1.1.6 | 1.1.7 | ✅ |
|
||||||
| SDK schema (`shared/src/version.rs`) | 1.5 | 1.6 | ✅ correctly bumped — `FilesService`, `PubsubService`, `FileMeta`, `NewFile`, `FileUpdate`, `topic_matches`, `validate_topic_pattern`, `TriggerEvent::{Files, Pubsub}` added to public surface |
|
| SDK schema (`shared/src/version.rs`) | 1.7 | 1.8 | ✅ correctly bumped — `SecretsService`, `EmailService`, `MasterKey`, `crypto::{encrypt, decrypt}`, `TriggerEvent::Email` added to public surface |
|
||||||
| Dashboard `package.json` | 0.10.0 | 0.11.0 | ✅ |
|
| Dashboard `package.json` | 0.12.0 | 0.13.0 | ✅ |
|
||||||
| Migrations | 0001..0017 | 0018..0020 added | ✅ sequential, no skips |
|
| Migrations | 0001..0022 | 0023..0025 added | ✅ sequential, no skips |
|
||||||
| CHANGELOG.md | v1.1.4 entry | v1.1.5 entry added | ✅ |
|
| CHANGELOG.md | v1.1.6 entry | v1.1.7 entry + retroactive dead_letter security note | ✅ Per prompt |
|
||||||
|
|
||||||
## 7. Recommended next steps (post-merge)
|
## 8. Recommended next steps (post-merge)
|
||||||
|
|
||||||
1. **Merge** `feat/v1.1.5-files-pubsub` into `main` (fast-forward; branch is linear ahead).
|
1. **Merge** `feat/v1.1.7-secrets-email` into `main` (fast-forward; branch is linear ahead).
|
||||||
2. **Pause** before dispatching v1.1.6 (Realtime Channels & Client Library — the co-shipped SSE + `@picloud/client` work).
|
2. **`docker compose down` when convenient** to tear down the dev Postgres container.
|
||||||
3. **For the v1.1.6 dispatch prompt**, consider folding in:
|
3. **Pause** before dispatching v1.1.8 (User Management).
|
||||||
- **Dispatcher end-to-end DB tests** for each trigger kind. This is broader than v1.1.5 — it's a workspace-wide hygiene task. Now that CI has a Postgres service (per v1.1.5's `.github/workflows/ci.yml`), gating these tests on `DATABASE_URL` lets them run in CI without breaking local `cargo test`. Cost is bounded; the goal is to catch dispatcher regressions before they surface as production trigger silence.
|
4. **For the v1.1.8 dispatch prompt**, fold in:
|
||||||
- **Empty-blob storage** — revisit whether `data: 0 bytes` should be a valid stored state (currently rejected as missing). Decide before v1.1.6 ships so the semantics across two releases stay consistent.
|
- **Drop the plaintext `realtime_signing_key` column** (the v1.1.7 phase-2 commitment). Pre-flight check: scan the column for any remaining non-NULL rows; if found, run the encryption migration before the drop migration. Add a CHANGELOG note that v1.1.8 requires v1.1.7 to have been applied first (no skipping versions).
|
||||||
- **Orphan file sweeper** — design + ship the simple `*.tmp.*` sweeper (defer the full DB-cross-check version to v1.3+). v1.1.6 is when the file storage will start to accumulate enough that operators notice.
|
- **Clippy --all-targets discipline refinement** (§3.3 finding). Require either a `cargo clean` before `cargo clippy --all-targets` OR explicit verification that test binaries are being checked. v1.1.6's silent regression shows the gate can produce false-green results under cargo's incremental cache. Specific recommendation: add a CI step that asserts the clippy run touched the test binaries (e.g. count `Checking` lines in the output and verify they include test crates).
|
||||||
4. **Awareness:** v1.1.5 is the first release where the CI workflow exists. If the project lands new contributors before v1.1.6, the workflow needs `secrets` review (none currently set) and possibly branch-protection rules pointing at the CI checks.
|
- **`auth_mode = 'session'` for realtime subscriber tokens** — v1.1.7's CHECK constraint on `topics.auth_mode` only allows `('public', 'token')`. v1.1.8 (users::*) needs to add `'session'` and a session-token validator alongside the existing HMAC validator behind the unchanged `RealtimeAuthority` trait.
|
||||||
|
- **Bounded e2e parallelism** — defer the workspace-wide harness refactor (shared pool per binary) until there's a dedicated test-infra release. Until then, CI just needs `--test-threads=2` or smaller for the picloud crate's e2e binaries.
|
||||||
|
5. **Awareness from §3.3**: the clippy regression in v1.1.6 was caught by v1.1.7's diligence, but every prior REVIEW.md from v1.1.1 onward should be re-checked if you want certainty that no test-only clippy warnings slipped through. The fix is forward-only — re-running clippy on v1.1.1 through v1.1.6 commits would just confirm the warnings were latent then too.
|
||||||
|
|
||||||
Branch is ready for merge. Verdict: **APPROVE**.
|
Branch is ready for merge. Verdict: **APPROVE**.
|
||||||
|
|||||||
3
clients/typescript/.gitignore
vendored
Normal file
3
clients/typescript/.gitignore
vendored
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
node_modules/
|
||||||
|
dist/
|
||||||
|
*.tsbuildinfo
|
||||||
111
clients/typescript/README.md
Normal file
111
clients/typescript/README.md
Normal file
@@ -0,0 +1,111 @@
|
|||||||
|
# @picloud/client
|
||||||
|
|
||||||
|
TypeScript client for [PiCloud](../../README.md). Three capabilities, all
|
||||||
|
**script-mediated** — there is no direct KV / docs / users access from the
|
||||||
|
browser (the hybrid model, by design):
|
||||||
|
|
||||||
|
1. **Typed HTTP** to dev-defined script endpoints.
|
||||||
|
2. **SSE realtime** subscriptions to externally-subscribable pub/sub topics.
|
||||||
|
3. **Auth-flow helpers** over your own dev-defined login/logout endpoints.
|
||||||
|
|
||||||
|
```ts
|
||||||
|
import { PicloudClient } from '@picloud/client';
|
||||||
|
|
||||||
|
const client = new PicloudClient({
|
||||||
|
baseURL: 'https://api.example.com',
|
||||||
|
getAuthToken: () => localStorage.getItem('auth_token')
|
||||||
|
});
|
||||||
|
|
||||||
|
// Typed HTTP
|
||||||
|
interface CreateUserReq { name: string; email?: string; role: string }
|
||||||
|
interface CreateUserRes { id: string; name: string; created_at: string }
|
||||||
|
const user = await client
|
||||||
|
.endpoint<CreateUserReq, CreateUserRes>('/api/users')
|
||||||
|
.post({ name: 'alice', role: 'admin' });
|
||||||
|
|
||||||
|
// SSE subscription
|
||||||
|
const unsubscribe = client.subscribe('chat-room-123', (event) => {
|
||||||
|
console.log('got event:', event.message);
|
||||||
|
});
|
||||||
|
unsubscribe();
|
||||||
|
|
||||||
|
// Token-gated topic (token obtained from one of YOUR script endpoints,
|
||||||
|
// which calls `pubsub::subscriber_token`)
|
||||||
|
client.subscribe('chat-room-123', cb, { token: 'eyJhbGc...' });
|
||||||
|
|
||||||
|
// Auth helpers (call dev-defined endpoints under the hood)
|
||||||
|
await client.auth.login('alice@example.com', 'password');
|
||||||
|
await client.auth.logout();
|
||||||
|
const token = client.auth.token;
|
||||||
|
```
|
||||||
|
|
||||||
|
## React
|
||||||
|
|
||||||
|
```tsx
|
||||||
|
import { PicloudProvider, useTopic, useEndpoint } from '@picloud/client/react';
|
||||||
|
|
||||||
|
// Wrap your tree once: <PicloudProvider client={client}>…</PicloudProvider>
|
||||||
|
|
||||||
|
function ChatRoom({ roomId }: { roomId: string }) {
|
||||||
|
const messages = useTopic<ChatMessage>(`chat-room-${roomId}`);
|
||||||
|
return <ul>{messages.map((m, i) => <li key={i}>{m.text}</li>)}</ul>;
|
||||||
|
}
|
||||||
|
|
||||||
|
function UserProfile({ id }: { id: string }) {
|
||||||
|
const { data, loading, error } = useEndpoint<UserRes>(`/api/users/${id}`).get();
|
||||||
|
if (loading) return <Spinner />;
|
||||||
|
if (error) return <ErrorView error={error} />;
|
||||||
|
return <div>{data?.name}</div>;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Svelte
|
||||||
|
|
||||||
|
```ts
|
||||||
|
import { topicStore, endpointStore } from '@picloud/client/svelte';
|
||||||
|
|
||||||
|
const messages = topicStore<ChatMessage>(client, `chat-room-${roomId}`);
|
||||||
|
// $messages is an array that grows as events arrive
|
||||||
|
|
||||||
|
const userQuery = endpointStore<UserRes>(client, `/api/users/${id}`).get();
|
||||||
|
// $userQuery is { data, loading, error }
|
||||||
|
```
|
||||||
|
|
||||||
|
> The Svelte helpers take the `client` explicitly (a store isn't a component,
|
||||||
|
> so there's no React-style context to read).
|
||||||
|
|
||||||
|
## Optional runtime validation (zod / valibot)
|
||||||
|
|
||||||
|
No hard dependency — the adapter is the `{ parse(input): T }` shape. A Zod
|
||||||
|
schema satisfies it directly; wrap Valibot in one line:
|
||||||
|
|
||||||
|
```ts
|
||||||
|
import { z } from 'zod';
|
||||||
|
const UserSchema = z.object({ id: z.string(), name: z.string() });
|
||||||
|
const user = await client.endpoint('/api/users/1').get({ validate: UserSchema });
|
||||||
|
|
||||||
|
// valibot:
|
||||||
|
import * as v from 'valibot';
|
||||||
|
const schema = v.object({ id: v.string() });
|
||||||
|
const adapter = { parse: (i: unknown) => v.parse(schema, i) };
|
||||||
|
```
|
||||||
|
|
||||||
|
## Transport notes
|
||||||
|
|
||||||
|
- SSE is implemented over streaming `fetch` (not native `EventSource`) so the
|
||||||
|
client can refresh an expired token on a 401, send `Last-Event-ID` on resume,
|
||||||
|
and apply its own exponential backoff (1s → 2s → 4s … capped at 30s).
|
||||||
|
- **React Native** has no native `EventSource`, but it also can't stream
|
||||||
|
`fetch` bodies on all engines — if you target RN, supply a streaming-capable
|
||||||
|
`fetch` polyfill via the `fetch` option, or use a `react-native-sse`-based
|
||||||
|
adapter. (Server-side `Last-Event-ID` replay is not implemented in v1.1.6;
|
||||||
|
the client sends the header so it's ready when the server adds replay.)
|
||||||
|
|
||||||
|
## Build / test
|
||||||
|
|
||||||
|
```sh
|
||||||
|
npm install
|
||||||
|
npm run lint # tsc --noEmit (strict)
|
||||||
|
npm run test # vitest
|
||||||
|
npm run build # tsup → dist/ (ESM + CJS + .d.ts)
|
||||||
|
```
|
||||||
3580
clients/typescript/package-lock.json
generated
Normal file
3580
clients/typescript/package-lock.json
generated
Normal file
File diff suppressed because it is too large
Load Diff
61
clients/typescript/package.json
Normal file
61
clients/typescript/package.json
Normal file
@@ -0,0 +1,61 @@
|
|||||||
|
{
|
||||||
|
"name": "@picloud/client",
|
||||||
|
"version": "1.0.0",
|
||||||
|
"description": "TypeScript client for PiCloud — typed HTTP to script endpoints, SSE realtime subscriptions, auth-flow helpers, and React/Svelte hooks.",
|
||||||
|
"license": "MIT OR Apache-2.0",
|
||||||
|
"type": "module",
|
||||||
|
"sideEffects": false,
|
||||||
|
"files": [
|
||||||
|
"dist"
|
||||||
|
],
|
||||||
|
"exports": {
|
||||||
|
".": {
|
||||||
|
"types": "./dist/index.d.ts",
|
||||||
|
"import": "./dist/index.js",
|
||||||
|
"require": "./dist/index.cjs"
|
||||||
|
},
|
||||||
|
"./react": {
|
||||||
|
"types": "./dist/react/index.d.ts",
|
||||||
|
"import": "./dist/react/index.js",
|
||||||
|
"require": "./dist/react/index.cjs"
|
||||||
|
},
|
||||||
|
"./svelte": {
|
||||||
|
"types": "./dist/svelte/index.d.ts",
|
||||||
|
"import": "./dist/svelte/index.js",
|
||||||
|
"require": "./dist/svelte/index.cjs"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"main": "./dist/index.cjs",
|
||||||
|
"module": "./dist/index.js",
|
||||||
|
"types": "./dist/index.d.ts",
|
||||||
|
"scripts": {
|
||||||
|
"build": "tsup",
|
||||||
|
"test": "vitest run",
|
||||||
|
"test:watch": "vitest",
|
||||||
|
"lint": "tsc --noEmit"
|
||||||
|
},
|
||||||
|
"peerDependencies": {
|
||||||
|
"react": ">=17",
|
||||||
|
"svelte": ">=4"
|
||||||
|
},
|
||||||
|
"peerDependenciesMeta": {
|
||||||
|
"react": {
|
||||||
|
"optional": true
|
||||||
|
},
|
||||||
|
"svelte": {
|
||||||
|
"optional": true
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"devDependencies": {
|
||||||
|
"@testing-library/dom": "^10.4.0",
|
||||||
|
"@testing-library/react": "^16.1.0",
|
||||||
|
"@types/react": "^18.3.0",
|
||||||
|
"jsdom": "^25.0.0",
|
||||||
|
"react": "^18.3.0",
|
||||||
|
"react-dom": "^18.3.0",
|
||||||
|
"svelte": "^4.2.0",
|
||||||
|
"tsup": "^8.3.0",
|
||||||
|
"typescript": "^5.6.0",
|
||||||
|
"vitest": "^2.1.0"
|
||||||
|
}
|
||||||
|
}
|
||||||
71
clients/typescript/src/auth.ts
Normal file
71
clients/typescript/src/auth.ts
Normal file
@@ -0,0 +1,71 @@
|
|||||||
|
import { Endpoint } from './endpoint.js';
|
||||||
|
import type { AuthTokenProvider } from './types.js';
|
||||||
|
|
||||||
|
export interface AuthClientConfig {
|
||||||
|
baseURL: string;
|
||||||
|
fetchImpl: typeof fetch;
|
||||||
|
/** Path of the dev-defined login endpoint (default `/api/auth/login`). */
|
||||||
|
loginPath?: string;
|
||||||
|
/** Path of the dev-defined logout endpoint (default `/api/auth/logout`). */
|
||||||
|
logoutPath?: string;
|
||||||
|
/** Called whenever the stored token changes (e.g. to persist it). */
|
||||||
|
onToken?: (token: string | null) => void;
|
||||||
|
}
|
||||||
|
|
||||||
|
interface LoginResponse {
|
||||||
|
token?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Auth-flow helpers. These call **dev-defined** endpoints under the hood
|
||||||
|
* (the script layer owns the actual auth); the lib only standardizes the
|
||||||
|
* dance + in-memory token storage. There is no built-in identity model —
|
||||||
|
* `login` POSTs credentials and stores whatever `token` comes back.
|
||||||
|
*/
|
||||||
|
export class AuthClient {
|
||||||
|
private current: string | null = null;
|
||||||
|
|
||||||
|
constructor(private readonly cfg: AuthClientConfig) {}
|
||||||
|
|
||||||
|
/** The current bearer token, or null. */
|
||||||
|
get token(): string | null {
|
||||||
|
return this.current;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Suitable as `PicloudClientOptions.getAuthToken`. */
|
||||||
|
readonly provider: AuthTokenProvider = () => this.current;
|
||||||
|
|
||||||
|
/** POST credentials to the login endpoint; store the returned token. */
|
||||||
|
async login(email: string, password: string): Promise<string | null> {
|
||||||
|
const ep = new Endpoint<{ email: string; password: string }, LoginResponse>({
|
||||||
|
baseURL: this.cfg.baseURL,
|
||||||
|
path: this.cfg.loginPath ?? '/api/auth/login',
|
||||||
|
fetchImpl: this.cfg.fetchImpl
|
||||||
|
});
|
||||||
|
const res = await ep.post({ email, password });
|
||||||
|
this.setToken(typeof res?.token === 'string' ? res.token : null);
|
||||||
|
return this.current;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** POST to the logout endpoint (best-effort) and clear the token. */
|
||||||
|
async logout(): Promise<void> {
|
||||||
|
const ep = new Endpoint<undefined, unknown>({
|
||||||
|
baseURL: this.cfg.baseURL,
|
||||||
|
path: this.cfg.logoutPath ?? '/api/auth/logout',
|
||||||
|
// Send the current token so the script can invalidate the session.
|
||||||
|
getAuthToken: () => this.current,
|
||||||
|
fetchImpl: this.cfg.fetchImpl
|
||||||
|
});
|
||||||
|
try {
|
||||||
|
await ep.post();
|
||||||
|
} finally {
|
||||||
|
this.setToken(null);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Manually set (or clear) the token — e.g. restoring from storage. */
|
||||||
|
setToken(token: string | null): void {
|
||||||
|
this.current = token;
|
||||||
|
this.cfg.onToken?.(token);
|
||||||
|
}
|
||||||
|
}
|
||||||
61
clients/typescript/src/client.ts
Normal file
61
clients/typescript/src/client.ts
Normal file
@@ -0,0 +1,61 @@
|
|||||||
|
import { AuthClient } from './auth.js';
|
||||||
|
import { Endpoint } from './endpoint.js';
|
||||||
|
import { subscribeTopic } from './subscribe.js';
|
||||||
|
import type {
|
||||||
|
PicloudClientOptions,
|
||||||
|
RealtimeEvent,
|
||||||
|
SubscribeOptions,
|
||||||
|
Unsubscribe
|
||||||
|
} from './types.js';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The PiCloud frontend client. Three capabilities, all script-mediated
|
||||||
|
* (the hybrid model — no direct KV/docs/users access from the browser):
|
||||||
|
*
|
||||||
|
* - `endpoint<Req, Res>(path)` — typed HTTP to a dev-defined route.
|
||||||
|
* - `subscribe(topic, cb, opts?)` — SSE realtime subscription.
|
||||||
|
* - `auth` — login/logout/token helpers over dev-defined endpoints.
|
||||||
|
*/
|
||||||
|
export class PicloudClient {
|
||||||
|
readonly auth: AuthClient;
|
||||||
|
private readonly baseURL: string;
|
||||||
|
private readonly fetchImpl: typeof fetch;
|
||||||
|
private readonly getAuthToken: PicloudClientOptions['getAuthToken'];
|
||||||
|
|
||||||
|
constructor(opts: PicloudClientOptions) {
|
||||||
|
if (!opts.baseURL) throw new Error('PicloudClient: baseURL is required');
|
||||||
|
this.baseURL = opts.baseURL;
|
||||||
|
const f = opts.fetch ?? globalThis.fetch;
|
||||||
|
if (typeof f !== 'function') {
|
||||||
|
throw new Error('PicloudClient: no fetch available — pass options.fetch');
|
||||||
|
}
|
||||||
|
// Bind to avoid "Illegal invocation" when calling a detached global.
|
||||||
|
this.fetchImpl = f.bind(globalThis);
|
||||||
|
this.getAuthToken = opts.getAuthToken;
|
||||||
|
this.auth = new AuthClient({ baseURL: this.baseURL, fetchImpl: this.fetchImpl });
|
||||||
|
}
|
||||||
|
|
||||||
|
/** A typed handle to a dev-defined endpoint. */
|
||||||
|
endpoint<Req = unknown, Res = unknown>(path: string): Endpoint<Req, Res> {
|
||||||
|
return new Endpoint<Req, Res>({
|
||||||
|
baseURL: this.baseURL,
|
||||||
|
path,
|
||||||
|
getAuthToken: this.getAuthToken,
|
||||||
|
fetchImpl: this.fetchImpl
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Subscribe to a realtime topic. Returns an unsubscribe function. */
|
||||||
|
subscribe<T = unknown>(
|
||||||
|
topic: string,
|
||||||
|
onMessage: (event: RealtimeEvent<T>) => void,
|
||||||
|
opts?: SubscribeOptions<T>
|
||||||
|
): Unsubscribe {
|
||||||
|
return subscribeTopic<T>(
|
||||||
|
{ baseURL: this.baseURL, fetchImpl: this.fetchImpl },
|
||||||
|
topic,
|
||||||
|
onMessage,
|
||||||
|
opts
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
106
clients/typescript/src/endpoint.ts
Normal file
106
clients/typescript/src/endpoint.ts
Normal file
@@ -0,0 +1,106 @@
|
|||||||
|
import { PicloudHttpError, type AuthTokenProvider, type Validator } from './types.js';
|
||||||
|
|
||||||
|
type Method = 'GET' | 'POST' | 'PUT' | 'PATCH' | 'DELETE';
|
||||||
|
|
||||||
|
export interface EndpointConfig {
|
||||||
|
baseURL: string;
|
||||||
|
path: string;
|
||||||
|
getAuthToken?: AuthTokenProvider;
|
||||||
|
fetchImpl: typeof fetch;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface RequestOptions<Res> {
|
||||||
|
/** Extra headers merged over the defaults. */
|
||||||
|
headers?: Record<string, string>;
|
||||||
|
/** Optional runtime validation of the parsed response. */
|
||||||
|
validate?: Validator<Res>;
|
||||||
|
/** AbortSignal to cancel the request. */
|
||||||
|
signal?: AbortSignal;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Typed HTTP to a dev-defined script endpoint. Auth header injection +
|
||||||
|
* structured errors; the request/response types are caller-supplied
|
||||||
|
* generics (`endpoint<Req, Res>('/path')`). No service access — every
|
||||||
|
* call hits a route a script binds (the hybrid model).
|
||||||
|
*/
|
||||||
|
export class Endpoint<Req = unknown, Res = unknown> {
|
||||||
|
constructor(private readonly cfg: EndpointConfig) {}
|
||||||
|
|
||||||
|
get(opts?: RequestOptions<Res>): Promise<Res> {
|
||||||
|
return this.send('GET', undefined, opts);
|
||||||
|
}
|
||||||
|
|
||||||
|
post(body?: Req, opts?: RequestOptions<Res>): Promise<Res> {
|
||||||
|
return this.send('POST', body, opts);
|
||||||
|
}
|
||||||
|
|
||||||
|
put(body?: Req, opts?: RequestOptions<Res>): Promise<Res> {
|
||||||
|
return this.send('PUT', body, opts);
|
||||||
|
}
|
||||||
|
|
||||||
|
patch(body?: Req, opts?: RequestOptions<Res>): Promise<Res> {
|
||||||
|
return this.send('PATCH', body, opts);
|
||||||
|
}
|
||||||
|
|
||||||
|
delete(opts?: RequestOptions<Res>): Promise<Res> {
|
||||||
|
return this.send('DELETE', undefined, opts);
|
||||||
|
}
|
||||||
|
|
||||||
|
private async send(method: Method, body: Req | undefined, opts?: RequestOptions<Res>): Promise<Res> {
|
||||||
|
const headers: Record<string, string> = {
|
||||||
|
Accept: 'application/json',
|
||||||
|
...(opts?.headers ?? {})
|
||||||
|
};
|
||||||
|
if (body !== undefined) {
|
||||||
|
headers['Content-Type'] ??= 'application/json';
|
||||||
|
}
|
||||||
|
const token = this.cfg.getAuthToken ? await this.cfg.getAuthToken() : undefined;
|
||||||
|
if (token) {
|
||||||
|
headers['Authorization'] ??= `Bearer ${token}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
const url = joinUrl(this.cfg.baseURL, this.cfg.path);
|
||||||
|
const init: RequestInit = { method, headers };
|
||||||
|
if (body !== undefined) {
|
||||||
|
init.body = JSON.stringify(body);
|
||||||
|
}
|
||||||
|
if (opts?.signal) {
|
||||||
|
init.signal = opts.signal;
|
||||||
|
}
|
||||||
|
|
||||||
|
const res = await this.cfg.fetchImpl(url, init);
|
||||||
|
const parsed = await parseBody(res);
|
||||||
|
if (!res.ok) {
|
||||||
|
const message =
|
||||||
|
(isRecord(parsed) && typeof parsed['error'] === 'string' && parsed['error']) ||
|
||||||
|
`${method} ${this.cfg.path} failed with ${res.status}`;
|
||||||
|
throw new PicloudHttpError(res.status, message, parsed);
|
||||||
|
}
|
||||||
|
return opts?.validate ? opts.validate.parse(parsed) : (parsed as Res);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function parseBody(res: Response): Promise<unknown> {
|
||||||
|
const text = await res.text();
|
||||||
|
if (text.length === 0) return null;
|
||||||
|
const ct = res.headers.get('content-type') ?? '';
|
||||||
|
if (ct.includes('application/json')) {
|
||||||
|
try {
|
||||||
|
return JSON.parse(text);
|
||||||
|
} catch {
|
||||||
|
return text;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return text;
|
||||||
|
}
|
||||||
|
|
||||||
|
function isRecord(v: unknown): v is Record<string, unknown> {
|
||||||
|
return typeof v === 'object' && v !== null;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function joinUrl(base: string, path: string): string {
|
||||||
|
const b = base.endsWith('/') ? base.slice(0, -1) : base;
|
||||||
|
const p = path.startsWith('/') ? path : `/${path}`;
|
||||||
|
return `${b}${p}`;
|
||||||
|
}
|
||||||
14
clients/typescript/src/index.ts
Normal file
14
clients/typescript/src/index.ts
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
export { PicloudClient } from './client.js';
|
||||||
|
export { Endpoint } from './endpoint.js';
|
||||||
|
export { AuthClient } from './auth.js';
|
||||||
|
export { subscribeTopic } from './subscribe.js';
|
||||||
|
export {
|
||||||
|
PicloudHttpError,
|
||||||
|
type PicloudClientOptions,
|
||||||
|
type AuthTokenProvider,
|
||||||
|
type RealtimeEvent,
|
||||||
|
type SubscribeOptions,
|
||||||
|
type Unsubscribe,
|
||||||
|
type Validator
|
||||||
|
} from './types.js';
|
||||||
|
export type { RequestOptions } from './endpoint.js';
|
||||||
101
clients/typescript/src/react/index.ts
Normal file
101
clients/typescript/src/react/index.ts
Normal file
@@ -0,0 +1,101 @@
|
|||||||
|
import {
|
||||||
|
createContext,
|
||||||
|
createElement,
|
||||||
|
useContext,
|
||||||
|
useEffect,
|
||||||
|
useState,
|
||||||
|
type ReactNode
|
||||||
|
} from 'react';
|
||||||
|
|
||||||
|
import type { PicloudClient } from '../client.js';
|
||||||
|
import type { SubscribeOptions } from '../types.js';
|
||||||
|
|
||||||
|
const PicloudContext = createContext<PicloudClient | null>(null);
|
||||||
|
|
||||||
|
export interface PicloudProviderProps {
|
||||||
|
client: PicloudClient;
|
||||||
|
children?: ReactNode;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Provides a `PicloudClient` to `useTopic` / `useEndpoint`. */
|
||||||
|
export function PicloudProvider(props: PicloudProviderProps) {
|
||||||
|
return createElement(PicloudContext.Provider, { value: props.client }, props.children);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** The client from the nearest `PicloudProvider`. Throws if absent. */
|
||||||
|
export function usePicloud(): PicloudClient {
|
||||||
|
const client = useContext(PicloudContext);
|
||||||
|
if (!client) {
|
||||||
|
throw new Error('usePicloud: wrap your tree in <PicloudProvider client={...}>');
|
||||||
|
}
|
||||||
|
return client;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Subscribe to a realtime topic; returns the accumulated messages in
|
||||||
|
* arrival order. Re-subscribes when `topic` changes; unsubscribes on
|
||||||
|
* unmount.
|
||||||
|
*/
|
||||||
|
export function useTopic<T = unknown>(topic: string, opts?: SubscribeOptions<T>): T[] {
|
||||||
|
const client = usePicloud();
|
||||||
|
const [messages, setMessages] = useState<T[]>([]);
|
||||||
|
useEffect(() => {
|
||||||
|
setMessages([]);
|
||||||
|
const unsubscribe = client.subscribe<T>(
|
||||||
|
topic,
|
||||||
|
(event) => setMessages((prev) => [...prev, event.message]),
|
||||||
|
opts
|
||||||
|
);
|
||||||
|
return () => unsubscribe();
|
||||||
|
// `opts` is intentionally excluded: a new object literal each render
|
||||||
|
// would otherwise resubscribe every render.
|
||||||
|
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||||
|
}, [client, topic]);
|
||||||
|
return messages;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface QueryState<T> {
|
||||||
|
data: T | null;
|
||||||
|
loading: boolean;
|
||||||
|
error: unknown;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface EndpointHook<Req, Res> {
|
||||||
|
get: () => QueryState<Res>;
|
||||||
|
post: (body?: Req) => QueryState<Res>;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Typed endpoint hook. `useEndpoint<Res>(path).get()` fires a GET and
|
||||||
|
* returns `{ data, loading, error }`, re-running when `path` changes.
|
||||||
|
* `.post(body)` is the mutation variant (auto-fires once per mount).
|
||||||
|
*/
|
||||||
|
export function useEndpoint<Res = unknown, Req = unknown>(path: string): EndpointHook<Req, Res> {
|
||||||
|
const client = usePicloud();
|
||||||
|
return {
|
||||||
|
get: () => useResource<Res>(() => client.endpoint<Req, Res>(path).get(), path, 'GET'),
|
||||||
|
post: (body?: Req) =>
|
||||||
|
useResource<Res>(() => client.endpoint<Req, Res>(path).post(body), path, 'POST')
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function useResource<Res>(run: () => Promise<Res>, key: string, method: string): QueryState<Res> {
|
||||||
|
const [state, setState] = useState<QueryState<Res>>({
|
||||||
|
data: null,
|
||||||
|
loading: true,
|
||||||
|
error: null
|
||||||
|
});
|
||||||
|
useEffect(() => {
|
||||||
|
let active = true;
|
||||||
|
setState({ data: null, loading: true, error: null });
|
||||||
|
run()
|
||||||
|
.then((data) => active && setState({ data, loading: false, error: null }))
|
||||||
|
.catch((error) => active && setState({ data: null, loading: false, error }));
|
||||||
|
return () => {
|
||||||
|
active = false;
|
||||||
|
};
|
||||||
|
// `run` is recreated each render; key it on path + method instead.
|
||||||
|
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||||
|
}, [key, method]);
|
||||||
|
return state;
|
||||||
|
}
|
||||||
194
clients/typescript/src/subscribe.ts
Normal file
194
clients/typescript/src/subscribe.ts
Normal file
@@ -0,0 +1,194 @@
|
|||||||
|
import { joinUrl } from './endpoint.js';
|
||||||
|
import type { RealtimeEvent, SubscribeOptions, Unsubscribe } from './types.js';
|
||||||
|
|
||||||
|
interface SubscribeConfig {
|
||||||
|
baseURL: string;
|
||||||
|
fetchImpl: typeof fetch;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Subscribe to an app pub/sub topic over SSE.
|
||||||
|
*
|
||||||
|
* Implemented over streaming `fetch` (not native `EventSource`) so the
|
||||||
|
* lib can: detect a 401 on (re)connect and refresh the token, send a
|
||||||
|
* `Last-Event-ID` header on resume, and apply its own exponential
|
||||||
|
* backoff. See HANDBACK for the rationale. Returns an unsubscribe
|
||||||
|
* function that aborts the connection and stops reconnecting.
|
||||||
|
*/
|
||||||
|
export function subscribeTopic<T = unknown>(
|
||||||
|
cfg: SubscribeConfig,
|
||||||
|
topic: string,
|
||||||
|
onMessage: (event: RealtimeEvent<T>) => void,
|
||||||
|
opts: SubscribeOptions<T> = {}
|
||||||
|
): Unsubscribe {
|
||||||
|
const baseBackoff = opts.baseBackoffMs ?? 1_000;
|
||||||
|
const maxBackoff = opts.maxBackoffMs ?? 30_000;
|
||||||
|
let token = opts.token;
|
||||||
|
let stopped = false;
|
||||||
|
let attempt = 0;
|
||||||
|
let lastEventId: string | undefined;
|
||||||
|
let controller: AbortController | null = null;
|
||||||
|
let backoffTimer: ReturnType<typeof setTimeout> | null = null;
|
||||||
|
|
||||||
|
const stop = () => {
|
||||||
|
stopped = true;
|
||||||
|
if (backoffTimer) clearTimeout(backoffTimer);
|
||||||
|
controller?.abort();
|
||||||
|
};
|
||||||
|
|
||||||
|
const scheduleReconnect = () => {
|
||||||
|
if (stopped) return;
|
||||||
|
// Exponential backoff: base, 2x, 4x… capped at maxBackoff.
|
||||||
|
const delay = Math.min(maxBackoff, baseBackoff * 2 ** attempt);
|
||||||
|
attempt += 1;
|
||||||
|
backoffTimer = setTimeout(() => void connect(), delay);
|
||||||
|
};
|
||||||
|
|
||||||
|
const connect = async (): Promise<void> => {
|
||||||
|
if (stopped) return;
|
||||||
|
controller = new AbortController();
|
||||||
|
const url = buildUrl(cfg.baseURL, topic, token);
|
||||||
|
const headers: Record<string, string> = { Accept: 'text/event-stream' };
|
||||||
|
if (lastEventId) headers['Last-Event-ID'] = lastEventId;
|
||||||
|
|
||||||
|
let res: Response;
|
||||||
|
try {
|
||||||
|
res = await cfg.fetchImpl(url, { headers, signal: controller.signal });
|
||||||
|
} catch (err) {
|
||||||
|
if (stopped || isAbort(err)) return;
|
||||||
|
scheduleReconnect();
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (res.status === 401) {
|
||||||
|
// Token expired / rejected — try to refresh, else give up.
|
||||||
|
const fresh = opts.onTokenExpired ? await opts.onTokenExpired() : null;
|
||||||
|
if (fresh) {
|
||||||
|
token = fresh;
|
||||||
|
attempt = 0; // fresh credential → reconnect immediately
|
||||||
|
void connect();
|
||||||
|
} else {
|
||||||
|
opts.onError?.(new Error('realtime subscribe unauthorized (401)'));
|
||||||
|
stop();
|
||||||
|
}
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!res.ok || !res.body) {
|
||||||
|
if (!stopped) scheduleReconnect();
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Connected — reset backoff and stream frames until the body ends.
|
||||||
|
attempt = 0;
|
||||||
|
try {
|
||||||
|
await readStream(res.body, (frame) => {
|
||||||
|
if (frame.id !== undefined) lastEventId = frame.id;
|
||||||
|
if (frame.data === undefined) return; // comment / heartbeat
|
||||||
|
const parsed = parseEvent<T>(frame.data, opts);
|
||||||
|
if (parsed) onMessage(parsed);
|
||||||
|
});
|
||||||
|
} catch (err) {
|
||||||
|
if (stopped || isAbort(err)) return;
|
||||||
|
}
|
||||||
|
// Stream ended (server closed, e.g. topic deleted) → reconnect.
|
||||||
|
if (!stopped) scheduleReconnect();
|
||||||
|
};
|
||||||
|
|
||||||
|
void connect();
|
||||||
|
return stop;
|
||||||
|
}
|
||||||
|
|
||||||
|
function buildUrl(baseURL: string, topic: string, token?: string): string {
|
||||||
|
const url = joinUrl(baseURL, `/realtime/topics/${encodeURIComponent(topic)}`);
|
||||||
|
// EventSource can't set headers, so the token rides in the query
|
||||||
|
// string — the same path a raw EventSource would use.
|
||||||
|
return token ? `${url}?token=${encodeURIComponent(token)}` : url;
|
||||||
|
}
|
||||||
|
|
||||||
|
function parseEvent<T>(data: string, opts: SubscribeOptions<T>): RealtimeEvent<T> | null {
|
||||||
|
let json: unknown;
|
||||||
|
try {
|
||||||
|
json = JSON.parse(data);
|
||||||
|
} catch {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
if (!isRealtimeShape(json)) return null;
|
||||||
|
const message = opts.validate ? opts.validate.parse(json.message) : (json.message as T);
|
||||||
|
return { topic: json.topic, message, published_at: json.published_at };
|
||||||
|
}
|
||||||
|
|
||||||
|
function isRealtimeShape(v: unknown): v is RealtimeEvent<unknown> {
|
||||||
|
return (
|
||||||
|
typeof v === 'object' &&
|
||||||
|
v !== null &&
|
||||||
|
typeof (v as Record<string, unknown>)['topic'] === 'string' &&
|
||||||
|
typeof (v as Record<string, unknown>)['published_at'] === 'string' &&
|
||||||
|
'message' in (v as Record<string, unknown>)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
interface SseFrame {
|
||||||
|
data?: string;
|
||||||
|
id?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Read an SSE response body, invoking `onFrame` per event. Minimal
|
||||||
|
* parser: accumulates `data:` lines (joined by `\n`) and `id:` until a
|
||||||
|
* blank line dispatches the frame. Lines starting with `:` are comments
|
||||||
|
* (heartbeats) — surfaced as a frame with no `data` so the id can still
|
||||||
|
* advance.
|
||||||
|
*/
|
||||||
|
async function readStream(
|
||||||
|
body: ReadableStream<Uint8Array>,
|
||||||
|
onFrame: (frame: SseFrame) => void
|
||||||
|
): Promise<void> {
|
||||||
|
const reader = body.getReader();
|
||||||
|
const decoder = new TextDecoder();
|
||||||
|
let buffer = '';
|
||||||
|
let dataLines: string[] = [];
|
||||||
|
let id: string | undefined;
|
||||||
|
let sawComment = false;
|
||||||
|
|
||||||
|
const dispatch = () => {
|
||||||
|
if (dataLines.length > 0) {
|
||||||
|
onFrame({ data: dataLines.join('\n'), id });
|
||||||
|
} else if (sawComment) {
|
||||||
|
onFrame({ id });
|
||||||
|
}
|
||||||
|
dataLines = [];
|
||||||
|
sawComment = false;
|
||||||
|
};
|
||||||
|
|
||||||
|
for (;;) {
|
||||||
|
const { value, done } = await reader.read();
|
||||||
|
if (done) break;
|
||||||
|
buffer += decoder.decode(value, { stream: true });
|
||||||
|
let nl: number;
|
||||||
|
while ((nl = buffer.indexOf('\n')) >= 0) {
|
||||||
|
const line = buffer.slice(0, nl).replace(/\r$/, '');
|
||||||
|
buffer = buffer.slice(nl + 1);
|
||||||
|
if (line === '') {
|
||||||
|
dispatch();
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (line.startsWith(':')) {
|
||||||
|
sawComment = true;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
const colon = line.indexOf(':');
|
||||||
|
const field = colon === -1 ? line : line.slice(0, colon);
|
||||||
|
const rawVal = colon === -1 ? '' : line.slice(colon + 1);
|
||||||
|
const val = rawVal.startsWith(' ') ? rawVal.slice(1) : rawVal;
|
||||||
|
if (field === 'data') dataLines.push(val);
|
||||||
|
else if (field === 'id') id = val;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Flush a trailing frame if the stream ended without a blank line.
|
||||||
|
dispatch();
|
||||||
|
}
|
||||||
|
|
||||||
|
function isAbort(err: unknown): boolean {
|
||||||
|
return typeof err === 'object' && err !== null && (err as { name?: string }).name === 'AbortError';
|
||||||
|
}
|
||||||
72
clients/typescript/src/svelte/index.ts
Normal file
72
clients/typescript/src/svelte/index.ts
Normal file
@@ -0,0 +1,72 @@
|
|||||||
|
import { readable, type Readable } from 'svelte/store';
|
||||||
|
|
||||||
|
import type { PicloudClient } from '../client.js';
|
||||||
|
import type { SubscribeOptions } from '../types.js';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A Svelte store of realtime messages for a topic. `$messages` is an
|
||||||
|
* array that grows as events arrive. The SSE connection opens on the
|
||||||
|
* first subscriber and closes when the last unsubscribes (standard
|
||||||
|
* `readable` lifecycle).
|
||||||
|
*
|
||||||
|
* The client is passed explicitly (Svelte stores aren't components, so
|
||||||
|
* there's no React-style context to read). See HANDBACK §7.
|
||||||
|
*/
|
||||||
|
export function topicStore<T = unknown>(
|
||||||
|
client: PicloudClient,
|
||||||
|
topic: string,
|
||||||
|
opts?: SubscribeOptions<T>
|
||||||
|
): Readable<T[]> {
|
||||||
|
return readable<T[]>([], (set) => {
|
||||||
|
let items: T[] = [];
|
||||||
|
const unsubscribe = client.subscribe<T>(
|
||||||
|
topic,
|
||||||
|
(event) => {
|
||||||
|
items = [...items, event.message];
|
||||||
|
set(items);
|
||||||
|
},
|
||||||
|
opts
|
||||||
|
);
|
||||||
|
return () => unsubscribe();
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface QueryState<T> {
|
||||||
|
data: T | null;
|
||||||
|
loading: boolean;
|
||||||
|
error: unknown;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface EndpointStore<Req, Res> {
|
||||||
|
get: () => Readable<QueryState<Res>>;
|
||||||
|
post: (body?: Req) => Readable<QueryState<Res>>;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A Svelte store wrapper over a typed endpoint. `$query` is
|
||||||
|
* `{ data, loading, error }`. The request fires when the store gains its
|
||||||
|
* first subscriber.
|
||||||
|
*/
|
||||||
|
export function endpointStore<Res = unknown, Req = unknown>(
|
||||||
|
client: PicloudClient,
|
||||||
|
path: string
|
||||||
|
): EndpointStore<Req, Res> {
|
||||||
|
const run = (exec: () => Promise<Res>): Readable<QueryState<Res>> =>
|
||||||
|
readable<QueryState<Res>>({ data: null, loading: true, error: null }, (set) => {
|
||||||
|
let active = true;
|
||||||
|
exec()
|
||||||
|
.then((data) => {
|
||||||
|
if (active) set({ data, loading: false, error: null });
|
||||||
|
})
|
||||||
|
.catch((error) => {
|
||||||
|
if (active) set({ data: null, loading: false, error });
|
||||||
|
});
|
||||||
|
return () => {
|
||||||
|
active = false;
|
||||||
|
};
|
||||||
|
});
|
||||||
|
return {
|
||||||
|
get: () => run(() => client.endpoint<Req, Res>(path).get()),
|
||||||
|
post: (body?: Req) => run(() => client.endpoint<Req, Res>(path).post(body))
|
||||||
|
};
|
||||||
|
}
|
||||||
73
clients/typescript/src/types.ts
Normal file
73
clients/typescript/src/types.ts
Normal file
@@ -0,0 +1,73 @@
|
|||||||
|
// Shared types for @picloud/client.
|
||||||
|
|
||||||
|
/** Returns the current bearer token (or null) before each HTTP request. */
|
||||||
|
export type AuthTokenProvider = () => string | null | undefined | Promise<string | null | undefined>;
|
||||||
|
|
||||||
|
export interface PicloudClientOptions {
|
||||||
|
/** Base URL of the PiCloud deployment, e.g. `https://api.example.com`. */
|
||||||
|
baseURL: string;
|
||||||
|
/**
|
||||||
|
* Optional: returns the current bearer token, called before each
|
||||||
|
* request. The client doesn't manage tokens — it just sends them.
|
||||||
|
*/
|
||||||
|
getAuthToken?: AuthTokenProvider;
|
||||||
|
/**
|
||||||
|
* Optional fetch implementation (defaults to the global `fetch`).
|
||||||
|
* Injected mainly for tests / non-browser runtimes.
|
||||||
|
*/
|
||||||
|
fetch?: typeof fetch;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** A realtime event as delivered over SSE. */
|
||||||
|
export interface RealtimeEvent<T = unknown> {
|
||||||
|
topic: string;
|
||||||
|
message: T;
|
||||||
|
published_at: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Minimal validator shape for the optional runtime-validation adapter.
|
||||||
|
* A Zod schema satisfies this directly (`schema.parse`); for Valibot,
|
||||||
|
* wrap it: `{ parse: (i) => v.parse(schema, i) }`. No hard dep on either.
|
||||||
|
*/
|
||||||
|
export interface Validator<T> {
|
||||||
|
parse: (input: unknown) => T;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Thrown when an endpoint call returns a non-2xx status. */
|
||||||
|
export class PicloudHttpError extends Error {
|
||||||
|
readonly status: number;
|
||||||
|
readonly body: unknown;
|
||||||
|
constructor(status: number, message: string, body: unknown) {
|
||||||
|
super(message);
|
||||||
|
this.name = 'PicloudHttpError';
|
||||||
|
this.status = status;
|
||||||
|
this.body = body;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface SubscribeOptions<T = unknown> {
|
||||||
|
/**
|
||||||
|
* Subscriber token for `auth_mode = 'token'` topics. Obtained from one
|
||||||
|
* of your app's script endpoints (which calls
|
||||||
|
* `pubsub::subscriber_token`). Sent as `?token=` (EventSource-parity).
|
||||||
|
*/
|
||||||
|
token?: string;
|
||||||
|
/**
|
||||||
|
* Called when a (re)connect is rejected with 401 — typically an
|
||||||
|
* expired token. Return a fresh token to retry immediately, or
|
||||||
|
* null/undefined to stop and surface the error.
|
||||||
|
*/
|
||||||
|
onTokenExpired?: () => string | null | undefined | Promise<string | null | undefined>;
|
||||||
|
/** Called on a terminal error (after retries are exhausted or aborted). */
|
||||||
|
onError?: (err: unknown) => void;
|
||||||
|
/** Optional runtime validation of each event's `message`. */
|
||||||
|
validate?: Validator<T>;
|
||||||
|
/** Max reconnect backoff in ms (default 30_000). */
|
||||||
|
maxBackoffMs?: number;
|
||||||
|
/** Base reconnect backoff in ms (default 1_000). */
|
||||||
|
baseBackoffMs?: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Cancels a realtime subscription. */
|
||||||
|
export type Unsubscribe = () => void;
|
||||||
41
clients/typescript/tests/auth.test.ts
Normal file
41
clients/typescript/tests/auth.test.ts
Normal file
@@ -0,0 +1,41 @@
|
|||||||
|
import { describe, expect, it, vi } from 'vitest';
|
||||||
|
|
||||||
|
import { PicloudClient } from '../src/index.js';
|
||||||
|
import { jsonResponse, lastUrl, type FetchArgs } from './helpers.js';
|
||||||
|
|
||||||
|
describe('auth', () => {
|
||||||
|
it('login POSTs credentials and stores the returned token', async () => {
|
||||||
|
const fetchMock = vi.fn(async (_u: FetchArgs[0], _i?: FetchArgs[1]) =>
|
||||||
|
jsonResponse({ token: 'session-abc' })
|
||||||
|
);
|
||||||
|
const client = new PicloudClient({ baseURL: 'https://api.test', fetch: fetchMock });
|
||||||
|
|
||||||
|
const token = await client.auth.login('alice@example.com', 'pw');
|
||||||
|
expect(token).toBe('session-abc');
|
||||||
|
expect(client.auth.token).toBe('session-abc');
|
||||||
|
expect(lastUrl(fetchMock)).toBe('https://api.test/api/auth/login');
|
||||||
|
|
||||||
|
const init = fetchMock.mock.calls[0]?.[1];
|
||||||
|
expect(JSON.parse(String(init?.body))).toEqual({
|
||||||
|
email: 'alice@example.com',
|
||||||
|
password: 'pw'
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('logout clears the stored token', async () => {
|
||||||
|
const fetchMock = vi.fn(async (_u: FetchArgs[0], _i?: FetchArgs[1]) => jsonResponse({}));
|
||||||
|
const client = new PicloudClient({ baseURL: 'https://api.test', fetch: fetchMock });
|
||||||
|
client.auth.setToken('existing');
|
||||||
|
await client.auth.logout();
|
||||||
|
expect(client.auth.token).toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('provider returns the current token for getAuthToken wiring', async () => {
|
||||||
|
const fetchMock = vi.fn(async (_u: FetchArgs[0], _i?: FetchArgs[1]) =>
|
||||||
|
jsonResponse({ token: 't' })
|
||||||
|
);
|
||||||
|
const client = new PicloudClient({ baseURL: 'https://api.test', fetch: fetchMock });
|
||||||
|
await client.auth.login('a@b.c', 'pw');
|
||||||
|
expect(client.auth.provider()).toBe('t');
|
||||||
|
});
|
||||||
|
});
|
||||||
82
clients/typescript/tests/endpoint.test.ts
Normal file
82
clients/typescript/tests/endpoint.test.ts
Normal file
@@ -0,0 +1,82 @@
|
|||||||
|
import { describe, expect, it, vi } from 'vitest';
|
||||||
|
|
||||||
|
import { PicloudClient, PicloudHttpError } from '../src/index.js';
|
||||||
|
import { headerOf, jsonResponse, lastInit, lastUrl, type FetchArgs } from './helpers.js';
|
||||||
|
|
||||||
|
describe('endpoint', () => {
|
||||||
|
it('post round-trips a typed request/response', async () => {
|
||||||
|
const fetchMock = vi.fn(async (_u: FetchArgs[0], _i?: FetchArgs[1]) =>
|
||||||
|
jsonResponse({ id: '1', name: 'alice', created_at: 'now' }, 201)
|
||||||
|
);
|
||||||
|
const client = new PicloudClient({ baseURL: 'https://api.test', fetch: fetchMock });
|
||||||
|
|
||||||
|
interface Req {
|
||||||
|
name: string;
|
||||||
|
role: string;
|
||||||
|
}
|
||||||
|
interface Res {
|
||||||
|
id: string;
|
||||||
|
name: string;
|
||||||
|
created_at: string;
|
||||||
|
}
|
||||||
|
const res = await client.endpoint<Req, Res>('/api/users').post({ name: 'alice', role: 'admin' });
|
||||||
|
|
||||||
|
expect(res).toEqual({ id: '1', name: 'alice', created_at: 'now' });
|
||||||
|
expect(lastUrl(fetchMock)).toBe('https://api.test/api/users');
|
||||||
|
const init = lastInit(fetchMock);
|
||||||
|
expect(init.method).toBe('POST');
|
||||||
|
expect(JSON.parse(String(init.body))).toEqual({ name: 'alice', role: 'admin' });
|
||||||
|
expect(headerOf(init, 'Content-Type')).toBe('application/json');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('get round-trips', async () => {
|
||||||
|
const fetchMock = vi.fn(async (_u: FetchArgs[0], _i?: FetchArgs[1]) =>
|
||||||
|
jsonResponse({ name: 'bob' })
|
||||||
|
);
|
||||||
|
const client = new PicloudClient({ baseURL: 'https://api.test', fetch: fetchMock });
|
||||||
|
const res = await client.endpoint<unknown, { name: string }>('/api/users/1').get();
|
||||||
|
expect(res.name).toBe('bob');
|
||||||
|
expect(lastInit(fetchMock).method).toBe('GET');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('injects the auth token from getAuthToken', async () => {
|
||||||
|
const fetchMock = vi.fn(async (_u: FetchArgs[0], _i?: FetchArgs[1]) => jsonResponse({ ok: true }));
|
||||||
|
const client = new PicloudClient({
|
||||||
|
baseURL: 'https://api.test',
|
||||||
|
fetch: fetchMock,
|
||||||
|
getAuthToken: () => 'tok-123'
|
||||||
|
});
|
||||||
|
await client.endpoint('/api/me').get();
|
||||||
|
expect(headerOf(lastInit(fetchMock), 'Authorization')).toBe('Bearer tok-123');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('throws PicloudHttpError with status + body on non-2xx', async () => {
|
||||||
|
const fetchMock = vi.fn(async (_u: FetchArgs[0], _i?: FetchArgs[1]) =>
|
||||||
|
jsonResponse({ error: 'bad input' }, 422)
|
||||||
|
);
|
||||||
|
const client = new PicloudClient({ baseURL: 'https://api.test', fetch: fetchMock });
|
||||||
|
const err = await client
|
||||||
|
.endpoint('/api/x')
|
||||||
|
.get()
|
||||||
|
.catch((e: unknown) => e);
|
||||||
|
expect(err).toBeInstanceOf(PicloudHttpError);
|
||||||
|
expect((err as PicloudHttpError).status).toBe(422);
|
||||||
|
expect((err as PicloudHttpError).message).toBe('bad input');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('applies an optional validator to the response', async () => {
|
||||||
|
const fetchMock = vi.fn(async (_u: FetchArgs[0], _i?: FetchArgs[1]) =>
|
||||||
|
jsonResponse({ id: 7 })
|
||||||
|
);
|
||||||
|
const client = new PicloudClient({ baseURL: 'https://api.test', fetch: fetchMock });
|
||||||
|
const validator = {
|
||||||
|
parse: (input: unknown) => {
|
||||||
|
const r = input as { id: number };
|
||||||
|
if (typeof r.id !== 'number') throw new Error('bad');
|
||||||
|
return r;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
const res = await client.endpoint<unknown, { id: number }>('/api/x').get({ validate: validator });
|
||||||
|
expect(res.id).toBe(7);
|
||||||
|
});
|
||||||
|
});
|
||||||
54
clients/typescript/tests/helpers.ts
Normal file
54
clients/typescript/tests/helpers.ts
Normal file
@@ -0,0 +1,54 @@
|
|||||||
|
// Test helpers: build JSON + SSE Response objects and a typed fetch mock.
|
||||||
|
|
||||||
|
export function jsonResponse(body: unknown, status = 200): Response {
|
||||||
|
return new Response(JSON.stringify(body), {
|
||||||
|
status,
|
||||||
|
headers: { 'content-type': 'application/json' }
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
export function emptyResponse(status = 200): Response {
|
||||||
|
return new Response(null, { status });
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Build a text/event-stream Response from raw SSE frame strings. */
|
||||||
|
export function sseResponse(frames: string[], status = 200): Response {
|
||||||
|
const encoder = new TextEncoder();
|
||||||
|
const stream = new ReadableStream<Uint8Array>({
|
||||||
|
start(controller) {
|
||||||
|
for (const frame of frames) controller.enqueue(encoder.encode(frame));
|
||||||
|
controller.close();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
return new Response(stream, {
|
||||||
|
status,
|
||||||
|
headers: { 'content-type': 'text/event-stream' }
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/** One SSE `data:` event frame for a realtime payload. */
|
||||||
|
export function dataFrame(topic: string, message: unknown, publishedAt = '2026-06-04T00:00:00Z'): string {
|
||||||
|
const payload = JSON.stringify({ topic, message, published_at: publishedAt });
|
||||||
|
return `data: ${payload}\n\n`;
|
||||||
|
}
|
||||||
|
|
||||||
|
export type FetchArgs = [string | URL | Request, RequestInit?];
|
||||||
|
|
||||||
|
type MockLike = { mock: { calls: ReadonlyArray<ReadonlyArray<unknown>> } };
|
||||||
|
|
||||||
|
export function lastInit(mock: MockLike, i = 0): RequestInit {
|
||||||
|
const call = mock.mock.calls[i];
|
||||||
|
if (!call) throw new Error(`no fetch call at index ${i}`);
|
||||||
|
return (call[1] as RequestInit | undefined) ?? {};
|
||||||
|
}
|
||||||
|
|
||||||
|
export function lastUrl(mock: MockLike, i = 0): string {
|
||||||
|
const call = mock.mock.calls[i];
|
||||||
|
if (!call) throw new Error(`no fetch call at index ${i}`);
|
||||||
|
return String(call[0]);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function headerOf(init: RequestInit, name: string): string | undefined {
|
||||||
|
const h = init.headers as Record<string, string> | undefined;
|
||||||
|
return h?.[name];
|
||||||
|
}
|
||||||
41
clients/typescript/tests/react.test.tsx
Normal file
41
clients/typescript/tests/react.test.tsx
Normal file
@@ -0,0 +1,41 @@
|
|||||||
|
import { act, renderHook } from '@testing-library/react';
|
||||||
|
import type { ReactNode } from 'react';
|
||||||
|
import { describe, expect, it, vi } from 'vitest';
|
||||||
|
|
||||||
|
import type { PicloudClient, RealtimeEvent, Unsubscribe } from '../src/index.js';
|
||||||
|
import { PicloudProvider, useTopic } from '../src/react/index.js';
|
||||||
|
|
||||||
|
type Cb = (e: RealtimeEvent<unknown>) => void;
|
||||||
|
|
||||||
|
function fakeClient() {
|
||||||
|
const unsubscribe = vi.fn();
|
||||||
|
let captured: Cb | null = null;
|
||||||
|
const subscribe = vi.fn(
|
||||||
|
(_topic: string, cb: Cb): Unsubscribe => {
|
||||||
|
captured = cb;
|
||||||
|
return unsubscribe as unknown as Unsubscribe;
|
||||||
|
}
|
||||||
|
);
|
||||||
|
const client = { subscribe } as unknown as PicloudClient;
|
||||||
|
return { client, subscribe, unsubscribe, emit: (e: RealtimeEvent<unknown>) => captured?.(e) };
|
||||||
|
}
|
||||||
|
|
||||||
|
describe('react useTopic', () => {
|
||||||
|
it('subscribes on mount, accumulates messages, unsubscribes on unmount', () => {
|
||||||
|
const { client, subscribe, unsubscribe, emit } = fakeClient();
|
||||||
|
const wrapper = ({ children }: { children: ReactNode }) =>
|
||||||
|
PicloudProvider({ client, children });
|
||||||
|
|
||||||
|
const { result, unmount } = renderHook(() => useTopic<{ n: number }>('chat'), { wrapper });
|
||||||
|
|
||||||
|
expect(subscribe).toHaveBeenCalledTimes(1);
|
||||||
|
expect(result.current).toEqual([]);
|
||||||
|
|
||||||
|
act(() => emit({ topic: 'chat', message: { n: 1 }, published_at: 't' }));
|
||||||
|
act(() => emit({ topic: 'chat', message: { n: 2 }, published_at: 't' }));
|
||||||
|
expect(result.current).toEqual([{ n: 1 }, { n: 2 }]);
|
||||||
|
|
||||||
|
unmount();
|
||||||
|
expect(unsubscribe).toHaveBeenCalledTimes(1);
|
||||||
|
});
|
||||||
|
});
|
||||||
99
clients/typescript/tests/subscribe.test.ts
Normal file
99
clients/typescript/tests/subscribe.test.ts
Normal file
@@ -0,0 +1,99 @@
|
|||||||
|
import { describe, expect, it, vi } from 'vitest';
|
||||||
|
|
||||||
|
import { PicloudClient, type RealtimeEvent } from '../src/index.js';
|
||||||
|
import { dataFrame, emptyResponse, lastUrl, sseResponse, type FetchArgs } from './helpers.js';
|
||||||
|
|
||||||
|
/** A fetch mock that plays through a queue of response factories. */
|
||||||
|
function queuedFetch(responders: Array<() => Promise<Response>>) {
|
||||||
|
let i = 0;
|
||||||
|
return vi.fn(async (_u: FetchArgs[0], _i?: FetchArgs[1]) => {
|
||||||
|
const idx = Math.min(i, responders.length - 1);
|
||||||
|
i += 1;
|
||||||
|
const r = responders[idx];
|
||||||
|
if (!r) throw new Error('no responder');
|
||||||
|
return r();
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
describe('subscribe', () => {
|
||||||
|
it('connects to the SSE endpoint and delivers events', async () => {
|
||||||
|
const fetchMock = queuedFetch([async () => sseResponse([dataFrame('chat', { hi: 1 })])]);
|
||||||
|
const client = new PicloudClient({ baseURL: 'https://api.test', fetch: fetchMock });
|
||||||
|
|
||||||
|
const received: Array<RealtimeEvent<{ hi: number }>> = [];
|
||||||
|
const unsubscribe = client.subscribe<{ hi: number }>('chat', (e) => received.push(e));
|
||||||
|
|
||||||
|
await vi.waitFor(() => expect(received.length).toBe(1));
|
||||||
|
unsubscribe();
|
||||||
|
|
||||||
|
expect(received[0]?.topic).toBe('chat');
|
||||||
|
expect(received[0]?.message).toEqual({ hi: 1 });
|
||||||
|
expect(lastUrl(fetchMock)).toBe('https://api.test/realtime/topics/chat');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('passes a token via the query string', async () => {
|
||||||
|
const fetchMock = queuedFetch([async () => sseResponse([dataFrame('chat', 1)])]);
|
||||||
|
const client = new PicloudClient({ baseURL: 'https://api.test', fetch: fetchMock });
|
||||||
|
const unsubscribe = client.subscribe('chat', () => {}, { token: 'abc.def' });
|
||||||
|
await vi.waitFor(() => expect(fetchMock).toHaveBeenCalled());
|
||||||
|
unsubscribe();
|
||||||
|
expect(lastUrl(fetchMock)).toBe('https://api.test/realtime/topics/chat?token=abc.def');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('reconnects with backoff after an initial connection failure', async () => {
|
||||||
|
const fetchMock = queuedFetch([
|
||||||
|
async () => {
|
||||||
|
throw new Error('network down');
|
||||||
|
},
|
||||||
|
async () => sseResponse([dataFrame('chat', { ok: true })])
|
||||||
|
]);
|
||||||
|
const client = new PicloudClient({ baseURL: 'https://api.test', fetch: fetchMock });
|
||||||
|
|
||||||
|
const received: unknown[] = [];
|
||||||
|
const unsubscribe = client.subscribe('chat', (e) => received.push(e.message), {
|
||||||
|
baseBackoffMs: 5,
|
||||||
|
maxBackoffMs: 20
|
||||||
|
});
|
||||||
|
|
||||||
|
await vi.waitFor(() => expect(received.length).toBeGreaterThanOrEqual(1), { timeout: 1000 });
|
||||||
|
unsubscribe();
|
||||||
|
expect(fetchMock.mock.calls.length).toBeGreaterThanOrEqual(2);
|
||||||
|
expect(received[0]).toEqual({ ok: true });
|
||||||
|
});
|
||||||
|
|
||||||
|
it('refreshes the token after a 401 and reconnects', async () => {
|
||||||
|
const fetchMock = queuedFetch([
|
||||||
|
async () => emptyResponse(401),
|
||||||
|
async () => sseResponse([dataFrame('chat', { v: 2 })])
|
||||||
|
]);
|
||||||
|
const client = new PicloudClient({ baseURL: 'https://api.test', fetch: fetchMock });
|
||||||
|
|
||||||
|
const onTokenExpired = vi.fn(() => 'fresh-token');
|
||||||
|
const received: unknown[] = [];
|
||||||
|
const unsubscribe = client.subscribe('chat', (e) => received.push(e.message), {
|
||||||
|
token: 'stale',
|
||||||
|
onTokenExpired,
|
||||||
|
baseBackoffMs: 5
|
||||||
|
});
|
||||||
|
|
||||||
|
await vi.waitFor(() => expect(received.length).toBeGreaterThanOrEqual(1), { timeout: 1000 });
|
||||||
|
unsubscribe();
|
||||||
|
|
||||||
|
expect(onTokenExpired).toHaveBeenCalled();
|
||||||
|
// Second connect carries the refreshed token.
|
||||||
|
expect(lastUrl(fetchMock, 1)).toContain('token=fresh-token');
|
||||||
|
expect(received[0]).toEqual({ v: 2 });
|
||||||
|
});
|
||||||
|
|
||||||
|
it('stops and reports when a 401 cannot be refreshed', async () => {
|
||||||
|
const fetchMock = queuedFetch([async () => emptyResponse(401)]);
|
||||||
|
const client = new PicloudClient({ baseURL: 'https://api.test', fetch: fetchMock });
|
||||||
|
const onError = vi.fn();
|
||||||
|
const unsubscribe = client.subscribe('chat', () => {}, {
|
||||||
|
onTokenExpired: () => null,
|
||||||
|
onError
|
||||||
|
});
|
||||||
|
await vi.waitFor(() => expect(onError).toHaveBeenCalled());
|
||||||
|
unsubscribe();
|
||||||
|
});
|
||||||
|
});
|
||||||
34
clients/typescript/tests/svelte.test.ts
Normal file
34
clients/typescript/tests/svelte.test.ts
Normal file
@@ -0,0 +1,34 @@
|
|||||||
|
import { get } from 'svelte/store';
|
||||||
|
import { describe, expect, it, vi } from 'vitest';
|
||||||
|
|
||||||
|
import type { PicloudClient, RealtimeEvent, Unsubscribe } from '../src/index.js';
|
||||||
|
import { topicStore } from '../src/svelte/index.js';
|
||||||
|
|
||||||
|
type Cb = (e: RealtimeEvent<unknown>) => void;
|
||||||
|
|
||||||
|
describe('svelte topicStore', () => {
|
||||||
|
it('subscribes on first subscriber and unsubscribes on last', () => {
|
||||||
|
const unsubscribe = vi.fn();
|
||||||
|
const holder: { cb: Cb | null } = { cb: null };
|
||||||
|
const subscribe = vi.fn((_topic: string, cb: Cb): Unsubscribe => {
|
||||||
|
holder.cb = cb;
|
||||||
|
return unsubscribe as unknown as Unsubscribe;
|
||||||
|
});
|
||||||
|
const client = { subscribe } as unknown as PicloudClient;
|
||||||
|
|
||||||
|
const store = topicStore<{ x: number }>(client, 'chat');
|
||||||
|
// No SSE connection until someone subscribes (readable lifecycle).
|
||||||
|
expect(subscribe).not.toHaveBeenCalled();
|
||||||
|
|
||||||
|
let value: { x: number }[] = [];
|
||||||
|
const stop = store.subscribe((v) => (value = v));
|
||||||
|
expect(subscribe).toHaveBeenCalledTimes(1);
|
||||||
|
|
||||||
|
holder.cb?.({ topic: 'chat', message: { x: 1 }, published_at: 't' });
|
||||||
|
expect(value).toEqual([{ x: 1 }]);
|
||||||
|
expect(get(store)).toEqual([{ x: 1 }]);
|
||||||
|
|
||||||
|
stop();
|
||||||
|
expect(unsubscribe).toHaveBeenCalledTimes(1);
|
||||||
|
});
|
||||||
|
});
|
||||||
21
clients/typescript/tsconfig.json
Normal file
21
clients/typescript/tsconfig.json
Normal file
@@ -0,0 +1,21 @@
|
|||||||
|
{
|
||||||
|
"compilerOptions": {
|
||||||
|
"target": "ES2020",
|
||||||
|
"module": "ESNext",
|
||||||
|
"moduleResolution": "Bundler",
|
||||||
|
"lib": ["ES2020", "DOM", "DOM.Iterable"],
|
||||||
|
"jsx": "react-jsx",
|
||||||
|
"strict": true,
|
||||||
|
"noUncheckedIndexedAccess": true,
|
||||||
|
"exactOptionalPropertyTypes": false,
|
||||||
|
"noImplicitOverride": true,
|
||||||
|
"noUnusedLocals": true,
|
||||||
|
"noUnusedParameters": true,
|
||||||
|
"esModuleInterop": true,
|
||||||
|
"skipLibCheck": true,
|
||||||
|
"declaration": true,
|
||||||
|
"noEmit": true,
|
||||||
|
"types": []
|
||||||
|
},
|
||||||
|
"include": ["src", "tests"]
|
||||||
|
}
|
||||||
18
clients/typescript/tsup.config.ts
Normal file
18
clients/typescript/tsup.config.ts
Normal file
@@ -0,0 +1,18 @@
|
|||||||
|
import { defineConfig } from 'tsup';
|
||||||
|
|
||||||
|
// Dual ESM + CJS emit with .d.ts for the main entry and the two
|
||||||
|
// framework subpath exports. React and Svelte are peer deps — kept
|
||||||
|
// external so the lib never bundles a framework copy.
|
||||||
|
export default defineConfig({
|
||||||
|
entry: {
|
||||||
|
index: 'src/index.ts',
|
||||||
|
'react/index': 'src/react/index.ts',
|
||||||
|
'svelte/index': 'src/svelte/index.ts'
|
||||||
|
},
|
||||||
|
format: ['esm', 'cjs'],
|
||||||
|
dts: true,
|
||||||
|
clean: true,
|
||||||
|
sourcemap: true,
|
||||||
|
treeshake: true,
|
||||||
|
external: ['react', 'svelte', 'svelte/store']
|
||||||
|
});
|
||||||
11
clients/typescript/vitest.config.ts
Normal file
11
clients/typescript/vitest.config.ts
Normal file
@@ -0,0 +1,11 @@
|
|||||||
|
import { defineConfig } from 'vitest/config';
|
||||||
|
|
||||||
|
export default defineConfig({
|
||||||
|
test: {
|
||||||
|
// jsdom so the React/Svelte hook tests have a DOM; the core
|
||||||
|
// endpoint/subscribe/auth tests are environment-agnostic.
|
||||||
|
environment: 'jsdom',
|
||||||
|
globals: true,
|
||||||
|
include: ['tests/**/*.test.ts', 'tests/**/*.test.tsx']
|
||||||
|
}
|
||||||
|
});
|
||||||
@@ -448,6 +448,40 @@ fn trigger_event_to_dynamic(event: &TriggerEvent) -> Dynamic {
|
|||||||
ps.insert("published_at".into(), published_at.to_rfc3339().into());
|
ps.insert("published_at".into(), published_at.to_rfc3339().into());
|
||||||
m.insert("pubsub".into(), ps.into());
|
m.insert("pubsub".into(), ps.into());
|
||||||
}
|
}
|
||||||
|
TriggerEvent::Email {
|
||||||
|
from,
|
||||||
|
to,
|
||||||
|
cc,
|
||||||
|
subject,
|
||||||
|
text,
|
||||||
|
html,
|
||||||
|
received_at,
|
||||||
|
message_id,
|
||||||
|
} => {
|
||||||
|
// `ctx.event.op` is always "receive" for inbound email.
|
||||||
|
m.insert("op".into(), "receive".into());
|
||||||
|
let mut em = Map::new();
|
||||||
|
em.insert("from".into(), from.clone().into());
|
||||||
|
let to_arr: rhai::Array = to.iter().map(|a| Dynamic::from(a.clone())).collect();
|
||||||
|
em.insert("to".into(), to_arr.into());
|
||||||
|
let cc_arr: rhai::Array = cc.iter().map(|a| Dynamic::from(a.clone())).collect();
|
||||||
|
em.insert("cc".into(), cc_arr.into());
|
||||||
|
em.insert("subject".into(), subject.clone().into());
|
||||||
|
em.insert(
|
||||||
|
"text".into(),
|
||||||
|
text.clone().map_or(Dynamic::UNIT, Dynamic::from),
|
||||||
|
);
|
||||||
|
em.insert(
|
||||||
|
"html".into(),
|
||||||
|
html.clone().map_or(Dynamic::UNIT, Dynamic::from),
|
||||||
|
);
|
||||||
|
em.insert("received_at".into(), received_at.to_rfc3339().into());
|
||||||
|
em.insert(
|
||||||
|
"message_id".into(),
|
||||||
|
message_id.clone().map_or(Dynamic::UNIT, Dynamic::from),
|
||||||
|
);
|
||||||
|
m.insert("email".into(), em.into());
|
||||||
|
}
|
||||||
TriggerEvent::DeadLetter {
|
TriggerEvent::DeadLetter {
|
||||||
dead_letter_id,
|
dead_letter_id,
|
||||||
original,
|
original,
|
||||||
|
|||||||
150
crates/executor-core/src/sdk/email.rs
Normal file
150
crates/executor-core/src/sdk/email.rs
Normal file
@@ -0,0 +1,150 @@
|
|||||||
|
//! `email::` Rhai bridge — outbound email (v1.1.7).
|
||||||
|
//!
|
||||||
|
//! ```rhai
|
||||||
|
//! email::send(#{
|
||||||
|
//! to: "alice@example.com", // String or Array of String
|
||||||
|
//! from: "alerts@myapp.com",
|
||||||
|
//! subject: "Build complete",
|
||||||
|
//! text: "Your deploy finished."
|
||||||
|
//! });
|
||||||
|
//!
|
||||||
|
//! email::send_html(#{
|
||||||
|
//! to: ["alice@x.com", "bob@y.com"],
|
||||||
|
//! cc: ["dave@z.com"],
|
||||||
|
//! bcc: ["audit@myapp.com"],
|
||||||
|
//! from: "alerts@myapp.com",
|
||||||
|
//! reply_to: "support@myapp.com", // optional; defaults to `from`
|
||||||
|
//! subject: "Build complete",
|
||||||
|
//! text: "Your deploy finished.", // plain-text fallback
|
||||||
|
//! html: "<p>Your deploy <b>finished</b>.</p>"
|
||||||
|
//! });
|
||||||
|
//! ```
|
||||||
|
//!
|
||||||
|
//! Both map onto `EmailService::send`. `email::send` forces a text-only
|
||||||
|
//! message (any `html` key is ignored); `email::send_html` requires an
|
||||||
|
//! `html` part. `app_id` is derived from `cx.app_id` in the service.
|
||||||
|
|
||||||
|
use std::sync::Arc;
|
||||||
|
|
||||||
|
use picloud_shared::{EmailError, OutboundEmail, SdkCallCx, Services};
|
||||||
|
use rhai::{Array, Engine as RhaiEngine, EvalAltResult, Map, Module};
|
||||||
|
use tokio::runtime::Handle as TokioHandle;
|
||||||
|
|
||||||
|
pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<SdkCallCx>) {
|
||||||
|
let svc = services.email.clone();
|
||||||
|
let mut module = Module::new();
|
||||||
|
|
||||||
|
// email::send(#{...}) — plain text (html ignored).
|
||||||
|
{
|
||||||
|
let svc = svc.clone();
|
||||||
|
let cx = cx.clone();
|
||||||
|
module.set_native_fn("send", move |opts: Map| -> Result<(), Box<EvalAltResult>> {
|
||||||
|
let mut email = parse_email(&opts)?;
|
||||||
|
email.html = None; // text-only path
|
||||||
|
let svc = svc.clone();
|
||||||
|
let cx = cx.clone();
|
||||||
|
block_on(async move { svc.send(&cx, email).await })
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
// email::send_html(#{...}) — multipart text + html (html required).
|
||||||
|
{
|
||||||
|
let svc = svc.clone();
|
||||||
|
let cx = cx.clone();
|
||||||
|
module.set_native_fn(
|
||||||
|
"send_html",
|
||||||
|
move |opts: Map| -> Result<(), Box<EvalAltResult>> {
|
||||||
|
let email = parse_email(&opts)?;
|
||||||
|
if email.html.as_ref().is_none_or(String::is_empty) {
|
||||||
|
return Err(runtime_err(
|
||||||
|
"email::send_html: an 'html' field is required (use email::send for text-only)",
|
||||||
|
));
|
||||||
|
}
|
||||||
|
let svc = svc.clone();
|
||||||
|
let cx = cx.clone();
|
||||||
|
block_on(async move { svc.send(&cx, email).await })
|
||||||
|
},
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
engine.register_static_module("email", module.into());
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Parse the Rhai options map into an [`OutboundEmail`]. Field-level
|
||||||
|
/// validation (required fields, address shape) happens in the service;
|
||||||
|
/// here we only do type coercion (String/Array → Vec<String>).
|
||||||
|
fn parse_email(opts: &Map) -> Result<OutboundEmail, Box<EvalAltResult>> {
|
||||||
|
Ok(OutboundEmail {
|
||||||
|
to: addresses(opts, "to")?,
|
||||||
|
cc: addresses(opts, "cc")?,
|
||||||
|
bcc: addresses(opts, "bcc")?,
|
||||||
|
from: string_field(opts, "from").unwrap_or_default(),
|
||||||
|
reply_to: string_field(opts, "reply_to"),
|
||||||
|
subject: string_field(opts, "subject").unwrap_or_default(),
|
||||||
|
text: string_field(opts, "text"),
|
||||||
|
html: string_field(opts, "html"),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Read a string field. Missing or `()` → `None`.
|
||||||
|
fn string_field(opts: &Map, key: &str) -> Option<String> {
|
||||||
|
match opts.get(key) {
|
||||||
|
None => None,
|
||||||
|
Some(d) if d.is_unit() => None,
|
||||||
|
Some(d) if d.is_string() => Some(d.clone().into_string().unwrap_or_default()),
|
||||||
|
// Coerce non-string scalars via display (numbers, etc.).
|
||||||
|
Some(d) => Some(d.to_string()),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Read an address list: a String becomes a one-element list; an Array
|
||||||
|
/// of Strings becomes a list; missing/`()` is empty.
|
||||||
|
fn addresses(opts: &Map, key: &str) -> Result<Vec<String>, Box<EvalAltResult>> {
|
||||||
|
match opts.get(key) {
|
||||||
|
None => Ok(Vec::new()),
|
||||||
|
Some(d) if d.is_unit() => Ok(Vec::new()),
|
||||||
|
Some(d) if d.is_string() => Ok(vec![d.clone().into_string().unwrap_or_default()]),
|
||||||
|
Some(d) => {
|
||||||
|
if let Some(arr) = d.clone().try_cast::<Array>() {
|
||||||
|
let mut out = Vec::with_capacity(arr.len());
|
||||||
|
for el in arr {
|
||||||
|
if !el.is_string() {
|
||||||
|
return Err(runtime_err(&format!(
|
||||||
|
"email: '{key}' array must contain only strings"
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
out.push(el.into_string().unwrap_or_default());
|
||||||
|
}
|
||||||
|
Ok(out)
|
||||||
|
} else {
|
||||||
|
Err(runtime_err(&format!(
|
||||||
|
"email: '{key}' must be a string or an array of strings"
|
||||||
|
)))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[allow(clippy::unnecessary_box_returns)]
|
||||||
|
fn runtime_err(msg: &str) -> Box<EvalAltResult> {
|
||||||
|
EvalAltResult::ErrorRuntime(msg.into(), rhai::Position::NONE).into()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Run an `EmailService` future inside the synchronous Rhai context,
|
||||||
|
/// mapping any `EmailError` to a Rhai runtime error. Mirrors
|
||||||
|
/// `kv::block_on`.
|
||||||
|
fn block_on<F>(fut: F) -> Result<(), Box<EvalAltResult>>
|
||||||
|
where
|
||||||
|
F: std::future::Future<Output = Result<(), EmailError>> + Send,
|
||||||
|
{
|
||||||
|
let handle = TokioHandle::try_current().map_err(|e| -> Box<EvalAltResult> {
|
||||||
|
EvalAltResult::ErrorRuntime(
|
||||||
|
format!("email: no tokio runtime available: {e}").into(),
|
||||||
|
rhai::Position::NONE,
|
||||||
|
)
|
||||||
|
.into()
|
||||||
|
})?;
|
||||||
|
handle.block_on(fut).map_err(|err| -> Box<EvalAltResult> {
|
||||||
|
EvalAltResult::ErrorRuntime(format!("email: {err}").into(), rhai::Position::NONE).into()
|
||||||
|
})
|
||||||
|
}
|
||||||
@@ -15,10 +15,12 @@ pub mod bridge;
|
|||||||
pub mod cx;
|
pub mod cx;
|
||||||
pub mod dead_letters;
|
pub mod dead_letters;
|
||||||
pub mod docs;
|
pub mod docs;
|
||||||
|
pub mod email;
|
||||||
pub mod files;
|
pub mod files;
|
||||||
pub mod http;
|
pub mod http;
|
||||||
pub mod kv;
|
pub mod kv;
|
||||||
pub mod pubsub;
|
pub mod pubsub;
|
||||||
|
pub mod secrets;
|
||||||
pub mod stdlib;
|
pub mod stdlib;
|
||||||
|
|
||||||
pub use bridge::{dynamic_to_json, json_to_dynamic};
|
pub use bridge::{dynamic_to_json, json_to_dynamic};
|
||||||
@@ -41,5 +43,7 @@ pub fn register_all(engine: &mut RhaiEngine, services: &Services, cx: Arc<SdkCal
|
|||||||
dead_letters::register(engine, services, cx.clone());
|
dead_letters::register(engine, services, cx.clone());
|
||||||
http::register(engine, services, cx.clone());
|
http::register(engine, services, cx.clone());
|
||||||
files::register(engine, services, cx.clone());
|
files::register(engine, services, cx.clone());
|
||||||
pubsub::register(engine, services, cx);
|
pubsub::register(engine, services, cx.clone());
|
||||||
|
secrets::register(engine, services, cx.clone());
|
||||||
|
email::register(engine, services, cx);
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -40,9 +40,85 @@ pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<Sdk
|
|||||||
},
|
},
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
// `pubsub::subscriber_token(topics)` — uses the configured default
|
||||||
|
// TTL.
|
||||||
|
{
|
||||||
|
let svc = svc.clone();
|
||||||
|
let cx = cx.clone();
|
||||||
|
module.set_native_fn(
|
||||||
|
"subscriber_token",
|
||||||
|
move |topics: Array| -> Result<String, Box<EvalAltResult>> {
|
||||||
|
mint_token(&svc, &cx, topics, None)
|
||||||
|
},
|
||||||
|
);
|
||||||
|
}
|
||||||
|
// `pubsub::subscriber_token(topics, ttl)` — `ttl` is an integer
|
||||||
|
// (seconds) or `()` for the default.
|
||||||
|
{
|
||||||
|
let svc = svc.clone();
|
||||||
|
let cx = cx.clone();
|
||||||
|
module.set_native_fn(
|
||||||
|
"subscriber_token",
|
||||||
|
move |topics: Array, ttl: Dynamic| -> Result<String, Box<EvalAltResult>> {
|
||||||
|
let ttl = ttl_from_dynamic(&ttl)?;
|
||||||
|
mint_token(&svc, &cx, topics, ttl)
|
||||||
|
},
|
||||||
|
);
|
||||||
|
}
|
||||||
engine.register_static_module("pubsub", module.into());
|
engine.register_static_module("pubsub", module.into());
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Interpret the optional `ttl` argument: `()` → use the default,
|
||||||
|
/// integer → that many seconds, anything else → throw.
|
||||||
|
fn ttl_from_dynamic(ttl: &Dynamic) -> Result<Option<i64>, Box<EvalAltResult>> {
|
||||||
|
if ttl.is_unit() {
|
||||||
|
return Ok(None);
|
||||||
|
}
|
||||||
|
ttl.as_int().map(Some).map_err(|_| -> Box<EvalAltResult> {
|
||||||
|
EvalAltResult::ErrorRuntime(
|
||||||
|
"pubsub::subscriber_token: ttl must be an integer (seconds) or ()".into(),
|
||||||
|
rhai::Position::NONE,
|
||||||
|
)
|
||||||
|
.into()
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
fn mint_token(
|
||||||
|
svc: &Arc<dyn picloud_shared::PubsubService>,
|
||||||
|
cx: &Arc<SdkCallCx>,
|
||||||
|
topics: Array,
|
||||||
|
ttl: Option<i64>,
|
||||||
|
) -> Result<String, Box<EvalAltResult>> {
|
||||||
|
// Every element must be a string; surface a clear error otherwise.
|
||||||
|
let mut names = Vec::with_capacity(topics.len());
|
||||||
|
for t in topics {
|
||||||
|
if !t.is_string() {
|
||||||
|
return Err(EvalAltResult::ErrorRuntime(
|
||||||
|
"pubsub::subscriber_token: topics must be an array of strings".into(),
|
||||||
|
rhai::Position::NONE,
|
||||||
|
)
|
||||||
|
.into());
|
||||||
|
}
|
||||||
|
names.push(t.into_string().unwrap_or_default());
|
||||||
|
}
|
||||||
|
let svc = svc.clone();
|
||||||
|
let cx = cx.clone();
|
||||||
|
let handle = TokioHandle::try_current().map_err(|e| -> Box<EvalAltResult> {
|
||||||
|
EvalAltResult::ErrorRuntime(
|
||||||
|
format!("pubsub: no tokio runtime available: {e}").into(),
|
||||||
|
rhai::Position::NONE,
|
||||||
|
)
|
||||||
|
.into()
|
||||||
|
})?;
|
||||||
|
// SubscriberToken errors already carry the full
|
||||||
|
// "pubsub::subscriber_token: …" wording, so surface them verbatim.
|
||||||
|
handle
|
||||||
|
.block_on(async move { svc.mint_subscriber_token(&cx, names, ttl).await })
|
||||||
|
.map_err(|err| -> Box<EvalAltResult> {
|
||||||
|
EvalAltResult::ErrorRuntime(format!("{err}").into(), rhai::Position::NONE).into()
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
/// Convert a Rhai `Dynamic` message into JSON, base64-encoding any
|
/// Convert a Rhai `Dynamic` message into JSON, base64-encoding any
|
||||||
/// `Blob` (at any nesting depth). Mirrors `bridge::dynamic_to_json` but
|
/// `Blob` (at any nesting depth). Mirrors `bridge::dynamic_to_json` but
|
||||||
/// adds the blob arm the pub/sub wire contract requires.
|
/// adds the blob arm the pub/sub wire contract requires.
|
||||||
|
|||||||
153
crates/executor-core/src/sdk/secrets.rs
Normal file
153
crates/executor-core/src/sdk/secrets.rs
Normal file
@@ -0,0 +1,153 @@
|
|||||||
|
//! `secrets::` Rhai bridge — encrypted per-app secrets (v1.1.7).
|
||||||
|
//!
|
||||||
|
//! ```rhai
|
||||||
|
//! secrets::set("stripe_key", "sk_live_xxx");
|
||||||
|
//! secrets::set("oauth", #{ client_id: "abc", client_secret: "xyz" });
|
||||||
|
//! let key = secrets::get("stripe_key"); // value or ()
|
||||||
|
//! let removed = secrets::delete("stripe_key"); // bool
|
||||||
|
//! let page = secrets::list(#{ cursor: (), limit: 100 });
|
||||||
|
//! // page = #{ names: [...], next_cursor: () | "..." }
|
||||||
|
//! ```
|
||||||
|
//!
|
||||||
|
//! Collection-less (secrets are per-app, like pubsub topics) so there's
|
||||||
|
//! no `::collection(...)`. Values are any JSON-serializable Rhai value
|
||||||
|
//! (String/Map/Array/number/bool); a String round-trips back as a
|
||||||
|
//! String. `app_id` is derived from `cx.app_id` in the service — it
|
||||||
|
//! never appears in the script-side signature, preserving cross-app
|
||||||
|
//! isolation.
|
||||||
|
|
||||||
|
use std::sync::Arc;
|
||||||
|
|
||||||
|
use picloud_shared::{SdkCallCx, SecretsError, SecretsListPage, Services};
|
||||||
|
use rhai::{Array, Dynamic, Engine as RhaiEngine, EvalAltResult, Map, Module};
|
||||||
|
use tokio::runtime::Handle as TokioHandle;
|
||||||
|
|
||||||
|
use super::bridge::{dynamic_to_json, json_to_dynamic};
|
||||||
|
|
||||||
|
pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<SdkCallCx>) {
|
||||||
|
let svc = services.secrets.clone();
|
||||||
|
let mut module = Module::new();
|
||||||
|
|
||||||
|
// secrets::set(name, value) — overwrites if present.
|
||||||
|
{
|
||||||
|
let svc = svc.clone();
|
||||||
|
let cx = cx.clone();
|
||||||
|
module.set_native_fn(
|
||||||
|
"set",
|
||||||
|
move |name: &str, value: Dynamic| -> Result<(), Box<EvalAltResult>> {
|
||||||
|
let json = dynamic_to_json(&value);
|
||||||
|
let svc = svc.clone();
|
||||||
|
let cx = cx.clone();
|
||||||
|
block_on(async move { svc.set(&cx, name, json).await })
|
||||||
|
},
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
// secrets::get(name) — decoded value, or () if missing.
|
||||||
|
{
|
||||||
|
let svc = svc.clone();
|
||||||
|
let cx = cx.clone();
|
||||||
|
module.set_native_fn(
|
||||||
|
"get",
|
||||||
|
move |name: &str| -> Result<Dynamic, Box<EvalAltResult>> {
|
||||||
|
let svc = svc.clone();
|
||||||
|
let cx = cx.clone();
|
||||||
|
let opt = block_on(async move { svc.get(&cx, name).await })?;
|
||||||
|
Ok(opt.map_or(Dynamic::UNIT, json_to_dynamic))
|
||||||
|
},
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
// secrets::delete(name) — bool was-present.
|
||||||
|
{
|
||||||
|
let svc = svc.clone();
|
||||||
|
let cx = cx.clone();
|
||||||
|
module.set_native_fn(
|
||||||
|
"delete",
|
||||||
|
move |name: &str| -> Result<bool, Box<EvalAltResult>> {
|
||||||
|
let svc = svc.clone();
|
||||||
|
let cx = cx.clone();
|
||||||
|
block_on(async move { svc.delete(&cx, name).await })
|
||||||
|
},
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
// secrets::list(#{ cursor, limit }) — names only, cursor-paginated.
|
||||||
|
{
|
||||||
|
let svc = svc.clone();
|
||||||
|
let cx = cx.clone();
|
||||||
|
module.set_native_fn(
|
||||||
|
"list",
|
||||||
|
move |opts: Map| -> Result<Map, Box<EvalAltResult>> {
|
||||||
|
let (cursor, limit) = parse_list_opts(&opts)?;
|
||||||
|
let svc = svc.clone();
|
||||||
|
let cx = cx.clone();
|
||||||
|
let page: SecretsListPage =
|
||||||
|
block_on(async move { svc.list(&cx, cursor.as_deref(), limit).await })?;
|
||||||
|
Ok(list_page_to_map(page))
|
||||||
|
},
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
engine.register_static_module("secrets", module.into());
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Pull `cursor` (string or `()`) and `limit` (int or `()`) out of the
|
||||||
|
/// options map. Unknown/extra keys are ignored.
|
||||||
|
fn parse_list_opts(opts: &Map) -> Result<(Option<String>, u32), Box<EvalAltResult>> {
|
||||||
|
let cursor = match opts.get("cursor") {
|
||||||
|
None => None,
|
||||||
|
Some(d) if d.is_unit() => None,
|
||||||
|
Some(d) if d.is_string() => Some(d.clone().into_string().unwrap_or_default()),
|
||||||
|
Some(_) => return Err(runtime_err("secrets::list: cursor must be a string or ()")),
|
||||||
|
};
|
||||||
|
let limit = match opts.get("limit") {
|
||||||
|
None => 0,
|
||||||
|
Some(d) if d.is_unit() => 0,
|
||||||
|
Some(d) => {
|
||||||
|
let n = d
|
||||||
|
.as_int()
|
||||||
|
.map_err(|_| runtime_err("secrets::list: limit must be an integer or ()"))?;
|
||||||
|
u32::try_from(n.max(0)).unwrap_or(u32::MAX)
|
||||||
|
}
|
||||||
|
};
|
||||||
|
Ok((cursor, limit))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn list_page_to_map(page: SecretsListPage) -> Map {
|
||||||
|
let mut m = Map::new();
|
||||||
|
let names: Array = page.names.into_iter().map(Dynamic::from).collect();
|
||||||
|
m.insert("names".into(), names.into());
|
||||||
|
m.insert(
|
||||||
|
"next_cursor".into(),
|
||||||
|
page.next_cursor.map_or(Dynamic::UNIT, Dynamic::from),
|
||||||
|
);
|
||||||
|
m
|
||||||
|
}
|
||||||
|
|
||||||
|
// Returns the boxed error directly because every caller needs a
|
||||||
|
// `Box<EvalAltResult>` (Rhai's error type), matching the other bridges.
|
||||||
|
#[allow(clippy::unnecessary_box_returns)]
|
||||||
|
fn runtime_err(msg: &str) -> Box<EvalAltResult> {
|
||||||
|
EvalAltResult::ErrorRuntime(msg.into(), rhai::Position::NONE).into()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Run a `SecretsService` future inside the synchronous Rhai context,
|
||||||
|
/// mapping any `SecretsError` to a Rhai runtime error. Mirrors
|
||||||
|
/// `kv::block_on` / `pubsub::block_on`.
|
||||||
|
fn block_on<T, F>(fut: F) -> Result<T, Box<EvalAltResult>>
|
||||||
|
where
|
||||||
|
F: std::future::Future<Output = Result<T, SecretsError>> + Send,
|
||||||
|
T: Send,
|
||||||
|
{
|
||||||
|
let handle = TokioHandle::try_current().map_err(|e| -> Box<EvalAltResult> {
|
||||||
|
EvalAltResult::ErrorRuntime(
|
||||||
|
format!("secrets: no tokio runtime available: {e}").into(),
|
||||||
|
rhai::Position::NONE,
|
||||||
|
)
|
||||||
|
.into()
|
||||||
|
})?;
|
||||||
|
handle.block_on(fut).map_err(|err| -> Box<EvalAltResult> {
|
||||||
|
EvalAltResult::ErrorRuntime(format!("secrets: {err}").into(), rhai::Position::NONE).into()
|
||||||
|
})
|
||||||
|
}
|
||||||
@@ -101,6 +101,8 @@ async fn original_backend_error_is_logged_at_error_level() {
|
|||||||
Arc::new(NoopHttpService),
|
Arc::new(NoopHttpService),
|
||||||
Arc::new(picloud_shared::NoopFilesService),
|
Arc::new(picloud_shared::NoopFilesService),
|
||||||
Arc::new(picloud_shared::NoopPubsubService),
|
Arc::new(picloud_shared::NoopPubsubService),
|
||||||
|
Arc::new(picloud_shared::NoopSecretsService),
|
||||||
|
Arc::new(picloud_shared::NoopEmailService),
|
||||||
);
|
);
|
||||||
let engine = Engine::new(Limits::default(), services);
|
let engine = Engine::new(Limits::default(), services);
|
||||||
|
|
||||||
|
|||||||
@@ -99,6 +99,8 @@ fn services_with(modules: Arc<dyn ModuleSource>) -> Services {
|
|||||||
Arc::new(NoopHttpService),
|
Arc::new(NoopHttpService),
|
||||||
Arc::new(picloud_shared::NoopFilesService),
|
Arc::new(picloud_shared::NoopFilesService),
|
||||||
Arc::new(picloud_shared::NoopPubsubService),
|
Arc::new(picloud_shared::NoopPubsubService),
|
||||||
|
Arc::new(picloud_shared::NoopSecretsService),
|
||||||
|
Arc::new(picloud_shared::NoopEmailService),
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -230,6 +230,8 @@ fn make_engine() -> Arc<Engine> {
|
|||||||
Arc::new(NoopHttpService),
|
Arc::new(NoopHttpService),
|
||||||
Arc::new(picloud_shared::NoopFilesService),
|
Arc::new(picloud_shared::NoopFilesService),
|
||||||
Arc::new(picloud_shared::NoopPubsubService),
|
Arc::new(picloud_shared::NoopPubsubService),
|
||||||
|
Arc::new(picloud_shared::NoopSecretsService),
|
||||||
|
Arc::new(picloud_shared::NoopEmailService),
|
||||||
);
|
);
|
||||||
Arc::new(Engine::new(Limits::default(), services))
|
Arc::new(Engine::new(Limits::default(), services))
|
||||||
}
|
}
|
||||||
|
|||||||
209
crates/executor-core/tests/sdk_email.rs
Normal file
209
crates/executor-core/tests/sdk_email.rs
Normal file
@@ -0,0 +1,209 @@
|
|||||||
|
//! `email::` SDK bridge integration tests — runs a real Rhai engine
|
||||||
|
//! against a recording `EmailService`. Verifies the Rhai map → DTO
|
||||||
|
//! plumbing (address coercion, the text-only vs multipart split). The
|
||||||
|
//! SMTP transport, validation, and authz are unit-tested at the service
|
||||||
|
//! layer in `manager-core::email_service`.
|
||||||
|
|
||||||
|
use std::collections::BTreeMap;
|
||||||
|
use std::sync::{Arc, Mutex};
|
||||||
|
|
||||||
|
use async_trait::async_trait;
|
||||||
|
use picloud_executor_core::{Engine, ExecRequest, InvocationType, Limits};
|
||||||
|
use picloud_shared::{
|
||||||
|
AppId, EmailError, EmailService, ExecutionId, NoopDeadLetterService, NoopDocsService,
|
||||||
|
NoopEventEmitter, NoopHttpService, NoopKvService, NoopModuleSource, OutboundEmail, RequestId,
|
||||||
|
ScriptId, ScriptSandbox, SdkCallCx, Services, TriggerEvent,
|
||||||
|
};
|
||||||
|
use serde_json::{json, Value};
|
||||||
|
|
||||||
|
#[derive(Default)]
|
||||||
|
struct RecordingEmail {
|
||||||
|
sent: Mutex<Vec<OutboundEmail>>,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[async_trait]
|
||||||
|
impl EmailService for RecordingEmail {
|
||||||
|
async fn send(&self, _cx: &SdkCallCx, email: OutboundEmail) -> Result<(), EmailError> {
|
||||||
|
self.sent.lock().unwrap().push(email);
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn engine_with(rec: Arc<RecordingEmail>) -> Arc<Engine> {
|
||||||
|
let services = Services::new(
|
||||||
|
Arc::new(NoopKvService),
|
||||||
|
Arc::new(NoopDocsService),
|
||||||
|
Arc::new(NoopDeadLetterService),
|
||||||
|
Arc::new(NoopEventEmitter),
|
||||||
|
Arc::new(NoopModuleSource),
|
||||||
|
Arc::new(NoopHttpService),
|
||||||
|
Arc::new(picloud_shared::NoopFilesService),
|
||||||
|
Arc::new(picloud_shared::NoopPubsubService),
|
||||||
|
Arc::new(picloud_shared::NoopSecretsService),
|
||||||
|
rec,
|
||||||
|
);
|
||||||
|
Arc::new(Engine::new(Limits::default(), services))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn baseline_request(app_id: AppId) -> ExecRequest {
|
||||||
|
let execution_id = ExecutionId::new();
|
||||||
|
ExecRequest {
|
||||||
|
execution_id,
|
||||||
|
request_id: RequestId::new(),
|
||||||
|
script_id: ScriptId::new(),
|
||||||
|
script_name: "email-test".into(),
|
||||||
|
invocation_type: InvocationType::Http,
|
||||||
|
path: "/email-test".into(),
|
||||||
|
headers: BTreeMap::new(),
|
||||||
|
body: Value::Null,
|
||||||
|
params: BTreeMap::new(),
|
||||||
|
query: BTreeMap::new(),
|
||||||
|
rest: String::new(),
|
||||||
|
sandbox_overrides: ScriptSandbox::default(),
|
||||||
|
app_id,
|
||||||
|
principal: None,
|
||||||
|
trigger_depth: 0,
|
||||||
|
root_execution_id: execution_id,
|
||||||
|
is_dead_letter_handler: false,
|
||||||
|
event: None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn run(engine: Arc<Engine>, src: &str) -> Result<(), ()> {
|
||||||
|
let src = src.to_string();
|
||||||
|
let app = AppId::new();
|
||||||
|
tokio::task::spawn_blocking(move || engine.execute(&src, baseline_request(app)))
|
||||||
|
.await
|
||||||
|
.expect("spawn_blocking")
|
||||||
|
.map(|_| ())
|
||||||
|
.map_err(|_| ())
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||||
|
async fn send_parses_single_recipient_text() {
|
||||||
|
let rec = Arc::new(RecordingEmail::default());
|
||||||
|
let engine = engine_with(rec.clone());
|
||||||
|
run(
|
||||||
|
engine,
|
||||||
|
r#"
|
||||||
|
email::send(#{
|
||||||
|
to: "alice@example.com",
|
||||||
|
from: "alerts@myapp.com",
|
||||||
|
subject: "Build complete",
|
||||||
|
text: "done"
|
||||||
|
});
|
||||||
|
#{ ok: true }
|
||||||
|
"#,
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
let g = rec.sent.lock().unwrap();
|
||||||
|
let e = g.last().unwrap();
|
||||||
|
assert_eq!(e.to, vec!["alice@example.com".to_string()]);
|
||||||
|
assert_eq!(e.from, "alerts@myapp.com");
|
||||||
|
assert_eq!(e.subject, "Build complete");
|
||||||
|
assert_eq!(e.text.as_deref(), Some("done"));
|
||||||
|
// email::send forces text-only even if html were present.
|
||||||
|
assert!(e.html.is_none());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||||
|
async fn send_html_carries_both_parts_and_lists() {
|
||||||
|
let rec = Arc::new(RecordingEmail::default());
|
||||||
|
let engine = engine_with(rec.clone());
|
||||||
|
run(
|
||||||
|
engine,
|
||||||
|
r#"
|
||||||
|
email::send_html(#{
|
||||||
|
to: ["alice@x.com", "bob@y.com"],
|
||||||
|
cc: ["dave@z.com"],
|
||||||
|
bcc: ["audit@myapp.com"],
|
||||||
|
from: "alerts@myapp.com",
|
||||||
|
reply_to: "support@myapp.com",
|
||||||
|
subject: "hi",
|
||||||
|
text: "plain",
|
||||||
|
html: "<p>rich</p>"
|
||||||
|
});
|
||||||
|
#{ ok: true }
|
||||||
|
"#,
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
let g = rec.sent.lock().unwrap();
|
||||||
|
let e = g.last().unwrap();
|
||||||
|
assert_eq!(
|
||||||
|
e.to,
|
||||||
|
vec!["alice@x.com".to_string(), "bob@y.com".to_string()]
|
||||||
|
);
|
||||||
|
assert_eq!(e.cc, vec!["dave@z.com".to_string()]);
|
||||||
|
assert_eq!(e.bcc, vec!["audit@myapp.com".to_string()]);
|
||||||
|
assert_eq!(e.reply_to.as_deref(), Some("support@myapp.com"));
|
||||||
|
assert_eq!(e.text.as_deref(), Some("plain"));
|
||||||
|
assert_eq!(e.html.as_deref(), Some("<p>rich</p>"));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||||
|
async fn inbound_email_event_visible_to_handler() {
|
||||||
|
// A handler invoked by an email:receive trigger sees the normalized
|
||||||
|
// message at ctx.event.email (built by the engine's ctx renderer).
|
||||||
|
let rec = Arc::new(RecordingEmail::default());
|
||||||
|
let engine = engine_with(rec);
|
||||||
|
let mut req = baseline_request(AppId::new());
|
||||||
|
req.event = Some(TriggerEvent::Email {
|
||||||
|
from: "sender@external.com".into(),
|
||||||
|
to: vec!["alice@myapp.com".into()],
|
||||||
|
cc: vec!["bob@myapp.com".into()],
|
||||||
|
subject: "Re: question".into(),
|
||||||
|
text: Some("hello".into()),
|
||||||
|
html: None,
|
||||||
|
received_at: chrono::DateTime::parse_from_rfc3339("2026-08-15T12:00:00Z")
|
||||||
|
.unwrap()
|
||||||
|
.with_timezone(&chrono::Utc),
|
||||||
|
message_id: Some("<abc@external.com>".into()),
|
||||||
|
});
|
||||||
|
let src = r#"
|
||||||
|
let e = ctx.event;
|
||||||
|
#{
|
||||||
|
source: e.source,
|
||||||
|
op: e.op,
|
||||||
|
from: e.email.from,
|
||||||
|
to0: e.email.to[0],
|
||||||
|
cc0: e.email.cc[0],
|
||||||
|
subject: e.email.subject,
|
||||||
|
text: e.email.text,
|
||||||
|
html_is_unit: type_of(e.email.html) == "()",
|
||||||
|
message_id: e.email.message_id
|
||||||
|
}
|
||||||
|
"#;
|
||||||
|
let src = src.to_string();
|
||||||
|
let body = tokio::task::spawn_blocking(move || engine.execute(&src, req))
|
||||||
|
.await
|
||||||
|
.unwrap()
|
||||||
|
.unwrap()
|
||||||
|
.body;
|
||||||
|
assert_eq!(body["source"], json!("email"));
|
||||||
|
assert_eq!(body["op"], json!("receive"));
|
||||||
|
assert_eq!(body["from"], json!("sender@external.com"));
|
||||||
|
assert_eq!(body["to0"], json!("alice@myapp.com"));
|
||||||
|
assert_eq!(body["cc0"], json!("bob@myapp.com"));
|
||||||
|
assert_eq!(body["subject"], json!("Re: question"));
|
||||||
|
assert_eq!(body["text"], json!("hello"));
|
||||||
|
assert_eq!(body["html_is_unit"], json!(true));
|
||||||
|
assert_eq!(body["message_id"], json!("<abc@external.com>"));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||||
|
async fn send_html_without_html_throws() {
|
||||||
|
let rec = Arc::new(RecordingEmail::default());
|
||||||
|
let engine = engine_with(rec.clone());
|
||||||
|
let res = run(
|
||||||
|
engine,
|
||||||
|
r#"
|
||||||
|
email::send_html(#{ to: "a@b.com", from: "c@d.com", subject: "x", text: "y" });
|
||||||
|
#{ ok: true }
|
||||||
|
"#,
|
||||||
|
)
|
||||||
|
.await;
|
||||||
|
assert!(res.is_err(), "send_html without html must throw");
|
||||||
|
assert!(rec.sent.lock().unwrap().is_empty());
|
||||||
|
}
|
||||||
@@ -167,6 +167,8 @@ fn make_engine() -> Arc<Engine> {
|
|||||||
Arc::new(NoopHttpService),
|
Arc::new(NoopHttpService),
|
||||||
Arc::new(InMemoryFiles::default()),
|
Arc::new(InMemoryFiles::default()),
|
||||||
Arc::new(picloud_shared::NoopPubsubService),
|
Arc::new(picloud_shared::NoopPubsubService),
|
||||||
|
Arc::new(picloud_shared::NoopSecretsService),
|
||||||
|
Arc::new(picloud_shared::NoopEmailService),
|
||||||
);
|
);
|
||||||
Arc::new(Engine::new(Limits::default(), services))
|
Arc::new(Engine::new(Limits::default(), services))
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -90,6 +90,8 @@ fn engine_with(http: Arc<dyn HttpService>) -> Arc<Engine> {
|
|||||||
http,
|
http,
|
||||||
Arc::new(picloud_shared::NoopFilesService),
|
Arc::new(picloud_shared::NoopFilesService),
|
||||||
Arc::new(picloud_shared::NoopPubsubService),
|
Arc::new(picloud_shared::NoopPubsubService),
|
||||||
|
Arc::new(picloud_shared::NoopSecretsService),
|
||||||
|
Arc::new(picloud_shared::NoopEmailService),
|
||||||
);
|
);
|
||||||
Arc::new(Engine::new(Limits::default(), services))
|
Arc::new(Engine::new(Limits::default(), services))
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -109,6 +109,8 @@ fn make_engine() -> Arc<Engine> {
|
|||||||
Arc::new(NoopHttpService),
|
Arc::new(NoopHttpService),
|
||||||
Arc::new(picloud_shared::NoopFilesService),
|
Arc::new(picloud_shared::NoopFilesService),
|
||||||
Arc::new(picloud_shared::NoopPubsubService),
|
Arc::new(picloud_shared::NoopPubsubService),
|
||||||
|
Arc::new(picloud_shared::NoopSecretsService),
|
||||||
|
Arc::new(picloud_shared::NoopEmailService),
|
||||||
);
|
);
|
||||||
Arc::new(Engine::new(Limits::default(), services))
|
Arc::new(Engine::new(Limits::default(), services))
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -47,6 +47,8 @@ fn make_engine(svc: Arc<RecordingPubsub>) -> Arc<Engine> {
|
|||||||
Arc::new(NoopHttpService),
|
Arc::new(NoopHttpService),
|
||||||
Arc::new(NoopFilesService),
|
Arc::new(NoopFilesService),
|
||||||
svc,
|
svc,
|
||||||
|
Arc::new(picloud_shared::NoopSecretsService),
|
||||||
|
Arc::new(picloud_shared::NoopEmailService),
|
||||||
);
|
);
|
||||||
Arc::new(Engine::new(Limits::default(), services))
|
Arc::new(Engine::new(Limits::default(), services))
|
||||||
}
|
}
|
||||||
|
|||||||
213
crates/executor-core/tests/sdk_secrets.rs
Normal file
213
crates/executor-core/tests/sdk_secrets.rs
Normal file
@@ -0,0 +1,213 @@
|
|||||||
|
//! `secrets::` SDK bridge integration tests — runs a real Rhai engine
|
||||||
|
//! against an in-memory `SecretsService` impl. Mirrors `sdk_kv.rs`: the
|
||||||
|
//! engine runs under `spawn_blocking` so the bridge's `block_on` has a
|
||||||
|
//! reachable runtime.
|
||||||
|
//!
|
||||||
|
//! This exercises the Rhai⇄JSON plumbing + the static `secrets` module
|
||||||
|
//! (set/get/delete/list, the missing→() contract, and the
|
||||||
|
//! String/Map/Array type round-trip). Encryption + authz + the
|
||||||
|
//! cross-app boundary are unit-tested at the service layer in
|
||||||
|
//! `manager-core::secrets_service`.
|
||||||
|
|
||||||
|
use std::collections::BTreeMap;
|
||||||
|
use std::sync::Arc;
|
||||||
|
|
||||||
|
use async_trait::async_trait;
|
||||||
|
use picloud_executor_core::{Engine, ExecRequest, InvocationType, Limits};
|
||||||
|
use picloud_shared::{
|
||||||
|
AppId, ExecutionId, NoopDeadLetterService, NoopDocsService, NoopEventEmitter, NoopHttpService,
|
||||||
|
NoopKvService, NoopModuleSource, RequestId, ScriptId, ScriptSandbox, SdkCallCx, SecretsError,
|
||||||
|
SecretsListPage, SecretsService, Services,
|
||||||
|
};
|
||||||
|
use serde_json::{json, Value};
|
||||||
|
use tokio::sync::Mutex;
|
||||||
|
|
||||||
|
/// In-memory secrets store keyed by `(app_id, name)`. Stores the JSON
|
||||||
|
/// value directly — the bridge test only cares about the Rhai plumbing,
|
||||||
|
/// not the at-rest encryption (which the service layer owns).
|
||||||
|
#[derive(Default)]
|
||||||
|
struct InMemorySecrets {
|
||||||
|
data: Mutex<BTreeMap<(AppId, String), Value>>,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[async_trait]
|
||||||
|
impl SecretsService for InMemorySecrets {
|
||||||
|
async fn get(&self, cx: &SdkCallCx, name: &str) -> Result<Option<Value>, SecretsError> {
|
||||||
|
picloud_shared::validate_secret_name(name)?;
|
||||||
|
Ok(self
|
||||||
|
.data
|
||||||
|
.lock()
|
||||||
|
.await
|
||||||
|
.get(&(cx.app_id, name.to_string()))
|
||||||
|
.cloned())
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn set(&self, cx: &SdkCallCx, name: &str, value: Value) -> Result<(), SecretsError> {
|
||||||
|
picloud_shared::validate_secret_name(name)?;
|
||||||
|
self.data
|
||||||
|
.lock()
|
||||||
|
.await
|
||||||
|
.insert((cx.app_id, name.to_string()), value);
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn delete(&self, cx: &SdkCallCx, name: &str) -> Result<bool, SecretsError> {
|
||||||
|
picloud_shared::validate_secret_name(name)?;
|
||||||
|
Ok(self
|
||||||
|
.data
|
||||||
|
.lock()
|
||||||
|
.await
|
||||||
|
.remove(&(cx.app_id, name.to_string()))
|
||||||
|
.is_some())
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn list(
|
||||||
|
&self,
|
||||||
|
cx: &SdkCallCx,
|
||||||
|
cursor: Option<&str>,
|
||||||
|
limit: u32,
|
||||||
|
) -> Result<SecretsListPage, SecretsError> {
|
||||||
|
let data = self.data.lock().await;
|
||||||
|
let mut names: Vec<String> = data
|
||||||
|
.iter()
|
||||||
|
.filter(|((a, _), _)| *a == cx.app_id)
|
||||||
|
.map(|((_, n), _)| n.clone())
|
||||||
|
.filter(|n| cursor.is_none_or(|c| n.as_str() > c))
|
||||||
|
.collect();
|
||||||
|
names.sort();
|
||||||
|
let take = if limit == 0 {
|
||||||
|
usize::MAX
|
||||||
|
} else {
|
||||||
|
limit as usize
|
||||||
|
};
|
||||||
|
let next_cursor = if names.len() > take {
|
||||||
|
names.truncate(take);
|
||||||
|
names.last().cloned()
|
||||||
|
} else {
|
||||||
|
None
|
||||||
|
};
|
||||||
|
Ok(SecretsListPage { names, next_cursor })
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn make_engine() -> Arc<Engine> {
|
||||||
|
let services = Services::new(
|
||||||
|
Arc::new(NoopKvService),
|
||||||
|
Arc::new(NoopDocsService),
|
||||||
|
Arc::new(NoopDeadLetterService),
|
||||||
|
Arc::new(NoopEventEmitter),
|
||||||
|
Arc::new(NoopModuleSource),
|
||||||
|
Arc::new(NoopHttpService),
|
||||||
|
Arc::new(picloud_shared::NoopFilesService),
|
||||||
|
Arc::new(picloud_shared::NoopPubsubService),
|
||||||
|
Arc::new(InMemorySecrets::default()),
|
||||||
|
Arc::new(picloud_shared::NoopEmailService),
|
||||||
|
);
|
||||||
|
Arc::new(Engine::new(Limits::default(), services))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn baseline_request(app_id: AppId) -> ExecRequest {
|
||||||
|
let execution_id = ExecutionId::new();
|
||||||
|
ExecRequest {
|
||||||
|
execution_id,
|
||||||
|
request_id: RequestId::new(),
|
||||||
|
script_id: ScriptId::new(),
|
||||||
|
script_name: "secrets-test".into(),
|
||||||
|
invocation_type: InvocationType::Http,
|
||||||
|
path: "/secrets-test".into(),
|
||||||
|
headers: BTreeMap::new(),
|
||||||
|
body: Value::Null,
|
||||||
|
params: BTreeMap::new(),
|
||||||
|
query: BTreeMap::new(),
|
||||||
|
rest: String::new(),
|
||||||
|
sandbox_overrides: ScriptSandbox::default(),
|
||||||
|
app_id,
|
||||||
|
principal: None,
|
||||||
|
trigger_depth: 0,
|
||||||
|
root_execution_id: execution_id,
|
||||||
|
is_dead_letter_handler: false,
|
||||||
|
event: None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn run_script(engine: Arc<Engine>, src: &str, req: ExecRequest) -> Value {
|
||||||
|
let src = src.to_string();
|
||||||
|
tokio::task::spawn_blocking(move || engine.execute(&src, req))
|
||||||
|
.await
|
||||||
|
.expect("spawn_blocking should not panic")
|
||||||
|
.expect("script execution should succeed")
|
||||||
|
.body
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||||
|
async fn set_then_get_string_round_trips() {
|
||||||
|
let engine = make_engine();
|
||||||
|
let src = r#"
|
||||||
|
secrets::set("stripe_key", "sk_live_xxx");
|
||||||
|
secrets::get("stripe_key")
|
||||||
|
"#;
|
||||||
|
let body = run_script(engine, src, baseline_request(AppId::new())).await;
|
||||||
|
// A String comes back a String, not a JSON-quoted "\"sk_live_xxx\"".
|
||||||
|
assert_eq!(body, json!("sk_live_xxx"));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||||
|
async fn set_then_get_map_round_trips() {
|
||||||
|
let engine = make_engine();
|
||||||
|
let src = r#"
|
||||||
|
secrets::set("oauth", #{ client_id: "abc", client_secret: "xyz" });
|
||||||
|
secrets::get("oauth")
|
||||||
|
"#;
|
||||||
|
let body = run_script(engine, src, baseline_request(AppId::new())).await;
|
||||||
|
assert_eq!(body, json!({ "client_id": "abc", "client_secret": "xyz" }));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||||
|
async fn get_missing_returns_unit() {
|
||||||
|
let engine = make_engine();
|
||||||
|
let src = r#"
|
||||||
|
let v = secrets::get("nope");
|
||||||
|
#{ is_unit: type_of(v) == "()" }
|
||||||
|
"#;
|
||||||
|
let body = run_script(engine, src, baseline_request(AppId::new())).await;
|
||||||
|
assert_eq!(body, json!({ "is_unit": true }));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||||
|
async fn delete_returns_was_present() {
|
||||||
|
let engine = make_engine();
|
||||||
|
let src = r#"
|
||||||
|
secrets::set("k", "v");
|
||||||
|
let first = secrets::delete("k");
|
||||||
|
let second = secrets::delete("k");
|
||||||
|
#{ first: first, second: second }
|
||||||
|
"#;
|
||||||
|
let body = run_script(engine, src, baseline_request(AppId::new())).await;
|
||||||
|
assert_eq!(body, json!({ "first": true, "second": false }));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||||
|
async fn list_returns_names_and_cursor() {
|
||||||
|
let engine = make_engine();
|
||||||
|
let src = r#"
|
||||||
|
secrets::set("a", 1);
|
||||||
|
secrets::set("b", 2);
|
||||||
|
secrets::set("c", 3);
|
||||||
|
let page = secrets::list(#{ cursor: (), limit: 2 });
|
||||||
|
page
|
||||||
|
"#;
|
||||||
|
let body = run_script(engine, src, baseline_request(AppId::new())).await;
|
||||||
|
assert_eq!(body["names"], json!(["a", "b"]));
|
||||||
|
assert_eq!(body["next_cursor"], json!("b"));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||||
|
async fn empty_name_throws() {
|
||||||
|
let engine = make_engine();
|
||||||
|
let src = r#" secrets::set("", "v"); #{ ok: true } "#;
|
||||||
|
let app = AppId::new();
|
||||||
|
let out = tokio::task::spawn_blocking(move || engine.execute(src, baseline_request(app)))
|
||||||
|
.await
|
||||||
|
.expect("spawn_blocking");
|
||||||
|
assert!(out.is_err(), "empty secret name must throw");
|
||||||
|
}
|
||||||
244
crates/executor-core/tests/sdk_subscriber_token.rs
Normal file
244
crates/executor-core/tests/sdk_subscriber_token.rs
Normal file
@@ -0,0 +1,244 @@
|
|||||||
|
//! `pubsub::subscriber_token` SDK bridge integration tests (v1.1.6).
|
||||||
|
//!
|
||||||
|
//! Runs a real Rhai engine against a fake `PubsubService` whose
|
||||||
|
//! `mint_subscriber_token` mirrors the production validation (principal
|
||||||
|
//! required, non-empty topics, ttl clamp, externally-subscribable check)
|
||||||
|
//! and signs a real token. These cover the bridge surface: array →
|
||||||
|
//! `Vec<String>` forwarding, the omitted/`()`/integer ttl handling, and
|
||||||
|
//! errors surfacing as thrown Rhai errors. The authoritative validation
|
||||||
|
//! logic is unit-tested in `manager-core::pubsub_service`.
|
||||||
|
|
||||||
|
use std::collections::BTreeMap;
|
||||||
|
use std::sync::Arc;
|
||||||
|
|
||||||
|
use async_trait::async_trait;
|
||||||
|
use picloud_executor_core::{Engine, ExecRequest, InvocationType, Limits};
|
||||||
|
use picloud_shared::subscriber_token::{self, TokenClaims};
|
||||||
|
use picloud_shared::{
|
||||||
|
AdminUserId, AppId, ExecutionId, InstanceRole, NoopDeadLetterService, NoopDocsService,
|
||||||
|
NoopEventEmitter, NoopFilesService, NoopHttpService, NoopKvService, NoopModuleSource,
|
||||||
|
Principal, PubsubError, PubsubService, RequestId, ScriptId, ScriptSandbox, SdkCallCx, Services,
|
||||||
|
};
|
||||||
|
use serde_json::Value;
|
||||||
|
|
||||||
|
const FAKE_KEY: [u8; 32] = [7u8; 32];
|
||||||
|
const MIN_TTL: i64 = 10;
|
||||||
|
const MAX_TTL: i64 = 86_400;
|
||||||
|
const DEFAULT_TTL: i64 = 3_600;
|
||||||
|
|
||||||
|
/// Fake that mirrors the production mint rules and signs with FAKE_KEY.
|
||||||
|
#[derive(Default)]
|
||||||
|
struct FakeMintPubsub;
|
||||||
|
|
||||||
|
#[async_trait]
|
||||||
|
impl PubsubService for FakeMintPubsub {
|
||||||
|
async fn publish_durable(
|
||||||
|
&self,
|
||||||
|
_cx: &SdkCallCx,
|
||||||
|
_topic: &str,
|
||||||
|
_message: Value,
|
||||||
|
) -> Result<(), PubsubError> {
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn mint_subscriber_token(
|
||||||
|
&self,
|
||||||
|
cx: &SdkCallCx,
|
||||||
|
topics: Vec<String>,
|
||||||
|
ttl_seconds: Option<i64>,
|
||||||
|
) -> Result<String, PubsubError> {
|
||||||
|
if cx.principal.is_none() {
|
||||||
|
return Err(PubsubError::SubscriberToken(
|
||||||
|
"pubsub::subscriber_token: requires an authenticated principal".into(),
|
||||||
|
));
|
||||||
|
}
|
||||||
|
if topics.is_empty() {
|
||||||
|
return Err(PubsubError::SubscriberToken(
|
||||||
|
"pubsub::subscriber_token: topics list must not be empty".into(),
|
||||||
|
));
|
||||||
|
}
|
||||||
|
let ttl = ttl_seconds.unwrap_or(DEFAULT_TTL);
|
||||||
|
if !(MIN_TTL..=MAX_TTL).contains(&ttl) {
|
||||||
|
return Err(PubsubError::SubscriberToken(format!(
|
||||||
|
"pubsub::subscriber_token: ttl_seconds must be between {MIN_TTL} and {MAX_TTL}"
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
for name in &topics {
|
||||||
|
// Only "chat" and "notify" are "registered" in this fake.
|
||||||
|
if name != "chat" && name != "notify" {
|
||||||
|
return Err(PubsubError::SubscriberToken(format!(
|
||||||
|
"pubsub::subscriber_token: topic {name} is not externally subscribable"
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let now = 1_000_000;
|
||||||
|
Ok(subscriber_token::sign(
|
||||||
|
&FAKE_KEY,
|
||||||
|
&TokenClaims {
|
||||||
|
app_id: cx.app_id,
|
||||||
|
topics,
|
||||||
|
exp: now + ttl,
|
||||||
|
iat: now,
|
||||||
|
},
|
||||||
|
))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn make_engine() -> Arc<Engine> {
|
||||||
|
let services = Services::new(
|
||||||
|
Arc::new(NoopKvService),
|
||||||
|
Arc::new(NoopDocsService),
|
||||||
|
Arc::new(NoopDeadLetterService),
|
||||||
|
Arc::new(NoopEventEmitter),
|
||||||
|
Arc::new(NoopModuleSource),
|
||||||
|
Arc::new(NoopHttpService),
|
||||||
|
Arc::new(NoopFilesService),
|
||||||
|
Arc::new(FakeMintPubsub),
|
||||||
|
Arc::new(picloud_shared::NoopSecretsService),
|
||||||
|
Arc::new(picloud_shared::NoopEmailService),
|
||||||
|
);
|
||||||
|
Arc::new(Engine::new(Limits::default(), services))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn request(app_id: AppId, with_principal: bool) -> ExecRequest {
|
||||||
|
let execution_id = ExecutionId::new();
|
||||||
|
ExecRequest {
|
||||||
|
execution_id,
|
||||||
|
request_id: RequestId::new(),
|
||||||
|
script_id: ScriptId::new(),
|
||||||
|
script_name: "token-test".into(),
|
||||||
|
invocation_type: InvocationType::Http,
|
||||||
|
path: "/token-test".into(),
|
||||||
|
headers: BTreeMap::new(),
|
||||||
|
body: Value::Null,
|
||||||
|
params: BTreeMap::new(),
|
||||||
|
query: BTreeMap::new(),
|
||||||
|
rest: String::new(),
|
||||||
|
sandbox_overrides: ScriptSandbox::default(),
|
||||||
|
app_id,
|
||||||
|
principal: with_principal.then(|| Principal {
|
||||||
|
user_id: AdminUserId::new(),
|
||||||
|
instance_role: InstanceRole::Owner,
|
||||||
|
scopes: None,
|
||||||
|
app_binding: None,
|
||||||
|
}),
|
||||||
|
trigger_depth: 0,
|
||||||
|
root_execution_id: execution_id,
|
||||||
|
is_dead_letter_handler: false,
|
||||||
|
event: None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn run_ok(engine: Arc<Engine>, src: &str, req: ExecRequest) -> Value {
|
||||||
|
let src = src.to_string();
|
||||||
|
tokio::task::spawn_blocking(move || engine.execute(&src, req))
|
||||||
|
.await
|
||||||
|
.expect("spawn_blocking should not panic")
|
||||||
|
.expect("script execution should succeed")
|
||||||
|
.body
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn run_err(engine: Arc<Engine>, src: &str, req: ExecRequest) {
|
||||||
|
let src = src.to_string();
|
||||||
|
let res = tokio::task::spawn_blocking(move || engine.execute(&src, req))
|
||||||
|
.await
|
||||||
|
.expect("spawn_blocking should not panic");
|
||||||
|
assert!(res.is_err(), "expected script to throw");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||||
|
async fn token_contains_topics_and_expiry() {
|
||||||
|
let app = AppId::new();
|
||||||
|
let body = run_ok(
|
||||||
|
make_engine(),
|
||||||
|
r#"#{ token: pubsub::subscriber_token(["chat", "notify"], 120) }"#,
|
||||||
|
request(app, true),
|
||||||
|
)
|
||||||
|
.await;
|
||||||
|
let token = body["token"].as_str().expect("token string");
|
||||||
|
let claims = subscriber_token::verify(&FAKE_KEY, token, 1_000_001).unwrap();
|
||||||
|
assert_eq!(claims.app_id, app);
|
||||||
|
assert_eq!(
|
||||||
|
claims.topics,
|
||||||
|
vec!["chat".to_string(), "notify".to_string()]
|
||||||
|
);
|
||||||
|
assert_eq!(claims.exp - claims.iat, 120);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||||
|
async fn omitted_ttl_uses_default() {
|
||||||
|
let app = AppId::new();
|
||||||
|
let body = run_ok(
|
||||||
|
make_engine(),
|
||||||
|
r#"#{ token: pubsub::subscriber_token(["chat"]) }"#,
|
||||||
|
request(app, true),
|
||||||
|
)
|
||||||
|
.await;
|
||||||
|
let token = body["token"].as_str().unwrap();
|
||||||
|
let claims = subscriber_token::verify(&FAKE_KEY, token, 1_000_001).unwrap();
|
||||||
|
assert_eq!(claims.exp - claims.iat, DEFAULT_TTL);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||||
|
async fn unit_ttl_uses_default() {
|
||||||
|
let app = AppId::new();
|
||||||
|
let body = run_ok(
|
||||||
|
make_engine(),
|
||||||
|
r#"#{ token: pubsub::subscriber_token(["chat"], ()) }"#,
|
||||||
|
request(app, true),
|
||||||
|
)
|
||||||
|
.await;
|
||||||
|
let token = body["token"].as_str().unwrap();
|
||||||
|
let claims = subscriber_token::verify(&FAKE_KEY, token, 1_000_001).unwrap();
|
||||||
|
assert_eq!(claims.exp - claims.iat, DEFAULT_TTL);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||||
|
async fn empty_topics_throws() {
|
||||||
|
run_err(
|
||||||
|
make_engine(),
|
||||||
|
r"pubsub::subscriber_token([], 60)",
|
||||||
|
request(AppId::new(), true),
|
||||||
|
)
|
||||||
|
.await;
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||||
|
async fn ttl_below_min_throws() {
|
||||||
|
run_err(
|
||||||
|
make_engine(),
|
||||||
|
r#"pubsub::subscriber_token(["chat"], 5)"#,
|
||||||
|
request(AppId::new(), true),
|
||||||
|
)
|
||||||
|
.await;
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||||
|
async fn ttl_above_max_throws() {
|
||||||
|
run_err(
|
||||||
|
make_engine(),
|
||||||
|
r#"pubsub::subscriber_token(["chat"], 90000)"#,
|
||||||
|
request(AppId::new(), true),
|
||||||
|
)
|
||||||
|
.await;
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||||
|
async fn anonymous_principal_throws() {
|
||||||
|
run_err(
|
||||||
|
make_engine(),
|
||||||
|
r#"pubsub::subscriber_token(["chat"], 60)"#,
|
||||||
|
request(AppId::new(), false),
|
||||||
|
)
|
||||||
|
.await;
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||||
|
async fn unregistered_topic_throws() {
|
||||||
|
run_err(
|
||||||
|
make_engine(),
|
||||||
|
r#"pubsub::subscriber_token(["chat", "secret"], 60)"#,
|
||||||
|
request(AppId::new(), true),
|
||||||
|
)
|
||||||
|
.await;
|
||||||
|
}
|
||||||
@@ -31,8 +31,13 @@ reqwest.workspace = true
|
|||||||
|
|
||||||
argon2.workspace = true
|
argon2.workspace = true
|
||||||
sha2.workspace = true
|
sha2.workspace = true
|
||||||
|
# HMAC-SHA256 verification of inbound-email provider signatures (v1.1.7).
|
||||||
|
hmac.workspace = true
|
||||||
|
hex.workspace = true
|
||||||
base64.workspace = true
|
base64.workspace = true
|
||||||
data-encoding.workspace = true
|
data-encoding.workspace = true
|
||||||
|
# Outbound SMTP email (v1.1.7 email::send / send_html).
|
||||||
|
lettre.workspace = true
|
||||||
|
|
||||||
[dev-dependencies]
|
[dev-dependencies]
|
||||||
tokio.workspace = true
|
tokio.workspace = true
|
||||||
|
|||||||
31
crates/manager-core/migrations/0021_topics.sql
Normal file
31
crates/manager-core/migrations/0021_topics.sql
Normal file
@@ -0,0 +1,31 @@
|
|||||||
|
-- v1.1.6: Explicit registration for externally-subscribable topics.
|
||||||
|
--
|
||||||
|
-- Internal-only topics remain implicit per the §5 design-notes
|
||||||
|
-- decision: anyone can publish_durable("any.topic", msg) and triggers
|
||||||
|
-- can subscribe without a row here. This table only holds topics that
|
||||||
|
-- have been explicitly externalized — external SSE subscribers can
|
||||||
|
-- only subscribe to topics with a row here AND external_subscribable
|
||||||
|
-- = TRUE.
|
||||||
|
--
|
||||||
|
-- The publish path (v1.1.5's publish_durable) does NOT consult this
|
||||||
|
-- table: publishing to a topic with no row still fans out to triggers
|
||||||
|
-- and to any in-process external subscribers (none exist for an
|
||||||
|
-- unregistered topic, since external subscribers can't subscribe to
|
||||||
|
-- one). The topics table is read by the SSE subscribe path only.
|
||||||
|
--
|
||||||
|
-- auth_mode values: 'public' + 'token' in v1.1.6. 'session' arrives in
|
||||||
|
-- v1.1.8 (users-SDK); 'script' arrives in v1.2 (script-mediated auth).
|
||||||
|
-- The CHECK constraint extends in those releases.
|
||||||
|
CREATE TABLE topics (
|
||||||
|
app_id UUID NOT NULL REFERENCES apps(id) ON DELETE CASCADE,
|
||||||
|
name TEXT NOT NULL,
|
||||||
|
external_subscribable BOOL NOT NULL DEFAULT FALSE,
|
||||||
|
auth_mode TEXT NOT NULL DEFAULT 'public'
|
||||||
|
CHECK (auth_mode IN ('public', 'token')),
|
||||||
|
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||||
|
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||||
|
PRIMARY KEY (app_id, name)
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Hot lookup: "is topic T in app X externally subscribable?" The PK
|
||||||
|
-- (app_id, name) already covers this; an explicit index is redundant.
|
||||||
19
crates/manager-core/migrations/0022_app_secrets.sql
Normal file
19
crates/manager-core/migrations/0022_app_secrets.sql
Normal file
@@ -0,0 +1,19 @@
|
|||||||
|
-- v1.1.6: per-app secret material. Currently holds the HMAC signing key
|
||||||
|
-- used to mint + verify realtime subscriber tokens
|
||||||
|
-- (pubsub::subscriber_token → SSE /realtime/topics handshake).
|
||||||
|
--
|
||||||
|
-- The key is:
|
||||||
|
-- * stable across restarts (issued tokens stay valid until expiry),
|
||||||
|
-- * per-app (a token signed by app A is rejected by app B),
|
||||||
|
-- * never script-accessible (scripts can't print/exfiltrate it — the
|
||||||
|
-- SDK only mints tokens, it never returns the key).
|
||||||
|
--
|
||||||
|
-- The row is created lazily on the first pubsub::subscriber_token call
|
||||||
|
-- for an app (32 random bytes). This table is the natural home for
|
||||||
|
-- v1.1.7's encrypted per-app secrets work.
|
||||||
|
CREATE TABLE app_secrets (
|
||||||
|
app_id UUID PRIMARY KEY REFERENCES apps(id) ON DELETE CASCADE,
|
||||||
|
realtime_signing_key BYTEA NOT NULL, -- 32 random bytes
|
||||||
|
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||||
|
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
|
||||||
|
);
|
||||||
24
crates/manager-core/migrations/0023_secrets.sql
Normal file
24
crates/manager-core/migrations/0023_secrets.sql
Normal file
@@ -0,0 +1,24 @@
|
|||||||
|
-- v1.1.7: encrypted per-app secrets.
|
||||||
|
--
|
||||||
|
-- Operational config (API keys, OAuth tokens, webhook signing keys)
|
||||||
|
-- encrypted at rest with the process master key (AES-256-GCM). Both the
|
||||||
|
-- ciphertext (16-byte GCM auth tag appended) and the 12-byte nonce are
|
||||||
|
-- stored; the master key itself never lives in the database. See
|
||||||
|
-- `picloud_shared::crypto` + `manager-core::secrets_service`.
|
||||||
|
--
|
||||||
|
-- This is the user-facing `secrets::*` store. It is intentionally
|
||||||
|
-- separate from `app_secrets` (the one-row-per-app realtime signing
|
||||||
|
-- key, 0022): different cardinality (many named rows per app), and the
|
||||||
|
-- realtime key is encrypted in place by migration 0025.
|
||||||
|
|
||||||
|
CREATE TABLE secrets (
|
||||||
|
app_id UUID NOT NULL REFERENCES apps(id) ON DELETE CASCADE,
|
||||||
|
name TEXT NOT NULL,
|
||||||
|
encrypted_value BYTEA NOT NULL, -- ciphertext incl. 16-byte GCM auth tag
|
||||||
|
nonce BYTEA NOT NULL, -- 12 bytes
|
||||||
|
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||||
|
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||||
|
PRIMARY KEY (app_id, name)
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE INDEX idx_secrets_app ON secrets (app_id);
|
||||||
32
crates/manager-core/migrations/0024_email_triggers.sql
Normal file
32
crates/manager-core/migrations/0024_email_triggers.sql
Normal file
@@ -0,0 +1,32 @@
|
|||||||
|
-- v1.1.7: inbound email triggers (email:receive).
|
||||||
|
--
|
||||||
|
-- A configured provider (Mailgun / Postmark / SendGrid / SES) POSTs
|
||||||
|
-- inbound email to POST /api/v1/email-inbound/{app_id}/{trigger_id};
|
||||||
|
-- the receiver normalizes it into a TriggerEvent::Email and enqueues an
|
||||||
|
-- outbox row for the trigger's handler. v1.1.7 ships the webhook path;
|
||||||
|
-- a native SMTP listener is v1.3+.
|
||||||
|
|
||||||
|
-- Widen the trigger-kind + outbox-source CHECK constraints to admit
|
||||||
|
-- 'email'.
|
||||||
|
ALTER TABLE triggers DROP CONSTRAINT triggers_kind_check;
|
||||||
|
ALTER TABLE triggers ADD CONSTRAINT triggers_kind_check
|
||||||
|
CHECK (kind IN ('kv', 'dead_letter', 'docs', 'cron',
|
||||||
|
'files', 'pubsub', 'email'));
|
||||||
|
|
||||||
|
ALTER TABLE outbox DROP CONSTRAINT outbox_source_kind_check;
|
||||||
|
ALTER TABLE outbox ADD CONSTRAINT outbox_source_kind_check
|
||||||
|
CHECK (source_kind IN ('http', 'kv', 'dead_letter', 'docs',
|
||||||
|
'cron', 'files', 'pubsub', 'email'));
|
||||||
|
|
||||||
|
-- Per-trigger inbound config. The HMAC secret used to verify provider
|
||||||
|
-- signatures is stored ENCRYPTED at rest (AES-256-GCM under the process
|
||||||
|
-- master key) — a deviation from the original brief's plaintext column,
|
||||||
|
-- chosen to keep all operationally-secret material encrypted. The
|
||||||
|
-- receiver decrypts it per inbound request. NULL columns mean the
|
||||||
|
-- trigger has no signature verification (accepts any POST to its URL —
|
||||||
|
-- relies on URL secrecy).
|
||||||
|
CREATE TABLE email_trigger_details (
|
||||||
|
trigger_id UUID PRIMARY KEY REFERENCES triggers(id) ON DELETE CASCADE,
|
||||||
|
inbound_secret_encrypted BYTEA, -- ciphertext incl. GCM auth tag (NULL = unsigned)
|
||||||
|
inbound_secret_nonce BYTEA -- 12 bytes (NULL = unsigned)
|
||||||
|
);
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
-- v1.1.7: encrypt the realtime signing key at rest (two-phase).
|
||||||
|
--
|
||||||
|
-- Phase 1 (this migration + the v1.1.7 startup task):
|
||||||
|
-- * add NULL-able encrypted columns,
|
||||||
|
-- * drop the NOT NULL on the plaintext column so newly-generated keys
|
||||||
|
-- can be stored encrypted-only,
|
||||||
|
-- * the application startup task `migrate_plaintext_keys` encrypts each
|
||||||
|
-- existing plaintext key into the new columns (plaintext is LEFT in
|
||||||
|
-- place during the compat window for rollback safety).
|
||||||
|
--
|
||||||
|
-- The `RealtimeAuthorityImpl` read path prefers the encrypted columns and
|
||||||
|
-- falls back to plaintext, so SSE keeps working throughout.
|
||||||
|
--
|
||||||
|
-- Phase 2 (v1.1.8): once all rows are migrated, a follow-up migration
|
||||||
|
-- drops the plaintext `realtime_signing_key` column.
|
||||||
|
|
||||||
|
ALTER TABLE app_secrets
|
||||||
|
ADD COLUMN realtime_signing_key_encrypted BYTEA,
|
||||||
|
ADD COLUMN realtime_signing_key_nonce BYTEA;
|
||||||
|
|
||||||
|
-- New keys (post-v1.1.7) are stored encrypted-only, so the plaintext
|
||||||
|
-- column must accept NULL.
|
||||||
|
ALTER TABLE app_secrets
|
||||||
|
ALTER COLUMN realtime_signing_key DROP NOT NULL;
|
||||||
241
crates/manager-core/src/app_secrets_repo.rs
Normal file
241
crates/manager-core/src/app_secrets_repo.rs
Normal file
@@ -0,0 +1,241 @@
|
|||||||
|
//! `AppSecretsRepo` — per-app secret material (v1.1.6, encrypted v1.1.7).
|
||||||
|
//!
|
||||||
|
//! Holds the HMAC signing key for realtime subscriber tokens. The key is
|
||||||
|
//! generated lazily (32 random bytes) on the first
|
||||||
|
//! `pubsub::subscriber_token` call for an app and never changes
|
||||||
|
//! thereafter (no rotation API yet). The key is never exposed to
|
||||||
|
//! scripts: the SDK mints tokens, it never returns the key.
|
||||||
|
//!
|
||||||
|
//! **v1.1.7 at-rest encryption (two-phase).** The key is now sealed with
|
||||||
|
//! the process master key (AES-256-GCM). New keys are written
|
||||||
|
//! encrypted-only; the startup task [`PostgresAppSecretsRepo::migrate_plaintext_keys`]
|
||||||
|
//! encrypts any pre-existing plaintext rows. The read path prefers the
|
||||||
|
//! encrypted columns and falls back to the plaintext column during the
|
||||||
|
//! compat window (migration 0025 made it NULL-able; v1.1.8 drops it).
|
||||||
|
|
||||||
|
use async_trait::async_trait;
|
||||||
|
use picloud_shared::{crypto, AppId, MasterKey};
|
||||||
|
use rand::RngCore;
|
||||||
|
use sqlx::PgPool;
|
||||||
|
use uuid::Uuid;
|
||||||
|
|
||||||
|
/// Length of a freshly-generated realtime signing key.
|
||||||
|
pub const SIGNING_KEY_LEN: usize = 32;
|
||||||
|
|
||||||
|
#[derive(Debug, thiserror::Error)]
|
||||||
|
pub enum AppSecretsRepoError {
|
||||||
|
#[error("database error: {0}")]
|
||||||
|
Db(#[from] sqlx::Error),
|
||||||
|
|
||||||
|
/// A stored encrypted signing key could not be decrypted — corrupted
|
||||||
|
/// row or a master-key mismatch (e.g. `PICLOUD_SECRET_KEY` changed).
|
||||||
|
#[error("realtime signing key could not be decrypted (corrupted row or master-key mismatch)")]
|
||||||
|
Crypto,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[async_trait]
|
||||||
|
pub trait AppSecretsRepo: Send + Sync {
|
||||||
|
/// Fetch the app's realtime signing key, generating + persisting one
|
||||||
|
/// (32 random bytes, encrypted) if absent. Idempotent under
|
||||||
|
/// concurrency: a racing creator's `ON CONFLICT DO NOTHING` insert is
|
||||||
|
/// a no-op and the existing key is returned.
|
||||||
|
async fn get_or_create_signing_key(
|
||||||
|
&self,
|
||||||
|
app_id: AppId,
|
||||||
|
) -> Result<Vec<u8>, AppSecretsRepoError>;
|
||||||
|
|
||||||
|
/// Fetch the signing key if it exists, WITHOUT creating one. The SSE
|
||||||
|
/// verify path uses this: a missing key means no token was ever
|
||||||
|
/// minted for the app, so any presented token must be rejected.
|
||||||
|
async fn signing_key(&self, app_id: AppId) -> Result<Option<Vec<u8>>, AppSecretsRepoError>;
|
||||||
|
}
|
||||||
|
|
||||||
|
pub struct PostgresAppSecretsRepo {
|
||||||
|
pool: PgPool,
|
||||||
|
master_key: MasterKey,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl PostgresAppSecretsRepo {
|
||||||
|
#[must_use]
|
||||||
|
pub fn new(pool: PgPool, master_key: MasterKey) -> Self {
|
||||||
|
Self { pool, master_key }
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Startup task (v1.1.7): encrypt every row that still has a
|
||||||
|
/// plaintext key but no encrypted key. Plaintext is left in place
|
||||||
|
/// (the read path prefers the encrypted columns); the plaintext
|
||||||
|
/// column is dropped in v1.1.8. Returns the number of rows migrated.
|
||||||
|
///
|
||||||
|
/// # Errors
|
||||||
|
///
|
||||||
|
/// Propagates database errors.
|
||||||
|
pub async fn migrate_plaintext_keys(&self) -> Result<usize, AppSecretsRepoError> {
|
||||||
|
let rows: Vec<(Uuid, Vec<u8>)> = sqlx::query_as(
|
||||||
|
"SELECT app_id, realtime_signing_key FROM app_secrets \
|
||||||
|
WHERE realtime_signing_key_encrypted IS NULL \
|
||||||
|
AND realtime_signing_key IS NOT NULL",
|
||||||
|
)
|
||||||
|
.fetch_all(&self.pool)
|
||||||
|
.await?;
|
||||||
|
|
||||||
|
let mut migrated = 0;
|
||||||
|
for (app_id, plaintext) in rows {
|
||||||
|
let enc = crypto::encrypt(&plaintext, self.master_key.as_bytes());
|
||||||
|
sqlx::query(
|
||||||
|
"UPDATE app_secrets \
|
||||||
|
SET realtime_signing_key_encrypted = $2, \
|
||||||
|
realtime_signing_key_nonce = $3, \
|
||||||
|
updated_at = NOW() \
|
||||||
|
WHERE app_id = $1 AND realtime_signing_key_encrypted IS NULL",
|
||||||
|
)
|
||||||
|
.bind(app_id)
|
||||||
|
.bind(&enc.ciphertext)
|
||||||
|
.bind(&enc.nonce[..])
|
||||||
|
.execute(&self.pool)
|
||||||
|
.await?;
|
||||||
|
migrated += 1;
|
||||||
|
}
|
||||||
|
Ok(migrated)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn decode(
|
||||||
|
&self,
|
||||||
|
encrypted: Option<Vec<u8>>,
|
||||||
|
nonce: Option<Vec<u8>>,
|
||||||
|
plaintext: Option<Vec<u8>>,
|
||||||
|
) -> Result<Option<Vec<u8>>, AppSecretsRepoError> {
|
||||||
|
decode_signing_key(&self.master_key, encrypted, nonce, plaintext)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Resolve the signing key from a row's three columns. **Encrypted wins**
|
||||||
|
/// when present; otherwise fall back to the plaintext column (compat for
|
||||||
|
/// un-migrated rows / the post-v1.1.8 dropped-plaintext state).
|
||||||
|
fn decode_signing_key(
|
||||||
|
master_key: &MasterKey,
|
||||||
|
encrypted: Option<Vec<u8>>,
|
||||||
|
nonce: Option<Vec<u8>>,
|
||||||
|
plaintext: Option<Vec<u8>>,
|
||||||
|
) -> Result<Option<Vec<u8>>, AppSecretsRepoError> {
|
||||||
|
match (encrypted, nonce) {
|
||||||
|
(Some(ct), Some(n)) => {
|
||||||
|
let key = crypto::decrypt(&ct, &n, master_key.as_bytes())
|
||||||
|
.map_err(|_| AppSecretsRepoError::Crypto)?;
|
||||||
|
Ok(Some(key))
|
||||||
|
}
|
||||||
|
_ => Ok(plaintext),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[async_trait]
|
||||||
|
impl AppSecretsRepo for PostgresAppSecretsRepo {
|
||||||
|
async fn get_or_create_signing_key(
|
||||||
|
&self,
|
||||||
|
app_id: AppId,
|
||||||
|
) -> Result<Vec<u8>, AppSecretsRepoError> {
|
||||||
|
let mut fresh = vec![0u8; SIGNING_KEY_LEN];
|
||||||
|
rand::thread_rng().fill_bytes(&mut fresh);
|
||||||
|
let enc = crypto::encrypt(&fresh, self.master_key.as_bytes());
|
||||||
|
|
||||||
|
// Insert-if-absent (encrypted-only). The racing-creator's insert
|
||||||
|
// is a no-op; the SELECT always returns the winning row.
|
||||||
|
sqlx::query(
|
||||||
|
"INSERT INTO app_secrets \
|
||||||
|
(app_id, realtime_signing_key_encrypted, realtime_signing_key_nonce) \
|
||||||
|
VALUES ($1, $2, $3) ON CONFLICT (app_id) DO NOTHING",
|
||||||
|
)
|
||||||
|
.bind(app_id.into_inner())
|
||||||
|
.bind(&enc.ciphertext)
|
||||||
|
.bind(&enc.nonce[..])
|
||||||
|
.execute(&self.pool)
|
||||||
|
.await?;
|
||||||
|
|
||||||
|
let row: (Option<Vec<u8>>, Option<Vec<u8>>, Option<Vec<u8>>) = sqlx::query_as(
|
||||||
|
"SELECT realtime_signing_key_encrypted, realtime_signing_key_nonce, \
|
||||||
|
realtime_signing_key \
|
||||||
|
FROM app_secrets WHERE app_id = $1",
|
||||||
|
)
|
||||||
|
.bind(app_id.into_inner())
|
||||||
|
.fetch_one(&self.pool)
|
||||||
|
.await?;
|
||||||
|
// A row exists by construction, so a key must decode.
|
||||||
|
self.decode(row.0, row.1, row.2)?
|
||||||
|
.ok_or(AppSecretsRepoError::Crypto)
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn signing_key(&self, app_id: AppId) -> Result<Option<Vec<u8>>, AppSecretsRepoError> {
|
||||||
|
let row: Option<(Option<Vec<u8>>, Option<Vec<u8>>, Option<Vec<u8>>)> = sqlx::query_as(
|
||||||
|
"SELECT realtime_signing_key_encrypted, realtime_signing_key_nonce, \
|
||||||
|
realtime_signing_key \
|
||||||
|
FROM app_secrets WHERE app_id = $1",
|
||||||
|
)
|
||||||
|
.bind(app_id.into_inner())
|
||||||
|
.fetch_optional(&self.pool)
|
||||||
|
.await?;
|
||||||
|
match row {
|
||||||
|
Some((e, n, p)) => self.decode(e, n, p),
|
||||||
|
None => Ok(None),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
fn key() -> MasterKey {
|
||||||
|
MasterKey::from_bytes([9u8; 32])
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn encrypted_wins_over_plaintext() {
|
||||||
|
let mk = key();
|
||||||
|
let secret = vec![1u8, 2, 3, 4];
|
||||||
|
let enc = crypto::encrypt(&secret, mk.as_bytes());
|
||||||
|
// Both present → the encrypted value is returned (not the bogus
|
||||||
|
// plaintext).
|
||||||
|
let got = decode_signing_key(
|
||||||
|
&mk,
|
||||||
|
Some(enc.ciphertext),
|
||||||
|
Some(enc.nonce.to_vec()),
|
||||||
|
Some(vec![0xff; 32]),
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(got, Some(secret));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn falls_back_to_plaintext_when_encrypted_absent() {
|
||||||
|
let mk = key();
|
||||||
|
let plaintext = vec![7u8; 32];
|
||||||
|
let got = decode_signing_key(&mk, None, None, Some(plaintext.clone())).unwrap();
|
||||||
|
assert_eq!(got, Some(plaintext));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn encrypted_present_plaintext_null_works() {
|
||||||
|
// Post-v1.1.8 state: only the encrypted columns are populated.
|
||||||
|
let mk = key();
|
||||||
|
let secret = vec![5u8; 32];
|
||||||
|
let enc = crypto::encrypt(&secret, mk.as_bytes());
|
||||||
|
let got =
|
||||||
|
decode_signing_key(&mk, Some(enc.ciphertext), Some(enc.nonce.to_vec()), None).unwrap();
|
||||||
|
assert_eq!(got, Some(secret));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn missing_everything_is_none() {
|
||||||
|
let got = decode_signing_key(&key(), None, None, None).unwrap();
|
||||||
|
assert_eq!(got, None);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn wrong_master_key_is_crypto_error() {
|
||||||
|
let secret = vec![3u8; 32];
|
||||||
|
let enc = crypto::encrypt(&secret, key().as_bytes());
|
||||||
|
let other = MasterKey::from_bytes([1u8; 32]);
|
||||||
|
let err = decode_signing_key(&other, Some(enc.ciphertext), Some(enc.nonce.to_vec()), None)
|
||||||
|
.unwrap_err();
|
||||||
|
assert!(matches!(err, AppSecretsRepoError::Crypto));
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -89,6 +89,18 @@ pub enum Capability {
|
|||||||
/// (v1.1.5). Maps to `script:write` on API keys (a publish is a
|
/// (v1.1.5). Maps to `script:write` on API keys (a publish is a
|
||||||
/// write that fans out to subscribers). Granted to `editor`+.
|
/// write that fans out to subscribers). Granted to `editor`+.
|
||||||
AppPubsubPublish(AppId),
|
AppPubsubPublish(AppId),
|
||||||
|
/// Read a decrypted secret from this app's secrets store (v1.1.7).
|
||||||
|
/// Same trust shape as KV/docs/files read — granted to `viewer`+,
|
||||||
|
/// maps to `script:read` on API keys. Honors the seven-scope
|
||||||
|
/// commitment.
|
||||||
|
AppSecretsRead(AppId),
|
||||||
|
/// Write (set/delete) a secret in this app's secrets store (v1.1.7).
|
||||||
|
/// Granted to `editor`+, maps to `script:write` on API keys.
|
||||||
|
AppSecretsWrite(AppId),
|
||||||
|
/// Send an outbound email from a script in this app (v1.1.7). Maps
|
||||||
|
/// to `script:write` on API keys (sending mail is an outbound
|
||||||
|
/// side-effect like an HTTP request). Granted to `editor`+.
|
||||||
|
AppEmailSend(AppId),
|
||||||
/// Create / list / delete triggers for this app (v1.1.1). Maps to
|
/// Create / list / delete triggers for this app (v1.1.1). Maps to
|
||||||
/// `app:admin` on API keys — triggers are app-configuration acts
|
/// `app:admin` on API keys — triggers are app-configuration acts
|
||||||
/// rather than data-plane access. Granted to `app_admin`+.
|
/// rather than data-plane access. Granted to `app_admin`+.
|
||||||
@@ -97,6 +109,12 @@ pub enum Capability {
|
|||||||
/// to `app:admin` on API keys. Public-HTTP scripts (principal None)
|
/// to `app:admin` on API keys. Public-HTTP scripts (principal None)
|
||||||
/// fail this check — managing dead letters is an admin act.
|
/// fail this check — managing dead letters is an admin act.
|
||||||
AppDeadLetterManage(AppId),
|
AppDeadLetterManage(AppId),
|
||||||
|
/// Register / list / update / delete externally-subscribable topics
|
||||||
|
/// for this app (v1.1.6). Maps to `app:admin` on API keys —
|
||||||
|
/// externalizing a topic is an app-configuration act with security
|
||||||
|
/// weight (it opens an internal pub/sub topic to outside SSE
|
||||||
|
/// subscribers). Granted to `app_admin`+.
|
||||||
|
AppTopicManage(AppId),
|
||||||
}
|
}
|
||||||
|
|
||||||
impl Capability {
|
impl Capability {
|
||||||
@@ -122,8 +140,12 @@ impl Capability {
|
|||||||
| Self::AppFilesRead(id)
|
| Self::AppFilesRead(id)
|
||||||
| Self::AppFilesWrite(id)
|
| Self::AppFilesWrite(id)
|
||||||
| Self::AppPubsubPublish(id)
|
| Self::AppPubsubPublish(id)
|
||||||
|
| Self::AppSecretsRead(id)
|
||||||
|
| Self::AppSecretsWrite(id)
|
||||||
|
| Self::AppEmailSend(id)
|
||||||
| Self::AppManageTriggers(id)
|
| Self::AppManageTriggers(id)
|
||||||
| Self::AppDeadLetterManage(id) => Some(id),
|
| Self::AppDeadLetterManage(id)
|
||||||
|
| Self::AppTopicManage(id) => Some(id),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -141,18 +163,22 @@ impl Capability {
|
|||||||
Self::AppRead(_)
|
Self::AppRead(_)
|
||||||
| Self::AppKvRead(_)
|
| Self::AppKvRead(_)
|
||||||
| Self::AppDocsRead(_)
|
| Self::AppDocsRead(_)
|
||||||
| Self::AppFilesRead(_) => Scope::ScriptRead,
|
| Self::AppFilesRead(_)
|
||||||
|
| Self::AppSecretsRead(_) => Scope::ScriptRead,
|
||||||
Self::AppWriteScript(_)
|
Self::AppWriteScript(_)
|
||||||
| Self::AppKvWrite(_)
|
| Self::AppKvWrite(_)
|
||||||
| Self::AppDocsWrite(_)
|
| Self::AppDocsWrite(_)
|
||||||
| Self::AppHttpRequest(_)
|
| Self::AppHttpRequest(_)
|
||||||
| Self::AppFilesWrite(_)
|
| Self::AppFilesWrite(_)
|
||||||
| Self::AppPubsubPublish(_) => Scope::ScriptWrite,
|
| Self::AppPubsubPublish(_)
|
||||||
|
| Self::AppSecretsWrite(_)
|
||||||
|
| Self::AppEmailSend(_) => Scope::ScriptWrite,
|
||||||
Self::AppWriteRoute(_) => Scope::RouteWrite,
|
Self::AppWriteRoute(_) => Scope::RouteWrite,
|
||||||
Self::AppManageDomains(_) => Scope::DomainManage,
|
Self::AppManageDomains(_) => Scope::DomainManage,
|
||||||
Self::AppAdmin(_) | Self::AppManageTriggers(_) | Self::AppDeadLetterManage(_) => {
|
Self::AppAdmin(_)
|
||||||
Scope::AppAdmin
|
| Self::AppManageTriggers(_)
|
||||||
}
|
| Self::AppDeadLetterManage(_)
|
||||||
|
| Self::AppTopicManage(_) => Scope::AppAdmin,
|
||||||
Self::AppLogRead(_) => Scope::LogRead,
|
Self::AppLogRead(_) => Scope::LogRead,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -297,6 +323,7 @@ const fn role_satisfies(role: AppRole, cap: Capability) -> bool {
|
|||||||
| Capability::AppKvRead(_)
|
| Capability::AppKvRead(_)
|
||||||
| Capability::AppDocsRead(_)
|
| Capability::AppDocsRead(_)
|
||||||
| Capability::AppFilesRead(_)
|
| Capability::AppFilesRead(_)
|
||||||
|
| Capability::AppSecretsRead(_)
|
||||||
);
|
);
|
||||||
let in_editor = in_viewer
|
let in_editor = in_viewer
|
||||||
|| matches!(
|
|| matches!(
|
||||||
@@ -308,6 +335,8 @@ const fn role_satisfies(role: AppRole, cap: Capability) -> bool {
|
|||||||
| Capability::AppHttpRequest(_)
|
| Capability::AppHttpRequest(_)
|
||||||
| Capability::AppFilesWrite(_)
|
| Capability::AppFilesWrite(_)
|
||||||
| Capability::AppPubsubPublish(_)
|
| Capability::AppPubsubPublish(_)
|
||||||
|
| Capability::AppSecretsWrite(_)
|
||||||
|
| Capability::AppEmailSend(_)
|
||||||
);
|
);
|
||||||
let in_app_admin = in_editor
|
let in_app_admin = in_editor
|
||||||
|| matches!(
|
|| matches!(
|
||||||
@@ -316,6 +345,7 @@ const fn role_satisfies(role: AppRole, cap: Capability) -> bool {
|
|||||||
| Capability::AppAdmin(_)
|
| Capability::AppAdmin(_)
|
||||||
| Capability::AppManageTriggers(_)
|
| Capability::AppManageTriggers(_)
|
||||||
| Capability::AppDeadLetterManage(_)
|
| Capability::AppDeadLetterManage(_)
|
||||||
|
| Capability::AppTopicManage(_)
|
||||||
);
|
);
|
||||||
match role {
|
match role {
|
||||||
AppRole::Viewer => in_viewer,
|
AppRole::Viewer => in_viewer,
|
||||||
@@ -659,6 +689,35 @@ mod tests {
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn topic_manage_requires_app_admin() {
|
||||||
|
let repo = InMemoryAuthzRepo::default();
|
||||||
|
let app = AppId::new();
|
||||||
|
// Maps to the app:admin scope, not a new one.
|
||||||
|
assert_eq!(
|
||||||
|
Capability::AppTopicManage(app).required_scope(),
|
||||||
|
Scope::AppAdmin
|
||||||
|
);
|
||||||
|
|
||||||
|
// Member with only Editor role cannot manage topics.
|
||||||
|
let p = principal(InstanceRole::Member);
|
||||||
|
repo.grant(p.user_id, app, AppRole::Editor).await;
|
||||||
|
assert_eq!(
|
||||||
|
can(&repo, &p, Capability::AppTopicManage(app))
|
||||||
|
.await
|
||||||
|
.unwrap(),
|
||||||
|
Decision::Deny,
|
||||||
|
);
|
||||||
|
|
||||||
|
// App-admin role can.
|
||||||
|
let admin = principal(InstanceRole::Member);
|
||||||
|
repo.grant(admin.user_id, app, AppRole::AppAdmin).await;
|
||||||
|
assert!(can(&repo, &admin, Capability::AppTopicManage(app))
|
||||||
|
.await
|
||||||
|
.unwrap()
|
||||||
|
.is_allow());
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn capability_app_id_extraction() {
|
fn capability_app_id_extraction() {
|
||||||
let app = AppId::new();
|
let app = AppId::new();
|
||||||
|
|||||||
@@ -23,19 +23,19 @@
|
|||||||
use std::sync::Arc;
|
use std::sync::Arc;
|
||||||
use std::time::Duration;
|
use std::time::Duration;
|
||||||
|
|
||||||
use chrono::Utc;
|
use chrono::{DateTime, Utc};
|
||||||
use picloud_executor_core::{ExecError, ExecRequest, ExecResponse, InvocationType};
|
use picloud_executor_core::{ExecError, ExecRequest, ExecResponse, InvocationType};
|
||||||
use picloud_orchestrator_core::{ExecutionGate, ExecutorClient};
|
use picloud_orchestrator_core::{ExecutionGate, ExecutorClient};
|
||||||
use picloud_shared::{
|
use picloud_shared::{
|
||||||
ExecResponseSummary, ExecutionId, HttpDispatchPayload, InboxDeliveryOutcome, InboxFailureKind,
|
DeadLetterId, ExecResponseSummary, ExecutionId, HttpDispatchPayload, InboxDeliveryOutcome,
|
||||||
InboxResolver, InboxResult, RequestId, ScriptId, ScriptSandbox, TriggerEvent,
|
InboxFailureKind, InboxResolver, InboxResult, RequestId, ScriptId, ScriptSandbox, TriggerEvent,
|
||||||
};
|
};
|
||||||
use rand::Rng;
|
use rand::Rng;
|
||||||
use uuid::Uuid;
|
use uuid::Uuid;
|
||||||
|
|
||||||
use crate::abandoned_repo::{AbandonedRepo, NewAbandonedExecution};
|
use crate::abandoned_repo::{AbandonedRepo, NewAbandonedExecution};
|
||||||
use crate::dead_letter_repo::{DeadLetterRepo, NewDeadLetter};
|
use crate::dead_letter_repo::{DeadLetterRepo, NewDeadLetter};
|
||||||
use crate::outbox_repo::{OutboxRepo, OutboxRow, OutboxSourceKind};
|
use crate::outbox_repo::{NewOutboxRow, OutboxRepo, OutboxRow, OutboxSourceKind};
|
||||||
use crate::principal_resolver::PrincipalResolver;
|
use crate::principal_resolver::PrincipalResolver;
|
||||||
use crate::repo::ScriptRepository;
|
use crate::repo::ScriptRepository;
|
||||||
use crate::trigger_config::{BackoffShape, TriggerConfig};
|
use crate::trigger_config::{BackoffShape, TriggerConfig};
|
||||||
@@ -168,7 +168,8 @@ impl Dispatcher {
|
|||||||
| OutboxSourceKind::DeadLetter
|
| OutboxSourceKind::DeadLetter
|
||||||
| OutboxSourceKind::Cron
|
| OutboxSourceKind::Cron
|
||||||
| OutboxSourceKind::Files
|
| OutboxSourceKind::Files
|
||||||
| OutboxSourceKind::Pubsub => {
|
| OutboxSourceKind::Pubsub
|
||||||
|
| OutboxSourceKind::Email => {
|
||||||
let resolved = self.resolve_trigger(&row).await?;
|
let resolved = self.resolve_trigger(&row).await?;
|
||||||
let req = match self.build_exec_request(&row, &resolved).await {
|
let req = match self.build_exec_request(&row, &resolved).await {
|
||||||
Ok(req) => req,
|
Ok(req) => req,
|
||||||
@@ -462,12 +463,12 @@ impl Dispatcher {
|
|||||||
// Exhausted retries → dead-letter.
|
// Exhausted retries → dead-letter.
|
||||||
let (op, source) = describe_event(&row.payload);
|
let (op, source) = describe_event(&row.payload);
|
||||||
let now = Utc::now();
|
let now = Utc::now();
|
||||||
if let Err(e) = self
|
let dl_id = match self
|
||||||
.dead_letters
|
.dead_letters
|
||||||
.insert(NewDeadLetter {
|
.insert(NewDeadLetter {
|
||||||
app_id: row.app_id,
|
app_id: row.app_id,
|
||||||
original_event_id: row.id,
|
original_event_id: row.id,
|
||||||
source,
|
source: source.clone(),
|
||||||
op,
|
op,
|
||||||
trigger_id: row.trigger_id,
|
trigger_id: row.trigger_id,
|
||||||
script_id: Some(resolved.script_id),
|
script_id: Some(resolved.script_id),
|
||||||
@@ -479,8 +480,26 @@ impl Dispatcher {
|
|||||||
})
|
})
|
||||||
.await
|
.await
|
||||||
{
|
{
|
||||||
|
Ok(id) => Some(id),
|
||||||
|
Err(e) => {
|
||||||
tracing::error!(?e, "failed to write dead-letter row");
|
tracing::error!(?e, "failed to write dead-letter row");
|
||||||
|
None
|
||||||
}
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
// v1.1.7 fix: fan the dead-letter out to matching handler triggers.
|
||||||
|
// This was missing since v1.1.1 — the row was written but
|
||||||
|
// `list_matching_dead_letter` had no production caller, so
|
||||||
|
// registered dead_letter handlers never fired. The recursion-stop
|
||||||
|
// (a dead-letter handler's own failure is not re-dead-lettered)
|
||||||
|
// is upheld by the `is_dead_letter_handler` short-circuit at the
|
||||||
|
// top of this function, so this fan-out is only reached for
|
||||||
|
// non-handler executions.
|
||||||
|
if let Some(dl_id) = dl_id {
|
||||||
|
self.fan_out_dead_letter(row, resolved, dl_id, &source, attempt, &err, now)
|
||||||
|
.await;
|
||||||
|
}
|
||||||
|
|
||||||
self.outbox
|
self.outbox
|
||||||
.delete(row.id)
|
.delete(row.id)
|
||||||
.await
|
.await
|
||||||
@@ -488,6 +507,82 @@ impl Dispatcher {
|
|||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Enqueue one outbox row per matching `dead_letter` trigger so its
|
||||||
|
/// handler script runs with the dead-letter event as `ctx.event`.
|
||||||
|
/// Best-effort: a lookup/insert failure is logged, not propagated
|
||||||
|
/// (the dead-letter row itself is already durably written).
|
||||||
|
#[allow(clippy::too_many_arguments)]
|
||||||
|
async fn fan_out_dead_letter(
|
||||||
|
&self,
|
||||||
|
row: &OutboxRow,
|
||||||
|
resolved: &ResolvedTrigger,
|
||||||
|
dead_letter_id: DeadLetterId,
|
||||||
|
source: &str,
|
||||||
|
attempt: u32,
|
||||||
|
err: &ExecError,
|
||||||
|
now: DateTime<Utc>,
|
||||||
|
) {
|
||||||
|
// The DL event nests the original verbatim; if the payload can't
|
||||||
|
// be decoded back into a TriggerEvent we can't build the nested
|
||||||
|
// `original`, so skip the fan-out (the DL row is still written).
|
||||||
|
let Ok(original) = serde_json::from_value::<TriggerEvent>(row.payload.clone()) else {
|
||||||
|
tracing::warn!(
|
||||||
|
outbox_id = %row.id,
|
||||||
|
"dead-letter payload is not a TriggerEvent; skipping handler fan-out"
|
||||||
|
);
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
|
||||||
|
let matches = match self
|
||||||
|
.triggers
|
||||||
|
.list_matching_dead_letter(row.app_id, source, row.trigger_id, Some(resolved.script_id))
|
||||||
|
.await
|
||||||
|
{
|
||||||
|
Ok(m) => m,
|
||||||
|
Err(e) => {
|
||||||
|
tracing::error!(?e, "dead-letter trigger lookup failed");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
for m in matches {
|
||||||
|
let event = TriggerEvent::DeadLetter {
|
||||||
|
dead_letter_id,
|
||||||
|
original: Box::new(original.clone()),
|
||||||
|
attempts: attempt,
|
||||||
|
last_error: err.to_string(),
|
||||||
|
trigger_id: row.trigger_id,
|
||||||
|
script_id: Some(resolved.script_id),
|
||||||
|
first_attempt_at: row.created_at,
|
||||||
|
last_attempt_at: now,
|
||||||
|
};
|
||||||
|
let payload = match serde_json::to_value(&event) {
|
||||||
|
Ok(p) => p,
|
||||||
|
Err(e) => {
|
||||||
|
tracing::error!(?e, "failed to serialize dead-letter event");
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
if let Err(e) = self
|
||||||
|
.outbox
|
||||||
|
.insert(NewOutboxRow {
|
||||||
|
app_id: row.app_id,
|
||||||
|
source_kind: OutboxSourceKind::DeadLetter,
|
||||||
|
trigger_id: Some(m.trigger_id),
|
||||||
|
script_id: Some(m.script_id),
|
||||||
|
reply_to: None,
|
||||||
|
payload,
|
||||||
|
origin_principal: Some(m.registered_by_principal),
|
||||||
|
trigger_depth: row.trigger_depth.saturating_add(1),
|
||||||
|
root_execution_id: row.root_execution_id,
|
||||||
|
})
|
||||||
|
.await
|
||||||
|
{
|
||||||
|
tracing::error!(?e, "failed to enqueue dead-letter handler delivery");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
async fn deliver_inbox(&self, row: &OutboxRow, inbox_id: Uuid, result: InboxResult) {
|
async fn deliver_inbox(&self, row: &OutboxRow, inbox_id: Uuid, result: InboxResult) {
|
||||||
match self.inbox.deliver(inbox_id, result.clone()).await {
|
match self.inbox.deliver(inbox_id, result.clone()).await {
|
||||||
InboxDeliveryOutcome::Delivered => {}
|
InboxDeliveryOutcome::Delivered => {}
|
||||||
|
|||||||
307
crates/manager-core/src/email_inbound_api.rs
Normal file
307
crates/manager-core/src/email_inbound_api.rs
Normal file
@@ -0,0 +1,307 @@
|
|||||||
|
//! `POST /api/v1/email-inbound/{app_id}/{trigger_id}` — the inbound-email
|
||||||
|
//! webhook receiver (v1.1.7).
|
||||||
|
//!
|
||||||
|
//! A configured provider (Mailgun / Postmark / SendGrid / SES) POSTs a
|
||||||
|
//! normalized JSON message here; the receiver verifies the optional HMAC
|
||||||
|
//! signature, builds a `TriggerEvent::Email`, and enqueues an outbox row
|
||||||
|
//! the dispatcher picks up like any other async trigger.
|
||||||
|
//!
|
||||||
|
//! This is a PUBLIC endpoint (no admin auth) — the trigger URL itself,
|
||||||
|
//! plus the per-trigger HMAC secret, are the security boundary. It is
|
||||||
|
//! mounted OUTSIDE the `require_authenticated` layer.
|
||||||
|
//!
|
||||||
|
//! Status codes:
|
||||||
|
//! * 202 — accepted + enqueued
|
||||||
|
//! * 401 — HMAC required but missing/invalid
|
||||||
|
//! * 404 — trigger missing, disabled, not `kind=email`, or app mismatch
|
||||||
|
//! * 422 — body is not the expected JSON shape
|
||||||
|
//!
|
||||||
|
//! Only the generic provider-agnostic JSON shape is accepted in v1.1.7
|
||||||
|
//! (see [`InboundPayload`]); provider-specific unmarshallers are v1.2.
|
||||||
|
|
||||||
|
use std::sync::Arc;
|
||||||
|
|
||||||
|
use axum::body::Bytes;
|
||||||
|
use axum::extract::{Path, State};
|
||||||
|
use axum::http::{HeaderMap, StatusCode};
|
||||||
|
use axum::response::{IntoResponse, Json, Response};
|
||||||
|
use axum::routing::post;
|
||||||
|
use axum::Router;
|
||||||
|
use hmac::{Hmac, Mac};
|
||||||
|
use picloud_shared::{AppId, MasterKey, TriggerEvent, TriggerId};
|
||||||
|
use serde::Deserialize;
|
||||||
|
use serde_json::json;
|
||||||
|
use sha2::Sha256;
|
||||||
|
|
||||||
|
use crate::outbox_repo::{NewOutboxRow, OutboxRepo, OutboxSourceKind};
|
||||||
|
use crate::secrets_repo::StoredSecret;
|
||||||
|
use crate::secrets_service::open;
|
||||||
|
use crate::trigger_repo::TriggerRepo;
|
||||||
|
|
||||||
|
type HmacSha256 = Hmac<Sha256>;
|
||||||
|
|
||||||
|
/// Header the provider's HMAC signature is read from. The signature is
|
||||||
|
/// the lowercase hex of `HMAC-SHA256(inbound_secret, raw_body)`.
|
||||||
|
const SIGNATURE_HEADER: &str = "x-picloud-signature";
|
||||||
|
|
||||||
|
#[derive(Clone)]
|
||||||
|
pub struct EmailInboundState {
|
||||||
|
pub triggers: Arc<dyn TriggerRepo>,
|
||||||
|
pub outbox: Arc<dyn OutboxRepo>,
|
||||||
|
pub master_key: MasterKey,
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn email_inbound_router(state: EmailInboundState) -> Router {
|
||||||
|
Router::new()
|
||||||
|
.route(
|
||||||
|
"/email-inbound/{app_id}/{trigger_id}",
|
||||||
|
post(receive_inbound_email),
|
||||||
|
)
|
||||||
|
.with_state(state)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The generic provider-agnostic inbound shape. Users configure their
|
||||||
|
/// provider's webhook templating to POST this. `from` is required;
|
||||||
|
/// everything else defaults.
|
||||||
|
#[derive(Debug, Deserialize)]
|
||||||
|
struct InboundPayload {
|
||||||
|
from: String,
|
||||||
|
#[serde(default)]
|
||||||
|
to: Vec<String>,
|
||||||
|
#[serde(default)]
|
||||||
|
cc: Vec<String>,
|
||||||
|
#[serde(default)]
|
||||||
|
subject: String,
|
||||||
|
#[serde(default)]
|
||||||
|
text: Option<String>,
|
||||||
|
#[serde(default)]
|
||||||
|
html: Option<String>,
|
||||||
|
#[serde(default)]
|
||||||
|
message_id: Option<String>,
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn receive_inbound_email(
|
||||||
|
State(s): State<EmailInboundState>,
|
||||||
|
Path((app_id, trigger_id)): Path<(AppId, TriggerId)>,
|
||||||
|
headers: HeaderMap,
|
||||||
|
body: Bytes,
|
||||||
|
) -> Result<StatusCode, EmailInboundError> {
|
||||||
|
// Resolve the trigger. 404 covers missing / wrong-kind / cross-app /
|
||||||
|
// disabled — all "this URL doesn't address a live email trigger".
|
||||||
|
let target = s
|
||||||
|
.triggers
|
||||||
|
.email_inbound_target(trigger_id)
|
||||||
|
.await
|
||||||
|
.map_err(|e| EmailInboundError::Backend(e.to_string()))?
|
||||||
|
.ok_or(EmailInboundError::NotFound)?;
|
||||||
|
if target.app_id != app_id || !target.enabled {
|
||||||
|
return Err(EmailInboundError::NotFound);
|
||||||
|
}
|
||||||
|
|
||||||
|
// HMAC verification (only when the trigger has a secret configured).
|
||||||
|
if let (Some(ct), Some(nonce)) = (
|
||||||
|
target.inbound_secret_encrypted.as_ref(),
|
||||||
|
target.inbound_secret_nonce.as_ref(),
|
||||||
|
) {
|
||||||
|
let secret = decrypt_secret(&s.master_key, ct, nonce)?;
|
||||||
|
verify_signature(&headers, &body, secret.as_bytes())?;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Parse the generic JSON shape. Malformed → 422.
|
||||||
|
let payload: InboundPayload =
|
||||||
|
serde_json::from_slice(&body).map_err(|e| EmailInboundError::Malformed(e.to_string()))?;
|
||||||
|
|
||||||
|
let event = TriggerEvent::Email {
|
||||||
|
from: payload.from,
|
||||||
|
to: payload.to,
|
||||||
|
cc: payload.cc,
|
||||||
|
subject: payload.subject,
|
||||||
|
text: payload.text,
|
||||||
|
html: payload.html,
|
||||||
|
received_at: chrono::Utc::now(),
|
||||||
|
message_id: payload.message_id,
|
||||||
|
};
|
||||||
|
let payload_json = serde_json::to_value(&event)
|
||||||
|
.map_err(|e| EmailInboundError::Backend(format!("serialize event: {e}")))?;
|
||||||
|
|
||||||
|
s.outbox
|
||||||
|
.insert(NewOutboxRow {
|
||||||
|
app_id,
|
||||||
|
source_kind: OutboxSourceKind::Email,
|
||||||
|
trigger_id: Some(trigger_id),
|
||||||
|
script_id: Some(target.script_id),
|
||||||
|
reply_to: None,
|
||||||
|
payload: payload_json,
|
||||||
|
origin_principal: Some(target.registered_by_principal),
|
||||||
|
// Inbound email is the root of a trigger chain (depth 1).
|
||||||
|
trigger_depth: 1,
|
||||||
|
root_execution_id: None,
|
||||||
|
})
|
||||||
|
.await
|
||||||
|
.map_err(|e| EmailInboundError::Backend(e.to_string()))?;
|
||||||
|
|
||||||
|
Ok(StatusCode::ACCEPTED)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Decrypt the stored inbound secret back to its raw string. It was
|
||||||
|
/// sealed as a JSON string by the admin layer, so `open` yields a
|
||||||
|
/// `Value::String`.
|
||||||
|
fn decrypt_secret(
|
||||||
|
master_key: &MasterKey,
|
||||||
|
ciphertext: &[u8],
|
||||||
|
nonce: &[u8],
|
||||||
|
) -> Result<String, EmailInboundError> {
|
||||||
|
let stored = StoredSecret {
|
||||||
|
encrypted_value: ciphertext.to_vec(),
|
||||||
|
nonce: nonce.to_vec(),
|
||||||
|
};
|
||||||
|
let value = open(master_key, &stored).map_err(|_| {
|
||||||
|
// Corrupted secret means we can't verify — fail closed (401).
|
||||||
|
EmailInboundError::Unauthorized
|
||||||
|
})?;
|
||||||
|
value
|
||||||
|
.as_str()
|
||||||
|
.map(str::to_string)
|
||||||
|
.ok_or(EmailInboundError::Unauthorized)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Constant-time HMAC-SHA256 verification of the body against the
|
||||||
|
/// `X-Picloud-Signature` header (lowercase hex).
|
||||||
|
fn verify_signature(
|
||||||
|
headers: &HeaderMap,
|
||||||
|
body: &[u8],
|
||||||
|
secret: &[u8],
|
||||||
|
) -> Result<(), EmailInboundError> {
|
||||||
|
let provided_hex = headers
|
||||||
|
.get(SIGNATURE_HEADER)
|
||||||
|
.and_then(|h| h.to_str().ok())
|
||||||
|
.ok_or(EmailInboundError::Unauthorized)?;
|
||||||
|
let provided = hex::decode(provided_hex.trim()).map_err(|_| EmailInboundError::Unauthorized)?;
|
||||||
|
let mut mac =
|
||||||
|
HmacSha256::new_from_slice(secret).map_err(|_| EmailInboundError::Unauthorized)?;
|
||||||
|
mac.update(body);
|
||||||
|
mac.verify_slice(&provided)
|
||||||
|
.map_err(|_| EmailInboundError::Unauthorized)
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, thiserror::Error)]
|
||||||
|
pub enum EmailInboundError {
|
||||||
|
#[error("trigger not found")]
|
||||||
|
NotFound,
|
||||||
|
#[error("invalid signature")]
|
||||||
|
Unauthorized,
|
||||||
|
#[error("malformed body: {0}")]
|
||||||
|
Malformed(String),
|
||||||
|
#[error("backend: {0}")]
|
||||||
|
Backend(String),
|
||||||
|
}
|
||||||
|
|
||||||
|
impl IntoResponse for EmailInboundError {
|
||||||
|
fn into_response(self) -> Response {
|
||||||
|
let (status, body) = match &self {
|
||||||
|
Self::NotFound => (
|
||||||
|
StatusCode::NOT_FOUND,
|
||||||
|
json!({ "error": "trigger not found" }),
|
||||||
|
),
|
||||||
|
Self::Unauthorized => (
|
||||||
|
StatusCode::UNAUTHORIZED,
|
||||||
|
json!({ "error": "invalid or missing signature" }),
|
||||||
|
),
|
||||||
|
Self::Malformed(m) => (
|
||||||
|
StatusCode::UNPROCESSABLE_ENTITY,
|
||||||
|
json!({ "error": format!("malformed inbound email body: {m}") }),
|
||||||
|
),
|
||||||
|
Self::Backend(e) => {
|
||||||
|
tracing::error!(error = %e, "inbound email receiver backend error");
|
||||||
|
(
|
||||||
|
StatusCode::INTERNAL_SERVER_ERROR,
|
||||||
|
json!({ "error": "internal error" }),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
};
|
||||||
|
(status, Json(body)).into_response()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
//! Unit tests for the security-critical helpers (HMAC verify, secret
|
||||||
|
//! round-trip, payload parsing). The full request flow — 202 / 401 /
|
||||||
|
//! 404 / 422 / cross-app — is exercised end-to-end against a real
|
||||||
|
//! Postgres in `crates/picloud/tests/email_inbound.rs`.
|
||||||
|
|
||||||
|
use super::*;
|
||||||
|
use crate::secrets_service::seal;
|
||||||
|
use crate::secrets_service::DEFAULT_SECRET_MAX_VALUE_BYTES;
|
||||||
|
|
||||||
|
fn sign(secret: &[u8], body: &[u8]) -> String {
|
||||||
|
let mut mac = HmacSha256::new_from_slice(secret).unwrap();
|
||||||
|
mac.update(body);
|
||||||
|
hex::encode(mac.finalize().into_bytes())
|
||||||
|
}
|
||||||
|
|
||||||
|
fn headers_with_sig(sig: &str) -> HeaderMap {
|
||||||
|
let mut h = HeaderMap::new();
|
||||||
|
h.insert(SIGNATURE_HEADER, sig.parse().unwrap());
|
||||||
|
h
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn valid_signature_verifies() {
|
||||||
|
let secret = b"shhh";
|
||||||
|
let body = br#"{"from":"a@b.com"}"#;
|
||||||
|
let sig = sign(secret, body);
|
||||||
|
assert!(verify_signature(&headers_with_sig(&sig), body, secret).is_ok());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn wrong_signature_rejected() {
|
||||||
|
let body = br#"{"from":"a@b.com"}"#;
|
||||||
|
let sig = sign(b"shhh", body);
|
||||||
|
let err = verify_signature(&headers_with_sig(&sig), body, b"different").unwrap_err();
|
||||||
|
assert!(matches!(err, EmailInboundError::Unauthorized));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn missing_signature_header_rejected() {
|
||||||
|
let err = verify_signature(&HeaderMap::new(), b"body", b"secret").unwrap_err();
|
||||||
|
assert!(matches!(err, EmailInboundError::Unauthorized));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn tampered_body_fails_verification() {
|
||||||
|
let secret = b"shhh";
|
||||||
|
let sig = sign(secret, b"original");
|
||||||
|
let err = verify_signature(&headers_with_sig(&sig), b"tampered", secret).unwrap_err();
|
||||||
|
assert!(matches!(err, EmailInboundError::Unauthorized));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn secret_round_trips_through_seal_open() {
|
||||||
|
let key = MasterKey::from_bytes([3u8; 32]);
|
||||||
|
let (ct, nonce) = seal(
|
||||||
|
&key,
|
||||||
|
&serde_json::Value::String("provider-secret".into()),
|
||||||
|
DEFAULT_SECRET_MAX_VALUE_BYTES,
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
let recovered = decrypt_secret(&key, &ct, &nonce).unwrap();
|
||||||
|
assert_eq!(recovered, "provider-secret");
|
||||||
|
// And a signature made with the recovered secret verifies.
|
||||||
|
let body = br#"{"from":"x@y.com"}"#;
|
||||||
|
let sig = sign(recovered.as_bytes(), body);
|
||||||
|
assert!(verify_signature(&headers_with_sig(&sig), body, recovered.as_bytes()).is_ok());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn payload_requires_from_but_defaults_rest() {
|
||||||
|
let ok: Result<InboundPayload, _> = serde_json::from_slice(br#"{"from":"a@b.com"}"#);
|
||||||
|
let p = ok.expect("from-only payload parses");
|
||||||
|
assert_eq!(p.from, "a@b.com");
|
||||||
|
assert!(p.to.is_empty() && p.cc.is_empty() && p.text.is_none());
|
||||||
|
|
||||||
|
// Missing `from` → malformed.
|
||||||
|
let bad: Result<InboundPayload, _> = serde_json::from_slice(br#"{"subject":"hi"}"#);
|
||||||
|
assert!(bad.is_err());
|
||||||
|
}
|
||||||
|
}
|
||||||
597
crates/manager-core/src/email_service.rs
Normal file
597
crates/manager-core/src/email_service.rs
Normal file
@@ -0,0 +1,597 @@
|
|||||||
|
//! `EmailServiceImpl` — outbound email over an SMTP relay (`lettre`),
|
||||||
|
//! behind the `picloud_shared::EmailService` trait scripts reach via the
|
||||||
|
//! Rhai `email::{send,send_html}` bridge.
|
||||||
|
//!
|
||||||
|
//! Layers added here:
|
||||||
|
//!
|
||||||
|
//! 1. **Script-as-gate authz**: `AppEmailSend` checked when
|
||||||
|
//! `cx.principal.is_some()`; skipped for public-HTTP (`None`).
|
||||||
|
//! 2. Required-field + RFC 5322-ish address validation at the boundary.
|
||||||
|
//! 3. Per-message size cap (default 25 MB).
|
||||||
|
//! 4. **Disabled mode**: if no SMTP relay is configured (HOST/USER/
|
||||||
|
//! PASSWORD not all set) every `send` returns `NotConfigured` and
|
||||||
|
//! startup logs a warning — there is no silent drop.
|
||||||
|
//!
|
||||||
|
//! Connection model: one connection per call (lettre's default). A
|
||||||
|
//! pooled transport is a v1.2+ optimization. Per-app `from` validation /
|
||||||
|
//! SPF / DKIM are the operator's responsibility at the relay (v1.1.7
|
||||||
|
//! does not restrict the `from` address).
|
||||||
|
|
||||||
|
use std::sync::Arc;
|
||||||
|
use std::time::Duration;
|
||||||
|
|
||||||
|
use async_trait::async_trait;
|
||||||
|
use lettre::message::{Mailbox, Message, MultiPart, SinglePart};
|
||||||
|
use lettre::transport::smtp::authentication::Credentials;
|
||||||
|
use lettre::{AsyncSmtpTransport, AsyncTransport, Tokio1Executor};
|
||||||
|
use picloud_shared::{EmailError, EmailService, OutboundEmail, SdkCallCx};
|
||||||
|
|
||||||
|
use crate::authz::{self, AuthzRepo, Capability};
|
||||||
|
|
||||||
|
/// Default per-message size cap (25 MB) — matches most providers.
|
||||||
|
/// Override with `PICLOUD_EMAIL_MAX_MESSAGE_BYTES`.
|
||||||
|
pub const DEFAULT_EMAIL_MAX_MESSAGE_BYTES: usize = 25 * 1024 * 1024;
|
||||||
|
|
||||||
|
/// Generous upper bound on a single address string (RFC 5321 caps the
|
||||||
|
/// path at 256; 320 covers local@domain comfortably).
|
||||||
|
const ADDRESS_MAX_LEN: usize = 320;
|
||||||
|
|
||||||
|
/// Process config for the email service.
|
||||||
|
#[derive(Debug, Clone, Copy)]
|
||||||
|
pub struct EmailConfig {
|
||||||
|
pub max_message_bytes: usize,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl EmailConfig {
|
||||||
|
#[must_use]
|
||||||
|
pub const fn conservative() -> Self {
|
||||||
|
Self {
|
||||||
|
max_message_bytes: DEFAULT_EMAIL_MAX_MESSAGE_BYTES,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[must_use]
|
||||||
|
pub fn from_env() -> Self {
|
||||||
|
let mut c = Self::conservative();
|
||||||
|
if let Ok(v) = std::env::var("PICLOUD_EMAIL_MAX_MESSAGE_BYTES") {
|
||||||
|
match v.trim().parse::<usize>() {
|
||||||
|
Ok(n) if n > 0 => c.max_message_bytes = n,
|
||||||
|
_ => tracing::warn!(
|
||||||
|
value = %v,
|
||||||
|
"ignoring invalid PICLOUD_EMAIL_MAX_MESSAGE_BYTES (want a positive integer)"
|
||||||
|
),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
c
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Default for EmailConfig {
|
||||||
|
fn default() -> Self {
|
||||||
|
Self::conservative()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// TLS mode for the SMTP relay connection.
|
||||||
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||||
|
pub enum SmtpTls {
|
||||||
|
/// STARTTLS upgrade on a plaintext port (typically 587). Default.
|
||||||
|
Starttls,
|
||||||
|
/// Implicit TLS from connect (typically 465).
|
||||||
|
Implicit,
|
||||||
|
/// No TLS — plaintext. Dev/test only.
|
||||||
|
None,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// SMTP relay connection settings, sourced from env.
|
||||||
|
#[derive(Debug, Clone)]
|
||||||
|
pub struct SmtpConfig {
|
||||||
|
pub host: String,
|
||||||
|
pub port: u16,
|
||||||
|
pub user: String,
|
||||||
|
pub password: String,
|
||||||
|
pub tls: SmtpTls,
|
||||||
|
pub timeout_secs: u64,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl SmtpConfig {
|
||||||
|
/// Read SMTP settings from env. Returns `None` (→ disabled mode) when
|
||||||
|
/// any of HOST / USER / PASSWORD is missing or empty.
|
||||||
|
#[must_use]
|
||||||
|
pub fn from_env() -> Option<Self> {
|
||||||
|
let host = non_empty_env("PICLOUD_SMTP_HOST")?;
|
||||||
|
let user = non_empty_env("PICLOUD_SMTP_USER")?;
|
||||||
|
let password = non_empty_env("PICLOUD_SMTP_PASSWORD")?;
|
||||||
|
let tls = match std::env::var("PICLOUD_SMTP_TLS")
|
||||||
|
.unwrap_or_default()
|
||||||
|
.trim()
|
||||||
|
.to_ascii_lowercase()
|
||||||
|
.as_str()
|
||||||
|
{
|
||||||
|
"implicit" => SmtpTls::Implicit,
|
||||||
|
"none" => SmtpTls::None,
|
||||||
|
// Default + explicit "starttls" + anything unrecognized.
|
||||||
|
_ => SmtpTls::Starttls,
|
||||||
|
};
|
||||||
|
let default_port = match tls {
|
||||||
|
SmtpTls::Implicit => 465,
|
||||||
|
SmtpTls::Starttls | SmtpTls::None => 587,
|
||||||
|
};
|
||||||
|
let port = std::env::var("PICLOUD_SMTP_PORT")
|
||||||
|
.ok()
|
||||||
|
.and_then(|v| v.trim().parse::<u16>().ok())
|
||||||
|
.unwrap_or(default_port);
|
||||||
|
let timeout_secs = std::env::var("PICLOUD_SMTP_TIMEOUT_SECS")
|
||||||
|
.ok()
|
||||||
|
.and_then(|v| v.trim().parse::<u64>().ok())
|
||||||
|
.filter(|n| *n > 0)
|
||||||
|
.unwrap_or(30);
|
||||||
|
Some(Self {
|
||||||
|
host,
|
||||||
|
port,
|
||||||
|
user,
|
||||||
|
password,
|
||||||
|
tls,
|
||||||
|
timeout_secs,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn non_empty_env(key: &str) -> Option<String> {
|
||||||
|
std::env::var(key).ok().filter(|v| !v.trim().is_empty())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Internal transport seam so the service can be tested without a live
|
||||||
|
/// SMTP server. The production impl is [`LettreEmailTransport`]; tests
|
||||||
|
/// use a recording fake.
|
||||||
|
#[async_trait]
|
||||||
|
pub trait EmailTransport: Send + Sync {
|
||||||
|
async fn send(&self, message: &Message) -> Result<(), EmailError>;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Production transport: a per-call lettre SMTP connection.
|
||||||
|
pub struct LettreEmailTransport {
|
||||||
|
inner: AsyncSmtpTransport<Tokio1Executor>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl LettreEmailTransport {
|
||||||
|
/// Build the transport from settings.
|
||||||
|
///
|
||||||
|
/// # Errors
|
||||||
|
///
|
||||||
|
/// Returns the lettre SMTP error string if the relay descriptor is
|
||||||
|
/// invalid (e.g. TLS setup fails).
|
||||||
|
pub fn build(cfg: &SmtpConfig) -> Result<Self, String> {
|
||||||
|
let builder = match cfg.tls {
|
||||||
|
SmtpTls::Implicit => {
|
||||||
|
AsyncSmtpTransport::<Tokio1Executor>::relay(&cfg.host).map_err(|e| e.to_string())?
|
||||||
|
}
|
||||||
|
SmtpTls::Starttls => AsyncSmtpTransport::<Tokio1Executor>::starttls_relay(&cfg.host)
|
||||||
|
.map_err(|e| e.to_string())?,
|
||||||
|
SmtpTls::None => AsyncSmtpTransport::<Tokio1Executor>::builder_dangerous(&cfg.host),
|
||||||
|
};
|
||||||
|
let inner = builder
|
||||||
|
.port(cfg.port)
|
||||||
|
.credentials(Credentials::new(cfg.user.clone(), cfg.password.clone()))
|
||||||
|
.timeout(Some(Duration::from_secs(cfg.timeout_secs)))
|
||||||
|
.build();
|
||||||
|
Ok(Self { inner })
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[async_trait]
|
||||||
|
impl EmailTransport for LettreEmailTransport {
|
||||||
|
async fn send(&self, message: &Message) -> Result<(), EmailError> {
|
||||||
|
// lettre's `AsyncTransport::send` consumes the `Message`; clone so
|
||||||
|
// the caller keeps ownership (it needs it for the size check).
|
||||||
|
self.inner
|
||||||
|
.send(message.clone())
|
||||||
|
.await
|
||||||
|
.map(|_| ())
|
||||||
|
.map_err(|e| EmailError::Transport(e.to_string()))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub struct EmailServiceImpl {
|
||||||
|
/// `None` → disabled mode (every send returns `NotConfigured`).
|
||||||
|
transport: Option<Arc<dyn EmailTransport>>,
|
||||||
|
authz: Arc<dyn AuthzRepo>,
|
||||||
|
config: EmailConfig,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl EmailServiceImpl {
|
||||||
|
#[must_use]
|
||||||
|
pub fn new(
|
||||||
|
transport: Option<Arc<dyn EmailTransport>>,
|
||||||
|
authz: Arc<dyn AuthzRepo>,
|
||||||
|
config: EmailConfig,
|
||||||
|
) -> Self {
|
||||||
|
Self {
|
||||||
|
transport,
|
||||||
|
authz,
|
||||||
|
config,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Construct from env: builds a lettre SMTP transport if the relay is
|
||||||
|
/// configured, otherwise runs in disabled mode (with a warning). A
|
||||||
|
/// malformed relay descriptor is logged and also yields disabled mode
|
||||||
|
/// — email is non-critical and must not block startup.
|
||||||
|
#[must_use]
|
||||||
|
pub fn from_env(authz: Arc<dyn AuthzRepo>) -> Self {
|
||||||
|
let config = EmailConfig::from_env();
|
||||||
|
let transport: Option<Arc<dyn EmailTransport>> = match SmtpConfig::from_env() {
|
||||||
|
None => {
|
||||||
|
tracing::warn!(
|
||||||
|
"email is DISABLED: set PICLOUD_SMTP_HOST/USER/PASSWORD to enable \
|
||||||
|
email::send. Scripts calling email::send will get an error."
|
||||||
|
);
|
||||||
|
None
|
||||||
|
}
|
||||||
|
Some(cfg) => match LettreEmailTransport::build(&cfg) {
|
||||||
|
Ok(t) => {
|
||||||
|
tracing::info!(host = %cfg.host, port = cfg.port, "outbound email enabled");
|
||||||
|
Some(Arc::new(t))
|
||||||
|
}
|
||||||
|
Err(e) => {
|
||||||
|
tracing::error!(error = %e, "failed to build SMTP transport; email DISABLED");
|
||||||
|
None
|
||||||
|
}
|
||||||
|
},
|
||||||
|
};
|
||||||
|
Self::new(transport, authz, config)
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn check_send(&self, cx: &SdkCallCx) -> Result<(), EmailError> {
|
||||||
|
if let Some(ref principal) = cx.principal {
|
||||||
|
authz::require(&*self.authz, principal, Capability::AppEmailSend(cx.app_id))
|
||||||
|
.await
|
||||||
|
.map_err(|_| EmailError::Forbidden)?;
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[async_trait]
|
||||||
|
impl EmailService for EmailServiceImpl {
|
||||||
|
async fn send(&self, cx: &SdkCallCx, email: OutboundEmail) -> Result<(), EmailError> {
|
||||||
|
self.check_send(cx).await?;
|
||||||
|
let Some(transport) = self.transport.as_ref() else {
|
||||||
|
return Err(EmailError::NotConfigured);
|
||||||
|
};
|
||||||
|
let message = build_message(&email)?;
|
||||||
|
let formatted = message.formatted();
|
||||||
|
if formatted.len() > self.config.max_message_bytes {
|
||||||
|
return Err(EmailError::TooLarge {
|
||||||
|
limit: self.config.max_message_bytes,
|
||||||
|
actual: formatted.len(),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
transport.send(&message).await
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Validate the required fields + addresses and assemble a lettre
|
||||||
|
/// `Message`. Pure (no I/O) so it's unit-testable on its own.
|
||||||
|
fn build_message(email: &OutboundEmail) -> Result<Message, EmailError> {
|
||||||
|
if email.from.trim().is_empty() {
|
||||||
|
return Err(EmailError::MissingField("from".into()));
|
||||||
|
}
|
||||||
|
if email.to.iter().all(|a| a.trim().is_empty()) {
|
||||||
|
return Err(EmailError::MissingField("to".into()));
|
||||||
|
}
|
||||||
|
if email.subject.trim().is_empty() {
|
||||||
|
return Err(EmailError::MissingField("subject".into()));
|
||||||
|
}
|
||||||
|
let has_text = email.text.as_ref().is_some_and(|t| !t.is_empty());
|
||||||
|
let has_html = email.html.as_ref().is_some_and(|h| !h.is_empty());
|
||||||
|
if !has_text && !has_html {
|
||||||
|
return Err(EmailError::MissingField("text or html".into()));
|
||||||
|
}
|
||||||
|
|
||||||
|
let mut builder = Message::builder()
|
||||||
|
.from(parse_address(&email.from)?)
|
||||||
|
.subject(email.subject.clone());
|
||||||
|
|
||||||
|
for addr in non_empty(&email.to) {
|
||||||
|
builder = builder.to(parse_address(addr)?);
|
||||||
|
}
|
||||||
|
for addr in non_empty(&email.cc) {
|
||||||
|
builder = builder.cc(parse_address(addr)?);
|
||||||
|
}
|
||||||
|
for addr in non_empty(&email.bcc) {
|
||||||
|
builder = builder.bcc(parse_address(addr)?);
|
||||||
|
}
|
||||||
|
// reply_to defaults to `from` when not supplied.
|
||||||
|
let reply_to = email.reply_to.as_deref().unwrap_or(&email.from);
|
||||||
|
builder = builder.reply_to(parse_address(reply_to)?);
|
||||||
|
|
||||||
|
// `has_text` / `has_html` were validated above (at least one is set).
|
||||||
|
let text = email.text.clone().unwrap_or_default();
|
||||||
|
let html = email.html.clone().unwrap_or_default();
|
||||||
|
let message = if has_text && has_html {
|
||||||
|
builder.multipart(MultiPart::alternative_plain_html(text, html))
|
||||||
|
} else if has_html {
|
||||||
|
builder.singlepart(SinglePart::html(html))
|
||||||
|
} else {
|
||||||
|
builder.singlepart(SinglePart::plain(text))
|
||||||
|
}
|
||||||
|
.map_err(|e| EmailError::Transport(e.to_string()))?;
|
||||||
|
Ok(message)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn non_empty(addrs: &[String]) -> impl Iterator<Item = &String> {
|
||||||
|
addrs.iter().filter(|a| !a.trim().is_empty())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Hand-rolled RFC 5322-ish address check, then a `lettre::Mailbox`
|
||||||
|
/// parse (the authoritative validator). We do NOT check deliverability —
|
||||||
|
/// that's the SMTP layer's job.
|
||||||
|
fn parse_address(addr: &str) -> Result<Mailbox, EmailError> {
|
||||||
|
let trimmed = addr.trim();
|
||||||
|
if trimmed.is_empty() {
|
||||||
|
return Err(EmailError::InvalidAddress("empty address".into()));
|
||||||
|
}
|
||||||
|
if trimmed.len() > ADDRESS_MAX_LEN {
|
||||||
|
return Err(EmailError::InvalidAddress(format!(
|
||||||
|
"address exceeds {ADDRESS_MAX_LEN} bytes"
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
// Must have a single-ish @ with a non-empty local part and a domain
|
||||||
|
// that contains a dot (rejects "a@b" and bare tokens).
|
||||||
|
match trimmed.rsplit_once('@') {
|
||||||
|
Some((local, domain)) if !local.is_empty() && domain.contains('.') => {}
|
||||||
|
_ => {
|
||||||
|
return Err(EmailError::InvalidAddress(format!(
|
||||||
|
"{trimmed:?} is not a valid email address"
|
||||||
|
)))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
trimmed.parse::<Mailbox>().map_err(|_| {
|
||||||
|
EmailError::InvalidAddress(format!("{trimmed:?} is not a valid email address"))
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// ----------------------------------------------------------------------------
|
||||||
|
// Tests — recording transport so unit tests need no live SMTP server.
|
||||||
|
// ----------------------------------------------------------------------------
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
use crate::authz::{AuthzError, AuthzRepo};
|
||||||
|
use async_trait::async_trait;
|
||||||
|
use picloud_shared::{
|
||||||
|
AdminUserId, AppId, AppRole, ExecutionId, InstanceRole, Principal, RequestId, ScriptId,
|
||||||
|
UserId,
|
||||||
|
};
|
||||||
|
use std::sync::Mutex as StdMutex;
|
||||||
|
|
||||||
|
#[derive(Default)]
|
||||||
|
struct RecordingTransport {
|
||||||
|
sent: StdMutex<Vec<Vec<u8>>>,
|
||||||
|
}
|
||||||
|
#[async_trait]
|
||||||
|
impl EmailTransport for RecordingTransport {
|
||||||
|
async fn send(&self, message: &Message) -> Result<(), EmailError> {
|
||||||
|
self.sent.lock().unwrap().push(message.formatted());
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Default)]
|
||||||
|
struct DenyAuthz;
|
||||||
|
#[async_trait]
|
||||||
|
impl AuthzRepo for DenyAuthz {
|
||||||
|
async fn membership(&self, _: UserId, _: AppId) -> Result<Option<AppRole>, AuthzError> {
|
||||||
|
Ok(None)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
struct GrantAuthz {
|
||||||
|
app: AppId,
|
||||||
|
role: AppRole,
|
||||||
|
}
|
||||||
|
#[async_trait]
|
||||||
|
impl AuthzRepo for GrantAuthz {
|
||||||
|
async fn membership(
|
||||||
|
&self,
|
||||||
|
_: UserId,
|
||||||
|
app_id: AppId,
|
||||||
|
) -> Result<Option<AppRole>, AuthzError> {
|
||||||
|
Ok((app_id == self.app).then_some(self.role))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn svc_with(
|
||||||
|
transport: Option<Arc<dyn EmailTransport>>,
|
||||||
|
authz: Arc<dyn AuthzRepo>,
|
||||||
|
) -> EmailServiceImpl {
|
||||||
|
EmailServiceImpl::new(transport, authz, EmailConfig::conservative())
|
||||||
|
}
|
||||||
|
|
||||||
|
fn recording() -> (EmailServiceImpl, Arc<RecordingTransport>) {
|
||||||
|
let rec = Arc::new(RecordingTransport::default());
|
||||||
|
let svc = svc_with(Some(rec.clone()), Arc::new(DenyAuthz));
|
||||||
|
(svc, rec)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn cx_with(app_id: AppId, principal: Option<Principal>) -> SdkCallCx {
|
||||||
|
SdkCallCx {
|
||||||
|
app_id,
|
||||||
|
script_id: ScriptId::new(),
|
||||||
|
principal,
|
||||||
|
execution_id: ExecutionId::new(),
|
||||||
|
request_id: RequestId::new(),
|
||||||
|
trigger_depth: 0,
|
||||||
|
root_execution_id: ExecutionId::new(),
|
||||||
|
is_dead_letter_handler: false,
|
||||||
|
event: None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn anon(app: AppId) -> SdkCallCx {
|
||||||
|
cx_with(app, None)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn principal(role: InstanceRole) -> Principal {
|
||||||
|
Principal {
|
||||||
|
user_id: AdminUserId::new(),
|
||||||
|
instance_role: role,
|
||||||
|
scopes: None,
|
||||||
|
app_binding: None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn base_email() -> OutboundEmail {
|
||||||
|
OutboundEmail {
|
||||||
|
to: vec!["alice@example.com".into()],
|
||||||
|
from: "alerts@myapp.com".into(),
|
||||||
|
subject: "Build complete".into(),
|
||||||
|
text: Some("Your deploy finished.".into()),
|
||||||
|
..Default::default()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn last_message(rec: &RecordingTransport) -> String {
|
||||||
|
let g = rec.sent.lock().unwrap();
|
||||||
|
String::from_utf8_lossy(g.last().expect("a message was sent")).into_owned()
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn send_text_includes_headers_and_body() {
|
||||||
|
let (svc, rec) = recording();
|
||||||
|
svc.send(&anon(AppId::new()), base_email()).await.unwrap();
|
||||||
|
let msg = last_message(&rec);
|
||||||
|
assert!(msg.contains("To: alice@example.com"), "{msg}");
|
||||||
|
assert!(msg.contains("From: alerts@myapp.com"), "{msg}");
|
||||||
|
assert!(msg.contains("Subject: Build complete"), "{msg}");
|
||||||
|
assert!(msg.contains("Your deploy finished."), "{msg}");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn send_html_is_multipart_with_both_parts() {
|
||||||
|
let (svc, rec) = recording();
|
||||||
|
let mut e = base_email();
|
||||||
|
e.text = Some("plain fallback".into());
|
||||||
|
e.html = Some("<p>rich <b>body</b></p>".into());
|
||||||
|
svc.send(&anon(AppId::new()), e).await.unwrap();
|
||||||
|
let msg = last_message(&rec);
|
||||||
|
assert!(msg.contains("multipart/alternative"), "{msg}");
|
||||||
|
assert!(msg.contains("plain fallback"), "{msg}");
|
||||||
|
// HTML part is quoted-printable encoded, but the tag survives.
|
||||||
|
assert!(msg.contains("text/html"), "{msg}");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn multiple_recipients_and_cc_bcc() {
|
||||||
|
let (svc, rec) = recording();
|
||||||
|
let mut e = base_email();
|
||||||
|
e.to = vec!["alice@x.com".into(), "bob@y.com".into()];
|
||||||
|
e.cc = vec!["dave@z.com".into()];
|
||||||
|
e.bcc = vec!["audit@myapp.com".into()];
|
||||||
|
svc.send(&anon(AppId::new()), e).await.unwrap();
|
||||||
|
let msg = last_message(&rec);
|
||||||
|
assert!(
|
||||||
|
msg.contains("alice@x.com") && msg.contains("bob@y.com"),
|
||||||
|
"{msg}"
|
||||||
|
);
|
||||||
|
assert!(msg.contains("Cc: dave@z.com"), "{msg}");
|
||||||
|
// Bcc is intentionally NOT serialized into the visible headers.
|
||||||
|
assert!(
|
||||||
|
!msg.contains("Bcc:"),
|
||||||
|
"bcc must not appear in headers: {msg}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn reply_to_populated() {
|
||||||
|
let (svc, rec) = recording();
|
||||||
|
let mut e = base_email();
|
||||||
|
e.reply_to = Some("support@myapp.com".into());
|
||||||
|
svc.send(&anon(AppId::new()), e).await.unwrap();
|
||||||
|
assert!(last_message(&rec).contains("Reply-To: support@myapp.com"));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn missing_required_field_throws() {
|
||||||
|
let (svc, _) = recording();
|
||||||
|
let mut e = base_email();
|
||||||
|
e.subject = String::new();
|
||||||
|
let err = svc.send(&anon(AppId::new()), e).await.unwrap_err();
|
||||||
|
assert!(matches!(err, EmailError::MissingField(f) if f == "subject"));
|
||||||
|
|
||||||
|
let (svc, _) = recording();
|
||||||
|
let mut e = base_email();
|
||||||
|
e.text = None;
|
||||||
|
e.html = None;
|
||||||
|
let err = svc.send(&anon(AppId::new()), e).await.unwrap_err();
|
||||||
|
assert!(matches!(err, EmailError::MissingField(_)));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn invalid_address_throws() {
|
||||||
|
let (svc, _) = recording();
|
||||||
|
let mut e = base_email();
|
||||||
|
e.to = vec!["not-an-email".into()];
|
||||||
|
let err = svc.send(&anon(AppId::new()), e).await.unwrap_err();
|
||||||
|
assert!(matches!(err, EmailError::InvalidAddress(_)));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn message_size_cap_enforced() {
|
||||||
|
let rec = Arc::new(RecordingTransport::default());
|
||||||
|
let svc = EmailServiceImpl::new(
|
||||||
|
Some(rec),
|
||||||
|
Arc::new(DenyAuthz),
|
||||||
|
EmailConfig {
|
||||||
|
max_message_bytes: 64,
|
||||||
|
},
|
||||||
|
);
|
||||||
|
let err = svc
|
||||||
|
.send(&anon(AppId::new()), base_email())
|
||||||
|
.await
|
||||||
|
.unwrap_err();
|
||||||
|
assert!(matches!(err, EmailError::TooLarge { limit: 64, .. }));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn not_configured_throws() {
|
||||||
|
let svc = svc_with(None, Arc::new(DenyAuthz));
|
||||||
|
let err = svc
|
||||||
|
.send(&anon(AppId::new()), base_email())
|
||||||
|
.await
|
||||||
|
.unwrap_err();
|
||||||
|
assert!(matches!(err, EmailError::NotConfigured));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn anonymous_skips_authz() {
|
||||||
|
// DenyAuthz would deny an authed principal; anon skips the check.
|
||||||
|
let (svc, _) = recording();
|
||||||
|
svc.send(&anon(AppId::new()), base_email()).await.unwrap();
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn member_with_editor_role_allowed() {
|
||||||
|
let app = AppId::new();
|
||||||
|
let rec = Arc::new(RecordingTransport::default());
|
||||||
|
let svc = svc_with(
|
||||||
|
Some(rec),
|
||||||
|
Arc::new(GrantAuthz {
|
||||||
|
app,
|
||||||
|
role: AppRole::Editor,
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
let cx = cx_with(app, Some(principal(InstanceRole::Member)));
|
||||||
|
svc.send(&cx, base_email()).await.unwrap();
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn member_without_role_forbidden() {
|
||||||
|
let (svc, _) = recording();
|
||||||
|
let cx = cx_with(AppId::new(), Some(principal(InstanceRole::Member)));
|
||||||
|
let err = svc.send(&cx, base_email()).await.unwrap_err();
|
||||||
|
assert!(matches!(err, EmailError::Forbidden));
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -633,20 +633,36 @@ mod tests {
|
|||||||
.await
|
.await
|
||||||
.unwrap_err();
|
.unwrap_err();
|
||||||
assert!(matches!(err, FilesError::MissingField("content_type")));
|
assert!(matches!(err, FilesError::MissingField("content_type")));
|
||||||
// data
|
// Empty `data` is NO LONGER rejected (v1.1.6 relaxation) — see
|
||||||
let err = files
|
// `empty_file_round_trips`.
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn empty_file_round_trips() {
|
||||||
|
// v1.1.6: a zero-byte blob is a valid stored state (sentinels,
|
||||||
|
// placeholders). Create with empty data, then read it back.
|
||||||
|
let files = svc();
|
||||||
|
let cx = anon_cx(AppId::new());
|
||||||
|
let id = files
|
||||||
.create(
|
.create(
|
||||||
&cx,
|
&cx,
|
||||||
"c",
|
"c",
|
||||||
NewFile {
|
NewFile {
|
||||||
name: "f".into(),
|
name: "empty.bin".into(),
|
||||||
content_type: "text/plain".into(),
|
content_type: "application/octet-stream".into(),
|
||||||
data: vec![],
|
data: vec![],
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
.await
|
.await
|
||||||
.unwrap_err();
|
.expect("empty file create should succeed");
|
||||||
assert!(matches!(err, FilesError::MissingField("data")));
|
let bytes = files.get(&cx, "c", &id.to_string()).await.unwrap();
|
||||||
|
assert_eq!(bytes, Some(Vec::new()));
|
||||||
|
let meta = files
|
||||||
|
.head(&cx, "c", &id.to_string())
|
||||||
|
.await
|
||||||
|
.unwrap()
|
||||||
|
.expect("metadata present");
|
||||||
|
assert_eq!(meta.size, 0);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[tokio::test]
|
#[tokio::test]
|
||||||
|
|||||||
185
crates/manager-core/src/files_sweep.rs
Normal file
185
crates/manager-core/src/files_sweep.rs
Normal file
@@ -0,0 +1,185 @@
|
|||||||
|
//! Orphan `*.tmp.*` blob sweeper (v1.1.6, v1.1.5 follow-up).
|
||||||
|
//!
|
||||||
|
//! The files repo writes blobs atomically: it streams into a
|
||||||
|
//! `<id>.tmp.<pid>-<seq>` temp file, fsyncs, then renames to the final
|
||||||
|
//! `<id>` path. A crash between create and rename leaves an orphan temp
|
||||||
|
//! file that is never read and never reclaimed. This sweeper deletes
|
||||||
|
//! those: every `PICLOUD_FILES_ORPHAN_SWEEP_INTERVAL_SEC` (default 6h) it
|
||||||
|
//! walks `<root>/files/` and unlinks any `*.tmp.*` file older than
|
||||||
|
//! `PICLOUD_FILES_ORPHAN_TMP_TTL_SEC` (default 1h).
|
||||||
|
//!
|
||||||
|
//! Deliberately bounded: it does NOT cross-check on-disk files against DB
|
||||||
|
//! rows (the full reconciling sweeper is v1.3+). It only targets the temp
|
||||||
|
//! files, which are unambiguously orphans once past the TTL — no live
|
||||||
|
//! writer keeps one around for an hour.
|
||||||
|
|
||||||
|
use std::path::{Path, PathBuf};
|
||||||
|
use std::time::{Duration, SystemTime};
|
||||||
|
|
||||||
|
const ENV_INTERVAL: &str = "PICLOUD_FILES_ORPHAN_SWEEP_INTERVAL_SEC";
|
||||||
|
const ENV_TMP_TTL: &str = "PICLOUD_FILES_ORPHAN_TMP_TTL_SEC";
|
||||||
|
const DEFAULT_INTERVAL_SECS: u64 = 21_600; // 6h
|
||||||
|
const DEFAULT_TMP_TTL_SECS: u64 = 3_600; // 1h
|
||||||
|
|
||||||
|
/// Marker that identifies a temp blob (`<id>.tmp.<pid>-<seq>`). A final
|
||||||
|
/// blob is named just `<id>` (a UUID), so it never contains this.
|
||||||
|
const TMP_MARKER: &str = ".tmp.";
|
||||||
|
|
||||||
|
#[derive(Debug, Default, Clone, Copy)]
|
||||||
|
pub struct SweepStats {
|
||||||
|
pub dirs_walked: u64,
|
||||||
|
pub files_deleted: u64,
|
||||||
|
pub bytes_reclaimed: u64,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Spawn the periodic orphan sweep. Spawned at startup alongside the
|
||||||
|
/// cron scheduler and the realtime/cache GC tasks.
|
||||||
|
pub fn spawn_files_orphan_sweep(files_root: PathBuf) {
|
||||||
|
let interval = Duration::from_secs(read_secs(ENV_INTERVAL, DEFAULT_INTERVAL_SECS));
|
||||||
|
let ttl = Duration::from_secs(read_secs(ENV_TMP_TTL, DEFAULT_TMP_TTL_SECS));
|
||||||
|
tokio::spawn(async move {
|
||||||
|
let mut ticker = tokio::time::interval(interval);
|
||||||
|
ticker.tick().await; // skip the immediate first fire
|
||||||
|
loop {
|
||||||
|
ticker.tick().await;
|
||||||
|
let root = files_root.clone();
|
||||||
|
// Blocking filesystem walk off the async worker.
|
||||||
|
let stats = tokio::task::spawn_blocking(move || sweep_orphan_tmp_files(&root, ttl))
|
||||||
|
.await
|
||||||
|
.unwrap_or_default();
|
||||||
|
tracing::info!(
|
||||||
|
dirs_walked = stats.dirs_walked,
|
||||||
|
files_deleted = stats.files_deleted,
|
||||||
|
bytes_reclaimed = stats.bytes_reclaimed,
|
||||||
|
"files orphan sweep complete"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Walk `<files_root>/files/` and delete `*.tmp.*` files older than
|
||||||
|
/// `ttl`. Missing root is not an error (returns zeroed stats). Pure +
|
||||||
|
/// synchronous so it's unit-testable without a runtime.
|
||||||
|
#[must_use]
|
||||||
|
pub fn sweep_orphan_tmp_files(files_root: &Path, ttl: Duration) -> SweepStats {
|
||||||
|
let mut stats = SweepStats::default();
|
||||||
|
let blobs_dir = files_root.join("files");
|
||||||
|
if !blobs_dir.is_dir() {
|
||||||
|
return stats;
|
||||||
|
}
|
||||||
|
let now = SystemTime::now();
|
||||||
|
walk(&blobs_dir, ttl, now, &mut stats);
|
||||||
|
stats
|
||||||
|
}
|
||||||
|
|
||||||
|
fn walk(dir: &Path, ttl: Duration, now: SystemTime, stats: &mut SweepStats) {
|
||||||
|
stats.dirs_walked += 1;
|
||||||
|
let Ok(entries) = std::fs::read_dir(dir) else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
for entry in entries.flatten() {
|
||||||
|
let Ok(ft) = entry.file_type() else {
|
||||||
|
continue;
|
||||||
|
};
|
||||||
|
let path = entry.path();
|
||||||
|
if ft.is_dir() {
|
||||||
|
walk(&path, ttl, now, stats);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if !ft.is_file() {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if !entry.file_name().to_string_lossy().contains(TMP_MARKER) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
let Ok(meta) = entry.metadata() else {
|
||||||
|
continue;
|
||||||
|
};
|
||||||
|
let age = meta
|
||||||
|
.modified()
|
||||||
|
.ok()
|
||||||
|
.and_then(|m| now.duration_since(m).ok())
|
||||||
|
.unwrap_or(Duration::ZERO);
|
||||||
|
if age >= ttl {
|
||||||
|
let size = meta.len();
|
||||||
|
if std::fs::remove_file(&path).is_ok() {
|
||||||
|
stats.files_deleted += 1;
|
||||||
|
stats.bytes_reclaimed += size;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn read_secs(key: &str, default: u64) -> u64 {
|
||||||
|
match std::env::var(key) {
|
||||||
|
Err(_) => default,
|
||||||
|
Ok(v) => match v.parse::<u64>() {
|
||||||
|
Ok(n) if n > 0 => n,
|
||||||
|
_ => {
|
||||||
|
tracing::warn!(env = key, value = %v, "invalid; using default");
|
||||||
|
default
|
||||||
|
}
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
use std::sync::atomic::{AtomicU64, Ordering};
|
||||||
|
|
||||||
|
static SEQ: AtomicU64 = AtomicU64::new(0);
|
||||||
|
|
||||||
|
fn tmp_root() -> PathBuf {
|
||||||
|
let n = SEQ.fetch_add(1, Ordering::Relaxed);
|
||||||
|
let dir =
|
||||||
|
std::env::temp_dir().join(format!("picloud-sweep-test-{}-{n}", std::process::id()));
|
||||||
|
std::fs::create_dir_all(dir.join("files").join("ab")).unwrap();
|
||||||
|
dir
|
||||||
|
}
|
||||||
|
|
||||||
|
fn touch(path: &Path) {
|
||||||
|
std::fs::write(path, b"x").unwrap();
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn deletes_old_tmp_files() {
|
||||||
|
let root = tmp_root();
|
||||||
|
let tmp = root.join("files/ab/uuid.tmp.123-0");
|
||||||
|
touch(&tmp);
|
||||||
|
// ttl 0 → any tmp file counts as old.
|
||||||
|
let stats = sweep_orphan_tmp_files(&root, Duration::ZERO);
|
||||||
|
assert_eq!(stats.files_deleted, 1);
|
||||||
|
assert!(!tmp.exists());
|
||||||
|
assert!(stats.bytes_reclaimed >= 1);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn keeps_young_tmp_files() {
|
||||||
|
let root = tmp_root();
|
||||||
|
let tmp = root.join("files/ab/uuid.tmp.123-0");
|
||||||
|
touch(&tmp);
|
||||||
|
// Large TTL → the just-created file is too young to reap.
|
||||||
|
let stats = sweep_orphan_tmp_files(&root, Duration::from_secs(3600));
|
||||||
|
assert_eq!(stats.files_deleted, 0);
|
||||||
|
assert!(tmp.exists());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn keeps_non_tmp_files() {
|
||||||
|
let root = tmp_root();
|
||||||
|
let blob = root.join("files/ab/0123456789abcdef");
|
||||||
|
touch(&blob);
|
||||||
|
let stats = sweep_orphan_tmp_files(&root, Duration::ZERO);
|
||||||
|
assert_eq!(stats.files_deleted, 0);
|
||||||
|
assert!(blob.exists());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn missing_root_does_not_panic() {
|
||||||
|
let root = std::env::temp_dir().join("picloud-sweep-nonexistent-xyz");
|
||||||
|
let stats = sweep_orphan_tmp_files(&root, Duration::ZERO);
|
||||||
|
assert_eq!(stats.files_deleted, 0);
|
||||||
|
assert_eq!(stats.dirs_walked, 0);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -16,6 +16,7 @@ pub mod app_domain_repo;
|
|||||||
pub mod app_members_api;
|
pub mod app_members_api;
|
||||||
pub mod app_members_repo;
|
pub mod app_members_repo;
|
||||||
pub mod app_repo;
|
pub mod app_repo;
|
||||||
|
pub mod app_secrets_repo;
|
||||||
pub mod apps_api;
|
pub mod apps_api;
|
||||||
pub mod auth;
|
pub mod auth;
|
||||||
pub mod auth_api;
|
pub mod auth_api;
|
||||||
@@ -30,9 +31,12 @@ pub mod dispatcher;
|
|||||||
pub mod docs_filter;
|
pub mod docs_filter;
|
||||||
pub mod docs_repo;
|
pub mod docs_repo;
|
||||||
pub mod docs_service;
|
pub mod docs_service;
|
||||||
|
pub mod email_inbound_api;
|
||||||
|
pub mod email_service;
|
||||||
pub mod files_api;
|
pub mod files_api;
|
||||||
pub mod files_repo;
|
pub mod files_repo;
|
||||||
pub mod files_service;
|
pub mod files_service;
|
||||||
|
pub mod files_sweep;
|
||||||
pub mod gc;
|
pub mod gc;
|
||||||
pub mod http_service;
|
pub mod http_service;
|
||||||
pub mod kv_repo;
|
pub mod kv_repo;
|
||||||
@@ -45,12 +49,18 @@ pub mod outbox_repo;
|
|||||||
pub mod principal_resolver;
|
pub mod principal_resolver;
|
||||||
pub mod pubsub_repo;
|
pub mod pubsub_repo;
|
||||||
pub mod pubsub_service;
|
pub mod pubsub_service;
|
||||||
|
pub mod realtime_authority;
|
||||||
pub mod repo;
|
pub mod repo;
|
||||||
pub mod route_admin;
|
pub mod route_admin;
|
||||||
pub mod route_repo;
|
pub mod route_repo;
|
||||||
pub mod sandbox;
|
pub mod sandbox;
|
||||||
pub mod scheduler;
|
pub mod scheduler;
|
||||||
|
pub mod secrets_api;
|
||||||
|
pub mod secrets_repo;
|
||||||
|
pub mod secrets_service;
|
||||||
pub mod ssrf;
|
pub mod ssrf;
|
||||||
|
pub mod topic_repo;
|
||||||
|
pub mod topics_api;
|
||||||
pub mod trigger_config;
|
pub mod trigger_config;
|
||||||
pub mod trigger_repo;
|
pub mod trigger_repo;
|
||||||
pub mod triggers_api;
|
pub mod triggers_api;
|
||||||
@@ -81,6 +91,9 @@ pub use app_members_repo::{
|
|||||||
PostgresAppMembersRepository,
|
PostgresAppMembersRepository,
|
||||||
};
|
};
|
||||||
pub use app_repo::{resolve_app, AppLookup, AppRepository, PostgresAppRepository};
|
pub use app_repo::{resolve_app, AppLookup, AppRepository, PostgresAppRepository};
|
||||||
|
pub use app_secrets_repo::{
|
||||||
|
AppSecretsRepo, AppSecretsRepoError, PostgresAppSecretsRepo, SIGNING_KEY_LEN,
|
||||||
|
};
|
||||||
pub use apps_api::{apps_router, AppsState};
|
pub use apps_api::{apps_router, AppsState};
|
||||||
pub use auth_api::auth_router;
|
pub use auth_api::auth_router;
|
||||||
pub use auth_bootstrap::{
|
pub use auth_bootstrap::{
|
||||||
@@ -101,9 +114,15 @@ pub use dead_letters_api::{dead_letters_router, DeadLettersApiError, DeadLetters
|
|||||||
pub use dispatcher::{compute_backoff, Dispatcher, DispatcherError};
|
pub use dispatcher::{compute_backoff, Dispatcher, DispatcherError};
|
||||||
pub use docs_repo::{DocsRepo, DocsRepoError, PostgresDocsRepo};
|
pub use docs_repo::{DocsRepo, DocsRepoError, PostgresDocsRepo};
|
||||||
pub use docs_service::DocsServiceImpl;
|
pub use docs_service::DocsServiceImpl;
|
||||||
|
pub use email_inbound_api::{email_inbound_router, EmailInboundError, EmailInboundState};
|
||||||
|
pub use email_service::{
|
||||||
|
EmailConfig, EmailServiceImpl, EmailTransport, LettreEmailTransport, SmtpConfig, SmtpTls,
|
||||||
|
DEFAULT_EMAIL_MAX_MESSAGE_BYTES,
|
||||||
|
};
|
||||||
pub use files_api::{files_admin_router, FilesAdminState};
|
pub use files_api::{files_admin_router, FilesAdminState};
|
||||||
pub use files_repo::{FilesConfig, FilesRepo, FilesRepoError, FsFilesRepo};
|
pub use files_repo::{FilesConfig, FilesRepo, FilesRepoError, FsFilesRepo};
|
||||||
pub use files_service::FilesServiceImpl;
|
pub use files_service::FilesServiceImpl;
|
||||||
|
pub use files_sweep::{spawn_files_orphan_sweep, sweep_orphan_tmp_files, SweepStats};
|
||||||
pub use gc::{spawn_abandoned_gc, spawn_dead_letter_gc};
|
pub use gc::{spawn_abandoned_gc, spawn_dead_letter_gc};
|
||||||
pub use http_service::{HttpConfig, HttpServiceImpl};
|
pub use http_service::{HttpConfig, HttpServiceImpl};
|
||||||
pub use kv_repo::{KvRepo, KvRepoError, PostgresKvRepo};
|
pub use kv_repo::{KvRepo, KvRepoError, PostgresKvRepo};
|
||||||
@@ -116,7 +135,8 @@ pub use outbox_repo::{
|
|||||||
};
|
};
|
||||||
pub use principal_resolver::{AdminPrincipalResolver, PrincipalResolver, PrincipalResolverError};
|
pub use principal_resolver::{AdminPrincipalResolver, PrincipalResolver, PrincipalResolverError};
|
||||||
pub use pubsub_repo::{PostgresPubsubRepo, PublishCtx, PubsubRepo, PubsubRepoError};
|
pub use pubsub_repo::{PostgresPubsubRepo, PublishCtx, PubsubRepo, PubsubRepoError};
|
||||||
pub use pubsub_service::PubsubServiceImpl;
|
pub use pubsub_service::{PubsubServiceImpl, SubscriberTokenConfig};
|
||||||
|
pub use realtime_authority::RealtimeAuthorityImpl;
|
||||||
pub use repo::{
|
pub use repo::{
|
||||||
ExecutionLogRepository, NewScript, PostgresExecutionLogRepository, PostgresScriptRepository,
|
ExecutionLogRepository, NewScript, PostgresExecutionLogRepository, PostgresScriptRepository,
|
||||||
RepoResolver, ScriptPatch, ScriptRepository, ScriptRepositoryError,
|
RepoResolver, ScriptPatch, ScriptRepository, ScriptRepositoryError,
|
||||||
@@ -124,11 +144,22 @@ pub use repo::{
|
|||||||
pub use route_admin::{compile_routes, route_admin_router, RouteAdminState};
|
pub use route_admin::{compile_routes, route_admin_router, RouteAdminState};
|
||||||
pub use route_repo::{NewRoute, PostgresRouteRepository, RouteRepository};
|
pub use route_repo::{NewRoute, PostgresRouteRepository, RouteRepository};
|
||||||
pub use sandbox::{CeilingError, SandboxCeiling};
|
pub use sandbox::{CeilingError, SandboxCeiling};
|
||||||
|
pub use secrets_api::{secrets_router, SecretsApiError, SecretsState};
|
||||||
|
pub use secrets_repo::{
|
||||||
|
PostgresSecretsRepo, SecretMeta, SecretsMetaPage, SecretsNamePage, SecretsRepo,
|
||||||
|
SecretsRepoError, StoredSecret,
|
||||||
|
};
|
||||||
|
pub use secrets_service::{
|
||||||
|
open as open_secret, seal as seal_secret, SecretsConfig, SecretsServiceImpl,
|
||||||
|
DEFAULT_SECRET_MAX_VALUE_BYTES,
|
||||||
|
};
|
||||||
|
pub use topic_repo::{PostgresTopicRepo, Topic, TopicAuthMode, TopicRepo, TopicRepoError};
|
||||||
|
pub use topics_api::{topics_router, TopicsApiError, TopicsState};
|
||||||
pub use trigger_config::{BackoffShape, TriggerConfig};
|
pub use trigger_config::{BackoffShape, TriggerConfig};
|
||||||
pub use trigger_repo::{
|
pub use trigger_repo::{
|
||||||
collection_matches, CreateDeadLetterTrigger, CreateDocsTrigger, CreateFilesTrigger,
|
collection_matches, CreateDeadLetterTrigger, CreateDocsTrigger, CreateEmailTrigger,
|
||||||
CreateKvTrigger, CreatePubsubTrigger, DeadLetterTriggerMatch, DocsTriggerMatch,
|
CreateFilesTrigger, CreateKvTrigger, CreatePubsubTrigger, DeadLetterTriggerMatch,
|
||||||
FilesTriggerMatch, KvTriggerMatch, PostgresTriggerRepo, Trigger, TriggerDetails,
|
DocsTriggerMatch, EmailInboundTarget, FilesTriggerMatch, KvTriggerMatch, PostgresTriggerRepo,
|
||||||
TriggerDispatchMode, TriggerKind, TriggerRepo, TriggerRepoError,
|
Trigger, TriggerDetails, TriggerDispatchMode, TriggerKind, TriggerRepo, TriggerRepoError,
|
||||||
};
|
};
|
||||||
pub use triggers_api::{triggers_router, TriggersApiError, TriggersState};
|
pub use triggers_api::{triggers_router, TriggersApiError, TriggersState};
|
||||||
|
|||||||
@@ -31,6 +31,8 @@ pub enum OutboxSourceKind {
|
|||||||
Files,
|
Files,
|
||||||
/// v1.1.5.
|
/// v1.1.5.
|
||||||
Pubsub,
|
Pubsub,
|
||||||
|
/// v1.1.7. Inbound email POSTed to the webhook receiver.
|
||||||
|
Email,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl OutboxSourceKind {
|
impl OutboxSourceKind {
|
||||||
@@ -44,6 +46,7 @@ impl OutboxSourceKind {
|
|||||||
Self::Cron => "cron",
|
Self::Cron => "cron",
|
||||||
Self::Files => "files",
|
Self::Files => "files",
|
||||||
Self::Pubsub => "pubsub",
|
Self::Pubsub => "pubsub",
|
||||||
|
Self::Email => "email",
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -57,6 +60,7 @@ impl OutboxSourceKind {
|
|||||||
"cron" => Some(Self::Cron),
|
"cron" => Some(Self::Cron),
|
||||||
"files" => Some(Self::Files),
|
"files" => Some(Self::Files),
|
||||||
"pubsub" => Some(Self::Pubsub),
|
"pubsub" => Some(Self::Pubsub),
|
||||||
|
"email" => Some(Self::Email),
|
||||||
_ => None,
|
_ => None,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -11,20 +11,106 @@
|
|||||||
use std::sync::Arc;
|
use std::sync::Arc;
|
||||||
|
|
||||||
use async_trait::async_trait;
|
use async_trait::async_trait;
|
||||||
use picloud_shared::{PubsubError, PubsubService, SdkCallCx, TriggerEvent};
|
use picloud_shared::subscriber_token::{self, TokenClaims};
|
||||||
|
use picloud_shared::{
|
||||||
|
PubsubError, PubsubService, RealtimeBroadcaster, RealtimeEvent, SdkCallCx, TriggerEvent,
|
||||||
|
};
|
||||||
|
|
||||||
|
use crate::app_secrets_repo::AppSecretsRepo;
|
||||||
use crate::authz::{self, AuthzRepo, Capability};
|
use crate::authz::{self, AuthzRepo, Capability};
|
||||||
use crate::pubsub_repo::{PublishCtx, PubsubRepo, PubsubRepoError};
|
use crate::pubsub_repo::{PublishCtx, PubsubRepo, PubsubRepoError};
|
||||||
|
use crate::topic_repo::TopicRepo;
|
||||||
|
|
||||||
|
/// TTL bounds (seconds) for `pubsub::subscriber_token`. Env-overridable
|
||||||
|
/// via `PICLOUD_SUBSCRIBER_TOKEN_TTL_{MIN,MAX,DEFAULT}_SEC`.
|
||||||
|
#[derive(Debug, Clone, Copy)]
|
||||||
|
pub struct SubscriberTokenConfig {
|
||||||
|
pub min_ttl: i64,
|
||||||
|
pub max_ttl: i64,
|
||||||
|
pub default_ttl: i64,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl SubscriberTokenConfig {
|
||||||
|
#[must_use]
|
||||||
|
pub const fn conservative() -> Self {
|
||||||
|
Self {
|
||||||
|
min_ttl: 10,
|
||||||
|
max_ttl: 86_400,
|
||||||
|
default_ttl: 3_600,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Load from env, falling back to the conservative defaults for any
|
||||||
|
/// missing / invalid value.
|
||||||
|
#[must_use]
|
||||||
|
pub fn from_env() -> Self {
|
||||||
|
let mut c = Self::conservative();
|
||||||
|
load_i64(&mut c.min_ttl, "PICLOUD_SUBSCRIBER_TOKEN_TTL_MIN_SEC");
|
||||||
|
load_i64(&mut c.max_ttl, "PICLOUD_SUBSCRIBER_TOKEN_TTL_MAX_SEC");
|
||||||
|
load_i64(
|
||||||
|
&mut c.default_ttl,
|
||||||
|
"PICLOUD_SUBSCRIBER_TOKEN_TTL_DEFAULT_SEC",
|
||||||
|
);
|
||||||
|
c
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Default for SubscriberTokenConfig {
|
||||||
|
fn default() -> Self {
|
||||||
|
Self::conservative()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn load_i64(dst: &mut i64, key: &str) {
|
||||||
|
if let Ok(v) = std::env::var(key) {
|
||||||
|
match v.parse::<i64>() {
|
||||||
|
Ok(n) if n > 0 => *dst = n,
|
||||||
|
_ => tracing::warn!(env = key, value = %v, "ignoring invalid token-ttl value"),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
pub struct PubsubServiceImpl {
|
pub struct PubsubServiceImpl {
|
||||||
repo: Arc<dyn PubsubRepo>,
|
repo: Arc<dyn PubsubRepo>,
|
||||||
authz: Arc<dyn AuthzRepo>,
|
authz: Arc<dyn AuthzRepo>,
|
||||||
|
// Realtime extras (v1.1.6) — optional so the existing two-arg
|
||||||
|
// constructor (and its unit tests) keep working without them. The
|
||||||
|
// production binary attaches them via `with_realtime`.
|
||||||
|
realtime: Option<Arc<dyn RealtimeBroadcaster>>,
|
||||||
|
topics: Option<Arc<dyn TopicRepo>>,
|
||||||
|
secrets: Option<Arc<dyn AppSecretsRepo>>,
|
||||||
|
token_config: SubscriberTokenConfig,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl PubsubServiceImpl {
|
impl PubsubServiceImpl {
|
||||||
#[must_use]
|
#[must_use]
|
||||||
pub fn new(repo: Arc<dyn PubsubRepo>, authz: Arc<dyn AuthzRepo>) -> Self {
|
pub fn new(repo: Arc<dyn PubsubRepo>, authz: Arc<dyn AuthzRepo>) -> Self {
|
||||||
Self { repo, authz }
|
Self {
|
||||||
|
repo,
|
||||||
|
authz,
|
||||||
|
realtime: None,
|
||||||
|
topics: None,
|
||||||
|
secrets: None,
|
||||||
|
token_config: SubscriberTokenConfig::conservative(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Attach the v1.1.6 realtime surface: the in-process broadcaster
|
||||||
|
/// (publish fan-out to SSE subscribers), the topic registry +
|
||||||
|
/// app-secrets repo (subscriber-token minting), and the TTL config.
|
||||||
|
#[must_use]
|
||||||
|
pub fn with_realtime(
|
||||||
|
mut self,
|
||||||
|
broadcaster: Arc<dyn RealtimeBroadcaster>,
|
||||||
|
topics: Arc<dyn TopicRepo>,
|
||||||
|
secrets: Arc<dyn AppSecretsRepo>,
|
||||||
|
token_config: SubscriberTokenConfig,
|
||||||
|
) -> Self {
|
||||||
|
self.realtime = Some(broadcaster);
|
||||||
|
self.topics = Some(topics);
|
||||||
|
self.secrets = Some(secrets);
|
||||||
|
self.token_config = token_config;
|
||||||
|
self
|
||||||
}
|
}
|
||||||
|
|
||||||
async fn check_publish(&self, cx: &SdkCallCx) -> Result<(), PubsubError> {
|
async fn check_publish(&self, cx: &SdkCallCx) -> Result<(), PubsubError> {
|
||||||
@@ -60,12 +146,15 @@ impl PubsubService for PubsubServiceImpl {
|
|||||||
}
|
}
|
||||||
self.check_publish(cx).await?;
|
self.check_publish(cx).await?;
|
||||||
|
|
||||||
// `published_at` is stamped on the manager side so every
|
// `published_at` is stamped once on the manager side so every
|
||||||
// delivery agrees on one instant.
|
// delivery path — durable triggers AND the realtime broadcast —
|
||||||
|
// agrees on one instant. The message is cloned into the trigger
|
||||||
|
// event so the realtime path can reuse the original.
|
||||||
|
let published_at = chrono::Utc::now();
|
||||||
let event = TriggerEvent::Pubsub {
|
let event = TriggerEvent::Pubsub {
|
||||||
topic: topic.to_string(),
|
topic: topic.to_string(),
|
||||||
message,
|
message: message.clone(),
|
||||||
published_at: chrono::Utc::now(),
|
published_at,
|
||||||
};
|
};
|
||||||
let payload = serde_json::to_value(&event)
|
let payload = serde_json::to_value(&event)
|
||||||
.map_err(|e| PubsubError::Rejected(format!("event serialize: {e}")))?;
|
.map_err(|e| PubsubError::Rejected(format!("event serialize: {e}")))?;
|
||||||
@@ -76,12 +165,115 @@ impl PubsubService for PubsubServiceImpl {
|
|||||||
trigger_depth: cx.trigger_depth,
|
trigger_depth: cx.trigger_depth,
|
||||||
root_execution_id: cx.root_execution_id,
|
root_execution_id: cx.root_execution_id,
|
||||||
};
|
};
|
||||||
|
// Order (design notes §8): transactional outbox fan-out + commit
|
||||||
|
// FIRST; only then the best-effort realtime broadcast. If the
|
||||||
|
// fan-out fails, the publish throws and no broadcast happens. If
|
||||||
|
// the broadcast fails after a committed fan-out, trigger
|
||||||
|
// deliveries still happen and only SSE subscribers miss this
|
||||||
|
// event (no replay in v1.1.6).
|
||||||
|
//
|
||||||
// No matching triggers → 0 rows written, publish still succeeds.
|
// No matching triggers → 0 rows written, publish still succeeds.
|
||||||
self.repo
|
self.repo
|
||||||
.fan_out_publish(publish_ctx, topic, payload)
|
.fan_out_publish(publish_ctx, topic, payload)
|
||||||
.await?;
|
.await?;
|
||||||
|
|
||||||
|
// Non-transactional, best-effort fan-out to in-process SSE
|
||||||
|
// subscribers. Run on a child task so a panicking broadcaster
|
||||||
|
// (or a future cluster-mode resolver fault) becomes a warn log,
|
||||||
|
// never a failed publish — the durable deliveries already
|
||||||
|
// committed above.
|
||||||
|
if let Some(realtime) = self.realtime.clone() {
|
||||||
|
let app_id = cx.app_id;
|
||||||
|
let topic_owned = topic.to_string();
|
||||||
|
let realtime_event = RealtimeEvent {
|
||||||
|
topic: topic_owned.clone(),
|
||||||
|
message,
|
||||||
|
published_at,
|
||||||
|
};
|
||||||
|
let handle = tokio::spawn(async move {
|
||||||
|
realtime.publish(app_id, &topic_owned, realtime_event).await;
|
||||||
|
});
|
||||||
|
if let Err(e) = handle.await {
|
||||||
|
tracing::warn!(error = %e, "realtime broadcast failed; publish unaffected");
|
||||||
|
}
|
||||||
|
}
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async fn mint_subscriber_token(
|
||||||
|
&self,
|
||||||
|
cx: &SdkCallCx,
|
||||||
|
topics: Vec<String>,
|
||||||
|
ttl_seconds: Option<i64>,
|
||||||
|
) -> Result<String, PubsubError> {
|
||||||
|
// Anonymous (public-HTTP) scripts can't mint — that would bypass
|
||||||
|
// the token-minting authz boundary.
|
||||||
|
let Some(principal) = cx.principal.as_ref() else {
|
||||||
|
return Err(PubsubError::SubscriberToken(
|
||||||
|
"pubsub::subscriber_token: requires an authenticated principal \
|
||||||
|
(a script on a public route cannot mint tokens)"
|
||||||
|
.into(),
|
||||||
|
));
|
||||||
|
};
|
||||||
|
// Minting reuses the existing pub/sub publish capability (no new
|
||||||
|
// scope — the seven-scope commitment holds).
|
||||||
|
authz::require(
|
||||||
|
&*self.authz,
|
||||||
|
principal,
|
||||||
|
Capability::AppPubsubPublish(cx.app_id),
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
.map_err(|_| PubsubError::Forbidden)?;
|
||||||
|
|
||||||
|
let (Some(topic_repo), Some(secrets)) = (self.topics.as_ref(), self.secrets.as_ref())
|
||||||
|
else {
|
||||||
|
return Err(PubsubError::Unavailable(
|
||||||
|
"subscriber tokens are not wired in".into(),
|
||||||
|
));
|
||||||
|
};
|
||||||
|
|
||||||
|
if topics.is_empty() {
|
||||||
|
return Err(PubsubError::SubscriberToken(
|
||||||
|
"pubsub::subscriber_token: topics list must not be empty".into(),
|
||||||
|
));
|
||||||
|
}
|
||||||
|
|
||||||
|
let ttl = ttl_seconds.unwrap_or(self.token_config.default_ttl);
|
||||||
|
if ttl < self.token_config.min_ttl || ttl > self.token_config.max_ttl {
|
||||||
|
return Err(PubsubError::SubscriberToken(format!(
|
||||||
|
"pubsub::subscriber_token: ttl_seconds must be between {} and {}",
|
||||||
|
self.token_config.min_ttl, self.token_config.max_ttl
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
|
||||||
|
// Every requested topic must be registered as externally
|
||||||
|
// subscribable in this app — fail fast rather than mint a token
|
||||||
|
// that won't work.
|
||||||
|
for name in &topics {
|
||||||
|
let registered = topic_repo
|
||||||
|
.get(cx.app_id, name)
|
||||||
|
.await
|
||||||
|
.map_err(|e| PubsubError::Unavailable(e.to_string()))?;
|
||||||
|
if !registered.is_some_and(|t| t.external_subscribable) {
|
||||||
|
return Err(PubsubError::SubscriberToken(format!(
|
||||||
|
"pubsub::subscriber_token: topic {name} is not externally subscribable"
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
let key = secrets
|
||||||
|
.get_or_create_signing_key(cx.app_id)
|
||||||
|
.await
|
||||||
|
.map_err(|e| PubsubError::Unavailable(e.to_string()))?;
|
||||||
|
let now = chrono::Utc::now().timestamp();
|
||||||
|
let claims = TokenClaims {
|
||||||
|
app_id: cx.app_id,
|
||||||
|
topics,
|
||||||
|
exp: now.saturating_add(ttl),
|
||||||
|
iat: now,
|
||||||
|
};
|
||||||
|
Ok(subscriber_token::sign(&key, &claims))
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// ----------------------------------------------------------------------------
|
// ----------------------------------------------------------------------------
|
||||||
@@ -317,4 +509,218 @@ mod tests {
|
|||||||
.await
|
.await
|
||||||
.unwrap();
|
.unwrap();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ------------------------------------------------------------------
|
||||||
|
// v1.1.6 realtime broadcast + subscriber-token minting
|
||||||
|
// ------------------------------------------------------------------
|
||||||
|
|
||||||
|
use crate::app_secrets_repo::{AppSecretsRepo, AppSecretsRepoError};
|
||||||
|
use crate::topic_repo::{Topic, TopicAuthMode, TopicRepo, TopicRepoError};
|
||||||
|
use picloud_orchestrator_core::InProcessBroadcaster;
|
||||||
|
use picloud_shared::{RealtimeBroadcaster, RealtimeEvent};
|
||||||
|
|
||||||
|
/// Topic repo fake: returns the configured topics as registered +
|
||||||
|
/// externally-subscribable (unless absent).
|
||||||
|
struct FakeTopicRepo(Vec<String>);
|
||||||
|
#[async_trait]
|
||||||
|
impl TopicRepo for FakeTopicRepo {
|
||||||
|
async fn create(
|
||||||
|
&self,
|
||||||
|
_: AppId,
|
||||||
|
_: &str,
|
||||||
|
_: bool,
|
||||||
|
_: TopicAuthMode,
|
||||||
|
) -> Result<Topic, TopicRepoError> {
|
||||||
|
unimplemented!()
|
||||||
|
}
|
||||||
|
async fn list(&self, _: AppId) -> Result<Vec<Topic>, TopicRepoError> {
|
||||||
|
unimplemented!()
|
||||||
|
}
|
||||||
|
async fn get(&self, _: AppId, name: &str) -> Result<Option<Topic>, TopicRepoError> {
|
||||||
|
Ok(self.0.iter().any(|t| t == name).then(|| Topic {
|
||||||
|
name: name.to_string(),
|
||||||
|
external_subscribable: true,
|
||||||
|
auth_mode: TopicAuthMode::Token,
|
||||||
|
created_at: chrono::Utc::now(),
|
||||||
|
updated_at: chrono::Utc::now(),
|
||||||
|
}))
|
||||||
|
}
|
||||||
|
async fn update(
|
||||||
|
&self,
|
||||||
|
_: AppId,
|
||||||
|
_: &str,
|
||||||
|
_: Option<bool>,
|
||||||
|
_: Option<TopicAuthMode>,
|
||||||
|
) -> Result<Option<Topic>, TopicRepoError> {
|
||||||
|
unimplemented!()
|
||||||
|
}
|
||||||
|
async fn delete(&self, _: AppId, _: &str) -> Result<bool, TopicRepoError> {
|
||||||
|
unimplemented!()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Default)]
|
||||||
|
struct FakeSecrets;
|
||||||
|
#[async_trait]
|
||||||
|
impl AppSecretsRepo for FakeSecrets {
|
||||||
|
async fn get_or_create_signing_key(
|
||||||
|
&self,
|
||||||
|
_: AppId,
|
||||||
|
) -> Result<Vec<u8>, AppSecretsRepoError> {
|
||||||
|
Ok(vec![42u8; 32])
|
||||||
|
}
|
||||||
|
async fn signing_key(&self, _: AppId) -> Result<Option<Vec<u8>>, AppSecretsRepoError> {
|
||||||
|
Ok(Some(vec![42u8; 32]))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Broadcaster that panics on publish — proves a broadcast fault
|
||||||
|
/// can't fail the publish.
|
||||||
|
struct PanicBroadcaster;
|
||||||
|
#[async_trait]
|
||||||
|
impl RealtimeBroadcaster for PanicBroadcaster {
|
||||||
|
async fn subscribe(
|
||||||
|
&self,
|
||||||
|
_: AppId,
|
||||||
|
_: &str,
|
||||||
|
) -> Result<tokio::sync::broadcast::Receiver<RealtimeEvent>, picloud_shared::BroadcasterError>
|
||||||
|
{
|
||||||
|
unimplemented!()
|
||||||
|
}
|
||||||
|
async fn publish(&self, _: AppId, _: &str, _: RealtimeEvent) {
|
||||||
|
panic!("boom");
|
||||||
|
}
|
||||||
|
async fn drop_topic(&self, _: AppId, _: &str) {}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn realtime_svc(
|
||||||
|
repo: Arc<dyn PubsubRepo>,
|
||||||
|
broadcaster: Arc<dyn RealtimeBroadcaster>,
|
||||||
|
topics: Vec<String>,
|
||||||
|
) -> PubsubServiceImpl {
|
||||||
|
PubsubServiceImpl::new(repo, Arc::new(EditorAuthzRepo)).with_realtime(
|
||||||
|
broadcaster,
|
||||||
|
Arc::new(FakeTopicRepo(topics)),
|
||||||
|
Arc::new(FakeSecrets),
|
||||||
|
SubscriberTokenConfig::conservative(),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||||
|
async fn publish_broadcasts_to_in_process_subscribers() {
|
||||||
|
let app = AppId::new();
|
||||||
|
let broadcaster = Arc::new(InProcessBroadcaster::new(16));
|
||||||
|
let mut rx = broadcaster.subscribe(app, "chat").await.unwrap();
|
||||||
|
let svc = realtime_svc(
|
||||||
|
Arc::new(InMemoryPubsubRepo::new(vec![])),
|
||||||
|
broadcaster,
|
||||||
|
vec![],
|
||||||
|
);
|
||||||
|
svc.publish_durable(&anon_cx(app), "chat", serde_json::json!({ "hi": 1 }))
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
let ev = rx.recv().await.unwrap();
|
||||||
|
assert_eq!(ev.topic, "chat");
|
||||||
|
assert_eq!(ev.message, serde_json::json!({ "hi": 1 }));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||||
|
async fn panicking_broadcaster_does_not_fail_publish() {
|
||||||
|
let app = AppId::new();
|
||||||
|
let svc = realtime_svc(
|
||||||
|
Arc::new(InMemoryPubsubRepo::new(vec![])),
|
||||||
|
Arc::new(PanicBroadcaster),
|
||||||
|
vec![],
|
||||||
|
);
|
||||||
|
// The outbox fan-out committed; the broadcast panic is swallowed.
|
||||||
|
svc.publish_durable(&anon_cx(app), "chat", serde_json::json!(1))
|
||||||
|
.await
|
||||||
|
.expect("publish must succeed despite broadcast panic");
|
||||||
|
}
|
||||||
|
|
||||||
|
fn mint_svc(topics: Vec<String>) -> PubsubServiceImpl {
|
||||||
|
realtime_svc(
|
||||||
|
Arc::new(InMemoryPubsubRepo::new(vec![])),
|
||||||
|
Arc::new(picloud_shared::NoopRealtimeBroadcaster),
|
||||||
|
topics,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn mint_returns_token_scoped_to_topics() {
|
||||||
|
let app = AppId::new();
|
||||||
|
let svc = mint_svc(vec!["chat".into(), "notify".into()]);
|
||||||
|
let token = svc
|
||||||
|
.mint_subscriber_token(
|
||||||
|
&member_cx(app),
|
||||||
|
vec!["chat".into(), "notify".into()],
|
||||||
|
Some(120),
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
// Verify with the fake key; claims carry the topics + expiry.
|
||||||
|
let claims = subscriber_token::verify(&[42u8; 32], &token, chrono::Utc::now().timestamp())
|
||||||
|
.expect("token verifies");
|
||||||
|
assert_eq!(claims.app_id, app);
|
||||||
|
assert!(claims.allows_topic("chat") && claims.allows_topic("notify"));
|
||||||
|
assert!(claims.exp > claims.iat);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn mint_anonymous_principal_throws() {
|
||||||
|
let app = AppId::new();
|
||||||
|
let svc = mint_svc(vec!["chat".into()]);
|
||||||
|
let err = svc
|
||||||
|
.mint_subscriber_token(&anon_cx(app), vec!["chat".into()], None)
|
||||||
|
.await
|
||||||
|
.unwrap_err();
|
||||||
|
assert!(matches!(err, PubsubError::SubscriberToken(_)));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn mint_empty_topics_throws() {
|
||||||
|
let app = AppId::new();
|
||||||
|
let svc = mint_svc(vec!["chat".into()]);
|
||||||
|
let err = svc
|
||||||
|
.mint_subscriber_token(&member_cx(app), vec![], None)
|
||||||
|
.await
|
||||||
|
.unwrap_err();
|
||||||
|
assert!(matches!(err, PubsubError::SubscriberToken(_)));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn mint_ttl_below_min_and_above_max_throw() {
|
||||||
|
let app = AppId::new();
|
||||||
|
let svc = mint_svc(vec!["chat".into()]);
|
||||||
|
for bad in [Some(5), Some(90_000)] {
|
||||||
|
let err = svc
|
||||||
|
.mint_subscriber_token(&member_cx(app), vec!["chat".into()], bad)
|
||||||
|
.await
|
||||||
|
.unwrap_err();
|
||||||
|
assert!(
|
||||||
|
matches!(err, PubsubError::SubscriberToken(_)),
|
||||||
|
"ttl {bad:?}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn mint_unregistered_topic_throws_with_message() {
|
||||||
|
let app = AppId::new();
|
||||||
|
// "chat" registered; "secret" is not.
|
||||||
|
let svc = mint_svc(vec!["chat".into()]);
|
||||||
|
let err = svc
|
||||||
|
.mint_subscriber_token(&member_cx(app), vec!["chat".into(), "secret".into()], None)
|
||||||
|
.await
|
||||||
|
.unwrap_err();
|
||||||
|
match err {
|
||||||
|
PubsubError::SubscriberToken(msg) => {
|
||||||
|
assert!(
|
||||||
|
msg.contains("topic secret is not externally subscribable"),
|
||||||
|
"got: {msg}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
other => panic!("expected SubscriberToken, got {other:?}"),
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
338
crates/manager-core/src/realtime_authority.rs
Normal file
338
crates/manager-core/src/realtime_authority.rs
Normal file
@@ -0,0 +1,338 @@
|
|||||||
|
//! `RealtimeAuthorityImpl` — the DB-backed SSE subscribe gate (v1.1.6).
|
||||||
|
//!
|
||||||
|
//! Backs the [`picloud_shared::RealtimeAuthority`] trait the SSE handler
|
||||||
|
//! in orchestrator-core calls. All `topics`-table reads and signing-key
|
||||||
|
//! material stay inside this impl so the data-plane crate never touches
|
||||||
|
//! the key.
|
||||||
|
//!
|
||||||
|
//! Verdict mapping (see [`SubscribeDenied`]):
|
||||||
|
//! * topic missing OR not externally subscribable → `NotFound` (404).
|
||||||
|
//! Both collapse to 404 so the endpoint can't probe internal topics.
|
||||||
|
//! * `auth_mode = 'public'` → allow.
|
||||||
|
//! * `auth_mode = 'token'` → verify the HMAC token (present, signed by
|
||||||
|
//! this app's key, unexpired, scoped to this topic) → allow, else
|
||||||
|
//! `Unauthorized` (401, generic — never says which check failed).
|
||||||
|
//!
|
||||||
|
//! Signing keys never change in v1.1.6 (no rotation API), so a small
|
||||||
|
//! in-memory cache avoids a per-subscribe DB read once an app's key has
|
||||||
|
//! been seen. The cache is purely an optimization — a cold miss reads
|
||||||
|
//! the row.
|
||||||
|
|
||||||
|
use std::collections::HashMap;
|
||||||
|
use std::sync::{Arc, Mutex};
|
||||||
|
|
||||||
|
use async_trait::async_trait;
|
||||||
|
use picloud_shared::{subscriber_token, AppId, RealtimeAuthority, SubscribeDenied};
|
||||||
|
|
||||||
|
use crate::app_secrets_repo::AppSecretsRepo;
|
||||||
|
use crate::topic_repo::{TopicAuthMode, TopicRepo};
|
||||||
|
|
||||||
|
pub struct RealtimeAuthorityImpl {
|
||||||
|
topics: Arc<dyn TopicRepo>,
|
||||||
|
secrets: Arc<dyn AppSecretsRepo>,
|
||||||
|
key_cache: Mutex<HashMap<AppId, Vec<u8>>>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl RealtimeAuthorityImpl {
|
||||||
|
#[must_use]
|
||||||
|
pub fn new(topics: Arc<dyn TopicRepo>, secrets: Arc<dyn AppSecretsRepo>) -> Self {
|
||||||
|
Self {
|
||||||
|
topics,
|
||||||
|
secrets,
|
||||||
|
key_cache: Mutex::new(HashMap::new()),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Fetch the app's signing key, consulting the cache first. Returns
|
||||||
|
/// `None` when the app has no key (no token ever minted) — which the
|
||||||
|
/// caller maps to `Unauthorized`.
|
||||||
|
async fn signing_key(&self, app_id: AppId) -> Result<Option<Vec<u8>>, SubscribeDenied> {
|
||||||
|
if let Ok(cache) = self.key_cache.lock() {
|
||||||
|
if let Some(k) = cache.get(&app_id) {
|
||||||
|
return Ok(Some(k.clone()));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let key = self
|
||||||
|
.secrets
|
||||||
|
.signing_key(app_id)
|
||||||
|
.await
|
||||||
|
.map_err(|e| SubscribeDenied::Backend(e.to_string()))?;
|
||||||
|
if let Some(ref k) = key {
|
||||||
|
if let Ok(mut cache) = self.key_cache.lock() {
|
||||||
|
cache.insert(app_id, k.clone());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Ok(key)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[async_trait]
|
||||||
|
impl RealtimeAuthority for RealtimeAuthorityImpl {
|
||||||
|
async fn authorize_subscribe(
|
||||||
|
&self,
|
||||||
|
app_id: AppId,
|
||||||
|
topic: &str,
|
||||||
|
token: Option<&str>,
|
||||||
|
) -> Result<(), SubscribeDenied> {
|
||||||
|
let registered = self
|
||||||
|
.topics
|
||||||
|
.get(app_id, topic)
|
||||||
|
.await
|
||||||
|
.map_err(|e| SubscribeDenied::Backend(e.to_string()))?;
|
||||||
|
|
||||||
|
// Missing topic AND internal-only topic both 404 — don't leak
|
||||||
|
// which internal topics exist.
|
||||||
|
let Some(t) = registered.filter(|t| t.external_subscribable) else {
|
||||||
|
return Err(SubscribeDenied::NotFound);
|
||||||
|
};
|
||||||
|
|
||||||
|
match t.auth_mode {
|
||||||
|
TopicAuthMode::Public => Ok(()),
|
||||||
|
TopicAuthMode::Token => {
|
||||||
|
let token = token.ok_or(SubscribeDenied::Unauthorized)?;
|
||||||
|
let key = self
|
||||||
|
.signing_key(app_id)
|
||||||
|
.await?
|
||||||
|
.ok_or(SubscribeDenied::Unauthorized)?;
|
||||||
|
let now = chrono::Utc::now().timestamp();
|
||||||
|
let claims = subscriber_token::verify(&key, token, now)
|
||||||
|
.map_err(|_| SubscribeDenied::Unauthorized)?;
|
||||||
|
// Per-app key already makes a cross-app token fail the
|
||||||
|
// signature check; this is belt-and-suspenders.
|
||||||
|
if claims.app_id != app_id || !claims.allows_topic(topic) {
|
||||||
|
return Err(SubscribeDenied::Unauthorized);
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
use crate::app_secrets_repo::AppSecretsRepoError;
|
||||||
|
use crate::topic_repo::{Topic, TopicRepoError};
|
||||||
|
use chrono::Utc;
|
||||||
|
use picloud_shared::subscriber_token::{sign, TokenClaims};
|
||||||
|
|
||||||
|
struct FakeTopics(Vec<(AppId, Topic)>);
|
||||||
|
#[async_trait]
|
||||||
|
impl TopicRepo for FakeTopics {
|
||||||
|
async fn create(
|
||||||
|
&self,
|
||||||
|
_: AppId,
|
||||||
|
_: &str,
|
||||||
|
_: bool,
|
||||||
|
_: TopicAuthMode,
|
||||||
|
) -> Result<Topic, TopicRepoError> {
|
||||||
|
unimplemented!()
|
||||||
|
}
|
||||||
|
async fn list(&self, _: AppId) -> Result<Vec<Topic>, TopicRepoError> {
|
||||||
|
unimplemented!()
|
||||||
|
}
|
||||||
|
async fn get(&self, app_id: AppId, name: &str) -> Result<Option<Topic>, TopicRepoError> {
|
||||||
|
Ok(self
|
||||||
|
.0
|
||||||
|
.iter()
|
||||||
|
.find(|(a, t)| *a == app_id && t.name == name)
|
||||||
|
.map(|(_, t)| t.clone()))
|
||||||
|
}
|
||||||
|
async fn update(
|
||||||
|
&self,
|
||||||
|
_: AppId,
|
||||||
|
_: &str,
|
||||||
|
_: Option<bool>,
|
||||||
|
_: Option<TopicAuthMode>,
|
||||||
|
) -> Result<Option<Topic>, TopicRepoError> {
|
||||||
|
unimplemented!()
|
||||||
|
}
|
||||||
|
async fn delete(&self, _: AppId, _: &str) -> Result<bool, TopicRepoError> {
|
||||||
|
unimplemented!()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
struct FakeSecrets(AppId, Vec<u8>);
|
||||||
|
#[async_trait]
|
||||||
|
impl AppSecretsRepo for FakeSecrets {
|
||||||
|
async fn get_or_create_signing_key(
|
||||||
|
&self,
|
||||||
|
_: AppId,
|
||||||
|
) -> Result<Vec<u8>, AppSecretsRepoError> {
|
||||||
|
Ok(self.1.clone())
|
||||||
|
}
|
||||||
|
async fn signing_key(&self, app_id: AppId) -> Result<Option<Vec<u8>>, AppSecretsRepoError> {
|
||||||
|
Ok((app_id == self.0).then(|| self.1.clone()))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn topic(name: &str, external: bool, mode: TopicAuthMode) -> Topic {
|
||||||
|
Topic {
|
||||||
|
name: name.to_string(),
|
||||||
|
external_subscribable: external,
|
||||||
|
auth_mode: mode,
|
||||||
|
created_at: Utc::now(),
|
||||||
|
updated_at: Utc::now(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn authority(
|
||||||
|
topics: Vec<(AppId, Topic)>,
|
||||||
|
key_app: AppId,
|
||||||
|
key: Vec<u8>,
|
||||||
|
) -> RealtimeAuthorityImpl {
|
||||||
|
RealtimeAuthorityImpl::new(
|
||||||
|
Arc::new(FakeTopics(topics)),
|
||||||
|
Arc::new(FakeSecrets(key_app, key)),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn missing_topic_is_not_found() {
|
||||||
|
let app = AppId::new();
|
||||||
|
let a = authority(vec![], app, vec![0u8; 32]);
|
||||||
|
assert_eq!(
|
||||||
|
a.authorize_subscribe(app, "ghost", None).await,
|
||||||
|
Err(SubscribeDenied::NotFound)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn internal_only_topic_is_not_found() {
|
||||||
|
let app = AppId::new();
|
||||||
|
let a = authority(
|
||||||
|
vec![(app, topic("internal", false, TopicAuthMode::Public))],
|
||||||
|
app,
|
||||||
|
vec![0u8; 32],
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
a.authorize_subscribe(app, "internal", None).await,
|
||||||
|
Err(SubscribeDenied::NotFound)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn public_topic_allows_without_token() {
|
||||||
|
let app = AppId::new();
|
||||||
|
let a = authority(
|
||||||
|
vec![(app, topic("news", true, TopicAuthMode::Public))],
|
||||||
|
app,
|
||||||
|
vec![0u8; 32],
|
||||||
|
);
|
||||||
|
assert!(a.authorize_subscribe(app, "news", None).await.is_ok());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn token_topic_without_token_is_unauthorized() {
|
||||||
|
let app = AppId::new();
|
||||||
|
let a = authority(
|
||||||
|
vec![(app, topic("chat", true, TopicAuthMode::Token))],
|
||||||
|
app,
|
||||||
|
vec![7u8; 32],
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
a.authorize_subscribe(app, "chat", None).await,
|
||||||
|
Err(SubscribeDenied::Unauthorized)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn token_topic_with_valid_token_allows() {
|
||||||
|
let app = AppId::new();
|
||||||
|
let key = vec![9u8; 32];
|
||||||
|
let a = authority(
|
||||||
|
vec![(app, topic("chat", true, TopicAuthMode::Token))],
|
||||||
|
app,
|
||||||
|
key.clone(),
|
||||||
|
);
|
||||||
|
let token = sign(
|
||||||
|
&key,
|
||||||
|
&TokenClaims {
|
||||||
|
app_id: app,
|
||||||
|
topics: vec!["chat".into()],
|
||||||
|
iat: Utc::now().timestamp(),
|
||||||
|
exp: Utc::now().timestamp() + 60,
|
||||||
|
},
|
||||||
|
);
|
||||||
|
assert!(a
|
||||||
|
.authorize_subscribe(app, "chat", Some(&token))
|
||||||
|
.await
|
||||||
|
.is_ok());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn token_for_other_topic_is_unauthorized() {
|
||||||
|
let app = AppId::new();
|
||||||
|
let key = vec![9u8; 32];
|
||||||
|
let a = authority(
|
||||||
|
vec![(app, topic("chat", true, TopicAuthMode::Token))],
|
||||||
|
app,
|
||||||
|
key.clone(),
|
||||||
|
);
|
||||||
|
let token = sign(
|
||||||
|
&key,
|
||||||
|
&TokenClaims {
|
||||||
|
app_id: app,
|
||||||
|
topics: vec!["other".into()],
|
||||||
|
iat: Utc::now().timestamp(),
|
||||||
|
exp: Utc::now().timestamp() + 60,
|
||||||
|
},
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
a.authorize_subscribe(app, "chat", Some(&token)).await,
|
||||||
|
Err(SubscribeDenied::Unauthorized)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn expired_token_is_unauthorized() {
|
||||||
|
let app = AppId::new();
|
||||||
|
let key = vec![9u8; 32];
|
||||||
|
let a = authority(
|
||||||
|
vec![(app, topic("chat", true, TopicAuthMode::Token))],
|
||||||
|
app,
|
||||||
|
key.clone(),
|
||||||
|
);
|
||||||
|
let token = sign(
|
||||||
|
&key,
|
||||||
|
&TokenClaims {
|
||||||
|
app_id: app,
|
||||||
|
topics: vec!["chat".into()],
|
||||||
|
iat: Utc::now().timestamp() - 120,
|
||||||
|
exp: Utc::now().timestamp() - 60,
|
||||||
|
},
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
a.authorize_subscribe(app, "chat", Some(&token)).await,
|
||||||
|
Err(SubscribeDenied::Unauthorized)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn token_signed_by_other_app_key_is_unauthorized() {
|
||||||
|
let app_a = AppId::new();
|
||||||
|
let app_b = AppId::new();
|
||||||
|
let key_a = vec![1u8; 32];
|
||||||
|
let key_b = vec![2u8; 32];
|
||||||
|
// Authority for app B; its key is key_b.
|
||||||
|
let a = authority(
|
||||||
|
vec![(app_b, topic("chat", true, TopicAuthMode::Token))],
|
||||||
|
app_b,
|
||||||
|
key_b,
|
||||||
|
);
|
||||||
|
// Token signed by app A's key, claiming app A.
|
||||||
|
let token = sign(
|
||||||
|
&key_a,
|
||||||
|
&TokenClaims {
|
||||||
|
app_id: app_a,
|
||||||
|
topics: vec!["chat".into()],
|
||||||
|
iat: Utc::now().timestamp(),
|
||||||
|
exp: Utc::now().timestamp() + 60,
|
||||||
|
},
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
a.authorize_subscribe(app_b, "chat", Some(&token)).await,
|
||||||
|
Err(SubscribeDenied::Unauthorized)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
232
crates/manager-core/src/secrets_api.rs
Normal file
232
crates/manager-core/src/secrets_api.rs
Normal file
@@ -0,0 +1,232 @@
|
|||||||
|
//! `/api/v1/admin/apps/{id}/secrets*` — secrets admin endpoints
|
||||||
|
//! (v1.1.7).
|
||||||
|
//!
|
||||||
|
//! * `GET /apps/{id}/secrets` — list names + updated_at
|
||||||
|
//! (NEVER values).
|
||||||
|
//! * `POST /apps/{id}/secrets` — set/overwrite a secret.
|
||||||
|
//! * `DELETE /apps/{id}/secrets/{name}` — delete a secret.
|
||||||
|
//!
|
||||||
|
//! Set/delete are gated by `AppSecretsWrite` (→ `script:write`); list by
|
||||||
|
//! `AppSecretsRead` (→ `script:read`). The list surface deliberately
|
||||||
|
//! returns only names + timestamps — the dashboard never receives
|
||||||
|
//! plaintext. Values are encrypted with the process master key before
|
||||||
|
//! they touch the database (same envelope as the script `secrets::set`).
|
||||||
|
|
||||||
|
use std::sync::Arc;
|
||||||
|
|
||||||
|
use axum::extract::{Path, Query, State};
|
||||||
|
use axum::http::StatusCode;
|
||||||
|
use axum::response::{IntoResponse, Json, Response};
|
||||||
|
use axum::routing::get;
|
||||||
|
use axum::{Extension, Router};
|
||||||
|
use picloud_shared::{validate_secret_name, AppId, MasterKey, Principal, SecretsError};
|
||||||
|
use serde::Deserialize;
|
||||||
|
use serde_json::json;
|
||||||
|
|
||||||
|
use crate::app_repo::AppRepository;
|
||||||
|
use crate::authz::{require, AuthzDenied, AuthzError, AuthzRepo, Capability};
|
||||||
|
use crate::secrets_repo::{SecretsRepo, SecretsRepoError};
|
||||||
|
use crate::secrets_service::seal;
|
||||||
|
|
||||||
|
#[derive(Clone)]
|
||||||
|
pub struct SecretsState {
|
||||||
|
pub repo: Arc<dyn SecretsRepo>,
|
||||||
|
pub apps: Arc<dyn AppRepository>,
|
||||||
|
pub authz: Arc<dyn AuthzRepo>,
|
||||||
|
pub master_key: MasterKey,
|
||||||
|
pub max_value_bytes: usize,
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn secrets_router(state: SecretsState) -> Router {
|
||||||
|
Router::new()
|
||||||
|
.route("/apps/{app_id}/secrets", get(list_secrets).post(set_secret))
|
||||||
|
.route(
|
||||||
|
"/apps/{app_id}/secrets/{name}",
|
||||||
|
axum::routing::delete(delete_secret),
|
||||||
|
)
|
||||||
|
.with_state(state)
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, Deserialize)]
|
||||||
|
pub struct ListQuery {
|
||||||
|
#[serde(default)]
|
||||||
|
pub cursor: Option<String>,
|
||||||
|
#[serde(default)]
|
||||||
|
pub limit: Option<u32>,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, serde::Serialize)]
|
||||||
|
struct SecretItem {
|
||||||
|
name: String,
|
||||||
|
updated_at: chrono::DateTime<chrono::Utc>,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, serde::Serialize)]
|
||||||
|
struct ListSecretsResponse {
|
||||||
|
secrets: Vec<SecretItem>,
|
||||||
|
next_cursor: Option<String>,
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn list_secrets(
|
||||||
|
State(s): State<SecretsState>,
|
||||||
|
Extension(principal): Extension<Principal>,
|
||||||
|
Path(app_id): Path<AppId>,
|
||||||
|
Query(q): Query<ListQuery>,
|
||||||
|
) -> Result<Json<ListSecretsResponse>, SecretsApiError> {
|
||||||
|
ensure_app_exists(&*s.apps, app_id).await?;
|
||||||
|
require(
|
||||||
|
s.authz.as_ref(),
|
||||||
|
&principal,
|
||||||
|
Capability::AppSecretsRead(app_id),
|
||||||
|
)
|
||||||
|
.await?;
|
||||||
|
let page = s
|
||||||
|
.repo
|
||||||
|
.list_meta(app_id, q.cursor.as_deref(), q.limit.unwrap_or(0))
|
||||||
|
.await?;
|
||||||
|
Ok(Json(ListSecretsResponse {
|
||||||
|
secrets: page
|
||||||
|
.items
|
||||||
|
.into_iter()
|
||||||
|
.map(|m| SecretItem {
|
||||||
|
name: m.name,
|
||||||
|
updated_at: m.updated_at,
|
||||||
|
})
|
||||||
|
.collect(),
|
||||||
|
next_cursor: page.next_cursor,
|
||||||
|
}))
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, Deserialize)]
|
||||||
|
pub struct SetSecretRequest {
|
||||||
|
pub name: String,
|
||||||
|
/// Any JSON value — the dashboard sends a single-line string, but
|
||||||
|
/// maps/arrays/numbers round-trip too (matching `secrets::set`).
|
||||||
|
pub value: serde_json::Value,
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn set_secret(
|
||||||
|
State(s): State<SecretsState>,
|
||||||
|
Extension(principal): Extension<Principal>,
|
||||||
|
Path(app_id): Path<AppId>,
|
||||||
|
Json(input): Json<SetSecretRequest>,
|
||||||
|
) -> Result<StatusCode, SecretsApiError> {
|
||||||
|
ensure_app_exists(&*s.apps, app_id).await?;
|
||||||
|
require(
|
||||||
|
s.authz.as_ref(),
|
||||||
|
&principal,
|
||||||
|
Capability::AppSecretsWrite(app_id),
|
||||||
|
)
|
||||||
|
.await?;
|
||||||
|
validate_secret_name(&input.name)?;
|
||||||
|
let (ciphertext, nonce) = seal(&s.master_key, &input.value, s.max_value_bytes)?;
|
||||||
|
s.repo.set(app_id, &input.name, &ciphertext, &nonce).await?;
|
||||||
|
Ok(StatusCode::NO_CONTENT)
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn delete_secret(
|
||||||
|
State(s): State<SecretsState>,
|
||||||
|
Extension(principal): Extension<Principal>,
|
||||||
|
Path((app_id, name)): Path<(AppId, String)>,
|
||||||
|
) -> Result<StatusCode, SecretsApiError> {
|
||||||
|
ensure_app_exists(&*s.apps, app_id).await?;
|
||||||
|
require(
|
||||||
|
s.authz.as_ref(),
|
||||||
|
&principal,
|
||||||
|
Capability::AppSecretsWrite(app_id),
|
||||||
|
)
|
||||||
|
.await?;
|
||||||
|
if !s.repo.delete(app_id, &name).await? {
|
||||||
|
return Err(SecretsApiError::NotFound);
|
||||||
|
}
|
||||||
|
Ok(StatusCode::NO_CONTENT)
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn ensure_app_exists(apps: &dyn AppRepository, app_id: AppId) -> Result<(), SecretsApiError> {
|
||||||
|
apps.get_by_id(app_id)
|
||||||
|
.await
|
||||||
|
.map_err(|e| SecretsApiError::Backend(e.to_string()))?
|
||||||
|
.ok_or(SecretsApiError::AppNotFound)?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, thiserror::Error)]
|
||||||
|
pub enum SecretsApiError {
|
||||||
|
#[error("app not found")]
|
||||||
|
AppNotFound,
|
||||||
|
#[error("secret not found")]
|
||||||
|
NotFound,
|
||||||
|
#[error("invalid request: {0}")]
|
||||||
|
Invalid(String),
|
||||||
|
#[error("forbidden")]
|
||||||
|
Forbidden,
|
||||||
|
#[error("authorization repo error: {0}")]
|
||||||
|
AuthzRepo(String),
|
||||||
|
#[error("secrets backend: {0}")]
|
||||||
|
Backend(String),
|
||||||
|
}
|
||||||
|
|
||||||
|
impl From<AuthzDenied> for SecretsApiError {
|
||||||
|
fn from(d: AuthzDenied) -> Self {
|
||||||
|
match d {
|
||||||
|
AuthzDenied::Denied => Self::Forbidden,
|
||||||
|
AuthzDenied::Repo(e) => Self::AuthzRepo(e.to_string()),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl From<AuthzError> for SecretsApiError {
|
||||||
|
fn from(e: AuthzError) -> Self {
|
||||||
|
Self::AuthzRepo(e.to_string())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl From<SecretsRepoError> for SecretsApiError {
|
||||||
|
fn from(e: SecretsRepoError) -> Self {
|
||||||
|
match e {
|
||||||
|
SecretsRepoError::InvalidCursor => Self::Invalid("invalid pagination cursor".into()),
|
||||||
|
SecretsRepoError::Db(e) => Self::Backend(e.to_string()),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl From<SecretsError> for SecretsApiError {
|
||||||
|
fn from(e: SecretsError) -> Self {
|
||||||
|
match e {
|
||||||
|
SecretsError::InvalidName(m) => Self::Invalid(m),
|
||||||
|
SecretsError::TooLarge { .. } => Self::Invalid(e.to_string()),
|
||||||
|
SecretsError::Forbidden => Self::Forbidden,
|
||||||
|
other => Self::Backend(other.to_string()),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl IntoResponse for SecretsApiError {
|
||||||
|
fn into_response(self) -> Response {
|
||||||
|
let (status, body) = match &self {
|
||||||
|
Self::AppNotFound | Self::NotFound => {
|
||||||
|
(StatusCode::NOT_FOUND, json!({ "error": self.to_string() }))
|
||||||
|
}
|
||||||
|
Self::Invalid(_) => (
|
||||||
|
StatusCode::UNPROCESSABLE_ENTITY,
|
||||||
|
json!({ "error": self.to_string() }),
|
||||||
|
),
|
||||||
|
Self::Forbidden => (StatusCode::FORBIDDEN, json!({ "error": self.to_string() })),
|
||||||
|
Self::AuthzRepo(e) => {
|
||||||
|
tracing::error!(error = %e, "secrets admin authz repo error");
|
||||||
|
(
|
||||||
|
StatusCode::INTERNAL_SERVER_ERROR,
|
||||||
|
json!({ "error": "internal error" }),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
Self::Backend(e) => {
|
||||||
|
tracing::error!(error = %e, "secrets admin backend error");
|
||||||
|
(
|
||||||
|
StatusCode::INTERNAL_SERVER_ERROR,
|
||||||
|
json!({ "error": "internal error" }),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
};
|
||||||
|
(status, Json(body)).into_response()
|
||||||
|
}
|
||||||
|
}
|
||||||
246
crates/manager-core/src/secrets_repo.rs
Normal file
246
crates/manager-core/src/secrets_repo.rs
Normal file
@@ -0,0 +1,246 @@
|
|||||||
|
//! Low-level Postgres CRUD over `secrets`. Storage-only: it moves
|
||||||
|
//! opaque ciphertext + nonce blobs in and out. Encryption, JSON
|
||||||
|
//! encoding, authorization, name validation, and the value-size cap all
|
||||||
|
//! live one layer up in `SecretsServiceImpl` / `secrets_api`.
|
||||||
|
|
||||||
|
use async_trait::async_trait;
|
||||||
|
use base64::engine::general_purpose::URL_SAFE_NO_PAD;
|
||||||
|
use base64::Engine as _;
|
||||||
|
use chrono::{DateTime, Utc};
|
||||||
|
use picloud_shared::AppId;
|
||||||
|
use sqlx::PgPool;
|
||||||
|
|
||||||
|
#[derive(Debug, thiserror::Error)]
|
||||||
|
pub enum SecretsRepoError {
|
||||||
|
#[error("database error: {0}")]
|
||||||
|
Db(#[from] sqlx::Error),
|
||||||
|
|
||||||
|
#[error("invalid pagination cursor")]
|
||||||
|
InvalidCursor,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// An encrypted secret as it lives on disk: ciphertext (auth tag
|
||||||
|
/// appended) plus the nonce it was sealed with.
|
||||||
|
#[derive(Debug, Clone)]
|
||||||
|
pub struct StoredSecret {
|
||||||
|
pub encrypted_value: Vec<u8>,
|
||||||
|
pub nonce: Vec<u8>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Admin-surface metadata for one secret. Values are never returned —
|
||||||
|
/// only the name and the last-modified timestamp.
|
||||||
|
#[derive(Debug, Clone)]
|
||||||
|
pub struct SecretMeta {
|
||||||
|
pub name: String,
|
||||||
|
pub updated_at: DateTime<Utc>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One page of names (SDK `list`).
|
||||||
|
#[derive(Debug, Clone)]
|
||||||
|
pub struct SecretsNamePage {
|
||||||
|
pub names: Vec<String>,
|
||||||
|
pub next_cursor: Option<String>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One page of name + updated_at (admin `GET`).
|
||||||
|
#[derive(Debug, Clone)]
|
||||||
|
pub struct SecretsMetaPage {
|
||||||
|
pub items: Vec<SecretMeta>,
|
||||||
|
pub next_cursor: Option<String>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Repo surface. Exposed as a trait so the service unit tests can
|
||||||
|
/// substitute an in-memory backing without Postgres.
|
||||||
|
#[async_trait]
|
||||||
|
pub trait SecretsRepo: Send + Sync {
|
||||||
|
async fn get(
|
||||||
|
&self,
|
||||||
|
app_id: AppId,
|
||||||
|
name: &str,
|
||||||
|
) -> Result<Option<StoredSecret>, SecretsRepoError>;
|
||||||
|
|
||||||
|
/// Upsert (overwrite if present).
|
||||||
|
async fn set(
|
||||||
|
&self,
|
||||||
|
app_id: AppId,
|
||||||
|
name: &str,
|
||||||
|
encrypted_value: &[u8],
|
||||||
|
nonce: &[u8],
|
||||||
|
) -> Result<(), SecretsRepoError>;
|
||||||
|
|
||||||
|
/// Delete; returns whether a row was present.
|
||||||
|
async fn delete(&self, app_id: AppId, name: &str) -> Result<bool, SecretsRepoError>;
|
||||||
|
|
||||||
|
/// Names only — the SDK `list` surface.
|
||||||
|
async fn list_names(
|
||||||
|
&self,
|
||||||
|
app_id: AppId,
|
||||||
|
cursor: Option<&str>,
|
||||||
|
limit: u32,
|
||||||
|
) -> Result<SecretsNamePage, SecretsRepoError>;
|
||||||
|
|
||||||
|
/// Name + updated_at — the admin `GET` surface.
|
||||||
|
async fn list_meta(
|
||||||
|
&self,
|
||||||
|
app_id: AppId,
|
||||||
|
cursor: Option<&str>,
|
||||||
|
limit: u32,
|
||||||
|
) -> Result<SecretsMetaPage, SecretsRepoError>;
|
||||||
|
}
|
||||||
|
|
||||||
|
pub struct PostgresSecretsRepo {
|
||||||
|
pool: PgPool,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl PostgresSecretsRepo {
|
||||||
|
#[must_use]
|
||||||
|
pub fn new(pool: PgPool) -> Self {
|
||||||
|
Self { pool }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const SECRETS_LIST_MAX_LIMIT: u32 = 1_000;
|
||||||
|
const SECRETS_LIST_DEFAULT_LIMIT: u32 = 100;
|
||||||
|
|
||||||
|
fn clamp_limit(limit: u32) -> u32 {
|
||||||
|
if limit == 0 {
|
||||||
|
SECRETS_LIST_DEFAULT_LIMIT
|
||||||
|
} else {
|
||||||
|
limit.min(SECRETS_LIST_MAX_LIMIT)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Opaque keyset cursor: base64url of the last `name` returned.
|
||||||
|
pub(crate) fn encode_cursor(last_name: &str) -> String {
|
||||||
|
URL_SAFE_NO_PAD.encode(last_name.as_bytes())
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn decode_cursor(cursor: &str) -> Result<String, SecretsRepoError> {
|
||||||
|
let bytes = URL_SAFE_NO_PAD
|
||||||
|
.decode(cursor)
|
||||||
|
.map_err(|_| SecretsRepoError::InvalidCursor)?;
|
||||||
|
String::from_utf8(bytes).map_err(|_| SecretsRepoError::InvalidCursor)
|
||||||
|
}
|
||||||
|
|
||||||
|
#[async_trait]
|
||||||
|
impl SecretsRepo for PostgresSecretsRepo {
|
||||||
|
async fn get(
|
||||||
|
&self,
|
||||||
|
app_id: AppId,
|
||||||
|
name: &str,
|
||||||
|
) -> Result<Option<StoredSecret>, SecretsRepoError> {
|
||||||
|
let row: Option<(Vec<u8>, Vec<u8>)> = sqlx::query_as(
|
||||||
|
"SELECT encrypted_value, nonce FROM secrets WHERE app_id = $1 AND name = $2",
|
||||||
|
)
|
||||||
|
.bind(app_id.into_inner())
|
||||||
|
.bind(name)
|
||||||
|
.fetch_optional(&self.pool)
|
||||||
|
.await?;
|
||||||
|
Ok(row.map(|(encrypted_value, nonce)| StoredSecret {
|
||||||
|
encrypted_value,
|
||||||
|
nonce,
|
||||||
|
}))
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn set(
|
||||||
|
&self,
|
||||||
|
app_id: AppId,
|
||||||
|
name: &str,
|
||||||
|
encrypted_value: &[u8],
|
||||||
|
nonce: &[u8],
|
||||||
|
) -> Result<(), SecretsRepoError> {
|
||||||
|
sqlx::query(
|
||||||
|
"INSERT INTO secrets (app_id, name, encrypted_value, nonce) \
|
||||||
|
VALUES ($1, $2, $3, $4) \
|
||||||
|
ON CONFLICT (app_id, name) DO UPDATE \
|
||||||
|
SET encrypted_value = EXCLUDED.encrypted_value, \
|
||||||
|
nonce = EXCLUDED.nonce, \
|
||||||
|
updated_at = NOW()",
|
||||||
|
)
|
||||||
|
.bind(app_id.into_inner())
|
||||||
|
.bind(name)
|
||||||
|
.bind(encrypted_value)
|
||||||
|
.bind(nonce)
|
||||||
|
.execute(&self.pool)
|
||||||
|
.await?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn delete(&self, app_id: AppId, name: &str) -> Result<bool, SecretsRepoError> {
|
||||||
|
let res = sqlx::query("DELETE FROM secrets WHERE app_id = $1 AND name = $2")
|
||||||
|
.bind(app_id.into_inner())
|
||||||
|
.bind(name)
|
||||||
|
.execute(&self.pool)
|
||||||
|
.await?;
|
||||||
|
Ok(res.rows_affected() > 0)
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn list_names(
|
||||||
|
&self,
|
||||||
|
app_id: AppId,
|
||||||
|
cursor: Option<&str>,
|
||||||
|
limit: u32,
|
||||||
|
) -> Result<SecretsNamePage, SecretsRepoError> {
|
||||||
|
let limit = clamp_limit(limit);
|
||||||
|
let last_name = match cursor {
|
||||||
|
Some(c) => Some(decode_cursor(c)?),
|
||||||
|
None => None,
|
||||||
|
};
|
||||||
|
let take = i64::from(limit) + 1;
|
||||||
|
let rows: Vec<(String,)> = sqlx::query_as(
|
||||||
|
"SELECT name FROM secrets \
|
||||||
|
WHERE app_id = $1 AND ($2::text IS NULL OR name > $2) \
|
||||||
|
ORDER BY name ASC LIMIT $3",
|
||||||
|
)
|
||||||
|
.bind(app_id.into_inner())
|
||||||
|
.bind(last_name.as_deref())
|
||||||
|
.bind(take)
|
||||||
|
.fetch_all(&self.pool)
|
||||||
|
.await?;
|
||||||
|
|
||||||
|
let mut names: Vec<String> = rows.into_iter().map(|(n,)| n).collect();
|
||||||
|
let next_cursor = if names.len() > limit as usize {
|
||||||
|
names.truncate(limit as usize);
|
||||||
|
names.last().map(|n| encode_cursor(n))
|
||||||
|
} else {
|
||||||
|
None
|
||||||
|
};
|
||||||
|
Ok(SecretsNamePage { names, next_cursor })
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn list_meta(
|
||||||
|
&self,
|
||||||
|
app_id: AppId,
|
||||||
|
cursor: Option<&str>,
|
||||||
|
limit: u32,
|
||||||
|
) -> Result<SecretsMetaPage, SecretsRepoError> {
|
||||||
|
let limit = clamp_limit(limit);
|
||||||
|
let last_name = match cursor {
|
||||||
|
Some(c) => Some(decode_cursor(c)?),
|
||||||
|
None => None,
|
||||||
|
};
|
||||||
|
let take = i64::from(limit) + 1;
|
||||||
|
let rows: Vec<(String, DateTime<Utc>)> = sqlx::query_as(
|
||||||
|
"SELECT name, updated_at FROM secrets \
|
||||||
|
WHERE app_id = $1 AND ($2::text IS NULL OR name > $2) \
|
||||||
|
ORDER BY name ASC LIMIT $3",
|
||||||
|
)
|
||||||
|
.bind(app_id.into_inner())
|
||||||
|
.bind(last_name.as_deref())
|
||||||
|
.bind(take)
|
||||||
|
.fetch_all(&self.pool)
|
||||||
|
.await?;
|
||||||
|
|
||||||
|
let mut items: Vec<SecretMeta> = rows
|
||||||
|
.into_iter()
|
||||||
|
.map(|(name, updated_at)| SecretMeta { name, updated_at })
|
||||||
|
.collect();
|
||||||
|
let next_cursor = if items.len() > limit as usize {
|
||||||
|
items.truncate(limit as usize);
|
||||||
|
items.last().map(|m| encode_cursor(&m.name))
|
||||||
|
} else {
|
||||||
|
None
|
||||||
|
};
|
||||||
|
Ok(SecretsMetaPage { items, next_cursor })
|
||||||
|
}
|
||||||
|
}
|
||||||
574
crates/manager-core/src/secrets_service.rs
Normal file
574
crates/manager-core/src/secrets_service.rs
Normal file
@@ -0,0 +1,574 @@
|
|||||||
|
//! `SecretsServiceImpl` — wires the `SecretsRepo` underneath the
|
||||||
|
//! `picloud_shared::SecretsService` trait that scripts see via the Rhai
|
||||||
|
//! bridge.
|
||||||
|
//!
|
||||||
|
//! Layers added here (vs the raw repo):
|
||||||
|
//!
|
||||||
|
//! 1. Name validation (non-empty, ≤255 bytes) at the SDK boundary.
|
||||||
|
//! 2. **Script-as-gate authz**: when `cx.principal.is_some()` we run
|
||||||
|
//! `authz::require(...)`; when it's `None` (public unauthenticated
|
||||||
|
//! HTTP) we skip the check. Cross-app isolation is unaffected — every
|
||||||
|
//! query is keyed by `cx.app_id`, never an argument.
|
||||||
|
//! 3. **JSON ⇄ ciphertext**: `set` serializes the value to JSON bytes,
|
||||||
|
//! enforces the per-secret size cap, and AES-256-GCM-seals it; `get`
|
||||||
|
//! decrypts and deserializes back to the same JSON shape (a String
|
||||||
|
//! round-trips to a String, not a JSON-quoted `"\"…\""`).
|
||||||
|
//!
|
||||||
|
//! Deliberately **no `ServiceEvent` emission** — secret writes do not
|
||||||
|
//! fire triggers (footgun avoidance; see `docs/sdk-shape.md` + the
|
||||||
|
//! v1.1.7 brief §2).
|
||||||
|
|
||||||
|
use std::sync::Arc;
|
||||||
|
|
||||||
|
use async_trait::async_trait;
|
||||||
|
use picloud_shared::{
|
||||||
|
crypto, validate_secret_name, MasterKey, SdkCallCx, SecretsError, SecretsListPage,
|
||||||
|
SecretsService,
|
||||||
|
};
|
||||||
|
|
||||||
|
use crate::authz::{self, AuthzRepo, Capability};
|
||||||
|
use crate::secrets_repo::{SecretsRepo, SecretsRepoError, StoredSecret};
|
||||||
|
|
||||||
|
/// Default per-secret plaintext cap (64 KB). Override with
|
||||||
|
/// `PICLOUD_SECRET_MAX_VALUE_BYTES`.
|
||||||
|
pub const DEFAULT_SECRET_MAX_VALUE_BYTES: usize = 64 * 1024;
|
||||||
|
|
||||||
|
/// Process config for the secrets service.
|
||||||
|
#[derive(Debug, Clone, Copy)]
|
||||||
|
pub struct SecretsConfig {
|
||||||
|
/// Maximum size of the JSON-encoded plaintext, in bytes.
|
||||||
|
pub max_value_bytes: usize,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl SecretsConfig {
|
||||||
|
#[must_use]
|
||||||
|
pub const fn conservative() -> Self {
|
||||||
|
Self {
|
||||||
|
max_value_bytes: DEFAULT_SECRET_MAX_VALUE_BYTES,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Read `PICLOUD_SECRET_MAX_VALUE_BYTES`; invalid values are ignored
|
||||||
|
/// with a warning (keeps the conservative default).
|
||||||
|
#[must_use]
|
||||||
|
pub fn from_env() -> Self {
|
||||||
|
let mut c = Self::conservative();
|
||||||
|
if let Ok(v) = std::env::var("PICLOUD_SECRET_MAX_VALUE_BYTES") {
|
||||||
|
match v.trim().parse::<usize>() {
|
||||||
|
Ok(n) if n > 0 => c.max_value_bytes = n,
|
||||||
|
_ => tracing::warn!(
|
||||||
|
value = %v,
|
||||||
|
"ignoring invalid PICLOUD_SECRET_MAX_VALUE_BYTES (want a positive integer)"
|
||||||
|
),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
c
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Default for SecretsConfig {
|
||||||
|
fn default() -> Self {
|
||||||
|
Self::conservative()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Serialize + size-check + encrypt a value into `(ciphertext, nonce)`.
|
||||||
|
///
|
||||||
|
/// # Errors
|
||||||
|
///
|
||||||
|
/// [`SecretsError::TooLarge`] when the encoded plaintext exceeds
|
||||||
|
/// `max_value_bytes`; [`SecretsError::Backend`] on a serialization
|
||||||
|
/// failure (should not happen for a `serde_json::Value`).
|
||||||
|
pub fn seal(
|
||||||
|
master_key: &MasterKey,
|
||||||
|
value: &serde_json::Value,
|
||||||
|
max_value_bytes: usize,
|
||||||
|
) -> Result<(Vec<u8>, [u8; crypto::NONCE_LEN]), SecretsError> {
|
||||||
|
let plaintext = serde_json::to_vec(value)
|
||||||
|
.map_err(|e| SecretsError::Backend(format!("encode secret value: {e}")))?;
|
||||||
|
if plaintext.len() > max_value_bytes {
|
||||||
|
return Err(SecretsError::TooLarge {
|
||||||
|
limit: max_value_bytes,
|
||||||
|
actual: plaintext.len(),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
let enc = crypto::encrypt(&plaintext, master_key.as_bytes());
|
||||||
|
Ok((enc.ciphertext, enc.nonce))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Decrypt + deserialize a stored secret back to its JSON value.
|
||||||
|
///
|
||||||
|
/// # Errors
|
||||||
|
///
|
||||||
|
/// [`SecretsError::Corrupted`] when decryption or JSON decoding fails.
|
||||||
|
pub fn open(
|
||||||
|
master_key: &MasterKey,
|
||||||
|
stored: &StoredSecret,
|
||||||
|
) -> Result<serde_json::Value, SecretsError> {
|
||||||
|
let plaintext = crypto::decrypt(
|
||||||
|
&stored.encrypted_value,
|
||||||
|
&stored.nonce,
|
||||||
|
master_key.as_bytes(),
|
||||||
|
)
|
||||||
|
.map_err(|_| SecretsError::Corrupted)?;
|
||||||
|
serde_json::from_slice(&plaintext).map_err(|_| SecretsError::Corrupted)
|
||||||
|
}
|
||||||
|
|
||||||
|
pub struct SecretsServiceImpl {
|
||||||
|
repo: Arc<dyn SecretsRepo>,
|
||||||
|
authz: Arc<dyn AuthzRepo>,
|
||||||
|
master_key: MasterKey,
|
||||||
|
max_value_bytes: usize,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl SecretsServiceImpl {
|
||||||
|
#[must_use]
|
||||||
|
pub fn new(
|
||||||
|
repo: Arc<dyn SecretsRepo>,
|
||||||
|
authz: Arc<dyn AuthzRepo>,
|
||||||
|
master_key: MasterKey,
|
||||||
|
config: SecretsConfig,
|
||||||
|
) -> Self {
|
||||||
|
Self {
|
||||||
|
repo,
|
||||||
|
authz,
|
||||||
|
master_key,
|
||||||
|
max_value_bytes: config.max_value_bytes,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn check_read(&self, cx: &SdkCallCx) -> Result<(), SecretsError> {
|
||||||
|
if let Some(ref principal) = cx.principal {
|
||||||
|
authz::require(
|
||||||
|
&*self.authz,
|
||||||
|
principal,
|
||||||
|
Capability::AppSecretsRead(cx.app_id),
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
.map_err(|_| SecretsError::Forbidden)?;
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn check_write(&self, cx: &SdkCallCx) -> Result<(), SecretsError> {
|
||||||
|
if let Some(ref principal) = cx.principal {
|
||||||
|
authz::require(
|
||||||
|
&*self.authz,
|
||||||
|
principal,
|
||||||
|
Capability::AppSecretsWrite(cx.app_id),
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
.map_err(|_| SecretsError::Forbidden)?;
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl From<SecretsRepoError> for SecretsError {
|
||||||
|
fn from(e: SecretsRepoError) -> Self {
|
||||||
|
Self::Backend(e.to_string())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[async_trait]
|
||||||
|
impl SecretsService for SecretsServiceImpl {
|
||||||
|
async fn get(
|
||||||
|
&self,
|
||||||
|
cx: &SdkCallCx,
|
||||||
|
name: &str,
|
||||||
|
) -> Result<Option<serde_json::Value>, SecretsError> {
|
||||||
|
validate_secret_name(name)?;
|
||||||
|
self.check_read(cx).await?;
|
||||||
|
let Some(stored) = self.repo.get(cx.app_id, name).await? else {
|
||||||
|
return Ok(None);
|
||||||
|
};
|
||||||
|
match open(&self.master_key, &stored) {
|
||||||
|
Ok(value) => Ok(Some(value)),
|
||||||
|
Err(e) => {
|
||||||
|
// A decrypt failure is operationally significant — surface
|
||||||
|
// the affected (app_id, name) so an operator can find the
|
||||||
|
// bad row, but never log the ciphertext or key material.
|
||||||
|
tracing::error!(
|
||||||
|
app_id = %cx.app_id,
|
||||||
|
secret = %name,
|
||||||
|
"secret could not be decrypted (corrupted row or master-key mismatch)"
|
||||||
|
);
|
||||||
|
Err(e)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn set(
|
||||||
|
&self,
|
||||||
|
cx: &SdkCallCx,
|
||||||
|
name: &str,
|
||||||
|
value: serde_json::Value,
|
||||||
|
) -> Result<(), SecretsError> {
|
||||||
|
validate_secret_name(name)?;
|
||||||
|
self.check_write(cx).await?;
|
||||||
|
let (ciphertext, nonce) = seal(&self.master_key, &value, self.max_value_bytes)?;
|
||||||
|
self.repo.set(cx.app_id, name, &ciphertext, &nonce).await?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn delete(&self, cx: &SdkCallCx, name: &str) -> Result<bool, SecretsError> {
|
||||||
|
validate_secret_name(name)?;
|
||||||
|
self.check_write(cx).await?;
|
||||||
|
Ok(self.repo.delete(cx.app_id, name).await?)
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn list(
|
||||||
|
&self,
|
||||||
|
cx: &SdkCallCx,
|
||||||
|
cursor: Option<&str>,
|
||||||
|
limit: u32,
|
||||||
|
) -> Result<SecretsListPage, SecretsError> {
|
||||||
|
self.check_read(cx).await?;
|
||||||
|
let page = self.repo.list_names(cx.app_id, cursor, limit).await?;
|
||||||
|
Ok(SecretsListPage {
|
||||||
|
names: page.names,
|
||||||
|
next_cursor: page.next_cursor,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ----------------------------------------------------------------------------
|
||||||
|
// Tests — in-memory SecretsRepo so unit tests don't need Postgres.
|
||||||
|
// ----------------------------------------------------------------------------
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
use crate::authz::{AuthzError, AuthzRepo};
|
||||||
|
use crate::secrets_repo::{SecretsMetaPage, SecretsNamePage};
|
||||||
|
use async_trait::async_trait;
|
||||||
|
use picloud_shared::{
|
||||||
|
AdminUserId, AppId, AppRole, ExecutionId, InstanceRole, Principal, RequestId, ScriptId,
|
||||||
|
UserId,
|
||||||
|
};
|
||||||
|
use std::collections::BTreeMap;
|
||||||
|
use tokio::sync::Mutex;
|
||||||
|
|
||||||
|
#[derive(Default)]
|
||||||
|
struct InMemorySecretsRepo {
|
||||||
|
data: Mutex<BTreeMap<(AppId, String), StoredSecret>>,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[async_trait]
|
||||||
|
impl SecretsRepo for InMemorySecretsRepo {
|
||||||
|
async fn get(
|
||||||
|
&self,
|
||||||
|
app_id: AppId,
|
||||||
|
name: &str,
|
||||||
|
) -> Result<Option<StoredSecret>, SecretsRepoError> {
|
||||||
|
Ok(self
|
||||||
|
.data
|
||||||
|
.lock()
|
||||||
|
.await
|
||||||
|
.get(&(app_id, name.to_string()))
|
||||||
|
.cloned())
|
||||||
|
}
|
||||||
|
async fn set(
|
||||||
|
&self,
|
||||||
|
app_id: AppId,
|
||||||
|
name: &str,
|
||||||
|
encrypted_value: &[u8],
|
||||||
|
nonce: &[u8],
|
||||||
|
) -> Result<(), SecretsRepoError> {
|
||||||
|
self.data.lock().await.insert(
|
||||||
|
(app_id, name.to_string()),
|
||||||
|
StoredSecret {
|
||||||
|
encrypted_value: encrypted_value.to_vec(),
|
||||||
|
nonce: nonce.to_vec(),
|
||||||
|
},
|
||||||
|
);
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
async fn delete(&self, app_id: AppId, name: &str) -> Result<bool, SecretsRepoError> {
|
||||||
|
Ok(self
|
||||||
|
.data
|
||||||
|
.lock()
|
||||||
|
.await
|
||||||
|
.remove(&(app_id, name.to_string()))
|
||||||
|
.is_some())
|
||||||
|
}
|
||||||
|
async fn list_names(
|
||||||
|
&self,
|
||||||
|
app_id: AppId,
|
||||||
|
cursor: Option<&str>,
|
||||||
|
limit: u32,
|
||||||
|
) -> Result<SecretsNamePage, SecretsRepoError> {
|
||||||
|
let data = self.data.lock().await;
|
||||||
|
let last = cursor.map(std::string::ToString::to_string);
|
||||||
|
let mut names: Vec<String> = data
|
||||||
|
.iter()
|
||||||
|
.filter(|((a, _), _)| *a == app_id)
|
||||||
|
.map(|((_, n), _)| n.clone())
|
||||||
|
.filter(|n| last.as_ref().is_none_or(|l| n > l))
|
||||||
|
.collect();
|
||||||
|
names.sort();
|
||||||
|
let take = (limit as usize).max(1);
|
||||||
|
let next_cursor = if names.len() > take {
|
||||||
|
names.truncate(take);
|
||||||
|
names.last().cloned()
|
||||||
|
} else {
|
||||||
|
None
|
||||||
|
};
|
||||||
|
Ok(SecretsNamePage { names, next_cursor })
|
||||||
|
}
|
||||||
|
async fn list_meta(
|
||||||
|
&self,
|
||||||
|
_app_id: AppId,
|
||||||
|
_cursor: Option<&str>,
|
||||||
|
_limit: u32,
|
||||||
|
) -> Result<SecretsMetaPage, SecretsRepoError> {
|
||||||
|
unimplemented!("admin-only; not exercised in service tests")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Default)]
|
||||||
|
struct DenyingAuthzRepo;
|
||||||
|
#[async_trait]
|
||||||
|
impl AuthzRepo for DenyingAuthzRepo {
|
||||||
|
async fn membership(
|
||||||
|
&self,
|
||||||
|
_user_id: UserId,
|
||||||
|
_app_id: AppId,
|
||||||
|
) -> Result<Option<AppRole>, AuthzError> {
|
||||||
|
Ok(None)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn key() -> MasterKey {
|
||||||
|
MasterKey::from_bytes([0x5au8; 32])
|
||||||
|
}
|
||||||
|
|
||||||
|
fn svc() -> SecretsServiceImpl {
|
||||||
|
SecretsServiceImpl::new(
|
||||||
|
Arc::new(InMemorySecretsRepo::default()),
|
||||||
|
Arc::new(DenyingAuthzRepo),
|
||||||
|
key(),
|
||||||
|
SecretsConfig::conservative(),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn cx_with(app_id: AppId, principal: Option<Principal>) -> SdkCallCx {
|
||||||
|
SdkCallCx {
|
||||||
|
app_id,
|
||||||
|
script_id: ScriptId::new(),
|
||||||
|
principal,
|
||||||
|
execution_id: ExecutionId::new(),
|
||||||
|
request_id: RequestId::new(),
|
||||||
|
trigger_depth: 0,
|
||||||
|
root_execution_id: ExecutionId::new(),
|
||||||
|
is_dead_letter_handler: false,
|
||||||
|
event: None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn anon_cx(app_id: AppId) -> SdkCallCx {
|
||||||
|
cx_with(app_id, None)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn member_no_role_cx(app_id: AppId) -> SdkCallCx {
|
||||||
|
cx_with(
|
||||||
|
app_id,
|
||||||
|
Some(Principal {
|
||||||
|
user_id: AdminUserId::new(),
|
||||||
|
instance_role: InstanceRole::Member,
|
||||||
|
scopes: None,
|
||||||
|
app_binding: None,
|
||||||
|
}),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn owner_cx(app_id: AppId) -> SdkCallCx {
|
||||||
|
cx_with(
|
||||||
|
app_id,
|
||||||
|
Some(Principal {
|
||||||
|
user_id: AdminUserId::new(),
|
||||||
|
instance_role: InstanceRole::Owner,
|
||||||
|
scopes: None,
|
||||||
|
app_binding: None,
|
||||||
|
}),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn set_get_delete_round_trip() {
|
||||||
|
let s = svc();
|
||||||
|
let cx = anon_cx(AppId::new());
|
||||||
|
s.set(&cx, "stripe_key", serde_json::json!("sk_live_xxx"))
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(
|
||||||
|
s.get(&cx, "stripe_key").await.unwrap(),
|
||||||
|
Some(serde_json::json!("sk_live_xxx"))
|
||||||
|
);
|
||||||
|
assert!(s.delete(&cx, "stripe_key").await.unwrap());
|
||||||
|
assert_eq!(s.get(&cx, "stripe_key").await.unwrap(), None);
|
||||||
|
// Idempotent delete.
|
||||||
|
assert!(!s.delete(&cx, "stripe_key").await.unwrap());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn get_missing_returns_none() {
|
||||||
|
let s = svc();
|
||||||
|
let cx = anon_cx(AppId::new());
|
||||||
|
assert_eq!(s.get(&cx, "nope").await.unwrap(), None);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn empty_name_rejected() {
|
||||||
|
let s = svc();
|
||||||
|
let cx = anon_cx(AppId::new());
|
||||||
|
let err = s.set(&cx, "", serde_json::json!("x")).await.unwrap_err();
|
||||||
|
assert!(matches!(err, SecretsError::InvalidName(_)));
|
||||||
|
let err = s.get(&cx, "").await.unwrap_err();
|
||||||
|
assert!(matches!(err, SecretsError::InvalidName(_)));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn name_length_capped() {
|
||||||
|
let s = svc();
|
||||||
|
let cx = anon_cx(AppId::new());
|
||||||
|
let long = "a".repeat(256);
|
||||||
|
let err = s.set(&cx, &long, serde_json::json!(1)).await.unwrap_err();
|
||||||
|
assert!(matches!(err, SecretsError::InvalidName(_)));
|
||||||
|
// Exactly 255 is allowed.
|
||||||
|
let ok = "b".repeat(255);
|
||||||
|
s.set(&cx, &ok, serde_json::json!(1)).await.unwrap();
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn value_over_cap_rejected() {
|
||||||
|
let s = SecretsServiceImpl::new(
|
||||||
|
Arc::new(InMemorySecretsRepo::default()),
|
||||||
|
Arc::new(DenyingAuthzRepo),
|
||||||
|
key(),
|
||||||
|
SecretsConfig {
|
||||||
|
max_value_bytes: 16,
|
||||||
|
},
|
||||||
|
);
|
||||||
|
let cx = anon_cx(AppId::new());
|
||||||
|
let big = serde_json::json!("x".repeat(64));
|
||||||
|
let err = s.set(&cx, "k", big).await.unwrap_err();
|
||||||
|
assert!(matches!(err, SecretsError::TooLarge { limit: 16, .. }));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn cross_app_isolation() {
|
||||||
|
let s = svc();
|
||||||
|
let a = AppId::new();
|
||||||
|
let b = AppId::new();
|
||||||
|
s.set(&anon_cx(a), "shared", serde_json::json!("from-a"))
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
s.set(&anon_cx(b), "shared", serde_json::json!("from-b"))
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(
|
||||||
|
s.get(&anon_cx(a), "shared").await.unwrap(),
|
||||||
|
Some(serde_json::json!("from-a"))
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
s.get(&anon_cx(b), "shared").await.unwrap(),
|
||||||
|
Some(serde_json::json!("from-b"))
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn anonymous_skips_authz() {
|
||||||
|
let s = svc();
|
||||||
|
// DenyingAuthzRepo would deny an authed principal; anon skips it.
|
||||||
|
s.set(&anon_cx(AppId::new()), "k", serde_json::json!(1))
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn authed_member_without_role_forbidden() {
|
||||||
|
let s = svc();
|
||||||
|
let err = s
|
||||||
|
.set(&member_no_role_cx(AppId::new()), "k", serde_json::json!(1))
|
||||||
|
.await
|
||||||
|
.unwrap_err();
|
||||||
|
assert!(matches!(err, SecretsError::Forbidden));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn owner_can_write() {
|
||||||
|
let s = svc();
|
||||||
|
s.set(&owner_cx(AppId::new()), "k", serde_json::json!(1))
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Type round-trip: a String comes back a String, a Map a Map, an
|
||||||
|
/// Array an Array — the JSON encoding is transparent.
|
||||||
|
#[tokio::test]
|
||||||
|
async fn type_round_trip_preserves_shape() {
|
||||||
|
let s = svc();
|
||||||
|
let cx = anon_cx(AppId::new());
|
||||||
|
|
||||||
|
s.set(&cx, "str", serde_json::json!("sk_live_xxx"))
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(
|
||||||
|
s.get(&cx, "str").await.unwrap(),
|
||||||
|
Some(serde_json::json!("sk_live_xxx"))
|
||||||
|
);
|
||||||
|
|
||||||
|
let map = serde_json::json!({ "client_id": "abc", "client_secret": "xyz" });
|
||||||
|
s.set(&cx, "oauth", map.clone()).await.unwrap();
|
||||||
|
assert_eq!(s.get(&cx, "oauth").await.unwrap(), Some(map));
|
||||||
|
|
||||||
|
let arr = serde_json::json!([1, 2, 3]);
|
||||||
|
s.set(&cx, "arr", arr.clone()).await.unwrap();
|
||||||
|
assert_eq!(s.get(&cx, "arr").await.unwrap(), Some(arr));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn corrupted_ciphertext_surfaces_error() {
|
||||||
|
let repo = Arc::new(InMemorySecretsRepo::default());
|
||||||
|
let s = SecretsServiceImpl::new(
|
||||||
|
repo.clone(),
|
||||||
|
Arc::new(DenyingAuthzRepo),
|
||||||
|
key(),
|
||||||
|
SecretsConfig::conservative(),
|
||||||
|
);
|
||||||
|
let app = AppId::new();
|
||||||
|
s.set(&anon_cx(app), "k", serde_json::json!("v"))
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
// Corrupt the stored ciphertext directly.
|
||||||
|
repo.data
|
||||||
|
.lock()
|
||||||
|
.await
|
||||||
|
.get_mut(&(app, "k".to_string()))
|
||||||
|
.unwrap()
|
||||||
|
.encrypted_value[0] ^= 0xff;
|
||||||
|
let err = s.get(&anon_cx(app), "k").await.unwrap_err();
|
||||||
|
assert!(matches!(err, SecretsError::Corrupted));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn list_returns_names_paginated() {
|
||||||
|
let s = svc();
|
||||||
|
let cx = anon_cx(AppId::new());
|
||||||
|
for i in 0..5 {
|
||||||
|
s.set(&cx, &format!("k{i:02}"), serde_json::json!(i))
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
}
|
||||||
|
let p1 = s.list(&cx, None, 2).await.unwrap();
|
||||||
|
assert_eq!(p1.names, vec!["k00".to_string(), "k01".to_string()]);
|
||||||
|
assert!(p1.next_cursor.is_some());
|
||||||
|
let p2 = s.list(&cx, p1.next_cursor.as_deref(), 10).await.unwrap();
|
||||||
|
assert_eq!(
|
||||||
|
p2.names,
|
||||||
|
vec!["k02".to_string(), "k03".to_string(), "k04".to_string()]
|
||||||
|
);
|
||||||
|
assert!(p2.next_cursor.is_none());
|
||||||
|
}
|
||||||
|
}
|
||||||
212
crates/manager-core/src/topic_repo.rs
Normal file
212
crates/manager-core/src/topic_repo.rs
Normal file
@@ -0,0 +1,212 @@
|
|||||||
|
//! `TopicRepo` — CRUD for the `topics` table (v1.1.6).
|
||||||
|
//!
|
||||||
|
//! This table holds ONLY topics that have been explicitly externalized
|
||||||
|
//! for SSE subscription (design notes §5). Internal-only pub/sub topics
|
||||||
|
//! stay implicit — they never get a row here, and the publish path never
|
||||||
|
//! consults this table. The two readers are the topic admin endpoints
|
||||||
|
//! ([`crate::topics_api`]) and the SSE subscribe authorization
|
||||||
|
//! ([`crate::realtime_authority`]).
|
||||||
|
|
||||||
|
use async_trait::async_trait;
|
||||||
|
use chrono::{DateTime, Utc};
|
||||||
|
use picloud_shared::AppId;
|
||||||
|
use serde::{Deserialize, Serialize};
|
||||||
|
use sqlx::PgPool;
|
||||||
|
|
||||||
|
/// External-subscriber auth gate for a topic. `'public'` + `'token'` in
|
||||||
|
/// v1.1.6; `'session'` (v1.1.8) and `'script'` (v1.2) extend the DB
|
||||||
|
/// CHECK constraint and this enum later.
|
||||||
|
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
#[serde(rename_all = "lowercase")]
|
||||||
|
pub enum TopicAuthMode {
|
||||||
|
Public,
|
||||||
|
Token,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl TopicAuthMode {
|
||||||
|
#[must_use]
|
||||||
|
pub const fn as_str(self) -> &'static str {
|
||||||
|
match self {
|
||||||
|
Self::Public => "public",
|
||||||
|
Self::Token => "token",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn from_db(s: &str) -> Result<Self, TopicRepoError> {
|
||||||
|
match s {
|
||||||
|
"public" => Ok(Self::Public),
|
||||||
|
"token" => Ok(Self::Token),
|
||||||
|
other => Err(TopicRepoError::Backend(format!(
|
||||||
|
"unknown auth_mode in DB: {other}"
|
||||||
|
))),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A registered, externally-subscribable topic row.
|
||||||
|
#[derive(Debug, Clone, Serialize)]
|
||||||
|
pub struct Topic {
|
||||||
|
pub name: String,
|
||||||
|
pub external_subscribable: bool,
|
||||||
|
pub auth_mode: TopicAuthMode,
|
||||||
|
pub created_at: DateTime<Utc>,
|
||||||
|
pub updated_at: DateTime<Utc>,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, thiserror::Error)]
|
||||||
|
pub enum TopicRepoError {
|
||||||
|
#[error("a topic named {0:?} already exists in this app")]
|
||||||
|
AlreadyExists(String),
|
||||||
|
#[error("database error: {0}")]
|
||||||
|
Db(#[from] sqlx::Error),
|
||||||
|
#[error("topic backend error: {0}")]
|
||||||
|
Backend(String),
|
||||||
|
}
|
||||||
|
|
||||||
|
#[async_trait]
|
||||||
|
pub trait TopicRepo: Send + Sync {
|
||||||
|
/// Register a topic. Errors `AlreadyExists` on PK conflict.
|
||||||
|
async fn create(
|
||||||
|
&self,
|
||||||
|
app_id: AppId,
|
||||||
|
name: &str,
|
||||||
|
external_subscribable: bool,
|
||||||
|
auth_mode: TopicAuthMode,
|
||||||
|
) -> Result<Topic, TopicRepoError>;
|
||||||
|
|
||||||
|
/// List every registered topic in the app, ordered by name.
|
||||||
|
async fn list(&self, app_id: AppId) -> Result<Vec<Topic>, TopicRepoError>;
|
||||||
|
|
||||||
|
/// Fetch one topic by name, `None` if not registered.
|
||||||
|
async fn get(&self, app_id: AppId, name: &str) -> Result<Option<Topic>, TopicRepoError>;
|
||||||
|
|
||||||
|
/// Update `external_subscribable` and/or `auth_mode` (each `None`
|
||||||
|
/// leaves the column unchanged). `None` return = no such topic.
|
||||||
|
async fn update(
|
||||||
|
&self,
|
||||||
|
app_id: AppId,
|
||||||
|
name: &str,
|
||||||
|
external_subscribable: Option<bool>,
|
||||||
|
auth_mode: Option<TopicAuthMode>,
|
||||||
|
) -> Result<Option<Topic>, TopicRepoError>;
|
||||||
|
|
||||||
|
/// Unregister a topic. Returns `true` if a row was removed.
|
||||||
|
async fn delete(&self, app_id: AppId, name: &str) -> Result<bool, TopicRepoError>;
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(sqlx::FromRow)]
|
||||||
|
struct TopicRow {
|
||||||
|
name: String,
|
||||||
|
external_subscribable: bool,
|
||||||
|
auth_mode: String,
|
||||||
|
created_at: DateTime<Utc>,
|
||||||
|
updated_at: DateTime<Utc>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl TopicRow {
|
||||||
|
fn into_topic(self) -> Result<Topic, TopicRepoError> {
|
||||||
|
Ok(Topic {
|
||||||
|
auth_mode: TopicAuthMode::from_db(&self.auth_mode)?,
|
||||||
|
name: self.name,
|
||||||
|
external_subscribable: self.external_subscribable,
|
||||||
|
created_at: self.created_at,
|
||||||
|
updated_at: self.updated_at,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const SELECT_COLS: &str = "name, external_subscribable, auth_mode, created_at, updated_at";
|
||||||
|
|
||||||
|
pub struct PostgresTopicRepo {
|
||||||
|
pool: PgPool,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl PostgresTopicRepo {
|
||||||
|
#[must_use]
|
||||||
|
pub fn new(pool: PgPool) -> Self {
|
||||||
|
Self { pool }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[async_trait]
|
||||||
|
impl TopicRepo for PostgresTopicRepo {
|
||||||
|
async fn create(
|
||||||
|
&self,
|
||||||
|
app_id: AppId,
|
||||||
|
name: &str,
|
||||||
|
external_subscribable: bool,
|
||||||
|
auth_mode: TopicAuthMode,
|
||||||
|
) -> Result<Topic, TopicRepoError> {
|
||||||
|
let row: Option<TopicRow> = sqlx::query_as(&format!(
|
||||||
|
"INSERT INTO topics (app_id, name, external_subscribable, auth_mode) \
|
||||||
|
VALUES ($1, $2, $3, $4) \
|
||||||
|
ON CONFLICT (app_id, name) DO NOTHING \
|
||||||
|
RETURNING {SELECT_COLS}"
|
||||||
|
))
|
||||||
|
.bind(app_id.into_inner())
|
||||||
|
.bind(name)
|
||||||
|
.bind(external_subscribable)
|
||||||
|
.bind(auth_mode.as_str())
|
||||||
|
.fetch_optional(&self.pool)
|
||||||
|
.await?;
|
||||||
|
match row {
|
||||||
|
Some(r) => r.into_topic(),
|
||||||
|
None => Err(TopicRepoError::AlreadyExists(name.to_string())),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn list(&self, app_id: AppId) -> Result<Vec<Topic>, TopicRepoError> {
|
||||||
|
let rows: Vec<TopicRow> = sqlx::query_as(&format!(
|
||||||
|
"SELECT {SELECT_COLS} FROM topics WHERE app_id = $1 ORDER BY name"
|
||||||
|
))
|
||||||
|
.bind(app_id.into_inner())
|
||||||
|
.fetch_all(&self.pool)
|
||||||
|
.await?;
|
||||||
|
rows.into_iter().map(TopicRow::into_topic).collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn get(&self, app_id: AppId, name: &str) -> Result<Option<Topic>, TopicRepoError> {
|
||||||
|
let row: Option<TopicRow> = sqlx::query_as(&format!(
|
||||||
|
"SELECT {SELECT_COLS} FROM topics WHERE app_id = $1 AND name = $2"
|
||||||
|
))
|
||||||
|
.bind(app_id.into_inner())
|
||||||
|
.bind(name)
|
||||||
|
.fetch_optional(&self.pool)
|
||||||
|
.await?;
|
||||||
|
row.map(TopicRow::into_topic).transpose()
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn update(
|
||||||
|
&self,
|
||||||
|
app_id: AppId,
|
||||||
|
name: &str,
|
||||||
|
external_subscribable: Option<bool>,
|
||||||
|
auth_mode: Option<TopicAuthMode>,
|
||||||
|
) -> Result<Option<Topic>, TopicRepoError> {
|
||||||
|
// COALESCE leaves a column untouched when its bind is NULL.
|
||||||
|
let row: Option<TopicRow> = sqlx::query_as(&format!(
|
||||||
|
"UPDATE topics SET \
|
||||||
|
external_subscribable = COALESCE($3, external_subscribable), \
|
||||||
|
auth_mode = COALESCE($4, auth_mode), \
|
||||||
|
updated_at = NOW() \
|
||||||
|
WHERE app_id = $1 AND name = $2 \
|
||||||
|
RETURNING {SELECT_COLS}"
|
||||||
|
))
|
||||||
|
.bind(app_id.into_inner())
|
||||||
|
.bind(name)
|
||||||
|
.bind(external_subscribable)
|
||||||
|
.bind(auth_mode.map(TopicAuthMode::as_str))
|
||||||
|
.fetch_optional(&self.pool)
|
||||||
|
.await?;
|
||||||
|
row.map(TopicRow::into_topic).transpose()
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn delete(&self, app_id: AppId, name: &str) -> Result<bool, TopicRepoError> {
|
||||||
|
let res = sqlx::query("DELETE FROM topics WHERE app_id = $1 AND name = $2")
|
||||||
|
.bind(app_id.into_inner())
|
||||||
|
.bind(name)
|
||||||
|
.execute(&self.pool)
|
||||||
|
.await?;
|
||||||
|
Ok(res.rows_affected() > 0)
|
||||||
|
}
|
||||||
|
}
|
||||||
629
crates/manager-core/src/topics_api.rs
Normal file
629
crates/manager-core/src/topics_api.rs
Normal file
@@ -0,0 +1,629 @@
|
|||||||
|
//! `/api/v1/admin/apps/{id}/topics*` — topic registration admin
|
||||||
|
//! endpoints (v1.1.6).
|
||||||
|
//!
|
||||||
|
//! These manage the `topics` table: the explicit registry of which
|
||||||
|
//! pub/sub topics are externally subscribable over SSE (design notes
|
||||||
|
//! §5). Internal-only topics never appear here.
|
||||||
|
//!
|
||||||
|
//! * `POST /apps/{id}/topics` — register a topic.
|
||||||
|
//! * `GET /apps/{id}/topics` — list registered topics.
|
||||||
|
//! * `PATCH /apps/{id}/topics/{name}` — flip external/auth_mode.
|
||||||
|
//! * `DELETE /apps/{id}/topics/{name}` — unregister + disconnect.
|
||||||
|
//!
|
||||||
|
//! The PATCH endpoint is deliberately its OWN surface (not folded into a
|
||||||
|
//! generic topic update) so every change to externally-subscribable
|
||||||
|
//! status is a discrete, watchable/auditable API call (§5 commitment).
|
||||||
|
//!
|
||||||
|
//! Create / update / delete are gated by `AppTopicManage` (→ `app:admin`
|
||||||
|
//! scope); list is gated by the existing `AppRead`. DELETE also drops
|
||||||
|
//! the topic's in-process broadcast channel so live SSE subscribers
|
||||||
|
//! disconnect cleanly.
|
||||||
|
|
||||||
|
use std::sync::Arc;
|
||||||
|
|
||||||
|
use axum::extract::{Path, State};
|
||||||
|
use axum::http::StatusCode;
|
||||||
|
use axum::response::{IntoResponse, Json, Response};
|
||||||
|
use axum::routing::{get, patch};
|
||||||
|
use axum::{Extension, Router};
|
||||||
|
use picloud_shared::{AppId, Principal, RealtimeBroadcaster};
|
||||||
|
use serde::Deserialize;
|
||||||
|
use serde_json::json;
|
||||||
|
|
||||||
|
use crate::app_repo::AppRepository;
|
||||||
|
use crate::authz::{require, AuthzDenied, AuthzError, AuthzRepo, Capability};
|
||||||
|
use crate::topic_repo::{Topic, TopicAuthMode, TopicRepo, TopicRepoError};
|
||||||
|
|
||||||
|
#[derive(Clone)]
|
||||||
|
pub struct TopicsState {
|
||||||
|
pub topics: Arc<dyn TopicRepo>,
|
||||||
|
pub apps: Arc<dyn AppRepository>,
|
||||||
|
pub authz: Arc<dyn AuthzRepo>,
|
||||||
|
pub broadcaster: Arc<dyn RealtimeBroadcaster>,
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn topics_router(state: TopicsState) -> Router {
|
||||||
|
Router::new()
|
||||||
|
.route("/apps/{app_id}/topics", get(list_topics).post(create_topic))
|
||||||
|
.route(
|
||||||
|
"/apps/{app_id}/topics/{name}",
|
||||||
|
patch(update_topic).delete(delete_topic),
|
||||||
|
)
|
||||||
|
.with_state(state)
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, Deserialize)]
|
||||||
|
pub struct CreateTopicRequest {
|
||||||
|
pub name: String,
|
||||||
|
#[serde(default)]
|
||||||
|
pub external_subscribable: bool,
|
||||||
|
#[serde(default = "default_auth_mode")]
|
||||||
|
pub auth_mode: TopicAuthMode,
|
||||||
|
}
|
||||||
|
|
||||||
|
const fn default_auth_mode() -> TopicAuthMode {
|
||||||
|
TopicAuthMode::Public
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, Deserialize)]
|
||||||
|
pub struct UpdateTopicRequest {
|
||||||
|
#[serde(default)]
|
||||||
|
pub external_subscribable: Option<bool>,
|
||||||
|
#[serde(default)]
|
||||||
|
pub auth_mode: Option<TopicAuthMode>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Topic names are concrete (external pattern subscription is v1.2), so
|
||||||
|
/// reject empties and `*` wildcards at registration.
|
||||||
|
fn validate_topic_name(name: &str) -> Result<(), TopicsApiError> {
|
||||||
|
if name.trim().is_empty() {
|
||||||
|
return Err(TopicsApiError::Invalid(
|
||||||
|
"topic name must not be empty".into(),
|
||||||
|
));
|
||||||
|
}
|
||||||
|
if name.contains('*') {
|
||||||
|
return Err(TopicsApiError::Invalid(
|
||||||
|
"topic name must be a concrete topic, not a pattern (no '*')".into(),
|
||||||
|
));
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn create_topic(
|
||||||
|
State(s): State<TopicsState>,
|
||||||
|
Extension(principal): Extension<Principal>,
|
||||||
|
Path(app_id): Path<AppId>,
|
||||||
|
Json(input): Json<CreateTopicRequest>,
|
||||||
|
) -> Result<(StatusCode, Json<Topic>), TopicsApiError> {
|
||||||
|
ensure_app_exists(&*s.apps, app_id).await?;
|
||||||
|
require(
|
||||||
|
s.authz.as_ref(),
|
||||||
|
&principal,
|
||||||
|
Capability::AppTopicManage(app_id),
|
||||||
|
)
|
||||||
|
.await?;
|
||||||
|
validate_topic_name(&input.name)?;
|
||||||
|
let topic = s
|
||||||
|
.topics
|
||||||
|
.create(
|
||||||
|
app_id,
|
||||||
|
input.name.trim(),
|
||||||
|
input.external_subscribable,
|
||||||
|
input.auth_mode,
|
||||||
|
)
|
||||||
|
.await?;
|
||||||
|
Ok((StatusCode::CREATED, Json(topic)))
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, serde::Serialize)]
|
||||||
|
struct ListTopicsResponse {
|
||||||
|
topics: Vec<Topic>,
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn list_topics(
|
||||||
|
State(s): State<TopicsState>,
|
||||||
|
Extension(principal): Extension<Principal>,
|
||||||
|
Path(app_id): Path<AppId>,
|
||||||
|
) -> Result<Json<ListTopicsResponse>, TopicsApiError> {
|
||||||
|
ensure_app_exists(&*s.apps, app_id).await?;
|
||||||
|
require(s.authz.as_ref(), &principal, Capability::AppRead(app_id)).await?;
|
||||||
|
let topics = s.topics.list(app_id).await?;
|
||||||
|
Ok(Json(ListTopicsResponse { topics }))
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn update_topic(
|
||||||
|
State(s): State<TopicsState>,
|
||||||
|
Extension(principal): Extension<Principal>,
|
||||||
|
Path((app_id, name)): Path<(AppId, String)>,
|
||||||
|
Json(input): Json<UpdateTopicRequest>,
|
||||||
|
) -> Result<Json<Topic>, TopicsApiError> {
|
||||||
|
ensure_app_exists(&*s.apps, app_id).await?;
|
||||||
|
require(
|
||||||
|
s.authz.as_ref(),
|
||||||
|
&principal,
|
||||||
|
Capability::AppTopicManage(app_id),
|
||||||
|
)
|
||||||
|
.await?;
|
||||||
|
let topic = s
|
||||||
|
.topics
|
||||||
|
.update(app_id, &name, input.external_subscribable, input.auth_mode)
|
||||||
|
.await?
|
||||||
|
.ok_or(TopicsApiError::NotFound)?;
|
||||||
|
Ok(Json(topic))
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn delete_topic(
|
||||||
|
State(s): State<TopicsState>,
|
||||||
|
Extension(principal): Extension<Principal>,
|
||||||
|
Path((app_id, name)): Path<(AppId, String)>,
|
||||||
|
) -> Result<StatusCode, TopicsApiError> {
|
||||||
|
ensure_app_exists(&*s.apps, app_id).await?;
|
||||||
|
require(
|
||||||
|
s.authz.as_ref(),
|
||||||
|
&principal,
|
||||||
|
Capability::AppTopicManage(app_id),
|
||||||
|
)
|
||||||
|
.await?;
|
||||||
|
if !s.topics.delete(app_id, &name).await? {
|
||||||
|
return Err(TopicsApiError::NotFound);
|
||||||
|
}
|
||||||
|
// Disconnect any live SSE subscribers for the now-unregistered topic.
|
||||||
|
s.broadcaster.drop_topic(app_id, &name).await;
|
||||||
|
Ok(StatusCode::NO_CONTENT)
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn ensure_app_exists(apps: &dyn AppRepository, app_id: AppId) -> Result<(), TopicsApiError> {
|
||||||
|
apps.get_by_id(app_id)
|
||||||
|
.await
|
||||||
|
.map_err(|e| TopicsApiError::Backend(e.to_string()))?
|
||||||
|
.ok_or(TopicsApiError::AppNotFound)?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, thiserror::Error)]
|
||||||
|
pub enum TopicsApiError {
|
||||||
|
#[error("app not found")]
|
||||||
|
AppNotFound,
|
||||||
|
#[error("topic not found")]
|
||||||
|
NotFound,
|
||||||
|
#[error("{0}")]
|
||||||
|
AlreadyExists(String),
|
||||||
|
#[error("invalid request: {0}")]
|
||||||
|
Invalid(String),
|
||||||
|
#[error("forbidden")]
|
||||||
|
Forbidden,
|
||||||
|
#[error("authorization repo error: {0}")]
|
||||||
|
AuthzRepo(String),
|
||||||
|
#[error("topics backend: {0}")]
|
||||||
|
Backend(String),
|
||||||
|
}
|
||||||
|
|
||||||
|
impl From<AuthzDenied> for TopicsApiError {
|
||||||
|
fn from(d: AuthzDenied) -> Self {
|
||||||
|
match d {
|
||||||
|
AuthzDenied::Denied => Self::Forbidden,
|
||||||
|
AuthzDenied::Repo(e) => Self::AuthzRepo(e.to_string()),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl From<AuthzError> for TopicsApiError {
|
||||||
|
fn from(e: AuthzError) -> Self {
|
||||||
|
Self::AuthzRepo(e.to_string())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl From<TopicRepoError> for TopicsApiError {
|
||||||
|
fn from(e: TopicRepoError) -> Self {
|
||||||
|
match e {
|
||||||
|
TopicRepoError::AlreadyExists(name) => {
|
||||||
|
Self::AlreadyExists(format!("a topic named {name:?} already exists in this app"))
|
||||||
|
}
|
||||||
|
other => Self::Backend(other.to_string()),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl IntoResponse for TopicsApiError {
|
||||||
|
fn into_response(self) -> Response {
|
||||||
|
let (status, body) = match &self {
|
||||||
|
Self::AppNotFound | Self::NotFound => {
|
||||||
|
(StatusCode::NOT_FOUND, json!({ "error": self.to_string() }))
|
||||||
|
}
|
||||||
|
Self::AlreadyExists(_) => (StatusCode::CONFLICT, json!({ "error": self.to_string() })),
|
||||||
|
Self::Invalid(_) => (
|
||||||
|
StatusCode::UNPROCESSABLE_ENTITY,
|
||||||
|
json!({ "error": self.to_string() }),
|
||||||
|
),
|
||||||
|
Self::Forbidden => (StatusCode::FORBIDDEN, json!({ "error": self.to_string() })),
|
||||||
|
Self::AuthzRepo(e) => {
|
||||||
|
tracing::error!(error = %e, "topics admin authz repo error");
|
||||||
|
(
|
||||||
|
StatusCode::INTERNAL_SERVER_ERROR,
|
||||||
|
json!({ "error": "internal error" }),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
Self::Backend(e) => {
|
||||||
|
tracing::error!(error = %e, "topics admin backend error");
|
||||||
|
(
|
||||||
|
StatusCode::INTERNAL_SERVER_ERROR,
|
||||||
|
json!({ "error": "internal error" }),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
};
|
||||||
|
(status, Json(body)).into_response()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
//! In-memory handler tests: capability enforcement, the
|
||||||
|
//! `external_subscribable` default, the flip being its own endpoint,
|
||||||
|
//! cross-app isolation, and DELETE disconnecting subscribers. The
|
||||||
|
//! Postgres repo is exercised by the schema + integration suites.
|
||||||
|
|
||||||
|
use super::*;
|
||||||
|
use crate::repo::ScriptRepositoryError;
|
||||||
|
use async_trait::async_trait;
|
||||||
|
use chrono::Utc;
|
||||||
|
use picloud_shared::{
|
||||||
|
AdminUserId, App, AppRole, BroadcasterError, InstanceRole, RealtimeEvent, UserId,
|
||||||
|
};
|
||||||
|
use std::collections::HashMap;
|
||||||
|
use std::sync::Mutex as StdMutex;
|
||||||
|
use tokio::sync::Mutex;
|
||||||
|
|
||||||
|
#[derive(Default)]
|
||||||
|
struct InMemoryTopicRepo {
|
||||||
|
inner: Mutex<HashMap<(AppId, String), Topic>>,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[async_trait]
|
||||||
|
impl TopicRepo for InMemoryTopicRepo {
|
||||||
|
async fn create(
|
||||||
|
&self,
|
||||||
|
app_id: AppId,
|
||||||
|
name: &str,
|
||||||
|
external_subscribable: bool,
|
||||||
|
auth_mode: TopicAuthMode,
|
||||||
|
) -> Result<Topic, TopicRepoError> {
|
||||||
|
let mut g = self.inner.lock().await;
|
||||||
|
if g.contains_key(&(app_id, name.to_string())) {
|
||||||
|
return Err(TopicRepoError::AlreadyExists(name.to_string()));
|
||||||
|
}
|
||||||
|
let now = Utc::now();
|
||||||
|
let t = Topic {
|
||||||
|
name: name.to_string(),
|
||||||
|
external_subscribable,
|
||||||
|
auth_mode,
|
||||||
|
created_at: now,
|
||||||
|
updated_at: now,
|
||||||
|
};
|
||||||
|
g.insert((app_id, name.to_string()), t.clone());
|
||||||
|
Ok(t)
|
||||||
|
}
|
||||||
|
async fn list(&self, app_id: AppId) -> Result<Vec<Topic>, TopicRepoError> {
|
||||||
|
let g = self.inner.lock().await;
|
||||||
|
let mut v: Vec<Topic> = g
|
||||||
|
.iter()
|
||||||
|
.filter(|((a, _), _)| *a == app_id)
|
||||||
|
.map(|(_, t)| t.clone())
|
||||||
|
.collect();
|
||||||
|
v.sort_by(|a, b| a.name.cmp(&b.name));
|
||||||
|
Ok(v)
|
||||||
|
}
|
||||||
|
async fn get(&self, app_id: AppId, name: &str) -> Result<Option<Topic>, TopicRepoError> {
|
||||||
|
Ok(self
|
||||||
|
.inner
|
||||||
|
.lock()
|
||||||
|
.await
|
||||||
|
.get(&(app_id, name.to_string()))
|
||||||
|
.cloned())
|
||||||
|
}
|
||||||
|
async fn update(
|
||||||
|
&self,
|
||||||
|
app_id: AppId,
|
||||||
|
name: &str,
|
||||||
|
external_subscribable: Option<bool>,
|
||||||
|
auth_mode: Option<TopicAuthMode>,
|
||||||
|
) -> Result<Option<Topic>, TopicRepoError> {
|
||||||
|
let mut g = self.inner.lock().await;
|
||||||
|
let Some(t) = g.get_mut(&(app_id, name.to_string())) else {
|
||||||
|
return Ok(None);
|
||||||
|
};
|
||||||
|
if let Some(e) = external_subscribable {
|
||||||
|
t.external_subscribable = e;
|
||||||
|
}
|
||||||
|
if let Some(m) = auth_mode {
|
||||||
|
t.auth_mode = m;
|
||||||
|
}
|
||||||
|
t.updated_at = Utc::now();
|
||||||
|
Ok(Some(t.clone()))
|
||||||
|
}
|
||||||
|
async fn delete(&self, app_id: AppId, name: &str) -> Result<bool, TopicRepoError> {
|
||||||
|
Ok(self
|
||||||
|
.inner
|
||||||
|
.lock()
|
||||||
|
.await
|
||||||
|
.remove(&(app_id, name.to_string()))
|
||||||
|
.is_some())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
struct InMemoryAppRepo(AppId);
|
||||||
|
#[async_trait]
|
||||||
|
impl AppRepository for InMemoryAppRepo {
|
||||||
|
async fn list(&self) -> Result<Vec<App>, ScriptRepositoryError> {
|
||||||
|
unimplemented!()
|
||||||
|
}
|
||||||
|
async fn list_for_user(&self, _: AdminUserId) -> Result<Vec<App>, ScriptRepositoryError> {
|
||||||
|
unimplemented!()
|
||||||
|
}
|
||||||
|
async fn get_by_id(&self, id: AppId) -> Result<Option<App>, ScriptRepositoryError> {
|
||||||
|
if id != self.0 {
|
||||||
|
return Ok(None);
|
||||||
|
}
|
||||||
|
let now = Utc::now();
|
||||||
|
Ok(Some(App {
|
||||||
|
id,
|
||||||
|
slug: "test".into(),
|
||||||
|
name: "test".into(),
|
||||||
|
description: None,
|
||||||
|
created_at: now,
|
||||||
|
updated_at: now,
|
||||||
|
}))
|
||||||
|
}
|
||||||
|
async fn get_by_slug(&self, _: &str) -> Result<Option<App>, ScriptRepositoryError> {
|
||||||
|
unimplemented!()
|
||||||
|
}
|
||||||
|
async fn get_by_slug_or_history(
|
||||||
|
&self,
|
||||||
|
_: &str,
|
||||||
|
) -> Result<Option<crate::app_repo::AppLookup>, ScriptRepositoryError> {
|
||||||
|
unimplemented!()
|
||||||
|
}
|
||||||
|
async fn slug_in_history(&self, _: &str) -> Result<Option<App>, ScriptRepositoryError> {
|
||||||
|
unimplemented!()
|
||||||
|
}
|
||||||
|
async fn create(
|
||||||
|
&self,
|
||||||
|
_: &str,
|
||||||
|
_: &str,
|
||||||
|
_: Option<&str>,
|
||||||
|
) -> Result<App, ScriptRepositoryError> {
|
||||||
|
unimplemented!()
|
||||||
|
}
|
||||||
|
async fn create_with_takeover(
|
||||||
|
&self,
|
||||||
|
_: &str,
|
||||||
|
_: &str,
|
||||||
|
_: Option<&str>,
|
||||||
|
) -> Result<App, ScriptRepositoryError> {
|
||||||
|
unimplemented!()
|
||||||
|
}
|
||||||
|
async fn update(
|
||||||
|
&self,
|
||||||
|
_: AppId,
|
||||||
|
_: Option<&str>,
|
||||||
|
_: Option<Option<&str>>,
|
||||||
|
) -> Result<App, ScriptRepositoryError> {
|
||||||
|
unimplemented!()
|
||||||
|
}
|
||||||
|
async fn rename_slug(
|
||||||
|
&self,
|
||||||
|
_: AppId,
|
||||||
|
_: &str,
|
||||||
|
_: bool,
|
||||||
|
) -> Result<App, ScriptRepositoryError> {
|
||||||
|
unimplemented!()
|
||||||
|
}
|
||||||
|
async fn delete(&self, _: AppId) -> Result<(), ScriptRepositoryError> {
|
||||||
|
unimplemented!()
|
||||||
|
}
|
||||||
|
async fn delete_cascade(&self, _: AppId) -> Result<(), ScriptRepositoryError> {
|
||||||
|
unimplemented!()
|
||||||
|
}
|
||||||
|
async fn count_scripts_in_app(&self, _: AppId) -> Result<i64, ScriptRepositoryError> {
|
||||||
|
unimplemented!()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Grants `AppAdmin` only for `granted_app`; denies elsewhere — used
|
||||||
|
/// for the cross-app isolation test.
|
||||||
|
struct PerAppAuthzRepo {
|
||||||
|
granted_app: AppId,
|
||||||
|
}
|
||||||
|
#[async_trait]
|
||||||
|
impl AuthzRepo for PerAppAuthzRepo {
|
||||||
|
async fn membership(
|
||||||
|
&self,
|
||||||
|
_: UserId,
|
||||||
|
app_id: AppId,
|
||||||
|
) -> Result<Option<AppRole>, AuthzError> {
|
||||||
|
Ok((app_id == self.granted_app).then_some(AppRole::AppAdmin))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
struct DenyAuthzRepo;
|
||||||
|
#[async_trait]
|
||||||
|
impl AuthzRepo for DenyAuthzRepo {
|
||||||
|
async fn membership(&self, _: UserId, _: AppId) -> Result<Option<AppRole>, AuthzError> {
|
||||||
|
Ok(None)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Default)]
|
||||||
|
struct RecordingBroadcaster {
|
||||||
|
dropped: StdMutex<Vec<(AppId, String)>>,
|
||||||
|
}
|
||||||
|
#[async_trait]
|
||||||
|
impl RealtimeBroadcaster for RecordingBroadcaster {
|
||||||
|
async fn subscribe(
|
||||||
|
&self,
|
||||||
|
_: AppId,
|
||||||
|
_: &str,
|
||||||
|
) -> Result<tokio::sync::broadcast::Receiver<RealtimeEvent>, BroadcasterError> {
|
||||||
|
unimplemented!()
|
||||||
|
}
|
||||||
|
async fn publish(&self, _: AppId, _: &str, _: RealtimeEvent) {}
|
||||||
|
async fn drop_topic(&self, app_id: AppId, topic: &str) {
|
||||||
|
self.dropped
|
||||||
|
.lock()
|
||||||
|
.unwrap()
|
||||||
|
.push((app_id, topic.to_string()));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn member() -> Principal {
|
||||||
|
Principal {
|
||||||
|
user_id: AdminUserId::new(),
|
||||||
|
instance_role: InstanceRole::Member,
|
||||||
|
scopes: None,
|
||||||
|
app_binding: None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn state(app_id: AppId, authz: Arc<dyn AuthzRepo>) -> (TopicsState, Arc<RecordingBroadcaster>) {
|
||||||
|
let bc = Arc::new(RecordingBroadcaster::default());
|
||||||
|
let state = TopicsState {
|
||||||
|
topics: Arc::new(InMemoryTopicRepo::default()),
|
||||||
|
apps: Arc::new(InMemoryAppRepo(app_id)),
|
||||||
|
authz,
|
||||||
|
broadcaster: bc.clone(),
|
||||||
|
};
|
||||||
|
(state, bc)
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn register_defaults_external_subscribable_false() {
|
||||||
|
let app = AppId::new();
|
||||||
|
let (s, _) = state(app, Arc::new(PerAppAuthzRepo { granted_app: app }));
|
||||||
|
let (status, Json(topic)) = create_topic(
|
||||||
|
State(s),
|
||||||
|
Extension(member()),
|
||||||
|
Path(app),
|
||||||
|
Json(CreateTopicRequest {
|
||||||
|
name: "chat".into(),
|
||||||
|
external_subscribable: false,
|
||||||
|
auth_mode: TopicAuthMode::Public,
|
||||||
|
}),
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(status, StatusCode::CREATED);
|
||||||
|
assert!(!topic.external_subscribable);
|
||||||
|
assert_eq!(topic.name, "chat");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn flip_requires_app_admin_role() {
|
||||||
|
let app = AppId::new();
|
||||||
|
// Topic exists; the caller has no role → PATCH is forbidden.
|
||||||
|
let (s, _) = state(app, Arc::new(DenyAuthzRepo));
|
||||||
|
s.topics
|
||||||
|
.create(app, "chat", false, TopicAuthMode::Public)
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
let err = update_topic(
|
||||||
|
State(s),
|
||||||
|
Extension(member()),
|
||||||
|
Path((app, "chat".to_string())),
|
||||||
|
Json(UpdateTopicRequest {
|
||||||
|
external_subscribable: Some(true),
|
||||||
|
auth_mode: None,
|
||||||
|
}),
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
.unwrap_err();
|
||||||
|
assert!(matches!(err, TopicsApiError::Forbidden));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn flip_is_its_own_endpoint_and_toggles_external() {
|
||||||
|
// The PATCH handler is a distinct surface from create; flipping
|
||||||
|
// external_subscribable false→true is a single discrete call.
|
||||||
|
let app = AppId::new();
|
||||||
|
let (s, _) = state(app, Arc::new(PerAppAuthzRepo { granted_app: app }));
|
||||||
|
s.topics
|
||||||
|
.create(app, "chat", false, TopicAuthMode::Public)
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
let Json(updated) = update_topic(
|
||||||
|
State(s),
|
||||||
|
Extension(member()),
|
||||||
|
Path((app, "chat".to_string())),
|
||||||
|
Json(UpdateTopicRequest {
|
||||||
|
external_subscribable: Some(true),
|
||||||
|
auth_mode: Some(TopicAuthMode::Token),
|
||||||
|
}),
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
assert!(updated.external_subscribable);
|
||||||
|
assert_eq!(updated.auth_mode, TopicAuthMode::Token);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn delete_disconnects_subscribers() {
|
||||||
|
let app = AppId::new();
|
||||||
|
let (s, bc) = state(app, Arc::new(PerAppAuthzRepo { granted_app: app }));
|
||||||
|
s.topics
|
||||||
|
.create(app, "chat", true, TopicAuthMode::Public)
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
let status = delete_topic(
|
||||||
|
State(s),
|
||||||
|
Extension(member()),
|
||||||
|
Path((app, "chat".to_string())),
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(status, StatusCode::NO_CONTENT);
|
||||||
|
assert_eq!(
|
||||||
|
bc.dropped.lock().unwrap().as_slice(),
|
||||||
|
&[(app, "chat".to_string())]
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn cross_app_admin_cannot_manage_other_app() {
|
||||||
|
let app_a = AppId::new();
|
||||||
|
let app_b = AppId::new();
|
||||||
|
// Caller is admin of app A only; both apps exist via separate state.
|
||||||
|
let authz = Arc::new(PerAppAuthzRepo { granted_app: app_a });
|
||||||
|
// App-B-scoped state, but the caller only has A's grant.
|
||||||
|
let (s, _) = state(app_b, authz);
|
||||||
|
let err = create_topic(
|
||||||
|
State(s),
|
||||||
|
Extension(member()),
|
||||||
|
Path(app_b),
|
||||||
|
Json(CreateTopicRequest {
|
||||||
|
name: "chat".into(),
|
||||||
|
external_subscribable: true,
|
||||||
|
auth_mode: TopicAuthMode::Public,
|
||||||
|
}),
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
.unwrap_err();
|
||||||
|
assert!(matches!(err, TopicsApiError::Forbidden));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn pattern_name_rejected() {
|
||||||
|
let app = AppId::new();
|
||||||
|
let (s, _) = state(app, Arc::new(PerAppAuthzRepo { granted_app: app }));
|
||||||
|
let err = create_topic(
|
||||||
|
State(s),
|
||||||
|
Extension(member()),
|
||||||
|
Path(app),
|
||||||
|
Json(CreateTopicRequest {
|
||||||
|
name: "user.*".into(),
|
||||||
|
external_subscribable: true,
|
||||||
|
auth_mode: TopicAuthMode::Public,
|
||||||
|
}),
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
.unwrap_err();
|
||||||
|
assert!(matches!(err, TopicsApiError::Invalid(_)));
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -57,6 +57,8 @@ pub enum TriggerKind {
|
|||||||
Files,
|
Files,
|
||||||
/// v1.1.5.
|
/// v1.1.5.
|
||||||
Pubsub,
|
Pubsub,
|
||||||
|
/// v1.1.7. Inbound email via the webhook receiver.
|
||||||
|
Email,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl TriggerKind {
|
impl TriggerKind {
|
||||||
@@ -69,6 +71,7 @@ impl TriggerKind {
|
|||||||
Self::Cron => "cron",
|
Self::Cron => "cron",
|
||||||
Self::Files => "files",
|
Self::Files => "files",
|
||||||
Self::Pubsub => "pubsub",
|
Self::Pubsub => "pubsub",
|
||||||
|
Self::Email => "email",
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -81,6 +84,7 @@ impl TriggerKind {
|
|||||||
"cron" => Some(Self::Cron),
|
"cron" => Some(Self::Cron),
|
||||||
"files" => Some(Self::Files),
|
"files" => Some(Self::Files),
|
||||||
"pubsub" => Some(Self::Pubsub),
|
"pubsub" => Some(Self::Pubsub),
|
||||||
|
"email" => Some(Self::Email),
|
||||||
_ => None,
|
_ => None,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -137,6 +141,10 @@ pub enum TriggerDetails {
|
|||||||
},
|
},
|
||||||
/// v1.1.5. A topic pattern: exact, `<prefix>.*`, or `*`.
|
/// v1.1.5. A topic pattern: exact, `<prefix>.*`, or `*`.
|
||||||
Pubsub { topic_pattern: String },
|
Pubsub { topic_pattern: String },
|
||||||
|
/// v1.1.7. Inbound email. The HMAC `inbound_secret` is never
|
||||||
|
/// surfaced (it's encrypted at rest); we expose only whether one is
|
||||||
|
/// configured so the admin UI can show "signed" vs "unsigned".
|
||||||
|
Email { has_inbound_secret: bool },
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Create payload for a KV trigger. Defaults applied at the admin
|
/// Create payload for a KV trigger. Defaults applied at the admin
|
||||||
@@ -232,6 +240,33 @@ pub struct CreatePubsubTrigger {
|
|||||||
pub registered_by_principal: AdminUserId,
|
pub registered_by_principal: AdminUserId,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Create payload for an email trigger (v1.1.7). `inbound_secret_*` is
|
||||||
|
/// the already-encrypted HMAC secret (sealed by the admin layer with the
|
||||||
|
/// process master key) or `None` for an unsigned trigger.
|
||||||
|
#[derive(Debug, Clone)]
|
||||||
|
pub struct CreateEmailTrigger {
|
||||||
|
pub script_id: ScriptId,
|
||||||
|
pub inbound_secret_encrypted: Option<Vec<u8>>,
|
||||||
|
pub inbound_secret_nonce: Option<Vec<u8>>,
|
||||||
|
pub registered_by_principal: AdminUserId,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What the inbound-email webhook receiver needs to verify + dispatch a
|
||||||
|
/// POST. Returned by `email_inbound_target`; `None` when the trigger
|
||||||
|
/// doesn't exist or isn't `kind = 'email'`.
|
||||||
|
#[derive(Debug, Clone)]
|
||||||
|
pub struct EmailInboundTarget {
|
||||||
|
pub app_id: AppId,
|
||||||
|
pub script_id: ScriptId,
|
||||||
|
pub enabled: bool,
|
||||||
|
pub dispatch_mode: TriggerDispatchMode,
|
||||||
|
pub registered_by_principal: AdminUserId,
|
||||||
|
/// Encrypted HMAC secret + nonce; both `None` for an unsigned
|
||||||
|
/// trigger (accepts any POST).
|
||||||
|
pub inbound_secret_encrypted: Option<Vec<u8>>,
|
||||||
|
pub inbound_secret_nonce: Option<Vec<u8>>,
|
||||||
|
}
|
||||||
|
|
||||||
/// One match for the dispatcher's "which KV triggers fire on this
|
/// One match for the dispatcher's "which KV triggers fire on this
|
||||||
/// event" lookup. Carries everything the dispatcher needs to construct
|
/// event" lookup. Carries everything the dispatcher needs to construct
|
||||||
/// the outbox row.
|
/// the outbox row.
|
||||||
@@ -313,6 +348,23 @@ pub trait TriggerRepo: Send + Sync {
|
|||||||
req: CreatePubsubTrigger,
|
req: CreatePubsubTrigger,
|
||||||
) -> Result<Trigger, TriggerRepoError>;
|
) -> Result<Trigger, TriggerRepoError>;
|
||||||
|
|
||||||
|
/// v1.1.7. Inbound email trigger. The `inbound_secret` is stored
|
||||||
|
/// already-encrypted (the admin layer seals it).
|
||||||
|
async fn create_email_trigger(
|
||||||
|
&self,
|
||||||
|
app_id: AppId,
|
||||||
|
req: CreateEmailTrigger,
|
||||||
|
) -> Result<Trigger, TriggerRepoError>;
|
||||||
|
|
||||||
|
/// v1.1.7. The webhook receiver's hot-path lookup: resolve a
|
||||||
|
/// `kind = 'email'` trigger to its app, handler script, dispatch
|
||||||
|
/// mode, and (encrypted) HMAC secret. Returns `None` when the
|
||||||
|
/// trigger doesn't exist or isn't an email trigger.
|
||||||
|
async fn email_inbound_target(
|
||||||
|
&self,
|
||||||
|
trigger_id: TriggerId,
|
||||||
|
) -> Result<Option<EmailInboundTarget>, TriggerRepoError>;
|
||||||
|
|
||||||
async fn list_for_app(&self, app_id: AppId) -> Result<Vec<Trigger>, TriggerRepoError>;
|
async fn list_for_app(&self, app_id: AppId) -> Result<Vec<Trigger>, TriggerRepoError>;
|
||||||
|
|
||||||
async fn get(&self, id: TriggerId) -> Result<Option<Trigger>, TriggerRepoError>;
|
async fn get(&self, id: TriggerId) -> Result<Option<Trigger>, TriggerRepoError>;
|
||||||
@@ -761,6 +813,89 @@ impl TriggerRepo for PostgresTriggerRepo {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async fn create_email_trigger(
|
||||||
|
&self,
|
||||||
|
app_id: AppId,
|
||||||
|
req: CreateEmailTrigger,
|
||||||
|
) -> Result<Trigger, TriggerRepoError> {
|
||||||
|
let has_inbound_secret = req.inbound_secret_encrypted.is_some();
|
||||||
|
let mut tx = self.pool.begin().await?;
|
||||||
|
// Inbound email is delivered async like every other fan-out
|
||||||
|
// event; the receiver enqueues an outbox row the dispatcher
|
||||||
|
// picks up. Retry settings use the standard defaults.
|
||||||
|
let parent: TriggerRow = sqlx::query_as(
|
||||||
|
"INSERT INTO triggers ( \
|
||||||
|
app_id, script_id, kind, enabled, dispatch_mode, \
|
||||||
|
retry_max_attempts, retry_backoff, retry_base_ms, \
|
||||||
|
registered_by_principal \
|
||||||
|
) VALUES ($1, $2, 'email', TRUE, 'async', 3, 'exponential', 1000, $3) \
|
||||||
|
RETURNING id, app_id, script_id, kind, enabled, dispatch_mode, \
|
||||||
|
retry_max_attempts, retry_backoff, retry_base_ms, \
|
||||||
|
registered_by_principal, created_at, updated_at",
|
||||||
|
)
|
||||||
|
.bind(app_id.into_inner())
|
||||||
|
.bind(req.script_id.into_inner())
|
||||||
|
.bind(req.registered_by_principal.into_inner())
|
||||||
|
.fetch_one(&mut *tx)
|
||||||
|
.await?;
|
||||||
|
|
||||||
|
sqlx::query(
|
||||||
|
"INSERT INTO email_trigger_details \
|
||||||
|
(trigger_id, inbound_secret_encrypted, inbound_secret_nonce) \
|
||||||
|
VALUES ($1, $2, $3)",
|
||||||
|
)
|
||||||
|
.bind(parent.id)
|
||||||
|
.bind(req.inbound_secret_encrypted.as_deref())
|
||||||
|
.bind(req.inbound_secret_nonce.as_deref())
|
||||||
|
.execute(&mut *tx)
|
||||||
|
.await?;
|
||||||
|
|
||||||
|
tx.commit().await?;
|
||||||
|
|
||||||
|
Ok(Trigger {
|
||||||
|
id: parent.id.into(),
|
||||||
|
app_id: parent.app_id.into(),
|
||||||
|
script_id: parent.script_id.into(),
|
||||||
|
kind: TriggerKind::Email,
|
||||||
|
enabled: parent.enabled,
|
||||||
|
dispatch_mode: dispatch_from_str(&parent.dispatch_mode),
|
||||||
|
retry_max_attempts: u32::try_from(parent.retry_max_attempts).unwrap_or(3),
|
||||||
|
retry_backoff: BackoffShape::from_wire(&parent.retry_backoff)
|
||||||
|
.unwrap_or(BackoffShape::Exponential),
|
||||||
|
retry_base_ms: u32::try_from(parent.retry_base_ms).unwrap_or(1000),
|
||||||
|
registered_by_principal: parent.registered_by_principal.into(),
|
||||||
|
created_at: parent.created_at,
|
||||||
|
updated_at: parent.updated_at,
|
||||||
|
details: TriggerDetails::Email { has_inbound_secret },
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn email_inbound_target(
|
||||||
|
&self,
|
||||||
|
trigger_id: TriggerId,
|
||||||
|
) -> Result<Option<EmailInboundTarget>, TriggerRepoError> {
|
||||||
|
let row: Option<EmailInboundRow> = sqlx::query_as(
|
||||||
|
"SELECT t.app_id, t.script_id, t.enabled, t.dispatch_mode, \
|
||||||
|
t.registered_by_principal, \
|
||||||
|
d.inbound_secret_encrypted, d.inbound_secret_nonce \
|
||||||
|
FROM triggers t \
|
||||||
|
JOIN email_trigger_details d ON d.trigger_id = t.id \
|
||||||
|
WHERE t.id = $1 AND t.kind = 'email'",
|
||||||
|
)
|
||||||
|
.bind(trigger_id.into_inner())
|
||||||
|
.fetch_optional(&self.pool)
|
||||||
|
.await?;
|
||||||
|
Ok(row.map(|r| EmailInboundTarget {
|
||||||
|
app_id: r.app_id.into(),
|
||||||
|
script_id: r.script_id.into(),
|
||||||
|
enabled: r.enabled,
|
||||||
|
dispatch_mode: dispatch_from_str(&r.dispatch_mode),
|
||||||
|
registered_by_principal: r.registered_by_principal.into(),
|
||||||
|
inbound_secret_encrypted: r.inbound_secret_encrypted,
|
||||||
|
inbound_secret_nonce: r.inbound_secret_nonce,
|
||||||
|
}))
|
||||||
|
}
|
||||||
|
|
||||||
async fn list_for_app(&self, app_id: AppId) -> Result<Vec<Trigger>, TriggerRepoError> {
|
async fn list_for_app(&self, app_id: AppId) -> Result<Vec<Trigger>, TriggerRepoError> {
|
||||||
let parents: Vec<TriggerRow> = sqlx::query_as(
|
let parents: Vec<TriggerRow> = sqlx::query_as(
|
||||||
"SELECT id, app_id, script_id, kind, enabled, dispatch_mode, \
|
"SELECT id, app_id, script_id, kind, enabled, dispatch_mode, \
|
||||||
@@ -1077,6 +1212,17 @@ async fn hydrate_one(pool: &PgPool, parent: TriggerRow) -> Result<Trigger, Trigg
|
|||||||
topic_pattern: row.topic_pattern,
|
topic_pattern: row.topic_pattern,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
TriggerKind::Email => {
|
||||||
|
let row: EmailDetailRow = sqlx::query_as(
|
||||||
|
"SELECT inbound_secret_encrypted FROM email_trigger_details WHERE trigger_id = $1",
|
||||||
|
)
|
||||||
|
.bind(parent.id)
|
||||||
|
.fetch_one(pool)
|
||||||
|
.await?;
|
||||||
|
TriggerDetails::Email {
|
||||||
|
has_inbound_secret: row.inbound_secret_encrypted.is_some(),
|
||||||
|
}
|
||||||
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
Ok(Trigger {
|
Ok(Trigger {
|
||||||
@@ -1154,6 +1300,22 @@ struct PubsubDetailRow {
|
|||||||
topic_pattern: String,
|
topic_pattern: String,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[derive(sqlx::FromRow)]
|
||||||
|
struct EmailDetailRow {
|
||||||
|
inbound_secret_encrypted: Option<Vec<u8>>,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(sqlx::FromRow)]
|
||||||
|
struct EmailInboundRow {
|
||||||
|
app_id: Uuid,
|
||||||
|
script_id: Uuid,
|
||||||
|
enabled: bool,
|
||||||
|
dispatch_mode: String,
|
||||||
|
registered_by_principal: Uuid,
|
||||||
|
inbound_secret_encrypted: Option<Vec<u8>>,
|
||||||
|
inbound_secret_nonce: Option<Vec<u8>>,
|
||||||
|
}
|
||||||
|
|
||||||
#[derive(sqlx::FromRow)]
|
#[derive(sqlx::FromRow)]
|
||||||
#[allow(clippy::struct_field_names)]
|
#[allow(clippy::struct_field_names)]
|
||||||
struct DlDetailRow {
|
struct DlDetailRow {
|
||||||
|
|||||||
@@ -17,7 +17,8 @@ use axum::response::{IntoResponse, Json, Response};
|
|||||||
use axum::routing::{delete, get, post};
|
use axum::routing::{delete, get, post};
|
||||||
use axum::{Extension, Router};
|
use axum::{Extension, Router};
|
||||||
use picloud_shared::{
|
use picloud_shared::{
|
||||||
AppId, DocsEventOp, FilesEventOp, KvEventOp, Principal, ScriptId, ScriptKind, TriggerId,
|
AppId, DocsEventOp, FilesEventOp, KvEventOp, MasterKey, Principal, ScriptId, ScriptKind,
|
||||||
|
TriggerId,
|
||||||
};
|
};
|
||||||
use serde::{Deserialize, Serialize};
|
use serde::{Deserialize, Serialize};
|
||||||
use serde_json::json;
|
use serde_json::json;
|
||||||
@@ -25,11 +26,12 @@ use serde_json::json;
|
|||||||
use crate::app_repo::AppRepository;
|
use crate::app_repo::AppRepository;
|
||||||
use crate::authz::{require, AuthzDenied, AuthzError, AuthzRepo, Capability};
|
use crate::authz::{require, AuthzDenied, AuthzError, AuthzRepo, Capability};
|
||||||
use crate::repo::{ScriptRepository, ScriptRepositoryError};
|
use crate::repo::{ScriptRepository, ScriptRepositoryError};
|
||||||
|
use crate::secrets_service::seal;
|
||||||
use crate::trigger_config::{BackoffShape, TriggerConfig};
|
use crate::trigger_config::{BackoffShape, TriggerConfig};
|
||||||
use crate::trigger_repo::{
|
use crate::trigger_repo::{
|
||||||
CreateCronTrigger, CreateDeadLetterTrigger, CreateDocsTrigger, CreateFilesTrigger,
|
CreateCronTrigger, CreateDeadLetterTrigger, CreateDocsTrigger, CreateEmailTrigger,
|
||||||
CreateKvTrigger, CreatePubsubTrigger, Trigger, TriggerDispatchMode, TriggerRepo,
|
CreateFilesTrigger, CreateKvTrigger, CreatePubsubTrigger, Trigger, TriggerDispatchMode,
|
||||||
TriggerRepoError,
|
TriggerRepo, TriggerRepoError,
|
||||||
};
|
};
|
||||||
|
|
||||||
#[derive(Clone)]
|
#[derive(Clone)]
|
||||||
@@ -46,6 +48,9 @@ pub struct TriggersState {
|
|||||||
/// retry settings. Kept on the state struct so tests can swap
|
/// retry settings. Kept on the state struct so tests can swap
|
||||||
/// in a stricter / looser config without env tinkering.
|
/// in a stricter / looser config without env tinkering.
|
||||||
pub config: TriggerConfig,
|
pub config: TriggerConfig,
|
||||||
|
/// v1.1.7: master key used to encrypt an email trigger's inbound HMAC
|
||||||
|
/// secret before it's stored.
|
||||||
|
pub master_key: MasterKey,
|
||||||
}
|
}
|
||||||
|
|
||||||
pub fn triggers_router(state: TriggersState) -> Router {
|
pub fn triggers_router(state: TriggersState) -> Router {
|
||||||
@@ -66,6 +71,7 @@ pub fn triggers_router(state: TriggersState) -> Router {
|
|||||||
"/apps/{app_id}/triggers/dead_letter",
|
"/apps/{app_id}/triggers/dead_letter",
|
||||||
post(create_dl_trigger),
|
post(create_dl_trigger),
|
||||||
)
|
)
|
||||||
|
.route("/apps/{app_id}/triggers/email", post(create_email_trigger))
|
||||||
.route(
|
.route(
|
||||||
"/apps/{app_id}/triggers/{trigger_id}",
|
"/apps/{app_id}/triggers/{trigger_id}",
|
||||||
delete(delete_trigger),
|
delete(delete_trigger),
|
||||||
@@ -467,6 +473,60 @@ async fn create_dl_trigger(
|
|||||||
Ok((StatusCode::CREATED, Json(created)))
|
Ok((StatusCode::CREATED, Json(created)))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, Deserialize)]
|
||||||
|
struct CreateEmailTriggerRequest {
|
||||||
|
script_id: ScriptId,
|
||||||
|
/// Shared HMAC secret the provider signs inbound POSTs with. `null`
|
||||||
|
/// (or absent) means the trigger accepts unsigned POSTs.
|
||||||
|
#[serde(default)]
|
||||||
|
inbound_secret: Option<String>,
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn create_email_trigger(
|
||||||
|
State(s): State<TriggersState>,
|
||||||
|
Extension(principal): Extension<Principal>,
|
||||||
|
Path(app_id): Path<AppId>,
|
||||||
|
Json(input): Json<CreateEmailTriggerRequest>,
|
||||||
|
) -> Result<(StatusCode, Json<Trigger>), TriggersApiError> {
|
||||||
|
ensure_app_exists(&*s.apps, app_id).await?;
|
||||||
|
require(
|
||||||
|
s.authz.as_ref(),
|
||||||
|
&principal,
|
||||||
|
Capability::AppManageTriggers(app_id),
|
||||||
|
)
|
||||||
|
.await?;
|
||||||
|
validate_trigger_target(&*s.scripts, app_id, input.script_id).await?;
|
||||||
|
|
||||||
|
// Encrypt the inbound HMAC secret at rest (user-approved deviation
|
||||||
|
// from the brief's plaintext column). An empty/whitespace secret is
|
||||||
|
// treated as "no secret" (unsigned trigger).
|
||||||
|
let (inbound_secret_encrypted, inbound_secret_nonce) = match input.inbound_secret {
|
||||||
|
Some(secret) if !secret.trim().is_empty() => {
|
||||||
|
// 64 KB cap is irrelevant for a signing secret, but `seal`
|
||||||
|
// takes one; reuse the secrets default.
|
||||||
|
let (ct, nonce) = seal(
|
||||||
|
&s.master_key,
|
||||||
|
&serde_json::Value::String(secret),
|
||||||
|
crate::secrets_service::DEFAULT_SECRET_MAX_VALUE_BYTES,
|
||||||
|
)
|
||||||
|
.map_err(|e| {
|
||||||
|
TriggersApiError::Invalid(format!("could not seal inbound_secret: {e}"))
|
||||||
|
})?;
|
||||||
|
(Some(ct), Some(nonce.to_vec()))
|
||||||
|
}
|
||||||
|
_ => (None, None),
|
||||||
|
};
|
||||||
|
|
||||||
|
let req = CreateEmailTrigger {
|
||||||
|
script_id: input.script_id,
|
||||||
|
inbound_secret_encrypted,
|
||||||
|
inbound_secret_nonce,
|
||||||
|
registered_by_principal: principal.user_id,
|
||||||
|
};
|
||||||
|
let created = s.triggers.create_email_trigger(app_id, req).await?;
|
||||||
|
Ok((StatusCode::CREATED, Json(created)))
|
||||||
|
}
|
||||||
|
|
||||||
async fn delete_trigger(
|
async fn delete_trigger(
|
||||||
State(s): State<TriggersState>,
|
State(s): State<TriggersState>,
|
||||||
Extension(principal): Extension<Principal>,
|
Extension(principal): Extension<Principal>,
|
||||||
@@ -598,9 +658,9 @@ mod tests {
|
|||||||
use super::*;
|
use super::*;
|
||||||
use crate::app_repo::{AppLookup, AppRepository};
|
use crate::app_repo::{AppLookup, AppRepository};
|
||||||
use crate::trigger_repo::{
|
use crate::trigger_repo::{
|
||||||
CreateCronTrigger, CreateFilesTrigger, CreatePubsubTrigger, DeadLetterTriggerMatch,
|
CreateCronTrigger, CreateEmailTrigger, CreateFilesTrigger, CreatePubsubTrigger,
|
||||||
DocsTriggerMatch, FilesTriggerMatch, KvTriggerMatch, Trigger, TriggerDetails, TriggerRepo,
|
DeadLetterTriggerMatch, DocsTriggerMatch, EmailInboundTarget, FilesTriggerMatch,
|
||||||
TriggerRepoError,
|
KvTriggerMatch, Trigger, TriggerDetails, TriggerKind, TriggerRepo, TriggerRepoError,
|
||||||
};
|
};
|
||||||
use async_trait::async_trait;
|
use async_trait::async_trait;
|
||||||
use chrono::Utc;
|
use chrono::Utc;
|
||||||
@@ -703,6 +763,50 @@ mod tests {
|
|||||||
self.inner.lock().await.insert(id, trigger.clone());
|
self.inner.lock().await.insert(id, trigger.clone());
|
||||||
Ok(trigger)
|
Ok(trigger)
|
||||||
}
|
}
|
||||||
|
async fn create_email_trigger(
|
||||||
|
&self,
|
||||||
|
app_id: AppId,
|
||||||
|
req: CreateEmailTrigger,
|
||||||
|
) -> Result<Trigger, TriggerRepoError> {
|
||||||
|
let now = Utc::now();
|
||||||
|
let id = TriggerId::new();
|
||||||
|
let trigger = Trigger {
|
||||||
|
id,
|
||||||
|
app_id,
|
||||||
|
script_id: req.script_id,
|
||||||
|
kind: TriggerKind::Email,
|
||||||
|
enabled: true,
|
||||||
|
dispatch_mode: TriggerDispatchMode::Async,
|
||||||
|
retry_max_attempts: 3,
|
||||||
|
retry_backoff: BackoffShape::Exponential,
|
||||||
|
retry_base_ms: 1000,
|
||||||
|
registered_by_principal: req.registered_by_principal,
|
||||||
|
created_at: now,
|
||||||
|
updated_at: now,
|
||||||
|
details: TriggerDetails::Email {
|
||||||
|
has_inbound_secret: req.inbound_secret_encrypted.is_some(),
|
||||||
|
},
|
||||||
|
};
|
||||||
|
self.inner.lock().await.insert(id, trigger.clone());
|
||||||
|
Ok(trigger)
|
||||||
|
}
|
||||||
|
async fn email_inbound_target(
|
||||||
|
&self,
|
||||||
|
trigger_id: TriggerId,
|
||||||
|
) -> Result<Option<EmailInboundTarget>, TriggerRepoError> {
|
||||||
|
let g = self.inner.lock().await;
|
||||||
|
Ok(g.get(&trigger_id)
|
||||||
|
.filter(|t| t.kind == TriggerKind::Email)
|
||||||
|
.map(|t| EmailInboundTarget {
|
||||||
|
app_id: t.app_id,
|
||||||
|
script_id: t.script_id,
|
||||||
|
enabled: t.enabled,
|
||||||
|
dispatch_mode: t.dispatch_mode,
|
||||||
|
registered_by_principal: t.registered_by_principal,
|
||||||
|
inbound_secret_encrypted: None,
|
||||||
|
inbound_secret_nonce: None,
|
||||||
|
}))
|
||||||
|
}
|
||||||
async fn create_cron_trigger(
|
async fn create_cron_trigger(
|
||||||
&self,
|
&self,
|
||||||
app_id: AppId,
|
app_id: AppId,
|
||||||
@@ -1101,6 +1205,7 @@ mod tests {
|
|||||||
authz,
|
authz,
|
||||||
scripts: InMemoryScriptRepo::empty(),
|
scripts: InMemoryScriptRepo::empty(),
|
||||||
config: TriggerConfig::conservative(),
|
config: TriggerConfig::conservative(),
|
||||||
|
master_key: picloud_shared::MasterKey::from_bytes([0u8; 32]),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1118,6 +1223,7 @@ mod tests {
|
|||||||
authz,
|
authz,
|
||||||
scripts: InMemoryScriptRepo::with_endpoint(app_id, script_id),
|
scripts: InMemoryScriptRepo::with_endpoint(app_id, script_id),
|
||||||
config: TriggerConfig::conservative(),
|
config: TriggerConfig::conservative(),
|
||||||
|
master_key: picloud_shared::MasterKey::from_bytes([0u8; 32]),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1390,6 +1496,7 @@ mod tests {
|
|||||||
authz: Arc::new(AlwaysAllowAuthzRepo),
|
authz: Arc::new(AlwaysAllowAuthzRepo),
|
||||||
scripts: InMemoryScriptRepo::with_module(app_id, script_id),
|
scripts: InMemoryScriptRepo::with_module(app_id, script_id),
|
||||||
config: TriggerConfig::conservative(),
|
config: TriggerConfig::conservative(),
|
||||||
|
master_key: picloud_shared::MasterKey::from_bytes([0u8; 32]),
|
||||||
};
|
};
|
||||||
let res = create_kv_trigger(
|
let res = create_kv_trigger(
|
||||||
State(state),
|
State(state),
|
||||||
@@ -1427,6 +1534,7 @@ mod tests {
|
|||||||
authz: Arc::new(AlwaysAllowAuthzRepo),
|
authz: Arc::new(AlwaysAllowAuthzRepo),
|
||||||
scripts: InMemoryScriptRepo::with_module(app_id, script_id),
|
scripts: InMemoryScriptRepo::with_module(app_id, script_id),
|
||||||
config: TriggerConfig::conservative(),
|
config: TriggerConfig::conservative(),
|
||||||
|
master_key: picloud_shared::MasterKey::from_bytes([0u8; 32]),
|
||||||
};
|
};
|
||||||
let res = create_docs_trigger(
|
let res = create_docs_trigger(
|
||||||
State(state),
|
State(state),
|
||||||
@@ -1461,6 +1569,7 @@ mod tests {
|
|||||||
authz: Arc::new(AlwaysAllowAuthzRepo),
|
authz: Arc::new(AlwaysAllowAuthzRepo),
|
||||||
scripts: InMemoryScriptRepo::with_module(app_id, script_id),
|
scripts: InMemoryScriptRepo::with_module(app_id, script_id),
|
||||||
config: TriggerConfig::conservative(),
|
config: TriggerConfig::conservative(),
|
||||||
|
master_key: picloud_shared::MasterKey::from_bytes([0u8; 32]),
|
||||||
};
|
};
|
||||||
let res = create_dl_trigger(
|
let res = create_dl_trigger(
|
||||||
State(state),
|
State(state),
|
||||||
@@ -1526,6 +1635,7 @@ mod tests {
|
|||||||
authz: Arc::new(AlwaysAllowAuthzRepo),
|
authz: Arc::new(AlwaysAllowAuthzRepo),
|
||||||
scripts,
|
scripts,
|
||||||
config: TriggerConfig::conservative(),
|
config: TriggerConfig::conservative(),
|
||||||
|
master_key: picloud_shared::MasterKey::from_bytes([0u8; 32]),
|
||||||
};
|
};
|
||||||
let res = create_kv_trigger(
|
let res = create_kv_trigger(
|
||||||
State(state),
|
State(state),
|
||||||
@@ -1656,6 +1766,7 @@ mod tests {
|
|||||||
authz: Arc::new(AlwaysAllowAuthzRepo),
|
authz: Arc::new(AlwaysAllowAuthzRepo),
|
||||||
scripts: InMemoryScriptRepo::with_module(app_id, script_id),
|
scripts: InMemoryScriptRepo::with_module(app_id, script_id),
|
||||||
config: TriggerConfig::conservative(),
|
config: TriggerConfig::conservative(),
|
||||||
|
master_key: picloud_shared::MasterKey::from_bytes([0u8; 32]),
|
||||||
};
|
};
|
||||||
let res = create_cron_trigger(
|
let res = create_cron_trigger(
|
||||||
State(state),
|
State(state),
|
||||||
@@ -1685,6 +1796,7 @@ mod tests {
|
|||||||
authz: Arc::new(AlwaysAllowAuthzRepo),
|
authz: Arc::new(AlwaysAllowAuthzRepo),
|
||||||
scripts: InMemoryScriptRepo::with_endpoint(app_b, script_id),
|
scripts: InMemoryScriptRepo::with_endpoint(app_b, script_id),
|
||||||
config: TriggerConfig::conservative(),
|
config: TriggerConfig::conservative(),
|
||||||
|
master_key: picloud_shared::MasterKey::from_bytes([0u8; 32]),
|
||||||
};
|
};
|
||||||
let res = create_cron_trigger(
|
let res = create_cron_trigger(
|
||||||
State(state),
|
State(state),
|
||||||
@@ -1813,6 +1925,7 @@ mod tests {
|
|||||||
authz: Arc::new(AlwaysAllowAuthzRepo),
|
authz: Arc::new(AlwaysAllowAuthzRepo),
|
||||||
scripts: InMemoryScriptRepo::with_module(app_id, script_id),
|
scripts: InMemoryScriptRepo::with_module(app_id, script_id),
|
||||||
config: TriggerConfig::conservative(),
|
config: TriggerConfig::conservative(),
|
||||||
|
master_key: picloud_shared::MasterKey::from_bytes([0u8; 32]),
|
||||||
};
|
};
|
||||||
let res = create_files_trigger(
|
let res = create_files_trigger(
|
||||||
State(state),
|
State(state),
|
||||||
@@ -1839,6 +1952,7 @@ mod tests {
|
|||||||
authz: Arc::new(AlwaysAllowAuthzRepo),
|
authz: Arc::new(AlwaysAllowAuthzRepo),
|
||||||
scripts: InMemoryScriptRepo::with_endpoint(app_b, script_id),
|
scripts: InMemoryScriptRepo::with_endpoint(app_b, script_id),
|
||||||
config: TriggerConfig::conservative(),
|
config: TriggerConfig::conservative(),
|
||||||
|
master_key: picloud_shared::MasterKey::from_bytes([0u8; 32]),
|
||||||
};
|
};
|
||||||
let res = create_files_trigger(
|
let res = create_files_trigger(
|
||||||
State(state),
|
State(state),
|
||||||
@@ -1936,6 +2050,7 @@ mod tests {
|
|||||||
authz: Arc::new(AlwaysAllowAuthzRepo),
|
authz: Arc::new(AlwaysAllowAuthzRepo),
|
||||||
scripts: InMemoryScriptRepo::with_module(app_id, script_id),
|
scripts: InMemoryScriptRepo::with_module(app_id, script_id),
|
||||||
config: TriggerConfig::conservative(),
|
config: TriggerConfig::conservative(),
|
||||||
|
master_key: picloud_shared::MasterKey::from_bytes([0u8; 32]),
|
||||||
};
|
};
|
||||||
let res = create_pubsub_trigger(
|
let res = create_pubsub_trigger(
|
||||||
State(state),
|
State(state),
|
||||||
@@ -1962,6 +2077,7 @@ mod tests {
|
|||||||
authz: Arc::new(AlwaysAllowAuthzRepo),
|
authz: Arc::new(AlwaysAllowAuthzRepo),
|
||||||
scripts: InMemoryScriptRepo::with_endpoint(app_b, script_id),
|
scripts: InMemoryScriptRepo::with_endpoint(app_b, script_id),
|
||||||
config: TriggerConfig::conservative(),
|
config: TriggerConfig::conservative(),
|
||||||
|
master_key: picloud_shared::MasterKey::from_bytes([0u8; 32]),
|
||||||
};
|
};
|
||||||
let res = create_pubsub_trigger(
|
let res = create_pubsub_trigger(
|
||||||
State(state),
|
State(state),
|
||||||
|
|||||||
@@ -58,6 +58,14 @@ table: app_members
|
|||||||
role: text NOT NULL
|
role: text NOT NULL
|
||||||
created_at: timestamp with time zone NOT NULL default=now()
|
created_at: timestamp with time zone NOT NULL default=now()
|
||||||
|
|
||||||
|
table: app_secrets
|
||||||
|
app_id: uuid NOT NULL
|
||||||
|
realtime_signing_key: bytea NULL
|
||||||
|
created_at: timestamp with time zone NOT NULL default=now()
|
||||||
|
updated_at: timestamp with time zone NOT NULL default=now()
|
||||||
|
realtime_signing_key_encrypted: bytea NULL
|
||||||
|
realtime_signing_key_nonce: bytea NULL
|
||||||
|
|
||||||
table: app_slug_history
|
table: app_slug_history
|
||||||
slug: text NOT NULL
|
slug: text NOT NULL
|
||||||
current_app_id: uuid NOT NULL
|
current_app_id: uuid NOT NULL
|
||||||
@@ -113,6 +121,11 @@ table: docs_trigger_details
|
|||||||
collection_glob: text NOT NULL
|
collection_glob: text NOT NULL
|
||||||
ops: ARRAY NOT NULL
|
ops: ARRAY NOT NULL
|
||||||
|
|
||||||
|
table: email_trigger_details
|
||||||
|
trigger_id: uuid NOT NULL
|
||||||
|
inbound_secret_encrypted: bytea NULL
|
||||||
|
inbound_secret_nonce: bytea NULL
|
||||||
|
|
||||||
table: execution_logs
|
table: execution_logs
|
||||||
id: uuid NOT NULL default=gen_random_uuid()
|
id: uuid NOT NULL default=gen_random_uuid()
|
||||||
script_id: uuid NOT NULL
|
script_id: uuid NOT NULL
|
||||||
@@ -211,6 +224,22 @@ table: scripts
|
|||||||
app_id: uuid NOT NULL
|
app_id: uuid NOT NULL
|
||||||
kind: text NOT NULL default='endpoint'::text
|
kind: text NOT NULL default='endpoint'::text
|
||||||
|
|
||||||
|
table: secrets
|
||||||
|
app_id: uuid NOT NULL
|
||||||
|
name: text NOT NULL
|
||||||
|
encrypted_value: bytea NOT NULL
|
||||||
|
nonce: bytea NOT NULL
|
||||||
|
created_at: timestamp with time zone NOT NULL default=now()
|
||||||
|
updated_at: timestamp with time zone NOT NULL default=now()
|
||||||
|
|
||||||
|
table: topics
|
||||||
|
app_id: uuid NOT NULL
|
||||||
|
name: text NOT NULL
|
||||||
|
external_subscribable: boolean NOT NULL default=false
|
||||||
|
auth_mode: text NOT NULL default='public'::text
|
||||||
|
created_at: timestamp with time zone NOT NULL default=now()
|
||||||
|
updated_at: timestamp with time zone NOT NULL default=now()
|
||||||
|
|
||||||
table: triggers
|
table: triggers
|
||||||
id: uuid NOT NULL default=gen_random_uuid()
|
id: uuid NOT NULL default=gen_random_uuid()
|
||||||
app_id: uuid NOT NULL
|
app_id: uuid NOT NULL
|
||||||
@@ -256,6 +285,9 @@ indexes on app_members:
|
|||||||
app_members_pkey: public.app_members USING btree (app_id, user_id)
|
app_members_pkey: public.app_members USING btree (app_id, user_id)
|
||||||
app_members_user_id_idx: public.app_members USING btree (user_id)
|
app_members_user_id_idx: public.app_members USING btree (user_id)
|
||||||
|
|
||||||
|
indexes on app_secrets:
|
||||||
|
app_secrets_pkey: public.app_secrets USING btree (app_id)
|
||||||
|
|
||||||
indexes on app_slug_history:
|
indexes on app_slug_history:
|
||||||
app_slug_history_pkey: public.app_slug_history USING btree (slug)
|
app_slug_history_pkey: public.app_slug_history USING btree (slug)
|
||||||
|
|
||||||
@@ -283,6 +315,9 @@ indexes on docs:
|
|||||||
indexes on docs_trigger_details:
|
indexes on docs_trigger_details:
|
||||||
docs_trigger_details_pkey: public.docs_trigger_details USING btree (trigger_id)
|
docs_trigger_details_pkey: public.docs_trigger_details USING btree (trigger_id)
|
||||||
|
|
||||||
|
indexes on email_trigger_details:
|
||||||
|
email_trigger_details_pkey: public.email_trigger_details USING btree (trigger_id)
|
||||||
|
|
||||||
indexes on execution_logs:
|
indexes on execution_logs:
|
||||||
execution_logs_app_id_created_at_idx: public.execution_logs USING btree (app_id, created_at DESC)
|
execution_logs_app_id_created_at_idx: public.execution_logs USING btree (app_id, created_at DESC)
|
||||||
execution_logs_pkey: public.execution_logs USING btree (id)
|
execution_logs_pkey: public.execution_logs USING btree (id)
|
||||||
@@ -328,6 +363,13 @@ indexes on scripts:
|
|||||||
scripts_name_uidx: public.scripts USING btree (app_id, lower(name))
|
scripts_name_uidx: public.scripts USING btree (app_id, lower(name))
|
||||||
scripts_pkey: public.scripts USING btree (id)
|
scripts_pkey: public.scripts USING btree (id)
|
||||||
|
|
||||||
|
indexes on secrets:
|
||||||
|
idx_secrets_app: public.secrets USING btree (app_id)
|
||||||
|
secrets_pkey: public.secrets USING btree (app_id, name)
|
||||||
|
|
||||||
|
indexes on topics:
|
||||||
|
topics_pkey: public.topics USING btree (app_id, name)
|
||||||
|
|
||||||
indexes on triggers:
|
indexes on triggers:
|
||||||
idx_triggers_app_kind_enabled: public.triggers USING btree (app_id, kind) WHERE (enabled = true)
|
idx_triggers_app_kind_enabled: public.triggers USING btree (app_id, kind) WHERE (enabled = true)
|
||||||
idx_triggers_app_pubsub_enabled: public.triggers USING btree (app_id, kind) WHERE ((enabled = true) AND (kind = 'pubsub'::text))
|
idx_triggers_app_pubsub_enabled: public.triggers USING btree (app_id, kind) WHERE ((enabled = true) AND (kind = 'pubsub'::text))
|
||||||
@@ -366,6 +408,10 @@ constraints on app_members:
|
|||||||
[FOREIGN KEY] app_members_user_id_fkey: FOREIGN KEY (user_id) REFERENCES admin_users(id) ON DELETE CASCADE
|
[FOREIGN KEY] app_members_user_id_fkey: FOREIGN KEY (user_id) REFERENCES admin_users(id) ON DELETE CASCADE
|
||||||
[PRIMARY KEY] app_members_pkey: PRIMARY KEY (app_id, user_id)
|
[PRIMARY KEY] app_members_pkey: PRIMARY KEY (app_id, user_id)
|
||||||
|
|
||||||
|
constraints on app_secrets:
|
||||||
|
[FOREIGN KEY] app_secrets_app_id_fkey: FOREIGN KEY (app_id) REFERENCES apps(id) ON DELETE CASCADE
|
||||||
|
[PRIMARY KEY] app_secrets_pkey: PRIMARY KEY (app_id)
|
||||||
|
|
||||||
constraints on app_slug_history:
|
constraints on app_slug_history:
|
||||||
[FOREIGN KEY] app_slug_history_current_app_id_fkey: FOREIGN KEY (current_app_id) REFERENCES apps(id) ON DELETE CASCADE
|
[FOREIGN KEY] app_slug_history_current_app_id_fkey: FOREIGN KEY (current_app_id) REFERENCES apps(id) ON DELETE CASCADE
|
||||||
[PRIMARY KEY] app_slug_history_pkey: PRIMARY KEY (slug)
|
[PRIMARY KEY] app_slug_history_pkey: PRIMARY KEY (slug)
|
||||||
@@ -395,6 +441,10 @@ constraints on docs_trigger_details:
|
|||||||
[FOREIGN KEY] docs_trigger_details_trigger_id_fkey: FOREIGN KEY (trigger_id) REFERENCES triggers(id) ON DELETE CASCADE
|
[FOREIGN KEY] docs_trigger_details_trigger_id_fkey: FOREIGN KEY (trigger_id) REFERENCES triggers(id) ON DELETE CASCADE
|
||||||
[PRIMARY KEY] docs_trigger_details_pkey: PRIMARY KEY (trigger_id)
|
[PRIMARY KEY] docs_trigger_details_pkey: PRIMARY KEY (trigger_id)
|
||||||
|
|
||||||
|
constraints on email_trigger_details:
|
||||||
|
[FOREIGN KEY] email_trigger_details_trigger_id_fkey: FOREIGN KEY (trigger_id) REFERENCES triggers(id) ON DELETE CASCADE
|
||||||
|
[PRIMARY KEY] email_trigger_details_pkey: PRIMARY KEY (trigger_id)
|
||||||
|
|
||||||
constraints on execution_logs:
|
constraints on execution_logs:
|
||||||
[CHECK] execution_logs_status_check: CHECK ((status = ANY (ARRAY['success'::text, 'error'::text, 'timeout'::text, 'budget_exceeded'::text])))
|
[CHECK] execution_logs_status_check: CHECK ((status = ANY (ARRAY['success'::text, 'error'::text, 'timeout'::text, 'budget_exceeded'::text])))
|
||||||
[FOREIGN KEY] execution_logs_app_id_fk: FOREIGN KEY (app_id) REFERENCES apps(id) ON DELETE CASCADE
|
[FOREIGN KEY] execution_logs_app_id_fk: FOREIGN KEY (app_id) REFERENCES apps(id) ON DELETE CASCADE
|
||||||
@@ -418,7 +468,7 @@ constraints on kv_trigger_details:
|
|||||||
[PRIMARY KEY] kv_trigger_details_pkey: PRIMARY KEY (trigger_id)
|
[PRIMARY KEY] kv_trigger_details_pkey: PRIMARY KEY (trigger_id)
|
||||||
|
|
||||||
constraints on outbox:
|
constraints on outbox:
|
||||||
[CHECK] outbox_source_kind_check: CHECK ((source_kind = ANY (ARRAY['http'::text, 'kv'::text, 'dead_letter'::text, 'docs'::text, 'cron'::text, 'files'::text, 'pubsub'::text])))
|
[CHECK] outbox_source_kind_check: CHECK ((source_kind = ANY (ARRAY['http'::text, 'kv'::text, 'dead_letter'::text, 'docs'::text, 'cron'::text, 'files'::text, 'pubsub'::text, 'email'::text])))
|
||||||
[FOREIGN KEY] outbox_app_id_fkey: FOREIGN KEY (app_id) REFERENCES apps(id) ON DELETE CASCADE
|
[FOREIGN KEY] outbox_app_id_fkey: FOREIGN KEY (app_id) REFERENCES apps(id) ON DELETE CASCADE
|
||||||
[PRIMARY KEY] outbox_pkey: PRIMARY KEY (id)
|
[PRIMARY KEY] outbox_pkey: PRIMARY KEY (id)
|
||||||
|
|
||||||
@@ -448,9 +498,18 @@ constraints on scripts:
|
|||||||
[FOREIGN KEY] scripts_app_id_fk: FOREIGN KEY (app_id) REFERENCES apps(id) ON DELETE RESTRICT
|
[FOREIGN KEY] scripts_app_id_fk: FOREIGN KEY (app_id) REFERENCES apps(id) ON DELETE RESTRICT
|
||||||
[PRIMARY KEY] scripts_pkey: PRIMARY KEY (id)
|
[PRIMARY KEY] scripts_pkey: PRIMARY KEY (id)
|
||||||
|
|
||||||
|
constraints on secrets:
|
||||||
|
[FOREIGN KEY] secrets_app_id_fkey: FOREIGN KEY (app_id) REFERENCES apps(id) ON DELETE CASCADE
|
||||||
|
[PRIMARY KEY] secrets_pkey: PRIMARY KEY (app_id, name)
|
||||||
|
|
||||||
|
constraints on topics:
|
||||||
|
[CHECK] topics_auth_mode_check: CHECK ((auth_mode = ANY (ARRAY['public'::text, 'token'::text])))
|
||||||
|
[FOREIGN KEY] topics_app_id_fkey: FOREIGN KEY (app_id) REFERENCES apps(id) ON DELETE CASCADE
|
||||||
|
[PRIMARY KEY] topics_pkey: PRIMARY KEY (app_id, name)
|
||||||
|
|
||||||
constraints on triggers:
|
constraints on triggers:
|
||||||
[CHECK] triggers_dispatch_mode_check: CHECK ((dispatch_mode = ANY (ARRAY['sync'::text, 'async'::text])))
|
[CHECK] triggers_dispatch_mode_check: CHECK ((dispatch_mode = ANY (ARRAY['sync'::text, 'async'::text])))
|
||||||
[CHECK] triggers_kind_check: CHECK ((kind = ANY (ARRAY['kv'::text, 'dead_letter'::text, 'docs'::text, 'cron'::text, 'files'::text, 'pubsub'::text])))
|
[CHECK] triggers_kind_check: CHECK ((kind = ANY (ARRAY['kv'::text, 'dead_letter'::text, 'docs'::text, 'cron'::text, 'files'::text, 'pubsub'::text, 'email'::text])))
|
||||||
[CHECK] triggers_retry_backoff_check: CHECK ((retry_backoff = ANY (ARRAY['exponential'::text, 'linear'::text, 'constant'::text])))
|
[CHECK] triggers_retry_backoff_check: CHECK ((retry_backoff = ANY (ARRAY['exponential'::text, 'linear'::text, 'constant'::text])))
|
||||||
[FOREIGN KEY] triggers_app_id_fkey: FOREIGN KEY (app_id) REFERENCES apps(id) ON DELETE CASCADE
|
[FOREIGN KEY] triggers_app_id_fkey: FOREIGN KEY (app_id) REFERENCES apps(id) ON DELETE CASCADE
|
||||||
[FOREIGN KEY] triggers_registered_by_principal_fkey: FOREIGN KEY (registered_by_principal) REFERENCES admin_users(id) ON DELETE CASCADE
|
[FOREIGN KEY] triggers_registered_by_principal_fkey: FOREIGN KEY (registered_by_principal) REFERENCES admin_users(id) ON DELETE CASCADE
|
||||||
@@ -478,3 +537,8 @@ constraints on triggers:
|
|||||||
0018: files
|
0018: files
|
||||||
0019: files triggers
|
0019: files triggers
|
||||||
0020: pubsub triggers
|
0020: pubsub triggers
|
||||||
|
0021: topics
|
||||||
|
0022: app secrets
|
||||||
|
0023: secrets
|
||||||
|
0024: email triggers
|
||||||
|
0025: encrypt realtime keys
|
||||||
|
|||||||
@@ -23,8 +23,13 @@ chrono.workspace = true
|
|||||||
reqwest.workspace = true
|
reqwest.workspace = true
|
||||||
rhai.workspace = true
|
rhai.workspace = true
|
||||||
tokio.workspace = true
|
tokio.workspace = true
|
||||||
|
tokio-stream.workspace = true
|
||||||
urlencoding.workspace = true
|
urlencoding.workspace = true
|
||||||
|
|
||||||
# v1.1.3 — top-level script AST cache lives in orchestrator-core's
|
# v1.1.3 — top-level script AST cache lives in orchestrator-core's
|
||||||
# LocalExecutorClient; key is ScriptId, value is `(updated_at, Arc<rhai::AST>)`.
|
# LocalExecutorClient; key is ScriptId, value is `(updated_at, Arc<rhai::AST>)`.
|
||||||
lru.workspace = true
|
lru.workspace = true
|
||||||
|
|
||||||
|
[dev-dependencies]
|
||||||
|
# `ServiceExt::oneshot` for driving the SSE router in unit tests.
|
||||||
|
tower.workspace = true
|
||||||
|
|||||||
@@ -12,6 +12,8 @@ pub mod api;
|
|||||||
pub mod client;
|
pub mod client;
|
||||||
pub mod gate;
|
pub mod gate;
|
||||||
pub mod inbox;
|
pub mod inbox;
|
||||||
|
pub mod realtime;
|
||||||
|
pub mod realtime_api;
|
||||||
pub mod resolver;
|
pub mod resolver;
|
||||||
pub mod routing;
|
pub mod routing;
|
||||||
|
|
||||||
@@ -19,4 +21,8 @@ pub use api::{data_plane_router, user_routes_router, DataPlaneState};
|
|||||||
pub use client::{ExecutorClient, LocalExecutorClient, RemoteExecutorClient, ScriptIdentity};
|
pub use client::{ExecutorClient, LocalExecutorClient, RemoteExecutorClient, ScriptIdentity};
|
||||||
pub use gate::{AcquireError, ExecutionGate};
|
pub use gate::{AcquireError, ExecutionGate};
|
||||||
pub use inbox::InboxRegistry;
|
pub use inbox::InboxRegistry;
|
||||||
|
pub use realtime::{spawn_realtime_gc, InProcessBroadcaster, DEFAULT_BROADCAST_CAPACITY};
|
||||||
|
pub use realtime_api::{
|
||||||
|
heartbeat_secs_from_env, realtime_router, RealtimeState, DEFAULT_HEARTBEAT_SECS,
|
||||||
|
};
|
||||||
pub use resolver::{ResolverError, ScriptResolver};
|
pub use resolver::{ResolverError, ScriptResolver};
|
||||||
|
|||||||
242
crates/orchestrator-core/src/realtime.rs
Normal file
242
crates/orchestrator-core/src/realtime.rs
Normal file
@@ -0,0 +1,242 @@
|
|||||||
|
//! In-process `RealtimeBroadcaster` — the SSE fan-out registry (v1.1.6).
|
||||||
|
//!
|
||||||
|
//! Sibling of [`crate::inbox::InboxRegistry`], but multi-receiver and
|
||||||
|
//! repeated-event: a `Mutex<HashMap<(AppId, topic), broadcast::Sender>>`
|
||||||
|
//! over `tokio::sync::broadcast` instead of a oneshot map. The publish
|
||||||
|
//! side ([`PubsubServiceImpl`]) and the SSE subscribe side both hold one
|
||||||
|
//! shared `Arc<InProcessBroadcaster>`.
|
||||||
|
//!
|
||||||
|
//! Delivery is best-effort: each channel has a bounded buffer
|
||||||
|
//! (`PICLOUD_REALTIME_BROADCAST_CAPACITY`, default 64); a slow consumer
|
||||||
|
//! that falls behind sees the oldest events dropped (standard
|
||||||
|
//! `broadcast` lag semantics — the receiver gets `RecvError::Lagged`).
|
||||||
|
//! SSE's transport-layer auto-reconnect is the recovery path; there's no
|
||||||
|
//! server-side replay in v1.1.6.
|
||||||
|
//!
|
||||||
|
//! Channels are created lazily on first subscribe. A periodic GC task
|
||||||
|
//! ([`spawn_realtime_gc`]) drops senders whose receiver count has fallen
|
||||||
|
//! to zero so one-shot subscribers don't grow the map unboundedly.
|
||||||
|
//!
|
||||||
|
//! Cluster mode (v1.3+) swaps this for a Postgres `LISTEN/NOTIFY`-backed
|
||||||
|
//! resolver behind the same `RealtimeBroadcaster` trait.
|
||||||
|
|
||||||
|
use std::collections::HashMap;
|
||||||
|
use std::sync::{Arc, Mutex};
|
||||||
|
use std::time::Duration;
|
||||||
|
|
||||||
|
use async_trait::async_trait;
|
||||||
|
use picloud_shared::{AppId, BroadcasterError, RealtimeBroadcaster, RealtimeEvent};
|
||||||
|
use tokio::sync::broadcast;
|
||||||
|
|
||||||
|
/// Default per-channel broadcast buffer depth.
|
||||||
|
pub const DEFAULT_BROADCAST_CAPACITY: usize = 64;
|
||||||
|
const ENV_CAPACITY: &str = "PICLOUD_REALTIME_BROADCAST_CAPACITY";
|
||||||
|
|
||||||
|
/// Default GC sweep interval for empty channels.
|
||||||
|
pub const DEFAULT_GC_INTERVAL_SECS: u64 = 60;
|
||||||
|
|
||||||
|
pub struct InProcessBroadcaster {
|
||||||
|
inner: Mutex<HashMap<(AppId, String), broadcast::Sender<RealtimeEvent>>>,
|
||||||
|
capacity: usize,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl InProcessBroadcaster {
|
||||||
|
#[must_use]
|
||||||
|
pub fn new(capacity: usize) -> Self {
|
||||||
|
Self {
|
||||||
|
inner: Mutex::new(HashMap::new()),
|
||||||
|
capacity: capacity.max(1),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Build from `PICLOUD_REALTIME_BROADCAST_CAPACITY` (default 64).
|
||||||
|
#[must_use]
|
||||||
|
pub fn from_env() -> Self {
|
||||||
|
let capacity = match std::env::var(ENV_CAPACITY) {
|
||||||
|
Err(_) => DEFAULT_BROADCAST_CAPACITY,
|
||||||
|
Ok(v) => match v.parse::<usize>() {
|
||||||
|
Ok(n) if n > 0 => n,
|
||||||
|
Ok(_) => {
|
||||||
|
tracing::warn!(env = ENV_CAPACITY, value = %v, "must be > 0; using default");
|
||||||
|
DEFAULT_BROADCAST_CAPACITY
|
||||||
|
}
|
||||||
|
Err(e) => {
|
||||||
|
tracing::warn!(env = ENV_CAPACITY, value = %v, error = %e, "invalid; using default");
|
||||||
|
DEFAULT_BROADCAST_CAPACITY
|
||||||
|
}
|
||||||
|
},
|
||||||
|
};
|
||||||
|
Self::new(capacity)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Number of live channels in the map (test/observability helper).
|
||||||
|
#[must_use]
|
||||||
|
pub fn channel_count(&self) -> usize {
|
||||||
|
self.inner.lock().map(|g| g.len()).unwrap_or(0)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Drop senders with zero receivers. Returns how many were removed.
|
||||||
|
/// Called periodically by [`spawn_realtime_gc`].
|
||||||
|
pub fn gc(&self) -> usize {
|
||||||
|
let Ok(mut g) = self.inner.lock() else {
|
||||||
|
return 0;
|
||||||
|
};
|
||||||
|
let before = g.len();
|
||||||
|
g.retain(|_, tx| tx.receiver_count() > 0);
|
||||||
|
before - g.len()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[async_trait]
|
||||||
|
impl RealtimeBroadcaster for InProcessBroadcaster {
|
||||||
|
async fn subscribe(
|
||||||
|
&self,
|
||||||
|
app_id: AppId,
|
||||||
|
topic: &str,
|
||||||
|
) -> Result<broadcast::Receiver<RealtimeEvent>, BroadcasterError> {
|
||||||
|
let mut g = self
|
||||||
|
.inner
|
||||||
|
.lock()
|
||||||
|
.map_err(|_| BroadcasterError::Unavailable("broadcaster map poisoned".into()))?;
|
||||||
|
let tx = g
|
||||||
|
.entry((app_id, topic.to_string()))
|
||||||
|
.or_insert_with(|| broadcast::channel(self.capacity).0);
|
||||||
|
Ok(tx.subscribe())
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn publish(&self, app_id: AppId, topic: &str, event: RealtimeEvent) {
|
||||||
|
let Ok(g) = self.inner.lock() else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
// Only fan out to an existing channel: a topic with no live
|
||||||
|
// subscribers has no sender (publish never creates one). `send`
|
||||||
|
// returns Err iff every receiver has dropped — a benign no-op.
|
||||||
|
if let Some(tx) = g.get(&(app_id, topic.to_string())) {
|
||||||
|
let _ = tx.send(event);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn drop_topic(&self, app_id: AppId, topic: &str) {
|
||||||
|
if let Ok(mut g) = self.inner.lock() {
|
||||||
|
// Removing the sender closes the channel; existing receivers
|
||||||
|
// observe `RecvError::Closed` and disconnect cleanly.
|
||||||
|
g.remove(&(app_id, topic.to_string()));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Spawn the background GC sweep that drops empty channels every
|
||||||
|
/// `interval_secs` (default [`DEFAULT_GC_INTERVAL_SECS`]). Spawned at
|
||||||
|
/// startup alongside the other housekeeping tasks.
|
||||||
|
pub fn spawn_realtime_gc(broadcaster: Arc<InProcessBroadcaster>, interval_secs: u64) {
|
||||||
|
let period = Duration::from_secs(interval_secs.max(1));
|
||||||
|
tokio::spawn(async move {
|
||||||
|
let mut ticker = tokio::time::interval(period);
|
||||||
|
ticker.tick().await; // skip the immediate first fire
|
||||||
|
loop {
|
||||||
|
ticker.tick().await;
|
||||||
|
let removed = broadcaster.gc();
|
||||||
|
if removed > 0 {
|
||||||
|
tracing::debug!(removed, "realtime broadcaster GC dropped empty channels");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
use chrono::Utc;
|
||||||
|
use serde_json::json;
|
||||||
|
|
||||||
|
fn event(topic: &str, n: i64) -> RealtimeEvent {
|
||||||
|
RealtimeEvent {
|
||||||
|
topic: topic.to_string(),
|
||||||
|
message: json!({ "n": n }),
|
||||||
|
published_at: Utc::now(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn multiple_subscribers_each_receive_each_event() {
|
||||||
|
let b = InProcessBroadcaster::new(16);
|
||||||
|
let app = AppId::new();
|
||||||
|
let mut rx1 = b.subscribe(app, "chat").await.unwrap();
|
||||||
|
let mut rx2 = b.subscribe(app, "chat").await.unwrap();
|
||||||
|
|
||||||
|
b.publish(app, "chat", event("chat", 1)).await;
|
||||||
|
b.publish(app, "chat", event("chat", 2)).await;
|
||||||
|
|
||||||
|
for rx in [&mut rx1, &mut rx2] {
|
||||||
|
assert_eq!(rx.recv().await.unwrap().message, json!({ "n": 1 }));
|
||||||
|
assert_eq!(rx.recv().await.unwrap().message, json!({ "n": 2 }));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn dropped_subscriber_does_not_leak_after_gc() {
|
||||||
|
let b = InProcessBroadcaster::new(16);
|
||||||
|
let app = AppId::new();
|
||||||
|
let rx = b.subscribe(app, "t").await.unwrap();
|
||||||
|
assert_eq!(b.channel_count(), 1);
|
||||||
|
drop(rx);
|
||||||
|
// GC reclaims the now-empty channel.
|
||||||
|
assert_eq!(b.gc(), 1);
|
||||||
|
assert_eq!(b.channel_count(), 0);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn drop_topic_disconnects_existing_subscribers() {
|
||||||
|
let b = InProcessBroadcaster::new(16);
|
||||||
|
let app = AppId::new();
|
||||||
|
let mut rx = b.subscribe(app, "t").await.unwrap();
|
||||||
|
b.drop_topic(app, "t").await;
|
||||||
|
// Sender gone → receiver observes a closed channel.
|
||||||
|
assert!(rx.recv().await.is_err());
|
||||||
|
assert_eq!(b.channel_count(), 0);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn slow_consumer_loses_oldest_events() {
|
||||||
|
// Capacity 2: a consumer that never drains sees the oldest
|
||||||
|
// events dropped (broadcast Lagged semantics).
|
||||||
|
let b = InProcessBroadcaster::new(2);
|
||||||
|
let app = AppId::new();
|
||||||
|
let mut rx = b.subscribe(app, "t").await.unwrap();
|
||||||
|
for i in 0..5 {
|
||||||
|
b.publish(app, "t", event("t", i)).await;
|
||||||
|
}
|
||||||
|
// First recv reports the lag rather than event 0.
|
||||||
|
let first = rx.recv().await;
|
||||||
|
assert!(
|
||||||
|
matches!(first, Err(broadcast::error::RecvError::Lagged(_))),
|
||||||
|
"expected Lagged, got {first:?}"
|
||||||
|
);
|
||||||
|
// Subsequent recvs return the most recent buffered events.
|
||||||
|
let next = rx.recv().await.unwrap();
|
||||||
|
assert_eq!(next.message, json!({ "n": 3 }));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn cross_app_isolation() {
|
||||||
|
let b = InProcessBroadcaster::new(16);
|
||||||
|
let app_a = AppId::new();
|
||||||
|
let app_b = AppId::new();
|
||||||
|
let mut rx_a = b.subscribe(app_a, "shared").await.unwrap();
|
||||||
|
let mut rx_b = b.subscribe(app_b, "shared").await.unwrap();
|
||||||
|
|
||||||
|
b.publish(app_a, "shared", event("shared", 1)).await;
|
||||||
|
// App B's subscriber must not see app A's publish.
|
||||||
|
assert_eq!(rx_a.recv().await.unwrap().message, json!({ "n": 1 }));
|
||||||
|
assert!(rx_b.try_recv().is_err());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn publish_with_no_subscribers_is_noop() {
|
||||||
|
let b = InProcessBroadcaster::new(16);
|
||||||
|
let app = AppId::new();
|
||||||
|
// No subscriber → no sender created → no panic, nothing fanned out.
|
||||||
|
b.publish(app, "ghost", event("ghost", 1)).await;
|
||||||
|
assert_eq!(b.channel_count(), 0);
|
||||||
|
}
|
||||||
|
}
|
||||||
407
crates/orchestrator-core/src/realtime_api.rs
Normal file
407
crates/orchestrator-core/src/realtime_api.rs
Normal file
@@ -0,0 +1,407 @@
|
|||||||
|
//! SSE realtime endpoint — `GET /realtime/topics/{topic}` (v1.1.6).
|
||||||
|
//!
|
||||||
|
//! This is a data-plane surface, deliberately NOT under `/api/`
|
||||||
|
//! (realtime is its own versioning surface per the path scheme). It is
|
||||||
|
//! merged at the router root by the `picloud` binary alongside
|
||||||
|
//! `/healthz`, `/version`, and the user-route fallback.
|
||||||
|
//!
|
||||||
|
//! Handshake:
|
||||||
|
//! 1. Resolve `Host` → `app_id` (two-phase dispatch). No app → 404.
|
||||||
|
//! 2. Extract the token from `Authorization: Bearer <t>` OR `?token=<t>`
|
||||||
|
//! (EventSource can't set custom headers, so the query form is the
|
||||||
|
//! browser-compatible path).
|
||||||
|
//! 3. Ask the injected [`RealtimeAuthority`]: missing/internal topic →
|
||||||
|
//! 404, bad/absent token on a token-gated topic → 401, otherwise OK.
|
||||||
|
//! 4. Acquire a `broadcast::Receiver` and stream events as SSE until
|
||||||
|
//! the client disconnects (dropping the receiver — the broadcaster
|
||||||
|
//! cleans up on its own).
|
||||||
|
//!
|
||||||
|
//! Heartbeats (`:` comment lines) keep idle proxies from closing the
|
||||||
|
//! connection; interval is `PICLOUD_REALTIME_HEARTBEAT_SEC` (default 30).
|
||||||
|
|
||||||
|
use std::sync::Arc;
|
||||||
|
use std::time::Duration;
|
||||||
|
|
||||||
|
use axum::extract::{Path, Query, State};
|
||||||
|
use axum::http::{HeaderMap, StatusCode};
|
||||||
|
use axum::response::sse::{Event, KeepAlive, Sse};
|
||||||
|
use axum::response::{IntoResponse, Response};
|
||||||
|
use axum::routing::get;
|
||||||
|
use axum::Router;
|
||||||
|
use picloud_shared::{RealtimeAuthority, RealtimeBroadcaster, SubscribeDenied};
|
||||||
|
use serde::Deserialize;
|
||||||
|
use tokio_stream::wrappers::BroadcastStream;
|
||||||
|
use tokio_stream::{Stream, StreamExt};
|
||||||
|
|
||||||
|
use crate::routing::AppDomainTable;
|
||||||
|
|
||||||
|
/// Default heartbeat interval (seconds) for idle SSE connections.
|
||||||
|
pub const DEFAULT_HEARTBEAT_SECS: u64 = 30;
|
||||||
|
const ENV_HEARTBEAT: &str = "PICLOUD_REALTIME_HEARTBEAT_SEC";
|
||||||
|
|
||||||
|
#[derive(Clone)]
|
||||||
|
pub struct RealtimeState {
|
||||||
|
/// Host → app_id resolver (shared with the rest of the data plane).
|
||||||
|
pub app_domains: Arc<AppDomainTable>,
|
||||||
|
pub broadcaster: Arc<dyn RealtimeBroadcaster>,
|
||||||
|
pub authority: Arc<dyn RealtimeAuthority>,
|
||||||
|
pub heartbeat: Duration,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl RealtimeState {
|
||||||
|
#[must_use]
|
||||||
|
pub fn new(
|
||||||
|
app_domains: Arc<AppDomainTable>,
|
||||||
|
broadcaster: Arc<dyn RealtimeBroadcaster>,
|
||||||
|
authority: Arc<dyn RealtimeAuthority>,
|
||||||
|
) -> Self {
|
||||||
|
Self {
|
||||||
|
app_domains,
|
||||||
|
broadcaster,
|
||||||
|
authority,
|
||||||
|
heartbeat: Duration::from_secs(heartbeat_secs_from_env()),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Read `PICLOUD_REALTIME_HEARTBEAT_SEC` (default 30, must be > 0).
|
||||||
|
#[must_use]
|
||||||
|
pub fn heartbeat_secs_from_env() -> u64 {
|
||||||
|
match std::env::var(ENV_HEARTBEAT) {
|
||||||
|
Err(_) => DEFAULT_HEARTBEAT_SECS,
|
||||||
|
Ok(v) => match v.parse::<u64>() {
|
||||||
|
Ok(n) if n > 0 => n,
|
||||||
|
_ => {
|
||||||
|
tracing::warn!(env = ENV_HEARTBEAT, value = %v, "invalid; using default");
|
||||||
|
DEFAULT_HEARTBEAT_SECS
|
||||||
|
}
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Router for the realtime SSE surface. Merged at the router root.
|
||||||
|
pub fn realtime_router(state: RealtimeState) -> Router {
|
||||||
|
Router::new()
|
||||||
|
.route("/realtime/topics/{topic}", get(sse_topic))
|
||||||
|
.with_state(state)
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, Deserialize)]
|
||||||
|
struct TokenQuery {
|
||||||
|
token: Option<String>,
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn sse_topic(
|
||||||
|
State(state): State<RealtimeState>,
|
||||||
|
Path(topic): Path<String>,
|
||||||
|
Query(q): Query<TokenQuery>,
|
||||||
|
headers: HeaderMap,
|
||||||
|
) -> Response {
|
||||||
|
// 1. Host → app.
|
||||||
|
let host = headers
|
||||||
|
.get("host")
|
||||||
|
.and_then(|h| h.to_str().ok())
|
||||||
|
.unwrap_or("");
|
||||||
|
let Some(app_id) = state.app_domains.resolve_app(host) else {
|
||||||
|
return not_found("no app claims this host");
|
||||||
|
};
|
||||||
|
|
||||||
|
// 2. Token: Authorization: Bearer <t> takes precedence, else ?token=.
|
||||||
|
let token = bearer_token(&headers).or(q.token);
|
||||||
|
|
||||||
|
// 3. Authorize.
|
||||||
|
match state
|
||||||
|
.authority
|
||||||
|
.authorize_subscribe(app_id, &topic, token.as_deref())
|
||||||
|
.await
|
||||||
|
{
|
||||||
|
Ok(()) => {}
|
||||||
|
Err(SubscribeDenied::NotFound) => return not_found("topic not found"),
|
||||||
|
Err(SubscribeDenied::Unauthorized) => return unauthorized(),
|
||||||
|
Err(SubscribeDenied::Backend(e)) => {
|
||||||
|
tracing::error!(error = %e, "realtime authority backend error");
|
||||||
|
return (
|
||||||
|
StatusCode::INTERNAL_SERVER_ERROR,
|
||||||
|
axum::Json(serde_json::json!({ "error": "internal error" })),
|
||||||
|
)
|
||||||
|
.into_response();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// 4. Subscribe + stream.
|
||||||
|
let rx = match state.broadcaster.subscribe(app_id, &topic).await {
|
||||||
|
Ok(rx) => rx,
|
||||||
|
Err(e) => {
|
||||||
|
tracing::error!(error = %e, "failed to acquire realtime subscription");
|
||||||
|
return (
|
||||||
|
StatusCode::INTERNAL_SERVER_ERROR,
|
||||||
|
axum::Json(serde_json::json!({ "error": "internal error" })),
|
||||||
|
)
|
||||||
|
.into_response();
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
let stream = event_stream(rx);
|
||||||
|
let sse =
|
||||||
|
Sse::new(stream).keep_alive(KeepAlive::new().interval(state.heartbeat).text("heartbeat"));
|
||||||
|
|
||||||
|
// Sse sets Content-Type: text/event-stream + Cache-Control: no-cache.
|
||||||
|
// Add X-Accel-Buffering: no so an intermediate nginx doesn't buffer
|
||||||
|
// the stream (ignored by other proxies). Connection management is
|
||||||
|
// hyper's concern (and is hop-by-hop on HTTP/1.1, server-managed on
|
||||||
|
// HTTP/2), so we don't set Connection ourselves.
|
||||||
|
let mut resp = sse.into_response();
|
||||||
|
resp.headers_mut().insert(
|
||||||
|
"X-Accel-Buffering",
|
||||||
|
axum::http::HeaderValue::from_static("no"),
|
||||||
|
);
|
||||||
|
resp
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Map the broadcast receiver into a stream of SSE events. Lagged
|
||||||
|
/// notifications (slow consumer) are skipped; a closed channel
|
||||||
|
/// (`drop_topic`, or all senders gone) ends the stream and the SSE
|
||||||
|
/// connection closes cleanly.
|
||||||
|
fn event_stream(
|
||||||
|
rx: tokio::sync::broadcast::Receiver<picloud_shared::RealtimeEvent>,
|
||||||
|
) -> impl Stream<Item = Result<Event, std::convert::Infallible>> {
|
||||||
|
BroadcastStream::new(rx).filter_map(|item| {
|
||||||
|
let ev = item.ok()?; // drop Lagged errors
|
||||||
|
let payload = serde_json::json!({
|
||||||
|
"topic": ev.topic,
|
||||||
|
"message": ev.message,
|
||||||
|
"published_at": ev.published_at.to_rfc3339(),
|
||||||
|
});
|
||||||
|
Some(Ok(Event::default().data(payload.to_string())))
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
fn bearer_token(headers: &HeaderMap) -> Option<String> {
|
||||||
|
let raw = headers
|
||||||
|
.get(axum::http::header::AUTHORIZATION)?
|
||||||
|
.to_str()
|
||||||
|
.ok()?;
|
||||||
|
raw.strip_prefix("Bearer ")
|
||||||
|
.map(|t| t.trim().to_string())
|
||||||
|
.filter(|t| !t.is_empty())
|
||||||
|
}
|
||||||
|
|
||||||
|
fn not_found(msg: &str) -> Response {
|
||||||
|
(
|
||||||
|
StatusCode::NOT_FOUND,
|
||||||
|
axum::Json(serde_json::json!({ "error": msg })),
|
||||||
|
)
|
||||||
|
.into_response()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn unauthorized() -> Response {
|
||||||
|
// Generic — never leaks which check failed.
|
||||||
|
(
|
||||||
|
StatusCode::UNAUTHORIZED,
|
||||||
|
axum::Json(serde_json::json!({ "error": "unauthorized" })),
|
||||||
|
)
|
||||||
|
.into_response()
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
use crate::realtime::InProcessBroadcaster;
|
||||||
|
use crate::routing::AppDomainTable;
|
||||||
|
use async_trait::async_trait;
|
||||||
|
use axum::body::Body;
|
||||||
|
use axum::http::Request;
|
||||||
|
use picloud_shared::{AppId, RealtimeEvent};
|
||||||
|
use tower::ServiceExt; // oneshot
|
||||||
|
|
||||||
|
/// Authority stub returning a fixed verdict.
|
||||||
|
struct StubAuthority(Result<(), SubscribeDenied>);
|
||||||
|
#[async_trait]
|
||||||
|
impl RealtimeAuthority for StubAuthority {
|
||||||
|
async fn authorize_subscribe(
|
||||||
|
&self,
|
||||||
|
_: AppId,
|
||||||
|
_: &str,
|
||||||
|
_: Option<&str>,
|
||||||
|
) -> Result<(), SubscribeDenied> {
|
||||||
|
self.0.clone()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// App-domain table that maps a fixed host to a fixed app.
|
||||||
|
fn domains(host: &str, app: AppId) -> Arc<AppDomainTable> {
|
||||||
|
use crate::routing::{parse_app_domain, CompiledAppDomain};
|
||||||
|
let d = parse_app_domain(host).unwrap();
|
||||||
|
let table = AppDomainTable::new();
|
||||||
|
table.replace(vec![CompiledAppDomain {
|
||||||
|
app_id: app,
|
||||||
|
pattern: d.pattern,
|
||||||
|
shape_key: d.shape_key,
|
||||||
|
}]);
|
||||||
|
Arc::new(table)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn state(
|
||||||
|
app: AppId,
|
||||||
|
host: &str,
|
||||||
|
verdict: Result<(), SubscribeDenied>,
|
||||||
|
broadcaster: Arc<dyn RealtimeBroadcaster>,
|
||||||
|
) -> RealtimeState {
|
||||||
|
RealtimeState {
|
||||||
|
app_domains: domains(host, app),
|
||||||
|
broadcaster,
|
||||||
|
authority: Arc::new(StubAuthority(verdict)),
|
||||||
|
heartbeat: Duration::from_millis(100),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn get_status(state: RealtimeState, host: &str, topic: &str) -> StatusCode {
|
||||||
|
let app = realtime_router(state);
|
||||||
|
let req = Request::builder()
|
||||||
|
.uri(format!("/realtime/topics/{topic}"))
|
||||||
|
.header("host", host)
|
||||||
|
.body(Body::empty())
|
||||||
|
.unwrap();
|
||||||
|
app.oneshot(req).await.unwrap().status()
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn unknown_host_is_404() {
|
||||||
|
let app = AppId::new();
|
||||||
|
let st = state(
|
||||||
|
app,
|
||||||
|
"app.example.com",
|
||||||
|
Ok(()),
|
||||||
|
Arc::new(InProcessBroadcaster::new(8)),
|
||||||
|
);
|
||||||
|
// Request a different host → no app claims it.
|
||||||
|
assert_eq!(
|
||||||
|
get_status(st, "other.example.com", "chat").await,
|
||||||
|
StatusCode::NOT_FOUND
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn not_found_topic_is_404() {
|
||||||
|
let app = AppId::new();
|
||||||
|
let st = state(
|
||||||
|
app,
|
||||||
|
"app.example.com",
|
||||||
|
Err(SubscribeDenied::NotFound),
|
||||||
|
Arc::new(InProcessBroadcaster::new(8)),
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
get_status(st, "app.example.com", "ghost").await,
|
||||||
|
StatusCode::NOT_FOUND
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn unauthorized_token_is_401() {
|
||||||
|
let app = AppId::new();
|
||||||
|
let st = state(
|
||||||
|
app,
|
||||||
|
"app.example.com",
|
||||||
|
Err(SubscribeDenied::Unauthorized),
|
||||||
|
Arc::new(InProcessBroadcaster::new(8)),
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
get_status(st, "app.example.com", "chat").await,
|
||||||
|
StatusCode::UNAUTHORIZED
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn public_topic_returns_event_stream() {
|
||||||
|
let app = AppId::new();
|
||||||
|
let st = state(
|
||||||
|
app,
|
||||||
|
"app.example.com",
|
||||||
|
Ok(()),
|
||||||
|
Arc::new(InProcessBroadcaster::new(8)),
|
||||||
|
);
|
||||||
|
let appr = realtime_router(st);
|
||||||
|
let req = Request::builder()
|
||||||
|
.uri("/realtime/topics/chat")
|
||||||
|
.header("host", "app.example.com")
|
||||||
|
.body(Body::empty())
|
||||||
|
.unwrap();
|
||||||
|
let resp = appr.oneshot(req).await.unwrap();
|
||||||
|
assert_eq!(resp.status(), StatusCode::OK);
|
||||||
|
let ct = resp
|
||||||
|
.headers()
|
||||||
|
.get(axum::http::header::CONTENT_TYPE)
|
||||||
|
.unwrap()
|
||||||
|
.to_str()
|
||||||
|
.unwrap();
|
||||||
|
assert!(ct.starts_with("text/event-stream"));
|
||||||
|
assert_eq!(resp.headers().get("x-accel-buffering").unwrap(), "no");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn subscribe_receives_published_event() {
|
||||||
|
let app = AppId::new();
|
||||||
|
let broadcaster = Arc::new(InProcessBroadcaster::new(8));
|
||||||
|
let st = state(app, "app.example.com", Ok(()), broadcaster.clone());
|
||||||
|
let appr = realtime_router(st);
|
||||||
|
let req = Request::builder()
|
||||||
|
.uri("/realtime/topics/chat")
|
||||||
|
.header("host", "app.example.com")
|
||||||
|
.body(Body::empty())
|
||||||
|
.unwrap();
|
||||||
|
let resp = appr.oneshot(req).await.unwrap();
|
||||||
|
assert_eq!(resp.status(), StatusCode::OK);
|
||||||
|
|
||||||
|
// The handler has subscribed; publish and read the first chunk.
|
||||||
|
// Give the streaming task a beat to register its receiver.
|
||||||
|
let mut body = resp.into_body().into_data_stream();
|
||||||
|
tokio::time::sleep(Duration::from_millis(50)).await;
|
||||||
|
broadcaster
|
||||||
|
.publish(
|
||||||
|
app,
|
||||||
|
"chat",
|
||||||
|
RealtimeEvent {
|
||||||
|
topic: "chat".into(),
|
||||||
|
message: serde_json::json!({ "hi": 1 }),
|
||||||
|
published_at: chrono::Utc::now(),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
.await;
|
||||||
|
|
||||||
|
let chunk = tokio::time::timeout(Duration::from_secs(2), body.next())
|
||||||
|
.await
|
||||||
|
.expect("a chunk within timeout")
|
||||||
|
.expect("stream item")
|
||||||
|
.expect("chunk ok");
|
||||||
|
let text = String::from_utf8_lossy(&chunk);
|
||||||
|
assert!(text.contains("data:"), "got: {text}");
|
||||||
|
assert!(text.contains("\"hi\":1"), "got: {text}");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn heartbeat_fires_on_idle_connection() {
|
||||||
|
let app = AppId::new();
|
||||||
|
let broadcaster = Arc::new(InProcessBroadcaster::new(8));
|
||||||
|
// Hold a clone so the channel's sender outlives the router (which
|
||||||
|
// oneshot consumes) — otherwise the stream closes immediately.
|
||||||
|
let _keepalive = broadcaster.clone();
|
||||||
|
let st = state(app, "app.example.com", Ok(()), broadcaster);
|
||||||
|
let appr = realtime_router(st);
|
||||||
|
let req = Request::builder()
|
||||||
|
.uri("/realtime/topics/chat")
|
||||||
|
.header("host", "app.example.com")
|
||||||
|
.body(Body::empty())
|
||||||
|
.unwrap();
|
||||||
|
let resp = appr.oneshot(req).await.unwrap();
|
||||||
|
let mut body = resp.into_body().into_data_stream();
|
||||||
|
// No publish — with a 100ms heartbeat, a keep-alive comment must
|
||||||
|
// arrive well within a second.
|
||||||
|
let chunk = tokio::time::timeout(Duration::from_secs(1), body.next())
|
||||||
|
.await
|
||||||
|
.expect("heartbeat within timeout")
|
||||||
|
.expect("stream item")
|
||||||
|
.expect("chunk ok");
|
||||||
|
let text = String::from_utf8_lossy(&chunk);
|
||||||
|
assert!(text.starts_with(':'), "expected SSE comment, got: {text}");
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -41,3 +41,7 @@ serde.workspace = true
|
|||||||
serde_json.workspace = true
|
serde_json.workspace = true
|
||||||
uuid.workspace = true
|
uuid.workspace = true
|
||||||
chrono.workspace = true
|
chrono.workspace = true
|
||||||
|
# Compute inbound-email HMAC signatures in the e2e receiver tests.
|
||||||
|
hmac.workspace = true
|
||||||
|
sha2.workspace = true
|
||||||
|
hex.workspace = true
|
||||||
|
|||||||
@@ -12,28 +12,34 @@ use picloud_executor_core::{Engine, Limits};
|
|||||||
use picloud_manager_core::{
|
use picloud_manager_core::{
|
||||||
admin_router, admins_router, api_keys_router, app_members_router, apps_api, apps_router,
|
admin_router, admins_router, api_keys_router, app_members_router, apps_api, apps_router,
|
||||||
attach_principal_if_present, auth_router, compile_routes, dead_letters_router,
|
attach_principal_if_present, auth_router, compile_routes, dead_letters_router,
|
||||||
files_admin_router, migrations, require_authenticated, route_admin_router, triggers_router,
|
email_inbound_router, files_admin_router, migrations, require_authenticated,
|
||||||
AbandonedRepo, AdminPrincipalResolver, AdminSessionRepository, AdminState, AdminUserRepository,
|
route_admin_router, secrets_router, topics_router, triggers_router, AbandonedRepo,
|
||||||
AdminsState, ApiKeyRepository, ApiKeysState, AppDomainRepository, AppMembersRepository,
|
AdminPrincipalResolver, AdminSessionRepository, AdminState, AdminUserRepository, AdminsState,
|
||||||
AppMembersState, AppRepository, AppsState, AuthState, AuthzRepo, DeadLetterRepo,
|
ApiKeyRepository, ApiKeysState, AppDomainRepository, AppMembersRepository, AppMembersState,
|
||||||
DeadLettersState, Dispatcher, DocsServiceImpl, FilesAdminState, FilesConfig, FilesServiceImpl,
|
AppRepository, AppsState, AuthState, AuthzRepo, DeadLetterRepo, DeadLettersState, Dispatcher,
|
||||||
FsFilesRepo, HttpConfig, HttpServiceImpl, KvServiceImpl, OutboxEventEmitter, OutboxRepo,
|
DocsServiceImpl, EmailInboundState, EmailServiceImpl, FilesAdminState, FilesConfig,
|
||||||
PostgresAbandonedRepo, PostgresAdminSessionRepository, PostgresAdminUserRepository,
|
FilesServiceImpl, FsFilesRepo, HttpConfig, HttpServiceImpl, KvServiceImpl, OutboxEventEmitter,
|
||||||
|
OutboxRepo, PostgresAbandonedRepo, PostgresAdminSessionRepository, PostgresAdminUserRepository,
|
||||||
PostgresApiKeyRepository, PostgresAppDomainRepository, PostgresAppMembersRepository,
|
PostgresApiKeyRepository, PostgresAppDomainRepository, PostgresAppMembersRepository,
|
||||||
PostgresAppRepository, PostgresDeadLetterRepo, PostgresDeadLetterService, PostgresDocsRepo,
|
PostgresAppRepository, PostgresAppSecretsRepo, PostgresDeadLetterRepo,
|
||||||
PostgresExecutionLogRepository, PostgresExecutionLogSink, PostgresKvRepo, PostgresOutboxRepo,
|
PostgresDeadLetterService, PostgresDocsRepo, PostgresExecutionLogRepository,
|
||||||
PostgresPubsubRepo, PostgresRouteRepository, PostgresScriptRepository, PostgresTriggerRepo,
|
PostgresExecutionLogSink, PostgresKvRepo, PostgresOutboxRepo, PostgresPubsubRepo,
|
||||||
PrincipalResolver, PubsubServiceImpl, RepoResolver, RouteAdminState, RouteRepository,
|
PostgresRouteRepository, PostgresScriptRepository, PostgresSecretsRepo, PostgresTopicRepo,
|
||||||
SandboxCeiling, ScriptRepository, TriggerConfig, TriggerRepo, TriggersState,
|
PostgresTriggerRepo, PrincipalResolver, PubsubServiceImpl, RealtimeAuthorityImpl, RepoResolver,
|
||||||
|
RouteAdminState, RouteRepository, SandboxCeiling, ScriptRepository, SecretsConfig,
|
||||||
|
SecretsServiceImpl, SecretsState, SubscriberTokenConfig, TopicRepo, TopicsState, TriggerConfig,
|
||||||
|
TriggerRepo, TriggersState,
|
||||||
};
|
};
|
||||||
|
use picloud_orchestrator_core::realtime::DEFAULT_GC_INTERVAL_SECS;
|
||||||
use picloud_orchestrator_core::routing::{AppDomainTable, RouteTable};
|
use picloud_orchestrator_core::routing::{AppDomainTable, RouteTable};
|
||||||
use picloud_orchestrator_core::{
|
use picloud_orchestrator_core::{
|
||||||
data_plane_router, user_routes_router, DataPlaneState, ExecutionGate, InboxRegistry,
|
data_plane_router, realtime_router, spawn_realtime_gc, user_routes_router, DataPlaneState,
|
||||||
LocalExecutorClient,
|
ExecutionGate, InProcessBroadcaster, InboxRegistry, LocalExecutorClient, RealtimeState,
|
||||||
};
|
};
|
||||||
use picloud_shared::{
|
use picloud_shared::{
|
||||||
DeadLetterService, DocsService, ExecutionLogSink, FilesService, HttpService, InboxResolver,
|
DeadLetterService, DocsService, EmailService, ExecutionLogSink, FilesService, HttpService,
|
||||||
KvService, OutboxWriter, PubsubService, ScriptValidator, ServiceEventEmitter, Services,
|
InboxResolver, KvService, MasterKey, OutboxWriter, PubsubService, RealtimeAuthority,
|
||||||
|
RealtimeBroadcaster, ScriptValidator, SecretsService, ServiceEventEmitter, Services,
|
||||||
API_VERSION, PRODUCT_VERSION, SDK_VERSION, WIRE_VERSION,
|
API_VERSION, PRODUCT_VERSION, SDK_VERSION, WIRE_VERSION,
|
||||||
};
|
};
|
||||||
use sqlx::postgres::PgPoolOptions;
|
use sqlx::postgres::PgPoolOptions;
|
||||||
@@ -90,7 +96,11 @@ fn read_session_ttl() -> Duration {
|
|||||||
/// (`/api/v1/execute/{id}`, the user-route fallthrough, `/healthz`,
|
/// (`/api/v1/execute/{id}`, the user-route fallthrough, `/healthz`,
|
||||||
/// `/version`) stays open — it's the public ingress for user scripts.
|
/// `/version`) stays open — it's the public ingress for user scripts.
|
||||||
#[allow(clippy::too_many_lines)]
|
#[allow(clippy::too_many_lines)]
|
||||||
pub async fn build_app(pool: PgPool, auth: AuthDeps) -> anyhow::Result<Router> {
|
pub async fn build_app(
|
||||||
|
pool: PgPool,
|
||||||
|
auth: AuthDeps,
|
||||||
|
master_key: MasterKey,
|
||||||
|
) -> anyhow::Result<Router> {
|
||||||
let script_repo = Arc::new(PostgresScriptRepository::new(pool.clone()));
|
let script_repo = Arc::new(PostgresScriptRepository::new(pool.clone()));
|
||||||
let log_repo = Arc::new(PostgresExecutionLogRepository::new(pool.clone()));
|
let log_repo = Arc::new(PostgresExecutionLogRepository::new(pool.clone()));
|
||||||
let log_sink: Arc<dyn ExecutionLogSink> = Arc::new(PostgresExecutionLogSink::new(pool.clone()));
|
let log_sink: Arc<dyn ExecutionLogSink> = Arc::new(PostgresExecutionLogSink::new(pool.clone()));
|
||||||
@@ -162,6 +172,8 @@ pub async fn build_app(pool: PgPool, auth: AuthDeps) -> anyhow::Result<Router> {
|
|||||||
// the bytes live on disk under `PICLOUD_FILES_ROOT` (default ./data).
|
// the bytes live on disk under `PICLOUD_FILES_ROOT` (default ./data).
|
||||||
let files_config = FilesConfig::from_env();
|
let files_config = FilesConfig::from_env();
|
||||||
let files_max_size = files_config.max_file_size_bytes;
|
let files_max_size = files_config.max_file_size_bytes;
|
||||||
|
// Kept for the v1.1.6 orphan sweeper (cleans stale `*.tmp.*` files).
|
||||||
|
let files_root = files_config.root.clone();
|
||||||
let files_repo = Arc::new(FsFilesRepo::new(pool.clone(), files_config));
|
let files_repo = Arc::new(FsFilesRepo::new(pool.clone(), files_config));
|
||||||
let files: Arc<dyn FilesService> = Arc::new(FilesServiceImpl::new(
|
let files: Arc<dyn FilesService> = Arc::new(FilesServiceImpl::new(
|
||||||
files_repo.clone(),
|
files_repo.clone(),
|
||||||
@@ -169,12 +181,69 @@ pub async fn build_app(pool: PgPool, auth: AuthDeps) -> anyhow::Result<Router> {
|
|||||||
events.clone(),
|
events.clone(),
|
||||||
files_max_size,
|
files_max_size,
|
||||||
));
|
));
|
||||||
// v1.1.5 durable pub/sub. Publishes fan out to matching pubsub
|
// v1.1.6 realtime: the in-process broadcaster is shared between the
|
||||||
// triggers at publish time (one outbox row each), delivered by the
|
// publish path (PubsubServiceImpl fans out to SSE subscribers after
|
||||||
// same dispatcher as every other async trigger.
|
// the durable outbox fan-out) and the SSE endpoint (subscribe side).
|
||||||
|
// The topic registry + app-secrets repo back the subscriber-token
|
||||||
|
// mint + SSE subscribe-authorization.
|
||||||
|
let broadcaster_concrete = Arc::new(InProcessBroadcaster::from_env());
|
||||||
|
let broadcaster: Arc<dyn RealtimeBroadcaster> = broadcaster_concrete.clone();
|
||||||
|
let topic_repo: Arc<dyn TopicRepo> = Arc::new(PostgresTopicRepo::new(pool.clone()));
|
||||||
|
let app_secrets_repo = Arc::new(PostgresAppSecretsRepo::new(
|
||||||
|
pool.clone(),
|
||||||
|
master_key.clone(),
|
||||||
|
));
|
||||||
|
// v1.1.7 two-phase migration: encrypt any plaintext realtime signing
|
||||||
|
// keys at rest. Idempotent — only touches rows not yet encrypted. The
|
||||||
|
// plaintext column is dropped in v1.1.8.
|
||||||
|
match app_secrets_repo.migrate_plaintext_keys().await {
|
||||||
|
Ok(0) => {}
|
||||||
|
Ok(n) => {
|
||||||
|
tracing::info!(
|
||||||
|
migrated = n,
|
||||||
|
"encrypted plaintext realtime signing keys at rest"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
Err(e) => {
|
||||||
|
tracing::error!(error = %e, "failed to encrypt realtime signing keys (continuing)");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let realtime_authority: Arc<dyn RealtimeAuthority> = Arc::new(RealtimeAuthorityImpl::new(
|
||||||
|
topic_repo.clone(),
|
||||||
|
app_secrets_repo.clone(),
|
||||||
|
));
|
||||||
|
|
||||||
|
// v1.1.5 durable pub/sub, extended in v1.1.6 with the realtime
|
||||||
|
// broadcast + subscriber-token mint. Publishes fan out to matching
|
||||||
|
// pubsub triggers at publish time (one outbox row each, delivered by
|
||||||
|
// the same dispatcher as every other async trigger) AND, best-effort,
|
||||||
|
// to in-process SSE subscribers.
|
||||||
let pubsub_repo = Arc::new(PostgresPubsubRepo::new(pool.clone()));
|
let pubsub_repo = Arc::new(PostgresPubsubRepo::new(pool.clone()));
|
||||||
let pubsub: Arc<dyn PubsubService> =
|
let pubsub: Arc<dyn PubsubService> = Arc::new(
|
||||||
Arc::new(PubsubServiceImpl::new(pubsub_repo, authz.clone()));
|
PubsubServiceImpl::new(pubsub_repo, authz.clone()).with_realtime(
|
||||||
|
broadcaster.clone(),
|
||||||
|
topic_repo.clone(),
|
||||||
|
app_secrets_repo,
|
||||||
|
SubscriberTokenConfig::from_env(),
|
||||||
|
),
|
||||||
|
);
|
||||||
|
// v1.1.7 encrypted per-app secrets. Values are AES-256-GCM-sealed
|
||||||
|
// with the process master key before they touch Postgres; the repo
|
||||||
|
// only ever sees ciphertext + nonce. The admin surface reuses the
|
||||||
|
// same repo + master key (see `secrets_state` below).
|
||||||
|
let secrets_config = SecretsConfig::from_env();
|
||||||
|
let secrets_max_value_bytes = secrets_config.max_value_bytes;
|
||||||
|
let secrets_repo: Arc<dyn picloud_manager_core::SecretsRepo> =
|
||||||
|
Arc::new(PostgresSecretsRepo::new(pool.clone()));
|
||||||
|
let secrets: Arc<dyn SecretsService> = Arc::new(SecretsServiceImpl::new(
|
||||||
|
secrets_repo.clone(),
|
||||||
|
authz.clone(),
|
||||||
|
master_key.clone(),
|
||||||
|
secrets_config,
|
||||||
|
));
|
||||||
|
// v1.1.7 outbound email. Builds a lettre SMTP transport from
|
||||||
|
// PICLOUD_SMTP_* env (disabled mode + warning if unconfigured).
|
||||||
|
let email: Arc<dyn EmailService> = Arc::new(EmailServiceImpl::from_env(authz.clone()));
|
||||||
let services = Services::new(
|
let services = Services::new(
|
||||||
kv,
|
kv,
|
||||||
docs,
|
docs,
|
||||||
@@ -184,6 +253,8 @@ pub async fn build_app(pool: PgPool, auth: AuthDeps) -> anyhow::Result<Router> {
|
|||||||
http,
|
http,
|
||||||
files,
|
files,
|
||||||
pubsub,
|
pubsub,
|
||||||
|
secrets,
|
||||||
|
email,
|
||||||
);
|
);
|
||||||
let engine = Arc::new(Engine::new(Limits::default(), services));
|
let engine = Arc::new(Engine::new(Limits::default(), services));
|
||||||
|
|
||||||
@@ -284,12 +355,24 @@ pub async fn build_app(pool: PgPool, auth: AuthDeps) -> anyhow::Result<Router> {
|
|||||||
// enqueues due triggers into the outbox; the dispatcher above
|
// enqueues due triggers into the outbox; the dispatcher above
|
||||||
// delivers them like any other async trigger.
|
// delivers them like any other async trigger.
|
||||||
picloud_manager_core::spawn_cron_scheduler(pool, trigger_config.cron_tick_interval_ms);
|
picloud_manager_core::spawn_cron_scheduler(pool, trigger_config.cron_tick_interval_ms);
|
||||||
|
// v1.1.6: GC empty realtime broadcast channels (one-shot subscribers)
|
||||||
|
// and sweep orphaned `*.tmp.*` blobs left by crashed file writes.
|
||||||
|
spawn_realtime_gc(broadcaster_concrete, DEFAULT_GC_INTERVAL_SECS);
|
||||||
|
picloud_manager_core::spawn_files_orphan_sweep(files_root);
|
||||||
let triggers_state = TriggersState {
|
let triggers_state = TriggersState {
|
||||||
triggers: trigger_repo,
|
triggers: trigger_repo.clone(),
|
||||||
apps: apps_repo.clone(),
|
apps: apps_repo.clone(),
|
||||||
authz: authz.clone(),
|
authz: authz.clone(),
|
||||||
scripts: Arc::new(PostgresScriptRepoHandle(script_repo.clone())),
|
scripts: Arc::new(PostgresScriptRepoHandle(script_repo.clone())),
|
||||||
config: trigger_config,
|
config: trigger_config,
|
||||||
|
master_key: master_key.clone(),
|
||||||
|
};
|
||||||
|
// v1.1.7 public inbound-email receiver. Outside the admin auth layer
|
||||||
|
// (the URL + per-trigger HMAC secret are the security boundary).
|
||||||
|
let email_inbound_state = EmailInboundState {
|
||||||
|
triggers: trigger_repo,
|
||||||
|
outbox: outbox_repo.clone(),
|
||||||
|
master_key: master_key.clone(),
|
||||||
};
|
};
|
||||||
let dead_letters_state = DeadLettersState {
|
let dead_letters_state = DeadLettersState {
|
||||||
repo: dl_repo,
|
repo: dl_repo,
|
||||||
@@ -302,11 +385,24 @@ pub async fn build_app(pool: PgPool, auth: AuthDeps) -> anyhow::Result<Router> {
|
|||||||
apps: apps_repo.clone(),
|
apps: apps_repo.clone(),
|
||||||
authz: authz.clone(),
|
authz: authz.clone(),
|
||||||
};
|
};
|
||||||
|
let topics_state = TopicsState {
|
||||||
|
topics: topic_repo,
|
||||||
|
apps: apps_repo.clone(),
|
||||||
|
authz: authz.clone(),
|
||||||
|
broadcaster: broadcaster.clone(),
|
||||||
|
};
|
||||||
|
let secrets_state = SecretsState {
|
||||||
|
repo: secrets_repo,
|
||||||
|
apps: apps_repo.clone(),
|
||||||
|
authz: authz.clone(),
|
||||||
|
master_key,
|
||||||
|
max_value_bytes: secrets_max_value_bytes,
|
||||||
|
};
|
||||||
let apps_state = AppsState {
|
let apps_state = AppsState {
|
||||||
apps: apps_repo,
|
apps: apps_repo,
|
||||||
domains: domains_repo,
|
domains: domains_repo,
|
||||||
routes: route_repo,
|
routes: route_repo,
|
||||||
domain_table: app_domain_table,
|
domain_table: app_domain_table.clone(),
|
||||||
authz: authz.clone(),
|
authz: authz.clone(),
|
||||||
};
|
};
|
||||||
|
|
||||||
@@ -345,6 +441,8 @@ pub async fn build_app(pool: PgPool, auth: AuthDeps) -> anyhow::Result<Router> {
|
|||||||
.merge(api_keys_router(api_keys_state))
|
.merge(api_keys_router(api_keys_state))
|
||||||
.merge(triggers_router(triggers_state))
|
.merge(triggers_router(triggers_state))
|
||||||
.merge(files_admin_router(files_admin_state))
|
.merge(files_admin_router(files_admin_state))
|
||||||
|
.merge(topics_router(topics_state))
|
||||||
|
.merge(secrets_router(secrets_state))
|
||||||
.merge(dead_letters_router(dead_letters_state))
|
.merge(dead_letters_router(dead_letters_state))
|
||||||
.layer(from_fn_with_state(
|
.layer(from_fn_with_state(
|
||||||
auth_state.clone(),
|
auth_state.clone(),
|
||||||
@@ -373,12 +471,24 @@ pub async fn build_app(pool: PgPool, auth: AuthDeps) -> anyhow::Result<Router> {
|
|||||||
let api_v1 = Router::new()
|
let api_v1 = Router::new()
|
||||||
.nest("/admin", auth_router(auth_state))
|
.nest("/admin", auth_router(auth_state))
|
||||||
.nest("/admin", guarded_admin)
|
.nest("/admin", guarded_admin)
|
||||||
|
.merge(email_inbound_router(email_inbound_state))
|
||||||
.merge(data_plane_routed);
|
.merge(data_plane_routed);
|
||||||
|
|
||||||
|
// v1.1.6 SSE realtime surface, merged at the root (deliberately NOT
|
||||||
|
// under /api/ — realtime is its own versioning surface). Public auth
|
||||||
|
// is per-topic; no principal middleware (token verification is the
|
||||||
|
// gate, handled inside the authority).
|
||||||
|
let realtime = realtime_router(RealtimeState::new(
|
||||||
|
app_domain_table,
|
||||||
|
broadcaster,
|
||||||
|
realtime_authority,
|
||||||
|
));
|
||||||
|
|
||||||
Ok(Router::new()
|
Ok(Router::new()
|
||||||
.route("/healthz", get(healthz))
|
.route("/healthz", get(healthz))
|
||||||
.route("/version", get(version))
|
.route("/version", get(version))
|
||||||
.nest(&format!("/api/v{API_VERSION}"), api_v1)
|
.nest(&format!("/api/v{API_VERSION}"), api_v1)
|
||||||
|
.merge(realtime)
|
||||||
.merge(user_routes)
|
.merge(user_routes)
|
||||||
.layer(TraceLayer::new_for_http()))
|
.layer(TraceLayer::new_for_http()))
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -39,6 +39,11 @@ async fn run_server() -> anyhow::Result<()> {
|
|||||||
let database_url =
|
let database_url =
|
||||||
std::env::var("DATABASE_URL").map_err(|_| anyhow::anyhow!("DATABASE_URL is required"))?;
|
std::env::var("DATABASE_URL").map_err(|_| anyhow::anyhow!("DATABASE_URL is required"))?;
|
||||||
|
|
||||||
|
// Source the process master key BEFORE doing any work — an unset or
|
||||||
|
// malformed PICLOUD_SECRET_KEY is fatal (v1.1.7). The only escape
|
||||||
|
// hatch is PICLOUD_DEV_MODE=true, which logs a prominent warning.
|
||||||
|
let master_key = picloud_shared::MasterKey::from_env()?;
|
||||||
|
|
||||||
let pool = init_db(&database_url).await?;
|
let pool = init_db(&database_url).await?;
|
||||||
migrations::run(&pool).await?;
|
migrations::run(&pool).await?;
|
||||||
tracing::info!("migrations applied");
|
tracing::info!("migrations applied");
|
||||||
@@ -69,7 +74,7 @@ async fn run_server() -> anyhow::Result<()> {
|
|||||||
// so a delayed sweep can't extend session lifetimes.
|
// so a delayed sweep can't extend session lifetimes.
|
||||||
spawn_session_pruner(auth.sessions.clone());
|
spawn_session_pruner(auth.sessions.clone());
|
||||||
|
|
||||||
let app = build_app(pool, auth).await?;
|
let app = build_app(pool, auth, master_key).await?;
|
||||||
|
|
||||||
let listener = tokio::net::TcpListener::bind(addr).await?;
|
let listener = tokio::net::TcpListener::bind(addr).await?;
|
||||||
tracing::info!(%addr, "picloud all-in-one listening");
|
tracing::info!(%addr, "picloud all-in-one listening");
|
||||||
|
|||||||
@@ -40,7 +40,13 @@ async fn server_with_app(pool: PgPool) -> (TestServer, String) {
|
|||||||
.await
|
.await
|
||||||
.expect("seed admin");
|
.expect("seed admin");
|
||||||
|
|
||||||
let app = picloud::build_app(pool, auth).await.expect("build_app");
|
let app = picloud::build_app(
|
||||||
|
pool,
|
||||||
|
auth,
|
||||||
|
picloud_shared::MasterKey::from_bytes([0x42u8; 32]),
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
.expect("build_app");
|
||||||
let mut server = TestServer::new(app).expect("TestServer should build");
|
let mut server = TestServer::new(app).expect("TestServer should build");
|
||||||
|
|
||||||
let resp = server
|
let resp = server
|
||||||
|
|||||||
@@ -57,7 +57,11 @@ async fn boot(pool: PgPool) -> Seeded {
|
|||||||
.await
|
.await
|
||||||
.expect("seed owner");
|
.expect("seed owner");
|
||||||
|
|
||||||
let app = picloud::build_app(pool.clone(), auth)
|
let app = picloud::build_app(
|
||||||
|
pool.clone(),
|
||||||
|
auth,
|
||||||
|
picloud_shared::MasterKey::from_bytes([0x42u8; 32]),
|
||||||
|
)
|
||||||
.await
|
.await
|
||||||
.expect("build_app");
|
.expect("build_app");
|
||||||
let server = TestServer::new(app).expect("TestServer");
|
let server = TestServer::new(app).expect("TestServer");
|
||||||
|
|||||||
489
crates/picloud/tests/dispatcher_e2e.rs
Normal file
489
crates/picloud/tests/dispatcher_e2e.rs
Normal file
@@ -0,0 +1,489 @@
|
|||||||
|
//! End-to-end dispatcher tests — one per trigger kind (v1.1.5 follow-up,
|
||||||
|
//! landed in v1.1.6). Each test wires the full all-in-one app via
|
||||||
|
//! `build_app` (which spawns the real dispatcher + cron scheduler +
|
||||||
|
//! executor), creates an app + a logging handler script + a trigger,
|
||||||
|
//! causes the originating event, and polls for the handler's side effect.
|
||||||
|
//!
|
||||||
|
//! ## Gating
|
||||||
|
//!
|
||||||
|
//! These need a Postgres reachable via `DATABASE_URL`. They follow the
|
||||||
|
//! `schema_snapshot` pattern (NOT `#[ignore]`): when `DATABASE_URL` is
|
||||||
|
//! unset the test prints a notice and returns early, so plain
|
||||||
|
//! `cargo test` stays green locally while CI (which sets `DATABASE_URL`)
|
||||||
|
//! runs them.
|
||||||
|
//!
|
||||||
|
//! ## How "the handler fired" is observed
|
||||||
|
//!
|
||||||
|
//! The dispatcher does not write `execution_log` rows for trigger
|
||||||
|
//! handlers, so each handler instead records its `ctx.event` into a KV
|
||||||
|
//! marker (`collection = "e2e_markers"`, which no trigger watches — no
|
||||||
|
//! recursion). The test polls `kv_entries` for that marker and asserts
|
||||||
|
//! the event shape. See HANDBACK §deviations for why this lives in
|
||||||
|
//! `picloud/tests/` rather than `manager-core/tests/` (build_app lives in
|
||||||
|
//! the `picloud` crate) and for the `dead_letter` reinterpretation.
|
||||||
|
|
||||||
|
#![allow(clippy::needless_pass_by_value)]
|
||||||
|
|
||||||
|
use std::time::Duration;
|
||||||
|
|
||||||
|
use axum_test::TestServer;
|
||||||
|
use serde_json::{json, Value};
|
||||||
|
use sqlx::postgres::PgPoolOptions;
|
||||||
|
use sqlx::PgPool;
|
||||||
|
use uuid::Uuid;
|
||||||
|
|
||||||
|
/// Connect + migrate, or return `None` (printing a skip notice) when
|
||||||
|
/// `DATABASE_URL` is unset — mirrors `schema_snapshot.rs`.
|
||||||
|
async fn pool_or_skip() -> Option<PgPool> {
|
||||||
|
let Ok(url) = std::env::var("DATABASE_URL") else {
|
||||||
|
eprintln!("dispatcher_e2e: DATABASE_URL unset — skipping");
|
||||||
|
return None;
|
||||||
|
};
|
||||||
|
let pool = PgPoolOptions::new()
|
||||||
|
.max_connections(5)
|
||||||
|
.connect(&url)
|
||||||
|
.await
|
||||||
|
.expect("connect to DATABASE_URL");
|
||||||
|
sqlx::migrate!("../manager-core/migrations")
|
||||||
|
.run(&pool)
|
||||||
|
.await
|
||||||
|
.expect("apply migrations");
|
||||||
|
Some(pool)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Build the app over the shared pool with a uniquely-named owner admin,
|
||||||
|
/// log in, and create a fresh app. `suffix` must be unique per test (the
|
||||||
|
/// pool is shared, so names must not collide).
|
||||||
|
async fn server_for(pool: PgPool, suffix: &str) -> (TestServer, String) {
|
||||||
|
use picloud_manager_core::auth::hash_password;
|
||||||
|
use picloud_shared::InstanceRole;
|
||||||
|
|
||||||
|
let unique = format!("{suffix}-{}", Uuid::new_v4().simple());
|
||||||
|
let auth = picloud::AuthDeps::from_pool(pool.clone());
|
||||||
|
let username = format!("e2e-{unique}");
|
||||||
|
let hash = hash_password("pw").expect("hash");
|
||||||
|
auth.users
|
||||||
|
.create(&username, &hash, InstanceRole::Owner, None)
|
||||||
|
.await
|
||||||
|
.expect("seed admin");
|
||||||
|
|
||||||
|
let app = picloud::build_app(
|
||||||
|
pool,
|
||||||
|
auth,
|
||||||
|
picloud_shared::MasterKey::from_bytes([0x42u8; 32]),
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
.expect("build_app");
|
||||||
|
let mut server = TestServer::new(app).expect("TestServer");
|
||||||
|
let resp = server
|
||||||
|
.post("/api/v1/admin/auth/login")
|
||||||
|
.json(&json!({ "username": username, "password": "pw" }))
|
||||||
|
.await;
|
||||||
|
resp.assert_status_ok();
|
||||||
|
let token = resp.json::<Value>()["token"]
|
||||||
|
.as_str()
|
||||||
|
.expect("login token")
|
||||||
|
.to_string();
|
||||||
|
server.add_header("authorization", format!("Bearer {token}"));
|
||||||
|
|
||||||
|
// A fresh app keeps each test's KV / events isolated from siblings.
|
||||||
|
let slug = format!("e2e-{unique}");
|
||||||
|
let created: Value = server
|
||||||
|
.post("/api/v1/admin/apps")
|
||||||
|
.json(&json!({ "slug": slug, "name": slug }))
|
||||||
|
.await
|
||||||
|
.json();
|
||||||
|
let app_id = created["id"].as_str().expect("app id").to_string();
|
||||||
|
(server, app_id)
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn create_script(server: &TestServer, app_id: &str, name: &str, source: &str) -> String {
|
||||||
|
let created: Value = server
|
||||||
|
.post("/api/v1/admin/scripts")
|
||||||
|
.json(&json!({ "app_id": app_id, "name": name, "source": source }))
|
||||||
|
.await
|
||||||
|
.json();
|
||||||
|
created["id"].as_str().expect("script id").to_string()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A handler that records its `ctx.event` into a KV marker the test can
|
||||||
|
/// observe. The marker collection is watched by no trigger.
|
||||||
|
const MARKER_HANDLER: &str = r#"
|
||||||
|
let e = ctx.event;
|
||||||
|
kv::collection("e2e_markers").set("marker", e);
|
||||||
|
#{ ok: true }
|
||||||
|
"#;
|
||||||
|
|
||||||
|
/// Poll the marker KV key until present (or ~10s timeout).
|
||||||
|
async fn poll_marker(pool: &PgPool, app_id: &str) -> Option<Value> {
|
||||||
|
poll_marker_n(pool, app_id, 100).await
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Poll the marker KV key for `iters` × 100ms.
|
||||||
|
async fn poll_marker_n(pool: &PgPool, app_id: &str, iters: u32) -> Option<Value> {
|
||||||
|
let app_uuid = Uuid::parse_str(app_id).expect("app uuid");
|
||||||
|
for _ in 0..iters {
|
||||||
|
let row: Option<(Value,)> = sqlx::query_as(
|
||||||
|
"SELECT value FROM kv_entries \
|
||||||
|
WHERE app_id = $1 AND collection = 'e2e_markers' AND key = 'marker'",
|
||||||
|
)
|
||||||
|
.bind(app_uuid)
|
||||||
|
.fetch_optional(pool)
|
||||||
|
.await
|
||||||
|
.expect("query marker");
|
||||||
|
if let Some((value,)) = row {
|
||||||
|
return Some(value);
|
||||||
|
}
|
||||||
|
tokio::time::sleep(Duration::from_millis(100)).await;
|
||||||
|
}
|
||||||
|
None
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn execute(server: &TestServer, script_id: &str) {
|
||||||
|
server
|
||||||
|
.post(&format!("/api/v1/execute/{script_id}"))
|
||||||
|
.json(&json!({}))
|
||||||
|
.await
|
||||||
|
.assert_status_ok();
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn dispatcher_delivers_kv_to_handler() {
|
||||||
|
let Some(pool) = pool_or_skip().await else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
let (server, app_id) = server_for(pool.clone(), "kv").await;
|
||||||
|
|
||||||
|
let handler = create_script(&server, &app_id, "kv-handler", MARKER_HANDLER).await;
|
||||||
|
server
|
||||||
|
.post(&format!("/api/v1/admin/apps/{app_id}/triggers/kv"))
|
||||||
|
.json(&json!({ "script_id": handler, "collection_glob": "src" }))
|
||||||
|
.await
|
||||||
|
.assert_status(axum::http::StatusCode::CREATED);
|
||||||
|
|
||||||
|
let source = create_script(
|
||||||
|
&server,
|
||||||
|
&app_id,
|
||||||
|
"kv-source",
|
||||||
|
r#"kv::collection("src").set("k", 42); #{ ok: true }"#,
|
||||||
|
)
|
||||||
|
.await;
|
||||||
|
execute(&server, &source).await;
|
||||||
|
|
||||||
|
let event = poll_marker(&pool, &app_id).await.expect("kv handler fired");
|
||||||
|
assert_eq!(event["source"], "kv");
|
||||||
|
assert_eq!(event["op"], "insert");
|
||||||
|
assert_eq!(event["kv"]["collection"], "src");
|
||||||
|
assert_eq!(event["kv"]["key"], "k");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn dispatcher_delivers_docs_to_handler() {
|
||||||
|
let Some(pool) = pool_or_skip().await else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
let (server, app_id) = server_for(pool.clone(), "docs").await;
|
||||||
|
|
||||||
|
let handler = create_script(&server, &app_id, "docs-handler", MARKER_HANDLER).await;
|
||||||
|
server
|
||||||
|
.post(&format!("/api/v1/admin/apps/{app_id}/triggers/docs"))
|
||||||
|
.json(&json!({ "script_id": handler, "collection_glob": "src" }))
|
||||||
|
.await
|
||||||
|
.assert_status(axum::http::StatusCode::CREATED);
|
||||||
|
|
||||||
|
let source = create_script(
|
||||||
|
&server,
|
||||||
|
&app_id,
|
||||||
|
"docs-source",
|
||||||
|
r#"docs::collection("src").create(#{ x: 1 }); #{ ok: true }"#,
|
||||||
|
)
|
||||||
|
.await;
|
||||||
|
execute(&server, &source).await;
|
||||||
|
|
||||||
|
let event = poll_marker(&pool, &app_id)
|
||||||
|
.await
|
||||||
|
.expect("docs handler fired");
|
||||||
|
assert_eq!(event["source"], "docs");
|
||||||
|
assert_eq!(event["op"], "create");
|
||||||
|
assert_eq!(event["docs"]["collection"], "src");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn dispatcher_delivers_cron_to_handler() {
|
||||||
|
let Some(pool) = pool_or_skip().await else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
let (server, app_id) = server_for(pool.clone(), "cron").await;
|
||||||
|
|
||||||
|
let handler = create_script(&server, &app_id, "cron-handler", MARKER_HANDLER).await;
|
||||||
|
// Fire every second (6-field cron, seconds-resolution).
|
||||||
|
server
|
||||||
|
.post(&format!("/api/v1/admin/apps/{app_id}/triggers/cron"))
|
||||||
|
.json(&json!({ "script_id": handler, "schedule": "* * * * * *", "timezone": "UTC" }))
|
||||||
|
.await
|
||||||
|
.assert_status(axum::http::StatusCode::CREATED);
|
||||||
|
|
||||||
|
// No source — the scheduler enqueues the due tick on its own. The
|
||||||
|
// scheduler skips its first tick and then ticks every
|
||||||
|
// PICLOUD_CRON_TICK_INTERVAL_MS (default 30s), so poll past that
|
||||||
|
// (set the env var lower to speed CI up if desired).
|
||||||
|
let event = poll_marker_n(&pool, &app_id, 450)
|
||||||
|
.await
|
||||||
|
.expect("cron handler fired");
|
||||||
|
assert_eq!(event["source"], "cron");
|
||||||
|
assert_eq!(event["op"], "tick");
|
||||||
|
assert_eq!(event["cron"]["timezone"], "UTC");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn dispatcher_delivers_files_to_handler() {
|
||||||
|
let Some(pool) = pool_or_skip().await else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
let (server, app_id) = server_for(pool.clone(), "files").await;
|
||||||
|
|
||||||
|
let handler = create_script(&server, &app_id, "files-handler", MARKER_HANDLER).await;
|
||||||
|
server
|
||||||
|
.post(&format!("/api/v1/admin/apps/{app_id}/triggers/files"))
|
||||||
|
.json(&json!({ "script_id": handler, "collection_glob": "src" }))
|
||||||
|
.await
|
||||||
|
.assert_status(axum::http::StatusCode::CREATED);
|
||||||
|
|
||||||
|
let source = create_script(
|
||||||
|
&server,
|
||||||
|
&app_id,
|
||||||
|
"files-source",
|
||||||
|
r#"
|
||||||
|
let data = base64::decode("aGk=");
|
||||||
|
files::collection("src").create(#{ name: "f.txt", content_type: "text/plain", data: data });
|
||||||
|
#{ ok: true }
|
||||||
|
"#,
|
||||||
|
)
|
||||||
|
.await;
|
||||||
|
execute(&server, &source).await;
|
||||||
|
|
||||||
|
let event = poll_marker(&pool, &app_id)
|
||||||
|
.await
|
||||||
|
.expect("files handler fired");
|
||||||
|
assert_eq!(event["source"], "files");
|
||||||
|
assert_eq!(event["op"], "create");
|
||||||
|
assert_eq!(event["files"]["collection"], "src");
|
||||||
|
assert_eq!(event["files"]["name"], "f.txt");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn dispatcher_delivers_pubsub_to_handler() {
|
||||||
|
let Some(pool) = pool_or_skip().await else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
let (server, app_id) = server_for(pool.clone(), "pubsub").await;
|
||||||
|
|
||||||
|
let handler = create_script(&server, &app_id, "pubsub-handler", MARKER_HANDLER).await;
|
||||||
|
server
|
||||||
|
.post(&format!("/api/v1/admin/apps/{app_id}/triggers/pubsub"))
|
||||||
|
.json(&json!({ "script_id": handler, "topic_pattern": "e2e.topic" }))
|
||||||
|
.await
|
||||||
|
.assert_status(axum::http::StatusCode::CREATED);
|
||||||
|
|
||||||
|
let source = create_script(
|
||||||
|
&server,
|
||||||
|
&app_id,
|
||||||
|
"pubsub-source",
|
||||||
|
r#"pubsub::publish_durable("e2e.topic", #{ hello: 1 }); #{ ok: true }"#,
|
||||||
|
)
|
||||||
|
.await;
|
||||||
|
execute(&server, &source).await;
|
||||||
|
|
||||||
|
let event = poll_marker(&pool, &app_id)
|
||||||
|
.await
|
||||||
|
.expect("pubsub handler fired");
|
||||||
|
assert_eq!(event["source"], "pubsub");
|
||||||
|
assert_eq!(event["op"], "publish");
|
||||||
|
assert_eq!(event["pubsub"]["topic"], "e2e.topic");
|
||||||
|
assert_eq!(event["pubsub"]["message"]["hello"], 1);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Count dead_letters rows for an app.
|
||||||
|
async fn dead_letter_count(pool: &PgPool, app_id: &str) -> i64 {
|
||||||
|
let app_uuid = Uuid::parse_str(app_id).unwrap();
|
||||||
|
sqlx::query_scalar("SELECT COUNT(*) FROM dead_letters WHERE app_id = $1")
|
||||||
|
.bind(app_uuid)
|
||||||
|
.fetch_one(pool)
|
||||||
|
.await
|
||||||
|
.expect("count dead_letters")
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn poll_dead_letter_count(pool: &PgPool, app_id: &str, want: i64) -> i64 {
|
||||||
|
let mut count = 0;
|
||||||
|
for _ in 0..100 {
|
||||||
|
count = dead_letter_count(pool, app_id).await;
|
||||||
|
if count >= want {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
tokio::time::sleep(Duration::from_millis(100)).await;
|
||||||
|
}
|
||||||
|
count
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Register a failing KV trigger on `dlsrc` (single attempt → immediate
|
||||||
|
/// dead-letter) and a `dead_letter` trigger pointing at the marker
|
||||||
|
/// handler, then cause the originating KV event. Returns when set up.
|
||||||
|
async fn setup_dead_letter(server: &TestServer, app_id: &str, dl_handler: &str) {
|
||||||
|
let failing = create_script(server, app_id, "dl-failing", r#"throw "boom";"#).await;
|
||||||
|
server
|
||||||
|
.post(&format!("/api/v1/admin/apps/{app_id}/triggers/kv"))
|
||||||
|
.json(&json!({
|
||||||
|
"script_id": failing,
|
||||||
|
"collection_glob": "dlsrc",
|
||||||
|
"retry_max_attempts": 1,
|
||||||
|
"retry_base_ms": 0
|
||||||
|
}))
|
||||||
|
.await
|
||||||
|
.assert_status(axum::http::StatusCode::CREATED);
|
||||||
|
// The dead_letter trigger (no filters → matches any dead-letter).
|
||||||
|
server
|
||||||
|
.post(&format!("/api/v1/admin/apps/{app_id}/triggers/dead_letter"))
|
||||||
|
.json(&json!({ "script_id": dl_handler }))
|
||||||
|
.await
|
||||||
|
.assert_status(axum::http::StatusCode::CREATED);
|
||||||
|
|
||||||
|
let source = create_script(
|
||||||
|
server,
|
||||||
|
app_id,
|
||||||
|
"dl-source",
|
||||||
|
r#"kv::collection("dlsrc").set("k", 1); #{ ok: true }"#,
|
||||||
|
)
|
||||||
|
.await;
|
||||||
|
execute(server, source.as_str()).await;
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn dispatcher_delivers_dead_letter_to_handler() {
|
||||||
|
// v1.1.7: the dead-letter fan-out is now wired
|
||||||
|
// (`dispatcher::handle_failure` → `list_matching_dead_letter` →
|
||||||
|
// outbox). This asserts BOTH that the `dead_letters` row is written
|
||||||
|
// AND that the registered `dead_letter`-kind handler actually fires
|
||||||
|
// (it was silently non-functional v1.1.1–v1.1.6).
|
||||||
|
let Some(pool) = pool_or_skip().await else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
let (server, app_id) = server_for(pool.clone(), "dl").await;
|
||||||
|
let handler = create_script(&server, &app_id, "dl-handler", MARKER_HANDLER).await;
|
||||||
|
setup_dead_letter(&server, &app_id, &handler).await;
|
||||||
|
|
||||||
|
// Row written.
|
||||||
|
assert!(
|
||||||
|
poll_dead_letter_count(&pool, &app_id, 1).await > 0,
|
||||||
|
"a dead-letter row should have been produced"
|
||||||
|
);
|
||||||
|
// Handler fired.
|
||||||
|
let event = poll_marker(&pool, &app_id)
|
||||||
|
.await
|
||||||
|
.expect("dead-letter handler fired");
|
||||||
|
assert_eq!(event["source"], "dead_letter");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn dispatcher_delivers_dead_letter_to_handler_actually_fires() {
|
||||||
|
// Focused on the handler-fire side: the marker handler receives a
|
||||||
|
// fully-shaped dead-letter event (the original KV event nested under
|
||||||
|
// `ctx.event.dead_letter.original`, plus the failure metadata).
|
||||||
|
let Some(pool) = pool_or_skip().await else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
let (server, app_id) = server_for(pool.clone(), "dlfire").await;
|
||||||
|
let handler = create_script(&server, &app_id, "dl-handler", MARKER_HANDLER).await;
|
||||||
|
setup_dead_letter(&server, &app_id, &handler).await;
|
||||||
|
|
||||||
|
let event = poll_marker(&pool, &app_id)
|
||||||
|
.await
|
||||||
|
.expect("dead-letter handler fired");
|
||||||
|
assert_eq!(event["source"], "dead_letter");
|
||||||
|
// The original KV event is nested verbatim.
|
||||||
|
assert_eq!(event["dead_letter"]["original"]["source"], "kv");
|
||||||
|
assert_eq!(
|
||||||
|
event["dead_letter"]["original"]["kv"]["collection"],
|
||||||
|
"dlsrc"
|
||||||
|
);
|
||||||
|
// Failure metadata is present.
|
||||||
|
assert!(event["dead_letter"]["last_error"]
|
||||||
|
.as_str()
|
||||||
|
.unwrap()
|
||||||
|
.contains("boom"));
|
||||||
|
assert!(event["dead_letter"]["attempts"].as_i64().unwrap() >= 1);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn dead_letter_source_filter_excludes_nonmatching() {
|
||||||
|
// `list_matching_dead_letter` filters by source (among trigger_id /
|
||||||
|
// script_id). A dead_letter trigger whose `source_filter` is "docs"
|
||||||
|
// must NOT fire for a "kv"-sourced dead-letter — the row is still
|
||||||
|
// written, but no handler delivery is enqueued.
|
||||||
|
let Some(pool) = pool_or_skip().await else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
let (server, app_id) = server_for(pool.clone(), "dlfilter").await;
|
||||||
|
let handler = create_script(&server, &app_id, "dl-handler", MARKER_HANDLER).await;
|
||||||
|
|
||||||
|
let failing = create_script(&server, &app_id, "dl-failing", r#"throw "boom";"#).await;
|
||||||
|
server
|
||||||
|
.post(&format!("/api/v1/admin/apps/{app_id}/triggers/kv"))
|
||||||
|
.json(&json!({
|
||||||
|
"script_id": failing,
|
||||||
|
"collection_glob": "dlsrc",
|
||||||
|
"retry_max_attempts": 1,
|
||||||
|
"retry_base_ms": 0
|
||||||
|
}))
|
||||||
|
.await
|
||||||
|
.assert_status(axum::http::StatusCode::CREATED);
|
||||||
|
// Filter to a different source so this handler must NOT match.
|
||||||
|
server
|
||||||
|
.post(&format!("/api/v1/admin/apps/{app_id}/triggers/dead_letter"))
|
||||||
|
.json(&json!({ "script_id": handler, "source_filter": "docs" }))
|
||||||
|
.await
|
||||||
|
.assert_status(axum::http::StatusCode::CREATED);
|
||||||
|
|
||||||
|
let source = create_script(
|
||||||
|
&server,
|
||||||
|
&app_id,
|
||||||
|
"dl-source",
|
||||||
|
r#"kv::collection("dlsrc").set("k", 1); #{ ok: true }"#,
|
||||||
|
)
|
||||||
|
.await;
|
||||||
|
execute(&server, &source).await;
|
||||||
|
|
||||||
|
// The dead-letter row is written…
|
||||||
|
assert!(poll_dead_letter_count(&pool, &app_id, 1).await >= 1);
|
||||||
|
// …but the source-filtered handler never fires.
|
||||||
|
let marker = poll_marker_n(&pool, &app_id, 8).await;
|
||||||
|
assert!(
|
||||||
|
marker.is_none(),
|
||||||
|
"source_filter='docs' must not fire for a kv dead-letter"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn dead_letter_handler_failure_does_not_recurse() {
|
||||||
|
// Recursion-stop (design notes §4): a dead_letter handler that itself
|
||||||
|
// throws must NOT produce a second dead-letter row. The
|
||||||
|
// `is_dead_letter_handler` short-circuit annotates the original row
|
||||||
|
// and drops the outbox row without re-dead-lettering.
|
||||||
|
let Some(pool) = pool_or_skip().await else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
let (server, app_id) = server_for(pool.clone(), "dlrec").await;
|
||||||
|
// The DL handler itself throws.
|
||||||
|
let throwing = create_script(&server, &app_id, "dl-throws", r#"throw "handler boom";"#).await;
|
||||||
|
setup_dead_letter(&server, &app_id, &throwing).await;
|
||||||
|
|
||||||
|
// One dead-letter row appears (the original). Give the throwing
|
||||||
|
// handler time to run + (not) recurse, then confirm the count stayed
|
||||||
|
// at exactly 1.
|
||||||
|
assert!(poll_dead_letter_count(&pool, &app_id, 1).await >= 1);
|
||||||
|
tokio::time::sleep(Duration::from_millis(800)).await;
|
||||||
|
assert_eq!(
|
||||||
|
dead_letter_count(&pool, &app_id).await,
|
||||||
|
1,
|
||||||
|
"a failing dead-letter handler must not create a new dead-letter row"
|
||||||
|
);
|
||||||
|
}
|
||||||
298
crates/picloud/tests/email_inbound.rs
Normal file
298
crates/picloud/tests/email_inbound.rs
Normal file
@@ -0,0 +1,298 @@
|
|||||||
|
//! End-to-end tests for the inbound-email webhook receiver (v1.1.7).
|
||||||
|
//!
|
||||||
|
//! Gated on `DATABASE_URL` like `dispatcher_e2e.rs`: when unset the test
|
||||||
|
//! prints a notice and returns early so plain `cargo test` stays green.
|
||||||
|
//!
|
||||||
|
//! Covers the receiver's status-code matrix (202 / 401 / 404 / 422),
|
||||||
|
//! cross-app path isolation, HMAC verification (signed + unsigned
|
||||||
|
//! triggers), the dispatcher routing the `email` outbox row, and the
|
||||||
|
//! handler actually firing with `ctx.event.email` populated. The
|
||||||
|
//! "handler fired" observation uses the same KV-marker pattern as
|
||||||
|
//! `dispatcher_e2e.rs`.
|
||||||
|
|
||||||
|
#![allow(clippy::needless_pass_by_value)]
|
||||||
|
|
||||||
|
use std::time::Duration;
|
||||||
|
|
||||||
|
use axum_test::TestServer;
|
||||||
|
use hmac::{Hmac, Mac};
|
||||||
|
use serde_json::{json, Value};
|
||||||
|
use sha2::Sha256;
|
||||||
|
use sqlx::postgres::PgPoolOptions;
|
||||||
|
use sqlx::PgPool;
|
||||||
|
use uuid::Uuid;
|
||||||
|
|
||||||
|
/// Fixed master key so the receiver decrypts the inbound_secret the
|
||||||
|
/// admin endpoint encrypted (same key feeds build_app + the admin path).
|
||||||
|
fn master_key() -> picloud_shared::MasterKey {
|
||||||
|
picloud_shared::MasterKey::from_bytes([0x42u8; 32])
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn pool_or_skip() -> Option<PgPool> {
|
||||||
|
let Ok(url) = std::env::var("DATABASE_URL") else {
|
||||||
|
eprintln!("email_inbound: DATABASE_URL unset — skipping");
|
||||||
|
return None;
|
||||||
|
};
|
||||||
|
let pool = PgPoolOptions::new()
|
||||||
|
.max_connections(5)
|
||||||
|
.connect(&url)
|
||||||
|
.await
|
||||||
|
.expect("connect to DATABASE_URL");
|
||||||
|
sqlx::migrate!("../manager-core/migrations")
|
||||||
|
.run(&pool)
|
||||||
|
.await
|
||||||
|
.expect("apply migrations");
|
||||||
|
Some(pool)
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn server_for(pool: PgPool, suffix: &str) -> (TestServer, String) {
|
||||||
|
use picloud_manager_core::auth::hash_password;
|
||||||
|
use picloud_shared::InstanceRole;
|
||||||
|
|
||||||
|
let unique = format!("{suffix}-{}", Uuid::new_v4().simple());
|
||||||
|
let auth = picloud::AuthDeps::from_pool(pool.clone());
|
||||||
|
let username = format!("eml-{unique}");
|
||||||
|
let hash = hash_password("pw").expect("hash");
|
||||||
|
auth.users
|
||||||
|
.create(&username, &hash, InstanceRole::Owner, None)
|
||||||
|
.await
|
||||||
|
.expect("seed admin");
|
||||||
|
|
||||||
|
let app = picloud::build_app(pool, auth, master_key())
|
||||||
|
.await
|
||||||
|
.expect("build_app");
|
||||||
|
let mut server = TestServer::new(app).expect("TestServer");
|
||||||
|
let resp = server
|
||||||
|
.post("/api/v1/admin/auth/login")
|
||||||
|
.json(&json!({ "username": username, "password": "pw" }))
|
||||||
|
.await;
|
||||||
|
resp.assert_status_ok();
|
||||||
|
let token = resp.json::<Value>()["token"]
|
||||||
|
.as_str()
|
||||||
|
.expect("login token")
|
||||||
|
.to_string();
|
||||||
|
server.add_header("authorization", format!("Bearer {token}"));
|
||||||
|
|
||||||
|
let slug = format!("eml-{unique}");
|
||||||
|
let created: Value = server
|
||||||
|
.post("/api/v1/admin/apps")
|
||||||
|
.json(&json!({ "slug": slug, "name": slug }))
|
||||||
|
.await
|
||||||
|
.json();
|
||||||
|
let app_id = created["id"].as_str().expect("app id").to_string();
|
||||||
|
(server, app_id)
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn create_script(server: &TestServer, app_id: &str, name: &str, source: &str) -> String {
|
||||||
|
let created: Value = server
|
||||||
|
.post("/api/v1/admin/scripts")
|
||||||
|
.json(&json!({ "app_id": app_id, "name": name, "source": source }))
|
||||||
|
.await
|
||||||
|
.json();
|
||||||
|
created["id"].as_str().expect("script id").to_string()
|
||||||
|
}
|
||||||
|
|
||||||
|
const MARKER_HANDLER: &str = r#"
|
||||||
|
let e = ctx.event;
|
||||||
|
kv::collection("e2e_markers").set("marker", e);
|
||||||
|
#{ ok: true }
|
||||||
|
"#;
|
||||||
|
|
||||||
|
async fn create_email_trigger(
|
||||||
|
server: &TestServer,
|
||||||
|
app_id: &str,
|
||||||
|
script_id: &str,
|
||||||
|
secret: Option<&str>,
|
||||||
|
) -> String {
|
||||||
|
let created: Value = server
|
||||||
|
.post(&format!("/api/v1/admin/apps/{app_id}/triggers/email"))
|
||||||
|
.json(&json!({ "script_id": script_id, "inbound_secret": secret }))
|
||||||
|
.await
|
||||||
|
.json();
|
||||||
|
created["id"].as_str().expect("trigger id").to_string()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn sign(secret: &str, body: &str) -> String {
|
||||||
|
let mut mac = Hmac::<Sha256>::new_from_slice(secret.as_bytes()).expect("hmac key");
|
||||||
|
mac.update(body.as_bytes());
|
||||||
|
hex::encode(mac.finalize().into_bytes())
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn poll_marker(pool: &PgPool, app_id: &str) -> Option<Value> {
|
||||||
|
let app_uuid = Uuid::parse_str(app_id).expect("app uuid");
|
||||||
|
for _ in 0..100 {
|
||||||
|
let row: Option<(Value,)> = sqlx::query_as(
|
||||||
|
"SELECT value FROM kv_entries \
|
||||||
|
WHERE app_id = $1 AND collection = 'e2e_markers' AND key = 'marker'",
|
||||||
|
)
|
||||||
|
.bind(app_uuid)
|
||||||
|
.fetch_optional(pool)
|
||||||
|
.await
|
||||||
|
.expect("query marker");
|
||||||
|
if let Some((value,)) = row {
|
||||||
|
return Some(value);
|
||||||
|
}
|
||||||
|
tokio::time::sleep(Duration::from_millis(100)).await;
|
||||||
|
}
|
||||||
|
None
|
||||||
|
}
|
||||||
|
|
||||||
|
const BODY: &str = r#"{"from":"sender@external.com","to":["alice@myapp.com"],"cc":["bob@myapp.com"],"subject":"Re: question","text":"hello there","message_id":"<abc@external.com>"}"#;
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn signed_post_accepts_and_fires_handler() {
|
||||||
|
let Some(pool) = pool_or_skip().await else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
let (server, app_id) = server_for(pool.clone(), "signed").await;
|
||||||
|
let handler = create_script(&server, &app_id, "eml-handler", MARKER_HANDLER).await;
|
||||||
|
let trigger = create_email_trigger(&server, &app_id, &handler, Some("topsecret")).await;
|
||||||
|
|
||||||
|
let sig = sign("topsecret", BODY);
|
||||||
|
server
|
||||||
|
.post(&format!("/api/v1/email-inbound/{app_id}/{trigger}"))
|
||||||
|
.add_header("x-picloud-signature", sig)
|
||||||
|
.text(BODY)
|
||||||
|
.await
|
||||||
|
.assert_status(axum::http::StatusCode::ACCEPTED);
|
||||||
|
|
||||||
|
// Outbox row landed with source_kind = 'email'.
|
||||||
|
let app_uuid = Uuid::parse_str(&app_id).unwrap();
|
||||||
|
// The dispatcher deletes the row after delivery; instead assert the
|
||||||
|
// handler fired (which proves the email row was dispatched).
|
||||||
|
let event = poll_marker(&pool, &app_id).await.expect("handler fired");
|
||||||
|
assert_eq!(event["source"], "email");
|
||||||
|
assert_eq!(event["op"], "receive");
|
||||||
|
assert_eq!(event["email"]["from"], "sender@external.com");
|
||||||
|
assert_eq!(event["email"]["to"][0], "alice@myapp.com");
|
||||||
|
assert_eq!(event["email"]["cc"][0], "bob@myapp.com");
|
||||||
|
assert_eq!(event["email"]["subject"], "Re: question");
|
||||||
|
assert_eq!(event["email"]["text"], "hello there");
|
||||||
|
assert_eq!(event["email"]["message_id"], "<abc@external.com>");
|
||||||
|
let _ = app_uuid;
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn missing_signature_is_401_when_secret_configured() {
|
||||||
|
let Some(pool) = pool_or_skip().await else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
let (server, app_id) = server_for(pool, "nosig").await;
|
||||||
|
let handler = create_script(&server, &app_id, "h", MARKER_HANDLER).await;
|
||||||
|
let trigger = create_email_trigger(&server, &app_id, &handler, Some("topsecret")).await;
|
||||||
|
|
||||||
|
server
|
||||||
|
.post(&format!("/api/v1/email-inbound/{app_id}/{trigger}"))
|
||||||
|
.text(BODY)
|
||||||
|
.await
|
||||||
|
.assert_status(axum::http::StatusCode::UNAUTHORIZED);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn wrong_signature_is_401() {
|
||||||
|
let Some(pool) = pool_or_skip().await else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
let (server, app_id) = server_for(pool, "wrongsig").await;
|
||||||
|
let handler = create_script(&server, &app_id, "h", MARKER_HANDLER).await;
|
||||||
|
let trigger = create_email_trigger(&server, &app_id, &handler, Some("topsecret")).await;
|
||||||
|
|
||||||
|
server
|
||||||
|
.post(&format!("/api/v1/email-inbound/{app_id}/{trigger}"))
|
||||||
|
.add_header("x-picloud-signature", sign("WRONG", BODY))
|
||||||
|
.text(BODY)
|
||||||
|
.await
|
||||||
|
.assert_status(axum::http::StatusCode::UNAUTHORIZED);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn unsigned_trigger_accepts_without_signature() {
|
||||||
|
let Some(pool) = pool_or_skip().await else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
let (server, app_id) = server_for(pool, "unsigned").await;
|
||||||
|
let handler = create_script(&server, &app_id, "h", MARKER_HANDLER).await;
|
||||||
|
let trigger = create_email_trigger(&server, &app_id, &handler, None).await;
|
||||||
|
|
||||||
|
server
|
||||||
|
.post(&format!("/api/v1/email-inbound/{app_id}/{trigger}"))
|
||||||
|
.text(BODY)
|
||||||
|
.await
|
||||||
|
.assert_status(axum::http::StatusCode::ACCEPTED);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn unknown_trigger_is_404() {
|
||||||
|
let Some(pool) = pool_or_skip().await else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
let (server, app_id) = server_for(pool, "missing").await;
|
||||||
|
let missing = Uuid::new_v4();
|
||||||
|
server
|
||||||
|
.post(&format!("/api/v1/email-inbound/{app_id}/{missing}"))
|
||||||
|
.text(BODY)
|
||||||
|
.await
|
||||||
|
.assert_status(axum::http::StatusCode::NOT_FOUND);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn wrong_kind_trigger_is_404() {
|
||||||
|
let Some(pool) = pool_or_skip().await else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
let (server, app_id) = server_for(pool, "wrongkind").await;
|
||||||
|
let handler = create_script(&server, &app_id, "h", MARKER_HANDLER).await;
|
||||||
|
// A KV trigger — not an email trigger.
|
||||||
|
let kv_trigger: Value = server
|
||||||
|
.post(&format!("/api/v1/admin/apps/{app_id}/triggers/kv"))
|
||||||
|
.json(&json!({ "script_id": handler, "collection_glob": "*" }))
|
||||||
|
.await
|
||||||
|
.json();
|
||||||
|
let kv_id = kv_trigger["id"].as_str().unwrap();
|
||||||
|
server
|
||||||
|
.post(&format!("/api/v1/email-inbound/{app_id}/{kv_id}"))
|
||||||
|
.text(BODY)
|
||||||
|
.await
|
||||||
|
.assert_status(axum::http::StatusCode::NOT_FOUND);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn malformed_body_is_422() {
|
||||||
|
let Some(pool) = pool_or_skip().await else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
let (server, app_id) = server_for(pool, "malformed").await;
|
||||||
|
let handler = create_script(&server, &app_id, "h", MARKER_HANDLER).await;
|
||||||
|
// Unsigned so we reach the parse step.
|
||||||
|
let trigger = create_email_trigger(&server, &app_id, &handler, None).await;
|
||||||
|
server
|
||||||
|
.post(&format!("/api/v1/email-inbound/{app_id}/{trigger}"))
|
||||||
|
.text("not json at all")
|
||||||
|
.await
|
||||||
|
.assert_status(axum::http::StatusCode::UNPROCESSABLE_ENTITY);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn cross_app_path_is_404() {
|
||||||
|
let Some(pool) = pool_or_skip().await else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
// Two apps under the same server. A trigger created in app B must
|
||||||
|
// not be reachable via app A's path segment.
|
||||||
|
let (server, app_a) = server_for(pool.clone(), "xa").await;
|
||||||
|
let app_b: Value = server
|
||||||
|
.post("/api/v1/admin/apps")
|
||||||
|
.json(&json!({ "slug": format!("xb-{}", Uuid::new_v4().simple()), "name": "xb" }))
|
||||||
|
.await
|
||||||
|
.json();
|
||||||
|
let app_b_id = app_b["id"].as_str().unwrap().to_string();
|
||||||
|
let handler_b = create_script(&server, &app_b_id, "hb", MARKER_HANDLER).await;
|
||||||
|
let trigger_b = create_email_trigger(&server, &app_b_id, &handler_b, None).await;
|
||||||
|
|
||||||
|
// POST to app A's path with app B's trigger id → 404 (path-bound).
|
||||||
|
server
|
||||||
|
.post(&format!("/api/v1/email-inbound/{app_a}/{trigger_b}"))
|
||||||
|
.text(BODY)
|
||||||
|
.await
|
||||||
|
.assert_status(axum::http::StatusCode::NOT_FOUND);
|
||||||
|
}
|
||||||
@@ -15,3 +15,16 @@ serde_json.workspace = true
|
|||||||
thiserror.workspace = true
|
thiserror.workspace = true
|
||||||
uuid.workspace = true
|
uuid.workspace = true
|
||||||
chrono.workspace = true
|
chrono.workspace = true
|
||||||
|
# Realtime broadcaster trait returns a broadcast::Receiver; subscriber
|
||||||
|
# tokens are HMAC-SHA256 over a base64url payload (v1.1.6).
|
||||||
|
tokio = { workspace = true, features = ["sync"] }
|
||||||
|
hmac.workspace = true
|
||||||
|
sha2.workspace = true
|
||||||
|
base64.workspace = true
|
||||||
|
# AES-256-GCM envelope + master-key sourcing (v1.1.7 crypto module).
|
||||||
|
aes-gcm.workspace = true
|
||||||
|
rand.workspace = true
|
||||||
|
tracing.workspace = true
|
||||||
|
|
||||||
|
[dev-dependencies]
|
||||||
|
tokio = { workspace = true, features = ["macros", "rt", "rt-multi-thread", "time", "sync"] }
|
||||||
|
|||||||
358
crates/shared/src/crypto.rs
Normal file
358
crates/shared/src/crypto.rs
Normal file
@@ -0,0 +1,358 @@
|
|||||||
|
//! AES-256-GCM encryption envelope + master-key sourcing (v1.1.7).
|
||||||
|
//!
|
||||||
|
//! Two responsibilities:
|
||||||
|
//!
|
||||||
|
//! 1. [`encrypt`] / [`decrypt`] — the at-rest envelope used by per-app
|
||||||
|
//! `secrets`, the encrypted `inbound_secret` on email triggers, and
|
||||||
|
//! the realtime signing key. `Aes256Gcm` with a 96-bit (12-byte)
|
||||||
|
//! random nonce and a 128-bit auth tag **appended to the
|
||||||
|
//! ciphertext** (the RustCrypto `Aead`-trait layout — `encrypt`
|
||||||
|
//! returns `ciphertext || tag`, `decrypt` consumes the same). Both
|
||||||
|
//! the ciphertext (tag included) and the nonce are stored.
|
||||||
|
//!
|
||||||
|
//! 2. [`MasterKey`] — the process-wide 32-byte key, sourced once at
|
||||||
|
//! startup from `PICLOUD_SECRET_KEY` (base64 of exactly 32 bytes).
|
||||||
|
//! A deterministic in-memory dev key is allowed ONLY when the env
|
||||||
|
//! var is unset AND `PICLOUD_DEV_MODE=true`; otherwise an unset key
|
||||||
|
//! is fatal (no quiet "your secrets are unencrypted" mode).
|
||||||
|
//!
|
||||||
|
//! **Key rotation is out of scope for v1.1.7.** Changing
|
||||||
|
//! `PICLOUD_SECRET_KEY` between deploys orphans every existing
|
||||||
|
//! ciphertext (it can no longer be decrypted). v1.2+ adds key-version
|
||||||
|
//! columns + a re-encryption pass.
|
||||||
|
|
||||||
|
use aes_gcm::aead::{Aead, KeyInit};
|
||||||
|
use aes_gcm::{Aes256Gcm, Key, Nonce};
|
||||||
|
use base64::engine::general_purpose::STANDARD as B64;
|
||||||
|
use base64::Engine as _;
|
||||||
|
use rand::RngCore;
|
||||||
|
use sha2::{Digest, Sha256};
|
||||||
|
use thiserror::Error;
|
||||||
|
|
||||||
|
/// Master-key length in bytes (AES-256 → 32-byte key).
|
||||||
|
pub const KEY_LEN: usize = 32;
|
||||||
|
|
||||||
|
/// GCM nonce length in bytes (96-bit nonce, the AES-GCM standard).
|
||||||
|
pub const NONCE_LEN: usize = 12;
|
||||||
|
|
||||||
|
/// Output of [`encrypt`]: the ciphertext (auth tag appended) plus the
|
||||||
|
/// randomly-generated nonce. Both must be persisted; `decrypt` needs
|
||||||
|
/// the nonce to recover the plaintext.
|
||||||
|
#[derive(Debug, Clone)]
|
||||||
|
pub struct EncryptResult {
|
||||||
|
/// Ciphertext with the 16-byte GCM auth tag appended.
|
||||||
|
pub ciphertext: Vec<u8>,
|
||||||
|
/// The 12-byte nonce used for this encryption.
|
||||||
|
pub nonce: [u8; NONCE_LEN],
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Errors from the encryption envelope.
|
||||||
|
#[derive(Debug, Error)]
|
||||||
|
pub enum CryptoError {
|
||||||
|
/// Authentication failed — wrong key, corrupted ciphertext, or a
|
||||||
|
/// tampered nonce/tag. GCM does not distinguish these (by design),
|
||||||
|
/// so neither do we.
|
||||||
|
#[error("decryption failed: authentication tag mismatch (wrong key, corrupted ciphertext, or tampered nonce)")]
|
||||||
|
Decrypt,
|
||||||
|
|
||||||
|
/// The stored nonce was not exactly [`NONCE_LEN`] bytes — a sign of
|
||||||
|
/// row corruption.
|
||||||
|
#[error("invalid nonce length: expected {NONCE_LEN} bytes, got {0}")]
|
||||||
|
InvalidNonce(usize),
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Encrypt `plaintext` under `key`, generating a fresh random nonce.
|
||||||
|
///
|
||||||
|
/// The auth tag is appended to the returned ciphertext (RustCrypto
|
||||||
|
/// `Aead` layout). Encryption with a valid 32-byte key and 12-byte
|
||||||
|
/// nonce is infallible in `aes-gcm`, so this returns a value rather
|
||||||
|
/// than a `Result`.
|
||||||
|
#[must_use]
|
||||||
|
pub fn encrypt(plaintext: &[u8], key: &[u8; KEY_LEN]) -> EncryptResult {
|
||||||
|
let cipher = Aes256Gcm::new(Key::<Aes256Gcm>::from_slice(key));
|
||||||
|
let mut nonce_bytes = [0u8; NONCE_LEN];
|
||||||
|
// CSPRNG nonce. `thread_rng` is seeded from the OS CSPRNG; a fresh
|
||||||
|
// 96-bit nonce per encryption keeps the (key, nonce) pair unique.
|
||||||
|
rand::thread_rng().fill_bytes(&mut nonce_bytes);
|
||||||
|
let nonce = Nonce::from_slice(&nonce_bytes);
|
||||||
|
let ciphertext = cipher
|
||||||
|
.encrypt(nonce, plaintext)
|
||||||
|
.expect("AES-256-GCM encryption is infallible for a valid key + 12-byte nonce");
|
||||||
|
EncryptResult {
|
||||||
|
ciphertext,
|
||||||
|
nonce: nonce_bytes,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Decrypt `ciphertext` (auth tag appended) with the stored `nonce`
|
||||||
|
/// under `key`.
|
||||||
|
///
|
||||||
|
/// # Errors
|
||||||
|
///
|
||||||
|
/// Returns [`CryptoError::InvalidNonce`] if `nonce` is the wrong length,
|
||||||
|
/// or [`CryptoError::Decrypt`] if authentication fails for any reason
|
||||||
|
/// (wrong key, corruption, tampering).
|
||||||
|
pub fn decrypt(
|
||||||
|
ciphertext: &[u8],
|
||||||
|
nonce: &[u8],
|
||||||
|
key: &[u8; KEY_LEN],
|
||||||
|
) -> Result<Vec<u8>, CryptoError> {
|
||||||
|
if nonce.len() != NONCE_LEN {
|
||||||
|
return Err(CryptoError::InvalidNonce(nonce.len()));
|
||||||
|
}
|
||||||
|
let cipher = Aes256Gcm::new(Key::<Aes256Gcm>::from_slice(key));
|
||||||
|
let nonce = Nonce::from_slice(nonce);
|
||||||
|
cipher
|
||||||
|
.decrypt(nonce, ciphertext)
|
||||||
|
.map_err(|_| CryptoError::Decrypt)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The process-wide master key. Sourced once at startup and threaded
|
||||||
|
/// into the secrets service, the email-trigger receiver, and the
|
||||||
|
/// realtime signing-key migration.
|
||||||
|
///
|
||||||
|
/// Cheap to clone (32 bytes). `Debug` is redacted so the key never
|
||||||
|
/// lands in a log line.
|
||||||
|
#[derive(Clone)]
|
||||||
|
pub struct MasterKey {
|
||||||
|
key: [u8; KEY_LEN],
|
||||||
|
}
|
||||||
|
|
||||||
|
impl std::fmt::Debug for MasterKey {
|
||||||
|
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||||
|
f.debug_struct("MasterKey")
|
||||||
|
.field("key", &"<redacted 32 bytes>")
|
||||||
|
.finish()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Failure modes for master-key sourcing. Every variant is a fatal
|
||||||
|
/// startup error — there is no fallback to a quiet plaintext mode.
|
||||||
|
#[derive(Debug, Error)]
|
||||||
|
pub enum MasterKeyError {
|
||||||
|
/// `PICLOUD_SECRET_KEY` is unset/empty and dev mode is off.
|
||||||
|
#[error(
|
||||||
|
"PICLOUD_SECRET_KEY is required but unset. Generate one with `openssl rand -base64 32`, \
|
||||||
|
or set PICLOUD_DEV_MODE=true to use an insecure deterministic dev key (never in production)."
|
||||||
|
)]
|
||||||
|
Missing,
|
||||||
|
|
||||||
|
/// `PICLOUD_SECRET_KEY` was not valid base64.
|
||||||
|
#[error("PICLOUD_SECRET_KEY is not valid base64 (expected base64 of 32 bytes — `openssl rand -base64 32`)")]
|
||||||
|
Malformed,
|
||||||
|
|
||||||
|
/// Decoded to the wrong number of bytes.
|
||||||
|
#[error("PICLOUD_SECRET_KEY must decode to exactly {KEY_LEN} bytes, got {0}")]
|
||||||
|
WrongLength(usize),
|
||||||
|
}
|
||||||
|
|
||||||
|
impl MasterKey {
|
||||||
|
/// Borrow the raw 32-byte key for the crypto envelope.
|
||||||
|
#[must_use]
|
||||||
|
pub const fn as_bytes(&self) -> &[u8; KEY_LEN] {
|
||||||
|
&self.key
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Build a key directly from 32 bytes (used by the realtime
|
||||||
|
/// migration's tests and by [`Self::from_base64`]).
|
||||||
|
#[must_use]
|
||||||
|
pub const fn from_bytes(key: [u8; KEY_LEN]) -> Self {
|
||||||
|
Self { key }
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Decode a base64-encoded 32-byte key.
|
||||||
|
///
|
||||||
|
/// # Errors
|
||||||
|
///
|
||||||
|
/// [`MasterKeyError::Malformed`] for non-base64 input,
|
||||||
|
/// [`MasterKeyError::WrongLength`] when the decoded length is not 32.
|
||||||
|
pub fn from_base64(s: &str) -> Result<Self, MasterKeyError> {
|
||||||
|
let decoded = B64
|
||||||
|
.decode(s.trim().as_bytes())
|
||||||
|
.map_err(|_| MasterKeyError::Malformed)?;
|
||||||
|
let len = decoded.len();
|
||||||
|
let key: [u8; KEY_LEN] = decoded
|
||||||
|
.try_into()
|
||||||
|
.map_err(|_| MasterKeyError::WrongLength(len))?;
|
||||||
|
Ok(Self { key })
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Source the master key from the process environment per the
|
||||||
|
/// v1.1.7 rules. See [`Self::resolve`] for the decision logic.
|
||||||
|
///
|
||||||
|
/// # Errors
|
||||||
|
///
|
||||||
|
/// Propagates [`MasterKeyError`] when the key is absent (and dev
|
||||||
|
/// mode is off) or malformed.
|
||||||
|
pub fn from_env() -> Result<Self, MasterKeyError> {
|
||||||
|
let secret = std::env::var("PICLOUD_SECRET_KEY").ok();
|
||||||
|
let dev_mode = std::env::var("PICLOUD_DEV_MODE")
|
||||||
|
.map(|v| is_truthy(&v))
|
||||||
|
.unwrap_or(false);
|
||||||
|
Self::resolve(secret.as_deref(), dev_mode)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Pure resolution logic, factored out of [`Self::from_env`] so it's
|
||||||
|
/// testable without mutating process-global env vars.
|
||||||
|
///
|
||||||
|
/// * `secret` present + non-empty → parse it (fatal if malformed).
|
||||||
|
/// * `secret` absent/empty + `dev_mode` → deterministic dev key +
|
||||||
|
/// a prominent warning.
|
||||||
|
/// * `secret` absent/empty + no dev mode → fatal.
|
||||||
|
///
|
||||||
|
/// # Errors
|
||||||
|
///
|
||||||
|
/// See [`Self::from_env`].
|
||||||
|
pub fn resolve(secret: Option<&str>, dev_mode: bool) -> Result<Self, MasterKeyError> {
|
||||||
|
match secret {
|
||||||
|
Some(v) if !v.trim().is_empty() => Self::from_base64(v),
|
||||||
|
_ if dev_mode => {
|
||||||
|
tracing::warn!(
|
||||||
|
"PICLOUD_SECRET_KEY is unset and PICLOUD_DEV_MODE=true: using a DETERMINISTIC \
|
||||||
|
in-memory dev master key. At-rest secrets are NOT secure in this mode. \
|
||||||
|
Never run a real deployment without PICLOUD_SECRET_KEY."
|
||||||
|
);
|
||||||
|
Ok(Self::dev_key())
|
||||||
|
}
|
||||||
|
_ => Err(MasterKeyError::Missing),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Deterministic dev key: SHA-256 of a fixed label. Stable across
|
||||||
|
/// restarts so dev secrets survive a reboot, but obviously not a
|
||||||
|
/// real secret (the input is public).
|
||||||
|
#[must_use]
|
||||||
|
fn dev_key() -> Self {
|
||||||
|
let digest = Sha256::digest(b"picloud-dev-master-key-v1.1.7");
|
||||||
|
let mut key = [0u8; KEY_LEN];
|
||||||
|
key.copy_from_slice(&digest);
|
||||||
|
Self { key }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Common env-var truthiness check shared with the other config knobs.
|
||||||
|
fn is_truthy(v: &str) -> bool {
|
||||||
|
matches!(v.trim().to_ascii_lowercase().as_str(), "1" | "true" | "yes")
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
fn test_key() -> [u8; KEY_LEN] {
|
||||||
|
let mut k = [0u8; KEY_LEN];
|
||||||
|
for (i, b) in k.iter_mut().enumerate() {
|
||||||
|
*b = u8::try_from(i).unwrap_or(0);
|
||||||
|
}
|
||||||
|
k
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn round_trip_recovers_plaintext() {
|
||||||
|
let key = test_key();
|
||||||
|
let plaintext = b"sk_live_super_secret_value";
|
||||||
|
let enc = encrypt(plaintext, &key);
|
||||||
|
let dec = decrypt(&enc.ciphertext, &enc.nonce, &key).unwrap();
|
||||||
|
assert_eq!(dec, plaintext);
|
||||||
|
// Tag is appended → ciphertext is longer than plaintext.
|
||||||
|
assert!(enc.ciphertext.len() > plaintext.len());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn round_trip_empty_plaintext() {
|
||||||
|
let key = test_key();
|
||||||
|
let enc = encrypt(b"", &key);
|
||||||
|
let dec = decrypt(&enc.ciphertext, &enc.nonce, &key).unwrap();
|
||||||
|
assert!(dec.is_empty());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn tampered_ciphertext_fails() {
|
||||||
|
let key = test_key();
|
||||||
|
let mut enc = encrypt(b"hello world", &key);
|
||||||
|
enc.ciphertext[0] ^= 0xff;
|
||||||
|
let err = decrypt(&enc.ciphertext, &enc.nonce, &key).unwrap_err();
|
||||||
|
assert!(matches!(err, CryptoError::Decrypt));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn tampered_nonce_fails() {
|
||||||
|
let key = test_key();
|
||||||
|
let enc = encrypt(b"hello world", &key);
|
||||||
|
let mut nonce = enc.nonce;
|
||||||
|
nonce[0] ^= 0xff;
|
||||||
|
let err = decrypt(&enc.ciphertext, &nonce, &key).unwrap_err();
|
||||||
|
assert!(matches!(err, CryptoError::Decrypt));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn wrong_key_fails() {
|
||||||
|
let key = test_key();
|
||||||
|
let mut other = test_key();
|
||||||
|
other[31] ^= 0xff;
|
||||||
|
let enc = encrypt(b"hello world", &key);
|
||||||
|
let err = decrypt(&enc.ciphertext, &enc.nonce, &other).unwrap_err();
|
||||||
|
assert!(matches!(err, CryptoError::Decrypt));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn wrong_length_nonce_rejected() {
|
||||||
|
let key = test_key();
|
||||||
|
let enc = encrypt(b"hi", &key);
|
||||||
|
let err = decrypt(&enc.ciphertext, &enc.nonce[..8], &key).unwrap_err();
|
||||||
|
assert!(matches!(err, CryptoError::InvalidNonce(8)));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn distinct_nonces_per_encryption() {
|
||||||
|
let key = test_key();
|
||||||
|
let a = encrypt(b"same plaintext", &key);
|
||||||
|
let b = encrypt(b"same plaintext", &key);
|
||||||
|
// Random nonce → ciphertext differs even for identical input.
|
||||||
|
assert_ne!(a.nonce, b.nonce);
|
||||||
|
assert_ne!(a.ciphertext, b.ciphertext);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn master_key_from_valid_base64() {
|
||||||
|
let raw = [7u8; KEY_LEN];
|
||||||
|
let b64 = B64.encode(raw);
|
||||||
|
let mk = MasterKey::from_base64(&b64).unwrap();
|
||||||
|
assert_eq!(mk.as_bytes(), &raw);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn master_key_malformed_base64() {
|
||||||
|
let err = MasterKey::from_base64("not valid base64 !!!").unwrap_err();
|
||||||
|
assert!(matches!(err, MasterKeyError::Malformed));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn master_key_wrong_length() {
|
||||||
|
let b64 = B64.encode([1u8; 16]); // 16 bytes, not 32
|
||||||
|
let err = MasterKey::from_base64(&b64).unwrap_err();
|
||||||
|
assert!(matches!(err, MasterKeyError::WrongLength(16)));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn resolve_missing_without_dev_is_fatal() {
|
||||||
|
let err = MasterKey::resolve(None, false).unwrap_err();
|
||||||
|
assert!(matches!(err, MasterKeyError::Missing));
|
||||||
|
// Empty string counts as missing too.
|
||||||
|
let err = MasterKey::resolve(Some(" "), false).unwrap_err();
|
||||||
|
assert!(matches!(err, MasterKeyError::Missing));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn resolve_dev_fallback_only_with_dev_mode() {
|
||||||
|
// Dev mode on + no key → deterministic dev key.
|
||||||
|
let a = MasterKey::resolve(None, true).unwrap();
|
||||||
|
let b = MasterKey::resolve(None, true).unwrap();
|
||||||
|
assert_eq!(a.as_bytes(), b.as_bytes(), "dev key must be deterministic");
|
||||||
|
// A real key always wins over dev mode.
|
||||||
|
let raw = [9u8; KEY_LEN];
|
||||||
|
let real = MasterKey::resolve(Some(&B64.encode(raw)), true).unwrap();
|
||||||
|
assert_eq!(real.as_bytes(), &raw);
|
||||||
|
assert_ne!(real.as_bytes(), a.as_bytes());
|
||||||
|
}
|
||||||
|
}
|
||||||
89
crates/shared/src/email.rs
Normal file
89
crates/shared/src/email.rs
Normal file
@@ -0,0 +1,89 @@
|
|||||||
|
//! `EmailService` — the v1.1.7 outbound email contract.
|
||||||
|
//!
|
||||||
|
//! Scripts get `email::send(#{...})` (plain text) and
|
||||||
|
//! `email::send_html(#{...})` (multipart text + HTML). Both route to the
|
||||||
|
//! single `send` trait method with an [`OutboundEmail`]; the bridge sets
|
||||||
|
//! `html` only for `send_html`.
|
||||||
|
//!
|
||||||
|
//! Lives in `picloud-shared` (not `manager-core`) so the Rhai bridge and
|
||||||
|
//! the impl share one trait. The impl (an SMTP relay over `lettre`)
|
||||||
|
//! lives in `manager-core::email_service`; `picloud-shared` stays free
|
||||||
|
//! of the `lettre` dependency.
|
||||||
|
//!
|
||||||
|
//! `app_id` is derived from `cx.app_id` (authz only — there is no
|
||||||
|
//! per-app `from` validation in v1.1.7; deliverability is the operator's
|
||||||
|
//! SMTP-relay concern).
|
||||||
|
|
||||||
|
use async_trait::async_trait;
|
||||||
|
use thiserror::Error;
|
||||||
|
|
||||||
|
use crate::SdkCallCx;
|
||||||
|
|
||||||
|
/// A single outbound message. `to`/`cc`/`bcc` are address lists (the
|
||||||
|
/// bridge accepts a String or an Array of Strings). At least one of
|
||||||
|
/// `text` / `html` must be present.
|
||||||
|
#[derive(Debug, Clone, Default)]
|
||||||
|
pub struct OutboundEmail {
|
||||||
|
pub to: Vec<String>,
|
||||||
|
pub cc: Vec<String>,
|
||||||
|
pub bcc: Vec<String>,
|
||||||
|
pub from: String,
|
||||||
|
/// Defaults to `from` when absent.
|
||||||
|
pub reply_to: Option<String>,
|
||||||
|
pub subject: String,
|
||||||
|
pub text: Option<String>,
|
||||||
|
pub html: Option<String>,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[async_trait]
|
||||||
|
pub trait EmailService: Send + Sync {
|
||||||
|
/// Validate, build, and send the message. Returns `Ok(())` once the
|
||||||
|
/// SMTP relay has accepted it for delivery (not on actual delivery —
|
||||||
|
/// that's the relay's job).
|
||||||
|
async fn send(&self, cx: &SdkCallCx, email: OutboundEmail) -> Result<(), EmailError>;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Failure modes surfaced to the Rhai bridge.
|
||||||
|
#[derive(Debug, Error)]
|
||||||
|
pub enum EmailError {
|
||||||
|
/// Caller principal lacked `AppEmailSend`. Only raised when
|
||||||
|
/// `cx.principal.is_some()` (script-as-gate semantics).
|
||||||
|
#[error("forbidden")]
|
||||||
|
Forbidden,
|
||||||
|
|
||||||
|
/// A required field (`to`, `from`, `subject`, or one of `text`/`html`)
|
||||||
|
/// was missing or empty.
|
||||||
|
#[error("missing required email field: {0}")]
|
||||||
|
MissingField(String),
|
||||||
|
|
||||||
|
/// An address failed basic RFC 5322-ish validation.
|
||||||
|
#[error("invalid email address: {0}")]
|
||||||
|
InvalidAddress(String),
|
||||||
|
|
||||||
|
/// The assembled message exceeded the per-message size cap.
|
||||||
|
#[error("email too large: {actual} bytes exceeds the {limit}-byte limit")]
|
||||||
|
TooLarge { limit: usize, actual: usize },
|
||||||
|
|
||||||
|
/// No SMTP relay is configured (HOST/USER/PASSWORD unset). Every
|
||||||
|
/// `send` fails until the operator configures one.
|
||||||
|
#[error(
|
||||||
|
"email is not configured: set PICLOUD_SMTP_HOST/USER/PASSWORD to enable outbound email"
|
||||||
|
)]
|
||||||
|
NotConfigured,
|
||||||
|
|
||||||
|
/// The SMTP relay rejected the message or the connection failed.
|
||||||
|
#[error("email transport error: {0}")]
|
||||||
|
Transport(String),
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Stub used by test harnesses that build a `Services` bundle without an
|
||||||
|
/// SMTP relay. Every call returns `EmailError::NotConfigured`.
|
||||||
|
#[derive(Debug, Default, Clone, Copy)]
|
||||||
|
pub struct NoopEmailService;
|
||||||
|
|
||||||
|
#[async_trait]
|
||||||
|
impl EmailService for NoopEmailService {
|
||||||
|
async fn send(&self, _cx: &SdkCallCx, _email: OutboundEmail) -> Result<(), EmailError> {
|
||||||
|
Err(EmailError::NotConfigured)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -177,8 +177,11 @@ pub enum FilesError {
|
|||||||
|
|
||||||
impl NewFile {
|
impl NewFile {
|
||||||
/// Validate required fields + length caps at the SDK boundary.
|
/// Validate required fields + length caps at the SDK boundary.
|
||||||
/// `data` must be non-empty (v1.1.5 treats an empty blob as a
|
///
|
||||||
/// missing `data` field — see HANDBACK §7).
|
/// Empty `data` is **accepted** as a valid stored state (v1.1.6
|
||||||
|
/// relaxed the v1.1.5 rejection — empty files are a legitimate use
|
||||||
|
/// case: sentinels, placeholders, zero-byte uploads. See HANDBACK
|
||||||
|
/// §7). `name` and `content_type` are still required.
|
||||||
///
|
///
|
||||||
/// # Errors
|
/// # Errors
|
||||||
///
|
///
|
||||||
@@ -191,9 +194,6 @@ impl NewFile {
|
|||||||
if self.content_type.trim().is_empty() {
|
if self.content_type.trim().is_empty() {
|
||||||
return Err(FilesError::MissingField("content_type"));
|
return Err(FilesError::MissingField("content_type"));
|
||||||
}
|
}
|
||||||
if self.data.is_empty() {
|
|
||||||
return Err(FilesError::MissingField("data"));
|
|
||||||
}
|
|
||||||
if self.name.len() > MAX_FILE_NAME_BYTES {
|
if self.name.len() > MAX_FILE_NAME_BYTES {
|
||||||
return Err(FilesError::NameTooLong(self.name.len()));
|
return Err(FilesError::NameTooLong(self.name.len()));
|
||||||
}
|
}
|
||||||
@@ -218,9 +218,9 @@ impl FileUpdate {
|
|||||||
/// Returns the field-specific [`FilesError`] for the first failing
|
/// Returns the field-specific [`FilesError`] for the first failing
|
||||||
/// check.
|
/// check.
|
||||||
pub fn validate(&self, max_size: usize) -> Result<(), FilesError> {
|
pub fn validate(&self, max_size: usize) -> Result<(), FilesError> {
|
||||||
if self.data.is_empty() {
|
// Empty replacement bytes are accepted (v1.1.6 relaxation —
|
||||||
return Err(FilesError::MissingField("data"));
|
// consistent with NewFile::validate; updating a file to zero
|
||||||
}
|
// bytes is as legitimate as creating one).
|
||||||
if let Some(name) = &self.name {
|
if let Some(name) = &self.name {
|
||||||
if name.trim().is_empty() {
|
if name.trim().is_empty() {
|
||||||
return Err(FilesError::MissingField("name"));
|
return Err(FilesError::MissingField("name"));
|
||||||
|
|||||||
@@ -6,8 +6,10 @@
|
|||||||
|
|
||||||
pub mod app;
|
pub mod app;
|
||||||
pub mod auth;
|
pub mod auth;
|
||||||
|
pub mod crypto;
|
||||||
pub mod dead_letters;
|
pub mod dead_letters;
|
||||||
pub mod docs;
|
pub mod docs;
|
||||||
|
pub mod email;
|
||||||
pub mod error;
|
pub mod error;
|
||||||
pub mod events;
|
pub mod events;
|
||||||
pub mod exec_summary;
|
pub mod exec_summary;
|
||||||
@@ -21,19 +23,25 @@ pub mod log_sink;
|
|||||||
pub mod modules;
|
pub mod modules;
|
||||||
pub mod outbox_writer;
|
pub mod outbox_writer;
|
||||||
pub mod pubsub;
|
pub mod pubsub;
|
||||||
|
pub mod realtime;
|
||||||
|
pub mod realtime_authority;
|
||||||
pub mod route;
|
pub mod route;
|
||||||
pub mod sandbox;
|
pub mod sandbox;
|
||||||
pub mod script;
|
pub mod script;
|
||||||
pub mod sdk_cx;
|
pub mod sdk_cx;
|
||||||
|
pub mod secrets;
|
||||||
pub mod services;
|
pub mod services;
|
||||||
|
pub mod subscriber_token;
|
||||||
pub mod trigger_event;
|
pub mod trigger_event;
|
||||||
pub mod validator;
|
pub mod validator;
|
||||||
pub mod version;
|
pub mod version;
|
||||||
|
|
||||||
pub use app::{App, AppDomain, DomainShape};
|
pub use app::{App, AppDomain, DomainShape};
|
||||||
pub use auth::{AppRole, InstanceRole, Principal, Scope, UserId};
|
pub use auth::{AppRole, InstanceRole, Principal, Scope, UserId};
|
||||||
|
pub use crypto::{decrypt, encrypt, CryptoError, EncryptResult, MasterKey, MasterKeyError};
|
||||||
pub use dead_letters::{DeadLetterError, DeadLetterId, DeadLetterService, NoopDeadLetterService};
|
pub use dead_letters::{DeadLetterError, DeadLetterId, DeadLetterService, NoopDeadLetterService};
|
||||||
pub use docs::{DocId, DocRow, DocsError, DocsListPage, DocsService, NoopDocsService};
|
pub use docs::{DocId, DocRow, DocsError, DocsListPage, DocsService, NoopDocsService};
|
||||||
|
pub use email::{EmailError, EmailService, NoopEmailService, OutboundEmail};
|
||||||
pub use error::Error;
|
pub use error::Error;
|
||||||
pub use events::{EmitError, NoopEventEmitter, ServiceEvent, ServiceEventEmitter};
|
pub use events::{EmitError, NoopEventEmitter, ServiceEvent, ServiceEventEmitter};
|
||||||
pub use exec_summary::ExecResponseSummary;
|
pub use exec_summary::ExecResponseSummary;
|
||||||
@@ -54,10 +62,16 @@ pub use outbox_writer::{HttpDispatchPayload, NewHttpOutbox, OutboxWriter, Outbox
|
|||||||
pub use pubsub::{
|
pub use pubsub::{
|
||||||
topic_matches, validate_topic_pattern, NoopPubsubService, PubsubError, PubsubService,
|
topic_matches, validate_topic_pattern, NoopPubsubService, PubsubError, PubsubService,
|
||||||
};
|
};
|
||||||
|
pub use realtime::{BroadcasterError, NoopRealtimeBroadcaster, RealtimeBroadcaster, RealtimeEvent};
|
||||||
|
pub use realtime_authority::{DenyAllRealtimeAuthority, RealtimeAuthority, SubscribeDenied};
|
||||||
pub use route::{DispatchMode, HostKind, PathKind, Route};
|
pub use route::{DispatchMode, HostKind, PathKind, Route};
|
||||||
pub use sandbox::ScriptSandbox;
|
pub use sandbox::ScriptSandbox;
|
||||||
pub use script::{Script, ScriptKind};
|
pub use script::{Script, ScriptKind};
|
||||||
pub use sdk_cx::SdkCallCx;
|
pub use sdk_cx::SdkCallCx;
|
||||||
|
pub use secrets::{
|
||||||
|
validate_secret_name, NoopSecretsService, SecretsError, SecretsListPage, SecretsService,
|
||||||
|
SECRET_NAME_MAX_BYTES,
|
||||||
|
};
|
||||||
pub use services::Services;
|
pub use services::Services;
|
||||||
pub use trigger_event::{
|
pub use trigger_event::{
|
||||||
DeadLetterEventDetail, DocsEventOp, FilesEventOp, KvEventOp, TriggerEvent,
|
DeadLetterEventDetail, DocsEventOp, FilesEventOp, KvEventOp, TriggerEvent,
|
||||||
|
|||||||
@@ -30,6 +30,32 @@ pub trait PubsubService: Send + Sync {
|
|||||||
topic: &str,
|
topic: &str,
|
||||||
message: serde_json::Value,
|
message: serde_json::Value,
|
||||||
) -> Result<(), PubsubError>;
|
) -> Result<(), PubsubError>;
|
||||||
|
|
||||||
|
/// Mint an HMAC-signed realtime subscriber token (v1.1.6). Backs the
|
||||||
|
/// `pubsub::subscriber_token(topics, ttl)` Rhai SDK call. The minted
|
||||||
|
/// token authorizes an external SSE client to subscribe to the given
|
||||||
|
/// `topics` for `ttl_seconds` (clamped to the configured bounds; the
|
||||||
|
/// configured default applies when `ttl_seconds` is `None`).
|
||||||
|
///
|
||||||
|
/// Every topic must already be registered as externally subscribable
|
||||||
|
/// in `cx.app_id`; `cx.principal` must be `Some` (anonymous
|
||||||
|
/// public-HTTP scripts can't mint). See [`PubsubError::SubscriberToken`]
|
||||||
|
/// for the rejection messages.
|
||||||
|
///
|
||||||
|
/// The default impl errors `Unavailable` so test fakes and the
|
||||||
|
/// `NoopPubsubService` keep compiling; the real minting lives in
|
||||||
|
/// manager-core's `PubsubServiceImpl`.
|
||||||
|
async fn mint_subscriber_token(
|
||||||
|
&self,
|
||||||
|
cx: &SdkCallCx,
|
||||||
|
topics: Vec<String>,
|
||||||
|
ttl_seconds: Option<i64>,
|
||||||
|
) -> Result<String, PubsubError> {
|
||||||
|
let _ = (cx, topics, ttl_seconds);
|
||||||
|
Err(PubsubError::Unavailable(
|
||||||
|
"subscriber tokens are not wired in".into(),
|
||||||
|
))
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
#[derive(Debug, Error)]
|
#[derive(Debug, Error)]
|
||||||
@@ -47,6 +73,13 @@ pub enum PubsubError {
|
|||||||
#[error("pubsub rejected: {0}")]
|
#[error("pubsub rejected: {0}")]
|
||||||
Rejected(String),
|
Rejected(String),
|
||||||
|
|
||||||
|
/// A `pubsub::subscriber_token` mint was rejected (empty topics,
|
||||||
|
/// unregistered topic, ttl out of range, anonymous caller). The
|
||||||
|
/// string is the full user-facing message; the SDK surfaces it
|
||||||
|
/// verbatim so scripts see the documented wording.
|
||||||
|
#[error("{0}")]
|
||||||
|
SubscriberToken(String),
|
||||||
|
|
||||||
/// Anything else — Postgres unavailable, etc.
|
/// Anything else — Postgres unavailable, etc.
|
||||||
#[error("pubsub backend error: {0}")]
|
#[error("pubsub backend error: {0}")]
|
||||||
Unavailable(String),
|
Unavailable(String),
|
||||||
|
|||||||
86
crates/shared/src/realtime.rs
Normal file
86
crates/shared/src/realtime.rs
Normal file
@@ -0,0 +1,86 @@
|
|||||||
|
//! `RealtimeBroadcaster` — the in-process fan-out seam for SSE realtime
|
||||||
|
//! delivery (v1.1.6).
|
||||||
|
//!
|
||||||
|
//! Structurally a sibling of [`crate::inbox::InboxResolver`]: the trait
|
||||||
|
//! lives here in `picloud-shared` because the publish side
|
||||||
|
//! (`PubsubServiceImpl` in manager-core) and the subscribe side (the SSE
|
||||||
|
//! handler in orchestrator-core) live in different crates and both need
|
||||||
|
//! one shared instance. The in-process impl lives in orchestrator-core
|
||||||
|
//! (`Mutex<HashMap<(AppId, topic), broadcast::Sender>>`); cluster mode
|
||||||
|
//! (v1.3+) swaps it for a Postgres `LISTEN/NOTIFY`-backed resolver behind
|
||||||
|
//! the same trait without touching either caller.
|
||||||
|
//!
|
||||||
|
//! Delivery is **best-effort, at-most-once**: this is the realtime path,
|
||||||
|
//! NOT the durable one. Durable trigger fan-out (retry / dead-letter)
|
||||||
|
//! goes through the outbox and is the publish caller's separate concern.
|
||||||
|
//! A slow SSE consumer loses the oldest events (bounded broadcast
|
||||||
|
//! buffer); SSE's own transport-layer auto-reconnect is the recovery
|
||||||
|
//! mechanism (no server-side replay in v1.1.6).
|
||||||
|
|
||||||
|
use async_trait::async_trait;
|
||||||
|
use chrono::{DateTime, Utc};
|
||||||
|
use thiserror::Error;
|
||||||
|
use tokio::sync::broadcast;
|
||||||
|
|
||||||
|
use crate::AppId;
|
||||||
|
|
||||||
|
/// A single realtime event delivered to in-process SSE subscribers. The
|
||||||
|
/// SSE handler serializes this to `data: {...}\n\n` on the wire.
|
||||||
|
#[derive(Debug, Clone)]
|
||||||
|
pub struct RealtimeEvent {
|
||||||
|
pub topic: String,
|
||||||
|
pub message: serde_json::Value,
|
||||||
|
pub published_at: DateTime<Utc>,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, Error)]
|
||||||
|
pub enum BroadcasterError {
|
||||||
|
/// Reserved for backends that can fail to register a subscriber
|
||||||
|
/// (e.g. the cluster-mode `LISTEN/NOTIFY` resolver). The in-process
|
||||||
|
/// impl never returns this.
|
||||||
|
#[error("realtime broadcaster unavailable: {0}")]
|
||||||
|
Unavailable(String),
|
||||||
|
}
|
||||||
|
|
||||||
|
#[async_trait]
|
||||||
|
pub trait RealtimeBroadcaster: Send + Sync {
|
||||||
|
/// Subscribe to events on `(app_id, topic)`. Returns a receiver that
|
||||||
|
/// yields events until dropped. Channels are created lazily on first
|
||||||
|
/// subscribe.
|
||||||
|
async fn subscribe(
|
||||||
|
&self,
|
||||||
|
app_id: AppId,
|
||||||
|
topic: &str,
|
||||||
|
) -> Result<broadcast::Receiver<RealtimeEvent>, BroadcasterError>;
|
||||||
|
|
||||||
|
/// Publish an event to in-process subscribers. NOT durable — the
|
||||||
|
/// outbox-backed durable fan-out is the publish caller's separate
|
||||||
|
/// concern. A publish with no live subscribers is a silent no-op.
|
||||||
|
async fn publish(&self, app_id: AppId, topic: &str, event: RealtimeEvent);
|
||||||
|
|
||||||
|
/// Drop every subscriber for a topic (called on topic DELETE). Live
|
||||||
|
/// receivers observe a closed channel and disconnect cleanly.
|
||||||
|
async fn drop_topic(&self, app_id: AppId, topic: &str);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Bootstrap / test impl: subscribe yields a receiver on a throwaway
|
||||||
|
/// channel, publish is a no-op. Lets a `Services`-style bundle build
|
||||||
|
/// without the real registry wired in.
|
||||||
|
#[derive(Debug, Default, Clone, Copy)]
|
||||||
|
pub struct NoopRealtimeBroadcaster;
|
||||||
|
|
||||||
|
#[async_trait]
|
||||||
|
impl RealtimeBroadcaster for NoopRealtimeBroadcaster {
|
||||||
|
async fn subscribe(
|
||||||
|
&self,
|
||||||
|
_app_id: AppId,
|
||||||
|
_topic: &str,
|
||||||
|
) -> Result<broadcast::Receiver<RealtimeEvent>, BroadcasterError> {
|
||||||
|
let (_tx, rx) = broadcast::channel(1);
|
||||||
|
Ok(rx)
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn publish(&self, _app_id: AppId, _topic: &str, _event: RealtimeEvent) {}
|
||||||
|
|
||||||
|
async fn drop_topic(&self, _app_id: AppId, _topic: &str) {}
|
||||||
|
}
|
||||||
70
crates/shared/src/realtime_authority.rs
Normal file
70
crates/shared/src/realtime_authority.rs
Normal file
@@ -0,0 +1,70 @@
|
|||||||
|
//! `RealtimeAuthority` — the SSE subscribe authorization seam (v1.1.6).
|
||||||
|
//!
|
||||||
|
//! The SSE endpoint (`GET /realtime/topics/{topic}`) is a data-plane
|
||||||
|
//! surface in orchestrator-core, but deciding whether a subscribe is
|
||||||
|
//! allowed needs a `topics` table read plus (for token-gated topics) an
|
||||||
|
//! HMAC verify against the app's signing key — both of which require DB
|
||||||
|
//! access and the signing-key material that must NOT leak into the
|
||||||
|
//! data-plane crate. This trait keeps all of that inside the manager-core
|
||||||
|
//! impl: orchestrator-core only ever sees the three-way verdict below.
|
||||||
|
//!
|
||||||
|
//! `NotFound` is deliberately returned for *both* "no such topic" and
|
||||||
|
//! "topic exists but isn't externally subscribable" so the endpoint
|
||||||
|
//! can't be used to probe which internal topics exist (design notes §5).
|
||||||
|
|
||||||
|
use async_trait::async_trait;
|
||||||
|
|
||||||
|
use crate::AppId;
|
||||||
|
|
||||||
|
/// Why a subscribe attempt was refused. The SSE handler maps these to
|
||||||
|
/// HTTP status codes.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||||
|
pub enum SubscribeDenied {
|
||||||
|
/// No externally-subscribable topic by that name in this app → 404.
|
||||||
|
/// Used for genuinely-missing topics AND internal-only ones, so the
|
||||||
|
/// endpoint doesn't leak which internal topics exist.
|
||||||
|
NotFound,
|
||||||
|
/// The topic is token-gated and the presented token was missing,
|
||||||
|
/// malformed, badly signed, expired, or not scoped to this topic →
|
||||||
|
/// 401 (generic; never says which check failed).
|
||||||
|
Unauthorized,
|
||||||
|
/// Backend failure (DB unavailable, etc.) → 500.
|
||||||
|
Backend(String),
|
||||||
|
}
|
||||||
|
|
||||||
|
#[async_trait]
|
||||||
|
pub trait RealtimeAuthority: Send + Sync {
|
||||||
|
/// Decide whether an external client may subscribe to
|
||||||
|
/// `(app_id, topic)`. `token` is the bearer/query token if the
|
||||||
|
/// client presented one (`None` otherwise).
|
||||||
|
///
|
||||||
|
/// Returns `Ok(())` when the subscribe is permitted (public topic,
|
||||||
|
/// or token-gated topic with a valid token scoped to it).
|
||||||
|
///
|
||||||
|
/// # Errors
|
||||||
|
///
|
||||||
|
/// [`SubscribeDenied`] — see the variants for the status mapping.
|
||||||
|
async fn authorize_subscribe(
|
||||||
|
&self,
|
||||||
|
app_id: AppId,
|
||||||
|
topic: &str,
|
||||||
|
token: Option<&str>,
|
||||||
|
) -> Result<(), SubscribeDenied>;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Bootstrap impl: denies everything as `NotFound`. Replaced in
|
||||||
|
/// `build_app` with the manager-core DB-backed authority.
|
||||||
|
#[derive(Debug, Default, Clone, Copy)]
|
||||||
|
pub struct DenyAllRealtimeAuthority;
|
||||||
|
|
||||||
|
#[async_trait]
|
||||||
|
impl RealtimeAuthority for DenyAllRealtimeAuthority {
|
||||||
|
async fn authorize_subscribe(
|
||||||
|
&self,
|
||||||
|
_app_id: AppId,
|
||||||
|
_topic: &str,
|
||||||
|
_token: Option<&str>,
|
||||||
|
) -> Result<(), SubscribeDenied> {
|
||||||
|
Err(SubscribeDenied::NotFound)
|
||||||
|
}
|
||||||
|
}
|
||||||
166
crates/shared/src/secrets.rs
Normal file
166
crates/shared/src/secrets.rs
Normal file
@@ -0,0 +1,166 @@
|
|||||||
|
//! `SecretsService` — the v1.1.7 encrypted per-app secrets contract.
|
||||||
|
//!
|
||||||
|
//! Collection-less (per-app, like pubsub): the script API is the bare
|
||||||
|
//! `secrets::{get,set,delete,list}(name)` — there is no
|
||||||
|
//! `secrets::collection(...)`. Secrets are operational config (API keys,
|
||||||
|
//! OAuth tokens, webhook signing keys), encrypted at rest with the
|
||||||
|
//! process master key.
|
||||||
|
//!
|
||||||
|
//! Lives in `picloud-shared` (not `executor-core`) so the Rhai bridge,
|
||||||
|
//! the manager-core Postgres impl, and test fakes can all depend on the
|
||||||
|
//! same trait. Implementations MUST derive every storage `app_id` from
|
||||||
|
//! `cx.app_id` — never from a script-passed argument. That is the
|
||||||
|
//! cross-app isolation boundary; see `docs/sdk-shape.md`.
|
||||||
|
//!
|
||||||
|
//! Values are JSON internally: `set` accepts any `serde_json::Value`
|
||||||
|
//! (the bridge maps a Rhai String/Map/Array to JSON), encrypts the
|
||||||
|
//! encoded bytes, and `get` decrypts + decodes back to the same JSON
|
||||||
|
//! shape — so a String round-trips to a String, not a JSON-quoted
|
||||||
|
//! `"\"…\""`. There is deliberately **no `ServiceEvent` emission**:
|
||||||
|
//! firing triggers on secret writes is a footgun (every rotation would
|
||||||
|
//! fan out handler executions that might log the new value).
|
||||||
|
|
||||||
|
use async_trait::async_trait;
|
||||||
|
use thiserror::Error;
|
||||||
|
|
||||||
|
use crate::SdkCallCx;
|
||||||
|
|
||||||
|
/// Maximum secret name length in bytes (matches the brief: 255).
|
||||||
|
pub const SECRET_NAME_MAX_BYTES: usize = 255;
|
||||||
|
|
||||||
|
/// `SecretsService` is collection-less and per-app. Every method derives
|
||||||
|
/// the owning `app_id` from `cx.app_id`.
|
||||||
|
#[async_trait]
|
||||||
|
pub trait SecretsService: Send + Sync {
|
||||||
|
/// Decrypt and return the secret, or `None` if no secret with this
|
||||||
|
/// name exists for the app.
|
||||||
|
async fn get(
|
||||||
|
&self,
|
||||||
|
cx: &SdkCallCx,
|
||||||
|
name: &str,
|
||||||
|
) -> Result<Option<serde_json::Value>, SecretsError>;
|
||||||
|
|
||||||
|
/// Encrypt and store the secret, overwriting any existing value for
|
||||||
|
/// this name.
|
||||||
|
async fn set(
|
||||||
|
&self,
|
||||||
|
cx: &SdkCallCx,
|
||||||
|
name: &str,
|
||||||
|
value: serde_json::Value,
|
||||||
|
) -> Result<(), SecretsError>;
|
||||||
|
|
||||||
|
/// Delete the secret. Returns whether a secret was present.
|
||||||
|
async fn delete(&self, cx: &SdkCallCx, name: &str) -> Result<bool, SecretsError>;
|
||||||
|
|
||||||
|
/// List secret **names only** (never values), cursor-paginated like
|
||||||
|
/// KV/files `list`. `cursor` is opaque; `None` starts from the
|
||||||
|
/// beginning.
|
||||||
|
async fn list(
|
||||||
|
&self,
|
||||||
|
cx: &SdkCallCx,
|
||||||
|
cursor: Option<&str>,
|
||||||
|
limit: u32,
|
||||||
|
) -> Result<SecretsListPage, SecretsError>;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One page of secret names from `SecretsService::list`. `next_cursor`
|
||||||
|
/// is `Some` when more pages exist.
|
||||||
|
#[derive(Debug, Clone)]
|
||||||
|
pub struct SecretsListPage {
|
||||||
|
pub names: Vec<String>,
|
||||||
|
pub next_cursor: Option<String>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Failure modes surfaced to the Rhai bridge. The bridge converts each
|
||||||
|
/// to a Rhai runtime error string.
|
||||||
|
#[derive(Debug, Error)]
|
||||||
|
pub enum SecretsError {
|
||||||
|
/// Empty name, or a name longer than [`SECRET_NAME_MAX_BYTES`].
|
||||||
|
#[error("{0}")]
|
||||||
|
InvalidName(String),
|
||||||
|
|
||||||
|
/// The encoded plaintext exceeded the configured per-secret cap.
|
||||||
|
#[error("secret value too large: {actual} bytes exceeds the {limit}-byte limit")]
|
||||||
|
TooLarge { limit: usize, actual: usize },
|
||||||
|
|
||||||
|
/// Caller principal lacked the required capability. Only raised when
|
||||||
|
/// `cx.principal.is_some()` — public-HTTP scripts (`principal: None`)
|
||||||
|
/// operate under script-as-gate semantics and skip the check.
|
||||||
|
#[error("forbidden")]
|
||||||
|
Forbidden,
|
||||||
|
|
||||||
|
/// The stored ciphertext could not be decrypted (corrupted row,
|
||||||
|
/// wrong master key, or tampering). The impl logs the affected
|
||||||
|
/// `(app_id, name)` at error level before returning this.
|
||||||
|
#[error("secret is corrupted or was encrypted with a different master key")]
|
||||||
|
Corrupted,
|
||||||
|
|
||||||
|
/// The process master key was unavailable. Startup should already
|
||||||
|
/// have failed; this is defense in depth.
|
||||||
|
#[error("master key is not configured")]
|
||||||
|
MasterKeyMissing,
|
||||||
|
|
||||||
|
/// Anything else — Postgres unavailable, serialization failure, etc.
|
||||||
|
#[error("secrets backend error: {0}")]
|
||||||
|
Backend(String),
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Stub used by the executor-core test harness (which doesn't touch
|
||||||
|
/// secrets) so a `Services` bundle can be built without Postgres. Every
|
||||||
|
/// call returns `SecretsError::Backend(...)` so accidental use surfaces.
|
||||||
|
#[derive(Debug, Default, Clone, Copy)]
|
||||||
|
pub struct NoopSecretsService;
|
||||||
|
|
||||||
|
#[async_trait]
|
||||||
|
impl SecretsService for NoopSecretsService {
|
||||||
|
async fn get(
|
||||||
|
&self,
|
||||||
|
_cx: &SdkCallCx,
|
||||||
|
_name: &str,
|
||||||
|
) -> Result<Option<serde_json::Value>, SecretsError> {
|
||||||
|
Err(SecretsError::Backend("secrets is not wired in".into()))
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn set(
|
||||||
|
&self,
|
||||||
|
_cx: &SdkCallCx,
|
||||||
|
_name: &str,
|
||||||
|
_value: serde_json::Value,
|
||||||
|
) -> Result<(), SecretsError> {
|
||||||
|
Err(SecretsError::Backend("secrets is not wired in".into()))
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn delete(&self, _cx: &SdkCallCx, _name: &str) -> Result<bool, SecretsError> {
|
||||||
|
Err(SecretsError::Backend("secrets is not wired in".into()))
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn list(
|
||||||
|
&self,
|
||||||
|
_cx: &SdkCallCx,
|
||||||
|
_cursor: Option<&str>,
|
||||||
|
_limit: u32,
|
||||||
|
) -> Result<SecretsListPage, SecretsError> {
|
||||||
|
Err(SecretsError::Backend("secrets is not wired in".into()))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Validate a secret name at the SDK/admin boundary: non-empty and at
|
||||||
|
/// most [`SECRET_NAME_MAX_BYTES`] bytes.
|
||||||
|
///
|
||||||
|
/// # Errors
|
||||||
|
///
|
||||||
|
/// Returns [`SecretsError::InvalidName`] when empty or too long.
|
||||||
|
pub fn validate_secret_name(name: &str) -> Result<(), SecretsError> {
|
||||||
|
if name.is_empty() {
|
||||||
|
return Err(SecretsError::InvalidName(
|
||||||
|
"secret name must not be empty".into(),
|
||||||
|
));
|
||||||
|
}
|
||||||
|
if name.len() > SECRET_NAME_MAX_BYTES {
|
||||||
|
return Err(SecretsError::InvalidName(format!(
|
||||||
|
"secret name must be at most {SECRET_NAME_MAX_BYTES} bytes, got {}",
|
||||||
|
name.len()
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
@@ -20,9 +20,10 @@
|
|||||||
use std::sync::Arc;
|
use std::sync::Arc;
|
||||||
|
|
||||||
use crate::{
|
use crate::{
|
||||||
DeadLetterService, DocsService, FilesService, HttpService, KvService, ModuleSource,
|
DeadLetterService, DocsService, EmailService, FilesService, HttpService, KvService,
|
||||||
NoopDeadLetterService, NoopDocsService, NoopEventEmitter, NoopFilesService, NoopHttpService,
|
ModuleSource, NoopDeadLetterService, NoopDocsService, NoopEmailService, NoopEventEmitter,
|
||||||
NoopKvService, NoopModuleSource, NoopPubsubService, PubsubService, ServiceEventEmitter,
|
NoopFilesService, NoopHttpService, NoopKvService, NoopModuleSource, NoopPubsubService,
|
||||||
|
NoopSecretsService, PubsubService, SecretsService, ServiceEventEmitter,
|
||||||
};
|
};
|
||||||
|
|
||||||
/// SDK service bundle. See module docs for the lifecycle and the v1.1.x
|
/// SDK service bundle. See module docs for the lifecycle and the v1.1.x
|
||||||
@@ -73,6 +74,18 @@ pub struct Services {
|
|||||||
/// publish-time outbox fan-out in the picloud binary;
|
/// publish-time outbox fan-out in the picloud binary;
|
||||||
/// `NoopPubsubService` in tests that don't publish.
|
/// `NoopPubsubService` in tests that don't publish.
|
||||||
pub pubsub: Arc<dyn PubsubService>,
|
pub pubsub: Arc<dyn PubsubService>,
|
||||||
|
|
||||||
|
/// Encrypted per-app secrets (v1.1.7). Scripts get
|
||||||
|
/// `secrets::{get,set,delete,list}(name)`. Backed by an
|
||||||
|
/// AES-256-GCM-at-rest Postgres repo in the picloud binary;
|
||||||
|
/// `NoopSecretsService` in tests that don't touch secrets.
|
||||||
|
pub secrets: Arc<dyn SecretsService>,
|
||||||
|
|
||||||
|
/// Outbound email (v1.1.7). Scripts get `email::{send,send_html}`.
|
||||||
|
/// Backed by an SMTP relay (lettre) in the picloud binary;
|
||||||
|
/// `NoopEmailService` (always `NotConfigured`) in tests that don't
|
||||||
|
/// send mail.
|
||||||
|
pub email: Arc<dyn EmailService>,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl Services {
|
impl Services {
|
||||||
@@ -90,6 +103,8 @@ impl Services {
|
|||||||
http: Arc<dyn HttpService>,
|
http: Arc<dyn HttpService>,
|
||||||
files: Arc<dyn FilesService>,
|
files: Arc<dyn FilesService>,
|
||||||
pubsub: Arc<dyn PubsubService>,
|
pubsub: Arc<dyn PubsubService>,
|
||||||
|
secrets: Arc<dyn SecretsService>,
|
||||||
|
email: Arc<dyn EmailService>,
|
||||||
) -> Self {
|
) -> Self {
|
||||||
Self {
|
Self {
|
||||||
kv,
|
kv,
|
||||||
@@ -100,6 +115,8 @@ impl Services {
|
|||||||
http,
|
http,
|
||||||
files,
|
files,
|
||||||
pubsub,
|
pubsub,
|
||||||
|
secrets,
|
||||||
|
email,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -119,6 +136,8 @@ impl Services {
|
|||||||
Arc::new(NoopHttpService),
|
Arc::new(NoopHttpService),
|
||||||
Arc::new(NoopFilesService),
|
Arc::new(NoopFilesService),
|
||||||
Arc::new(NoopPubsubService),
|
Arc::new(NoopPubsubService),
|
||||||
|
Arc::new(NoopSecretsService),
|
||||||
|
Arc::new(NoopEmailService),
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
200
crates/shared/src/subscriber_token.rs
Normal file
200
crates/shared/src/subscriber_token.rs
Normal file
@@ -0,0 +1,200 @@
|
|||||||
|
//! HMAC-signed realtime subscriber tokens (v1.1.6, design notes §5).
|
||||||
|
//!
|
||||||
|
//! A token is a compact, URL-safe, two-part string:
|
||||||
|
//!
|
||||||
|
//! ```text
|
||||||
|
//! <base64url(payload)>.<base64url(signature)>
|
||||||
|
//! ```
|
||||||
|
//!
|
||||||
|
//! where `payload` is the JSON [`TokenClaims`] and `signature` is
|
||||||
|
//! `HMAC-SHA256(app_signing_key, base64url(payload))`. Tokens are minted
|
||||||
|
//! by scripts via `pubsub::subscriber_token` (the minter lives in
|
||||||
|
//! manager-core's `PubsubServiceImpl`) and verified by the SSE subscribe
|
||||||
|
//! path (the verifier lives in manager-core's `RealtimeAuthority` impl).
|
||||||
|
//! Both sides depend on this module so the byte-for-byte contract has a
|
||||||
|
//! single home.
|
||||||
|
//!
|
||||||
|
//! There is no per-token revocation in v1.1.6 by design: HMAC bearers
|
||||||
|
//! can't be individually revoked. Rotating an app's signing key
|
||||||
|
//! invalidates every token for that app wholesale; short TTLs are the
|
||||||
|
//! safety mechanism.
|
||||||
|
|
||||||
|
use base64::engine::general_purpose::URL_SAFE_NO_PAD;
|
||||||
|
use base64::Engine as _;
|
||||||
|
use hmac::{Hmac, Mac};
|
||||||
|
use serde::{Deserialize, Serialize};
|
||||||
|
use sha2::Sha256;
|
||||||
|
use thiserror::Error;
|
||||||
|
|
||||||
|
use crate::AppId;
|
||||||
|
|
||||||
|
type HmacSha256 = Hmac<Sha256>;
|
||||||
|
|
||||||
|
/// The signed payload. `exp` / `iat` are Unix seconds.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
pub struct TokenClaims {
|
||||||
|
pub app_id: AppId,
|
||||||
|
pub topics: Vec<String>,
|
||||||
|
pub exp: i64,
|
||||||
|
pub iat: i64,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl TokenClaims {
|
||||||
|
/// Does this token grant access to `topic`?
|
||||||
|
#[must_use]
|
||||||
|
pub fn allows_topic(&self, topic: &str) -> bool {
|
||||||
|
self.topics.iter().any(|t| t == topic)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Is the token expired relative to `now_unix` (Unix seconds)?
|
||||||
|
#[must_use]
|
||||||
|
pub fn is_expired(&self, now_unix: i64) -> bool {
|
||||||
|
now_unix >= self.exp
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, Error, PartialEq, Eq)]
|
||||||
|
pub enum TokenError {
|
||||||
|
#[error("token is malformed")]
|
||||||
|
Malformed,
|
||||||
|
#[error("token signature is invalid")]
|
||||||
|
BadSignature,
|
||||||
|
#[error("token has expired")]
|
||||||
|
Expired,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Sign `claims` with `key`, producing the `payload.signature` string.
|
||||||
|
#[must_use]
|
||||||
|
pub fn sign(key: &[u8], claims: &TokenClaims) -> String {
|
||||||
|
// `serde_json` on a fixed-field struct never fails to serialize.
|
||||||
|
let payload_json = serde_json::to_vec(claims).expect("TokenClaims serialize");
|
||||||
|
let payload_b64 = URL_SAFE_NO_PAD.encode(&payload_json);
|
||||||
|
let sig = mac_sign(key, payload_b64.as_bytes());
|
||||||
|
let sig_b64 = URL_SAFE_NO_PAD.encode(sig);
|
||||||
|
format!("{payload_b64}.{sig_b64}")
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Verify `token` against `key` and check expiry against `now_unix`
|
||||||
|
/// (Unix seconds). Returns the decoded [`TokenClaims`] on success.
|
||||||
|
///
|
||||||
|
/// Topic-scope checking (is the requested topic in the token's list?)
|
||||||
|
/// is the caller's responsibility via [`TokenClaims::allows_topic`] —
|
||||||
|
/// this function proves authenticity + liveness only.
|
||||||
|
///
|
||||||
|
/// # Errors
|
||||||
|
///
|
||||||
|
/// [`TokenError::Malformed`] if the shape / base64 / JSON is wrong,
|
||||||
|
/// [`TokenError::BadSignature`] if the HMAC doesn't match, or
|
||||||
|
/// [`TokenError::Expired`] if `now_unix >= exp`.
|
||||||
|
pub fn verify(key: &[u8], token: &str, now_unix: i64) -> Result<TokenClaims, TokenError> {
|
||||||
|
let (payload_b64, sig_b64) = token.split_once('.').ok_or(TokenError::Malformed)?;
|
||||||
|
|
||||||
|
let provided_sig = URL_SAFE_NO_PAD
|
||||||
|
.decode(sig_b64)
|
||||||
|
.map_err(|_| TokenError::Malformed)?;
|
||||||
|
|
||||||
|
// Constant-time verify of the MAC over the exact payload bytes.
|
||||||
|
let mut mac = HmacSha256::new_from_slice(key).expect("HMAC accepts any key length");
|
||||||
|
mac.update(payload_b64.as_bytes());
|
||||||
|
mac.verify_slice(&provided_sig)
|
||||||
|
.map_err(|_| TokenError::BadSignature)?;
|
||||||
|
|
||||||
|
// Signature good → decode the claims and check expiry.
|
||||||
|
let payload_json = URL_SAFE_NO_PAD
|
||||||
|
.decode(payload_b64)
|
||||||
|
.map_err(|_| TokenError::Malformed)?;
|
||||||
|
let claims: TokenClaims =
|
||||||
|
serde_json::from_slice(&payload_json).map_err(|_| TokenError::Malformed)?;
|
||||||
|
|
||||||
|
if claims.is_expired(now_unix) {
|
||||||
|
return Err(TokenError::Expired);
|
||||||
|
}
|
||||||
|
Ok(claims)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn mac_sign(key: &[u8], data: &[u8]) -> Vec<u8> {
|
||||||
|
let mut mac = HmacSha256::new_from_slice(key).expect("HMAC accepts any key length");
|
||||||
|
mac.update(data);
|
||||||
|
mac.finalize().into_bytes().to_vec()
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
fn claims(app: AppId, topics: &[&str], exp: i64) -> TokenClaims {
|
||||||
|
TokenClaims {
|
||||||
|
app_id: app,
|
||||||
|
topics: topics.iter().map(|s| (*s).to_string()).collect(),
|
||||||
|
iat: 1000,
|
||||||
|
exp,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn round_trip_verifies() {
|
||||||
|
let key = b"super-secret-key-bytes-0123456789";
|
||||||
|
let app = AppId::new();
|
||||||
|
let c = claims(app, &["chat.room.1", "user.notify"], 5000);
|
||||||
|
let token = sign(key, &c);
|
||||||
|
let got = verify(key, &token, 2000).expect("valid token verifies");
|
||||||
|
assert_eq!(got, c);
|
||||||
|
assert!(got.allows_topic("chat.room.1"));
|
||||||
|
assert!(!got.allows_topic("chat.room.2"));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn tampered_payload_fails() {
|
||||||
|
let key = b"super-secret-key-bytes-0123456789";
|
||||||
|
let app = AppId::new();
|
||||||
|
let token = sign(key, &claims(app, &["t"], 5000));
|
||||||
|
// Flip a character in the payload half.
|
||||||
|
let (payload, sig) = token.split_once('.').unwrap();
|
||||||
|
let mut bytes = payload.as_bytes().to_vec();
|
||||||
|
bytes[0] ^= 0x01;
|
||||||
|
let tampered = format!("{}.{sig}", String::from_utf8_lossy(&bytes));
|
||||||
|
assert_eq!(verify(key, &tampered, 2000), Err(TokenError::BadSignature));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn tampered_signature_fails() {
|
||||||
|
let key = b"super-secret-key-bytes-0123456789";
|
||||||
|
let app = AppId::new();
|
||||||
|
let token = sign(key, &claims(app, &["t"], 5000));
|
||||||
|
let (payload, _sig) = token.split_once('.').unwrap();
|
||||||
|
// A valid-base64 but wrong signature.
|
||||||
|
let bogus = URL_SAFE_NO_PAD.encode([0u8; 32]);
|
||||||
|
let tampered = format!("{payload}.{bogus}");
|
||||||
|
assert_eq!(verify(key, &tampered, 2000), Err(TokenError::BadSignature));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn different_key_fails() {
|
||||||
|
let app = AppId::new();
|
||||||
|
let token = sign(
|
||||||
|
b"key-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
||||||
|
&claims(app, &["t"], 5000),
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
verify(b"key-bbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", &token, 2000),
|
||||||
|
Err(TokenError::BadSignature)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn expired_token_fails_at_expiry_check() {
|
||||||
|
let key = b"super-secret-key-bytes-0123456789";
|
||||||
|
let app = AppId::new();
|
||||||
|
let token = sign(key, &claims(app, &["t"], 5000));
|
||||||
|
// now == exp → expired (>= boundary).
|
||||||
|
assert_eq!(verify(key, &token, 5000), Err(TokenError::Expired));
|
||||||
|
assert_eq!(verify(key, &token, 9999), Err(TokenError::Expired));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn malformed_token_fails() {
|
||||||
|
let key = b"super-secret-key-bytes-0123456789";
|
||||||
|
assert_eq!(verify(key, "no-dot-here", 0), Err(TokenError::Malformed));
|
||||||
|
assert_eq!(verify(key, "a.b.c", 0), Err(TokenError::Malformed));
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -186,6 +186,27 @@ pub enum TriggerEvent {
|
|||||||
published_at: DateTime<Utc>,
|
published_at: DateTime<Utc>,
|
||||||
},
|
},
|
||||||
|
|
||||||
|
/// An inbound email (POSTed to the webhook receiver by a configured
|
||||||
|
/// provider) fired this handler. v1.1.7. Carries the normalized
|
||||||
|
/// message; `text`/`html` are absent when the provider sent only the
|
||||||
|
/// other. Surfaced to scripts as `ctx.event.email`. Attachments are
|
||||||
|
/// deferred to v1.2.
|
||||||
|
Email {
|
||||||
|
from: String,
|
||||||
|
to: Vec<String>,
|
||||||
|
#[serde(default)]
|
||||||
|
cc: Vec<String>,
|
||||||
|
subject: String,
|
||||||
|
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||||
|
text: Option<String>,
|
||||||
|
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||||
|
html: Option<String>,
|
||||||
|
received_at: DateTime<Utc>,
|
||||||
|
/// RFC 5322 Message-ID, when the provider supplied one.
|
||||||
|
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||||
|
message_id: Option<String>,
|
||||||
|
},
|
||||||
|
|
||||||
/// A dead-letter row fired this handler. The original event is
|
/// A dead-letter row fired this handler. The original event is
|
||||||
/// nested verbatim plus the dead-letter metadata the design notes
|
/// nested verbatim plus the dead-letter metadata the design notes
|
||||||
/// §4 require.
|
/// §4 require.
|
||||||
@@ -213,6 +234,7 @@ impl TriggerEvent {
|
|||||||
Self::Cron { .. } => "cron",
|
Self::Cron { .. } => "cron",
|
||||||
Self::Files { .. } => "files",
|
Self::Files { .. } => "files",
|
||||||
Self::Pubsub { .. } => "pubsub",
|
Self::Pubsub { .. } => "pubsub",
|
||||||
|
Self::Email { .. } => "email",
|
||||||
Self::DeadLetter { .. } => "dead_letter",
|
Self::DeadLetter { .. } => "dead_letter",
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -49,7 +49,24 @@ pub const PRODUCT_VERSION: &str = env!("CARGO_PKG_VERSION");
|
|||||||
/// publish-time fan-out and `ctx.event.pubsub` for pubsub-trigger
|
/// publish-time fan-out and `ctx.event.pubsub` for pubsub-trigger
|
||||||
/// handlers. The `Services` bundle gains `files: Arc<dyn FilesService>`
|
/// handlers. The `Services` bundle gains `files: Arc<dyn FilesService>`
|
||||||
/// and `pubsub: Arc<dyn PubsubService>`.
|
/// and `pubsub: Arc<dyn PubsubService>`.
|
||||||
pub const SDK_VERSION: &str = "1.6";
|
///
|
||||||
|
/// 1.7 additions (v1.1.6): `pubsub::subscriber_token(topics, ttl)` —
|
||||||
|
/// mints an HMAC-signed realtime subscriber token for externally-
|
||||||
|
/// subscribable topics (requires an authenticated principal). This is
|
||||||
|
/// the only new script-visible surface; the rest of the release is
|
||||||
|
/// server-side (the SSE `/realtime/topics/{topic}` endpoint; the
|
||||||
|
/// `RealtimeBroadcaster` / `RealtimeEvent` / `RealtimeAuthority` traits;
|
||||||
|
/// the `topics` registry + admin endpoints; the `@picloud/client`
|
||||||
|
/// TypeScript package).
|
||||||
|
///
|
||||||
|
/// 1.8 additions (v1.1.7): `secrets::{get,set,delete,list}(name)` —
|
||||||
|
/// encrypted per-app secrets (AES-256-GCM at rest under the process
|
||||||
|
/// master key); `email::{send,send_html}(#{...})` — outbound email via
|
||||||
|
/// an env-configured SMTP relay; and `ctx.event.email` for
|
||||||
|
/// `email:receive`-trigger handlers (inbound email POSTed to the webhook
|
||||||
|
/// receiver). The `Services` bundle gains `secrets: Arc<dyn
|
||||||
|
/// SecretsService>` and `email: Arc<dyn EmailService>`.
|
||||||
|
pub const SDK_VERSION: &str = "1.8";
|
||||||
|
|
||||||
/// HTTP API major version. Appears in URL paths as `/api/v{N}/...`.
|
/// HTTP API major version. Appears in URL paths as `/api/v{N}/...`.
|
||||||
/// Bump (new integer + new URL prefix) when the request/response
|
/// Bump (new integer + new URL prefix) when the request/response
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "picloud-dashboard",
|
"name": "picloud-dashboard",
|
||||||
"version": "0.11.0",
|
"version": "0.13.0",
|
||||||
"private": true,
|
"private": true,
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"scripts": {
|
"scripts": {
|
||||||
|
|||||||
@@ -211,7 +211,14 @@ export interface DeadLetterRow {
|
|||||||
resolution: 'replayed' | 'ignored' | 'handled_by_script' | 'handler_failed' | null;
|
resolution: 'replayed' | 'ignored' | 'handled_by_script' | 'handler_failed' | null;
|
||||||
}
|
}
|
||||||
|
|
||||||
export type TriggerKind = 'kv' | 'docs' | 'dead_letter' | 'cron' | 'files' | 'pubsub';
|
export type TriggerKind =
|
||||||
|
| 'kv'
|
||||||
|
| 'docs'
|
||||||
|
| 'dead_letter'
|
||||||
|
| 'cron'
|
||||||
|
| 'files'
|
||||||
|
| 'pubsub'
|
||||||
|
| 'email';
|
||||||
export type TriggerDispatchMode = 'sync' | 'async';
|
export type TriggerDispatchMode = 'sync' | 'async';
|
||||||
|
|
||||||
/// Per-kind detail, tagged by `kind` to match the Rust serde shape.
|
/// Per-kind detail, tagged by `kind` to match the Rust serde shape.
|
||||||
@@ -221,7 +228,15 @@ export type TriggerDetails =
|
|||||||
| { kind: 'dead_letter'; source_filter?: string; trigger_id_filter?: string; script_id_filter?: string }
|
| { kind: 'dead_letter'; source_filter?: string; trigger_id_filter?: string; script_id_filter?: string }
|
||||||
| { kind: 'cron'; schedule: string; timezone: string; last_fired_at?: string | null }
|
| { kind: 'cron'; schedule: string; timezone: string; last_fired_at?: string | null }
|
||||||
| { kind: 'files'; collection_glob: string; ops: string[] }
|
| { kind: 'files'; collection_glob: string; ops: string[] }
|
||||||
| { kind: 'pubsub'; topic_pattern: string };
|
| { kind: 'pubsub'; topic_pattern: string }
|
||||||
|
| { kind: 'email'; has_inbound_secret: boolean };
|
||||||
|
|
||||||
|
export interface CreateEmailTriggerInput {
|
||||||
|
script_id: string;
|
||||||
|
/// Shared HMAC secret; null/omitted means the receiver accepts
|
||||||
|
/// unsigned POSTs (URL secrecy is then the only guard).
|
||||||
|
inbound_secret?: string | null;
|
||||||
|
}
|
||||||
|
|
||||||
/// v1.1.5 file metadata as the admin files endpoint returns it.
|
/// v1.1.5 file metadata as the admin files endpoint returns it.
|
||||||
export interface FileMeta {
|
export interface FileMeta {
|
||||||
@@ -270,6 +285,33 @@ export interface CreatePubsubTriggerInput {
|
|||||||
retry_base_ms?: number;
|
retry_base_ms?: number;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// v1.1.6 — externally-subscribable realtime topics.
|
||||||
|
export type TopicAuthMode = 'public' | 'token';
|
||||||
|
|
||||||
|
export interface Topic {
|
||||||
|
name: string;
|
||||||
|
external_subscribable: boolean;
|
||||||
|
auth_mode: TopicAuthMode;
|
||||||
|
created_at: string;
|
||||||
|
updated_at: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface CreateTopicInput {
|
||||||
|
name: string;
|
||||||
|
external_subscribable: boolean;
|
||||||
|
auth_mode: TopicAuthMode;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface UpdateTopicInput {
|
||||||
|
external_subscribable?: boolean;
|
||||||
|
auth_mode?: TopicAuthMode;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface SecretListItem {
|
||||||
|
name: string;
|
||||||
|
updated_at: string;
|
||||||
|
}
|
||||||
|
|
||||||
export interface ExecutionResult {
|
export interface ExecutionResult {
|
||||||
status: number;
|
status: number;
|
||||||
headers: Record<string, string>;
|
headers: Record<string, string>;
|
||||||
@@ -646,6 +688,11 @@ export const api = {
|
|||||||
`/api/v1/admin/apps/${encodeURIComponent(idOrSlug)}/triggers/pubsub`,
|
`/api/v1/admin/apps/${encodeURIComponent(idOrSlug)}/triggers/pubsub`,
|
||||||
{ method: 'POST', body: JSON.stringify(input) }
|
{ method: 'POST', body: JSON.stringify(input) }
|
||||||
),
|
),
|
||||||
|
createEmail: (idOrSlug: string, input: CreateEmailTriggerInput) =>
|
||||||
|
adminRequest<Trigger>(
|
||||||
|
`/api/v1/admin/apps/${encodeURIComponent(idOrSlug)}/triggers/email`,
|
||||||
|
{ method: 'POST', body: JSON.stringify(input) }
|
||||||
|
),
|
||||||
remove: (idOrSlug: string, triggerId: string) =>
|
remove: (idOrSlug: string, triggerId: string) =>
|
||||||
adminRequest<null>(
|
adminRequest<null>(
|
||||||
`/api/v1/admin/apps/${encodeURIComponent(idOrSlug)}/triggers/${triggerId}`,
|
`/api/v1/admin/apps/${encodeURIComponent(idOrSlug)}/triggers/${triggerId}`,
|
||||||
@@ -653,6 +700,28 @@ export const api = {
|
|||||||
)
|
)
|
||||||
},
|
},
|
||||||
|
|
||||||
|
topics: {
|
||||||
|
list: (idOrSlug: string) =>
|
||||||
|
adminRequest<{ topics: Topic[] }>(
|
||||||
|
`/api/v1/admin/apps/${encodeURIComponent(idOrSlug)}/topics`
|
||||||
|
),
|
||||||
|
create: (idOrSlug: string, input: CreateTopicInput) =>
|
||||||
|
adminRequest<Topic>(`/api/v1/admin/apps/${encodeURIComponent(idOrSlug)}/topics`, {
|
||||||
|
method: 'POST',
|
||||||
|
body: JSON.stringify(input)
|
||||||
|
}),
|
||||||
|
update: (idOrSlug: string, name: string, input: UpdateTopicInput) =>
|
||||||
|
adminRequest<Topic>(
|
||||||
|
`/api/v1/admin/apps/${encodeURIComponent(idOrSlug)}/topics/${encodeURIComponent(name)}`,
|
||||||
|
{ method: 'PATCH', body: JSON.stringify(input) }
|
||||||
|
),
|
||||||
|
remove: (idOrSlug: string, name: string) =>
|
||||||
|
adminRequest<null>(
|
||||||
|
`/api/v1/admin/apps/${encodeURIComponent(idOrSlug)}/topics/${encodeURIComponent(name)}`,
|
||||||
|
{ method: 'DELETE' }
|
||||||
|
)
|
||||||
|
},
|
||||||
|
|
||||||
files: {
|
files: {
|
||||||
list: (idOrSlug: string, collection: string, opts: { cursor?: string; limit?: number } = {}) => {
|
list: (idOrSlug: string, collection: string, opts: { cursor?: string; limit?: number } = {}) => {
|
||||||
const params = new URLSearchParams();
|
const params = new URLSearchParams();
|
||||||
@@ -670,6 +739,27 @@ export const api = {
|
|||||||
)
|
)
|
||||||
},
|
},
|
||||||
|
|
||||||
|
secrets: {
|
||||||
|
// List returns names + last-modified ONLY — values never leave the
|
||||||
|
// server (v1.1.7).
|
||||||
|
list: (idOrSlug: string) =>
|
||||||
|
adminRequest<{ secrets: SecretListItem[]; next_cursor: string | null }>(
|
||||||
|
`/api/v1/admin/apps/${encodeURIComponent(idOrSlug)}/secrets`
|
||||||
|
),
|
||||||
|
// `value` is any JSON value; the dashboard sends a single-line
|
||||||
|
// string. Overwrites if the name already exists.
|
||||||
|
set: (idOrSlug: string, name: string, value: unknown) =>
|
||||||
|
adminRequest<null>(`/api/v1/admin/apps/${encodeURIComponent(idOrSlug)}/secrets`, {
|
||||||
|
method: 'POST',
|
||||||
|
body: JSON.stringify({ name, value })
|
||||||
|
}),
|
||||||
|
remove: (idOrSlug: string, name: string) =>
|
||||||
|
adminRequest<null>(
|
||||||
|
`/api/v1/admin/apps/${encodeURIComponent(idOrSlug)}/secrets/${encodeURIComponent(name)}`,
|
||||||
|
{ method: 'DELETE' }
|
||||||
|
)
|
||||||
|
},
|
||||||
|
|
||||||
execute: async (
|
execute: async (
|
||||||
id: string,
|
id: string,
|
||||||
body: unknown,
|
body: unknown,
|
||||||
|
|||||||
@@ -11,7 +11,10 @@
|
|||||||
type AppMemberDto,
|
type AppMemberDto,
|
||||||
type AppRole,
|
type AppRole,
|
||||||
type Script,
|
type Script,
|
||||||
type Trigger
|
type Trigger,
|
||||||
|
type Topic,
|
||||||
|
type TopicAuthMode,
|
||||||
|
type SecretListItem
|
||||||
} from '$lib/api';
|
} from '$lib/api';
|
||||||
import CodeEditor from '$lib/CodeEditor.svelte';
|
import CodeEditor from '$lib/CodeEditor.svelte';
|
||||||
import ConfirmModal from '$lib/ConfirmModal.svelte';
|
import ConfirmModal from '$lib/ConfirmModal.svelte';
|
||||||
@@ -25,7 +28,7 @@
|
|||||||
const SAMPLE_SOURCE =
|
const SAMPLE_SOURCE =
|
||||||
'#{\n statusCode: 200,\n body: #{ ok: true, echo: ctx.request.body }\n}';
|
'#{\n statusCode: 200,\n body: #{ ok: true, echo: ctx.request.body }\n}';
|
||||||
|
|
||||||
type Tab = 'scripts' | 'domains' | 'members' | 'settings' | 'triggers';
|
type Tab = 'scripts' | 'domains' | 'members' | 'settings' | 'triggers' | 'topics' | 'secrets';
|
||||||
|
|
||||||
// Common IANA timezones offered in the cron form dropdown. Not
|
// Common IANA timezones offered in the cron form dropdown. Not
|
||||||
// exhaustive — the backend validates any IANA name via chrono-tz.
|
// exhaustive — the backend validates any IANA name via chrono-tz.
|
||||||
@@ -123,6 +126,11 @@
|
|||||||
let createPubsubTopic = $state('');
|
let createPubsubTopic = $state('');
|
||||||
let creatingPubsub = $state(false);
|
let creatingPubsub = $state(false);
|
||||||
let createPubsubError = $state<string | null>(null);
|
let createPubsubError = $state<string | null>(null);
|
||||||
|
// Email triggers (v1.1.7).
|
||||||
|
let createEmailScriptId = $state('');
|
||||||
|
let createEmailSecret = $state('');
|
||||||
|
let creatingEmail = $state(false);
|
||||||
|
let createEmailError = $state<string | null>(null);
|
||||||
let triggerToRemove = $state<Trigger | null>(null);
|
let triggerToRemove = $state<Trigger | null>(null);
|
||||||
let removingTrigger = $state(false);
|
let removingTrigger = $state(false);
|
||||||
// Endpoint scripts only — modules can't be trigger targets.
|
// Endpoint scripts only — modules can't be trigger targets.
|
||||||
@@ -179,6 +187,34 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async function submitCreateEmail(e: SubmitEvent) {
|
||||||
|
e.preventDefault();
|
||||||
|
if (!app) return;
|
||||||
|
creatingEmail = true;
|
||||||
|
createEmailError = null;
|
||||||
|
try {
|
||||||
|
await api.triggers.createEmail(app.id, {
|
||||||
|
script_id: createEmailScriptId,
|
||||||
|
inbound_secret: createEmailSecret.trim() === '' ? null : createEmailSecret
|
||||||
|
});
|
||||||
|
createEmailScriptId = '';
|
||||||
|
createEmailSecret = '';
|
||||||
|
await loadTriggers(app.id);
|
||||||
|
} catch (err) {
|
||||||
|
createEmailError =
|
||||||
|
err instanceof ApiError ? err.message : err instanceof Error ? err.message : String(err);
|
||||||
|
} finally {
|
||||||
|
creatingEmail = false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The inbound-email webhook URL for a given email trigger (shown so
|
||||||
|
// the operator can configure their provider).
|
||||||
|
function emailInboundUrl(triggerId: string): string {
|
||||||
|
if (!app) return '';
|
||||||
|
return `${window.location.origin}/api/v1/email-inbound/${app.id}/${triggerId}`;
|
||||||
|
}
|
||||||
|
|
||||||
async function confirmRemoveTrigger() {
|
async function confirmRemoveTrigger() {
|
||||||
if (!app || !triggerToRemove) return;
|
if (!app || !triggerToRemove) return;
|
||||||
removingTrigger = true;
|
removingTrigger = true;
|
||||||
@@ -194,6 +230,177 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Topics tab (v1.1.6 — externally-subscribable realtime topics). Admin-gated.
|
||||||
|
let topics = $state<Topic[]>([]);
|
||||||
|
let createTopicName = $state('');
|
||||||
|
let createTopicExternal = $state(false);
|
||||||
|
let createTopicAuthMode = $state<TopicAuthMode>('public');
|
||||||
|
let creatingTopic = $state(false);
|
||||||
|
let createTopicError = $state<string | null>(null);
|
||||||
|
// Edit modal.
|
||||||
|
let topicToEdit = $state<Topic | null>(null);
|
||||||
|
let editTopicExternal = $state(false);
|
||||||
|
let editTopicAuthMode = $state<TopicAuthMode>('public');
|
||||||
|
let savingTopic = $state(false);
|
||||||
|
let editTopicError = $state<string | null>(null);
|
||||||
|
// Flipping internal → external is the security-sensitive change.
|
||||||
|
const editFlipToExternal = $derived(
|
||||||
|
!!topicToEdit && !topicToEdit.external_subscribable && editTopicExternal
|
||||||
|
);
|
||||||
|
// Delete confirm.
|
||||||
|
let topicToRemove = $state<Topic | null>(null);
|
||||||
|
let removingTopic = $state(false);
|
||||||
|
|
||||||
|
async function loadTopics(idOrSlug: string) {
|
||||||
|
try {
|
||||||
|
const r = await api.topics.list(idOrSlug);
|
||||||
|
topics = r.topics;
|
||||||
|
} catch {
|
||||||
|
topics = [];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function submitCreateTopic(e: SubmitEvent) {
|
||||||
|
e.preventDefault();
|
||||||
|
if (!app) return;
|
||||||
|
creatingTopic = true;
|
||||||
|
createTopicError = null;
|
||||||
|
try {
|
||||||
|
await api.topics.create(app.id, {
|
||||||
|
name: createTopicName.trim(),
|
||||||
|
external_subscribable: createTopicExternal,
|
||||||
|
auth_mode: createTopicAuthMode
|
||||||
|
});
|
||||||
|
createTopicName = '';
|
||||||
|
createTopicExternal = false;
|
||||||
|
createTopicAuthMode = 'public';
|
||||||
|
await loadTopics(app.id);
|
||||||
|
} catch (err) {
|
||||||
|
createTopicError =
|
||||||
|
err instanceof ApiError ? err.message : err instanceof Error ? err.message : String(err);
|
||||||
|
} finally {
|
||||||
|
creatingTopic = false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function openEditTopic(t: Topic) {
|
||||||
|
topicToEdit = t;
|
||||||
|
editTopicExternal = t.external_subscribable;
|
||||||
|
editTopicAuthMode = t.auth_mode;
|
||||||
|
editTopicError = null;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function confirmEditTopic() {
|
||||||
|
if (!app || !topicToEdit) return;
|
||||||
|
savingTopic = true;
|
||||||
|
editTopicError = null;
|
||||||
|
try {
|
||||||
|
await api.topics.update(app.id, topicToEdit.name, {
|
||||||
|
external_subscribable: editTopicExternal,
|
||||||
|
auth_mode: editTopicAuthMode
|
||||||
|
});
|
||||||
|
topicToEdit = null;
|
||||||
|
await loadTopics(app.id);
|
||||||
|
} catch (err) {
|
||||||
|
editTopicError =
|
||||||
|
err instanceof ApiError ? err.message : err instanceof Error ? err.message : String(err);
|
||||||
|
} finally {
|
||||||
|
savingTopic = false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function confirmRemoveTopic() {
|
||||||
|
if (!app || !topicToRemove) return;
|
||||||
|
removingTopic = true;
|
||||||
|
try {
|
||||||
|
await api.topics.remove(app.id, topicToRemove.name);
|
||||||
|
topicToRemove = null;
|
||||||
|
await loadTopics(app.id);
|
||||||
|
} catch (err) {
|
||||||
|
createTopicError =
|
||||||
|
err instanceof ApiError ? err.message : err instanceof Error ? err.message : String(err);
|
||||||
|
} finally {
|
||||||
|
removingTopic = false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Secrets tab (v1.1.7). The dashboard only ever sees names +
|
||||||
|
// last-modified — values never leave the server. The create form's
|
||||||
|
// value input is masked by default; revealing it requires a confirm.
|
||||||
|
let secrets = $state<SecretListItem[]>([]);
|
||||||
|
let createSecretName = $state('');
|
||||||
|
let createSecretValue = $state('');
|
||||||
|
let showSecretValue = $state(false);
|
||||||
|
let revealConfirm = $state(false);
|
||||||
|
let creatingSecret = $state(false);
|
||||||
|
let createSecretError = $state<string | null>(null);
|
||||||
|
let secretToRemove = $state<SecretListItem | null>(null);
|
||||||
|
let removingSecret = $state(false);
|
||||||
|
// True when the name already exists — set is an overwrite.
|
||||||
|
const secretNameExists = $derived(
|
||||||
|
secrets.some((s) => s.name === createSecretName.trim())
|
||||||
|
);
|
||||||
|
|
||||||
|
async function loadSecrets(idOrSlug: string) {
|
||||||
|
try {
|
||||||
|
const r = await api.secrets.list(idOrSlug);
|
||||||
|
secrets = r.secrets;
|
||||||
|
} catch {
|
||||||
|
secrets = [];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function toggleShowSecretValue(e: Event) {
|
||||||
|
const target = e.currentTarget as HTMLInputElement;
|
||||||
|
if (target.checked) {
|
||||||
|
// Revealing a secret on screen is sensitive — gate behind a
|
||||||
|
// confirm. Revert the checkbox until the user confirms.
|
||||||
|
target.checked = false;
|
||||||
|
revealConfirm = true;
|
||||||
|
} else {
|
||||||
|
showSecretValue = false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function confirmRevealSecret() {
|
||||||
|
showSecretValue = true;
|
||||||
|
revealConfirm = false;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function submitCreateSecret(e: SubmitEvent) {
|
||||||
|
e.preventDefault();
|
||||||
|
if (!app) return;
|
||||||
|
creatingSecret = true;
|
||||||
|
createSecretError = null;
|
||||||
|
try {
|
||||||
|
await api.secrets.set(app.id, createSecretName.trim(), createSecretValue);
|
||||||
|
createSecretName = '';
|
||||||
|
createSecretValue = '';
|
||||||
|
showSecretValue = false;
|
||||||
|
await loadSecrets(app.id);
|
||||||
|
} catch (err) {
|
||||||
|
createSecretError =
|
||||||
|
err instanceof ApiError ? err.message : err instanceof Error ? err.message : String(err);
|
||||||
|
} finally {
|
||||||
|
creatingSecret = false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function confirmRemoveSecret() {
|
||||||
|
if (!app || !secretToRemove) return;
|
||||||
|
removingSecret = true;
|
||||||
|
try {
|
||||||
|
await api.secrets.remove(app.id, secretToRemove.name);
|
||||||
|
secretToRemove = null;
|
||||||
|
await loadSecrets(app.id);
|
||||||
|
} catch (err) {
|
||||||
|
createSecretError =
|
||||||
|
err instanceof ApiError ? err.message : err instanceof Error ? err.message : String(err);
|
||||||
|
} finally {
|
||||||
|
removingSecret = false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// Members tab
|
// Members tab
|
||||||
let eligibleUsers = $state<AdminDto[]>([]);
|
let eligibleUsers = $state<AdminDto[]>([]);
|
||||||
let eligibleLoadError = $state<string | null>(null);
|
let eligibleLoadError = $state<string | null>(null);
|
||||||
@@ -234,7 +441,13 @@
|
|||||||
loadDeadLetterCount(app.id)
|
loadDeadLetterCount(app.id)
|
||||||
];
|
];
|
||||||
if (canAdmin) {
|
if (canAdmin) {
|
||||||
loaders.push(loadMembers(app.id), loadEligibleUsers(), loadTriggers(app.id));
|
loaders.push(
|
||||||
|
loadMembers(app.id),
|
||||||
|
loadEligibleUsers(),
|
||||||
|
loadTriggers(app.id),
|
||||||
|
loadTopics(app.id),
|
||||||
|
loadSecrets(app.id)
|
||||||
|
);
|
||||||
}
|
}
|
||||||
await Promise.all(loaders);
|
await Promise.all(loaders);
|
||||||
} catch (e) {
|
} catch (e) {
|
||||||
@@ -503,7 +716,11 @@
|
|||||||
$effect(() => {
|
$effect(() => {
|
||||||
if (
|
if (
|
||||||
!canAdmin &&
|
!canAdmin &&
|
||||||
(activeTab === 'settings' || activeTab === 'members' || activeTab === 'triggers')
|
(activeTab === 'settings' ||
|
||||||
|
activeTab === 'members' ||
|
||||||
|
activeTab === 'triggers' ||
|
||||||
|
activeTab === 'topics' ||
|
||||||
|
activeTab === 'secrets')
|
||||||
) {
|
) {
|
||||||
activeTab = 'scripts';
|
activeTab = 'scripts';
|
||||||
}
|
}
|
||||||
@@ -551,6 +768,16 @@
|
|||||||
class:active={activeTab === 'triggers'}
|
class:active={activeTab === 'triggers'}
|
||||||
onclick={() => (activeTab = 'triggers')}>Triggers ({triggers.length})</button
|
onclick={() => (activeTab = 'triggers')}>Triggers ({triggers.length})</button
|
||||||
>
|
>
|
||||||
|
<button
|
||||||
|
type="button"
|
||||||
|
class:active={activeTab === 'topics'}
|
||||||
|
onclick={() => (activeTab = 'topics')}>Topics ({topics.length})</button
|
||||||
|
>
|
||||||
|
<button
|
||||||
|
type="button"
|
||||||
|
class:active={activeTab === 'secrets'}
|
||||||
|
onclick={() => (activeTab = 'secrets')}>Secrets ({secrets.length})</button
|
||||||
|
>
|
||||||
<button
|
<button
|
||||||
type="button"
|
type="button"
|
||||||
class:active={activeTab === 'settings'}
|
class:active={activeTab === 'settings'}
|
||||||
@@ -905,6 +1132,59 @@
|
|||||||
</div>
|
</div>
|
||||||
</form>
|
</form>
|
||||||
|
|
||||||
|
|
||||||
|
<h2>Email triggers</h2>
|
||||||
|
<p class="muted">
|
||||||
|
Fire an endpoint script when your email provider POSTs an inbound
|
||||||
|
message to PiCloud. Configure your provider (Mailgun / Postmark /
|
||||||
|
SendGrid / SES) to POST the generic JSON shape below to the trigger's
|
||||||
|
webhook URL. Set a shared secret to require an
|
||||||
|
<code>X-Picloud-Signature</code> HMAC-SHA256 (hex of the request body);
|
||||||
|
leave it blank to accept unsigned POSTs (URL secrecy only).
|
||||||
|
</p>
|
||||||
|
<details class="muted small">
|
||||||
|
<summary>Expected inbound JSON shape</summary>
|
||||||
|
<pre>{`{
|
||||||
|
"from": "sender@external.com",
|
||||||
|
"to": ["alice@myapp.com"],
|
||||||
|
"cc": [],
|
||||||
|
"subject": "...",
|
||||||
|
"text": "...",
|
||||||
|
"html": "...",
|
||||||
|
"message_id": "<abc@external.com>"
|
||||||
|
}`}</pre>
|
||||||
|
</details>
|
||||||
|
|
||||||
|
<form class="create-form" onsubmit={submitCreateEmail}>
|
||||||
|
<div class="row">
|
||||||
|
<label>
|
||||||
|
<span>Target script</span>
|
||||||
|
<select bind:value={createEmailScriptId} required>
|
||||||
|
<option value="" disabled>Select an endpoint script…</option>
|
||||||
|
{#each endpointScripts as s (s.id)}
|
||||||
|
<option value={s.id}>{s.name}</option>
|
||||||
|
{/each}
|
||||||
|
</select>
|
||||||
|
</label>
|
||||||
|
<label class="grow">
|
||||||
|
<span>Inbound HMAC secret (optional)</span>
|
||||||
|
<input
|
||||||
|
type="password"
|
||||||
|
bind:value={createEmailSecret}
|
||||||
|
placeholder="leave blank to accept unsigned POSTs"
|
||||||
|
/>
|
||||||
|
</label>
|
||||||
|
</div>
|
||||||
|
{#if createEmailError}
|
||||||
|
<div class="error">{createEmailError}</div>
|
||||||
|
{/if}
|
||||||
|
<div class="actions">
|
||||||
|
<button type="submit" disabled={creatingEmail || !createEmailScriptId}>
|
||||||
|
{creatingEmail ? 'Creating…' : 'Create email trigger'}
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
</form>
|
||||||
|
|
||||||
{#if triggers.length === 0}
|
{#if triggers.length === 0}
|
||||||
<p class="muted">No triggers in this app yet.</p>
|
<p class="muted">No triggers in this app yet.</p>
|
||||||
{:else}
|
{:else}
|
||||||
@@ -924,6 +1204,11 @@
|
|||||||
<span class="muted">— {t.details.ops.join(', ') || 'any op'}</span>
|
<span class="muted">— {t.details.ops.join(', ') || 'any op'}</span>
|
||||||
{:else if t.details.kind === 'pubsub'}
|
{:else if t.details.kind === 'pubsub'}
|
||||||
<code>{t.details.topic_pattern}</code>
|
<code>{t.details.topic_pattern}</code>
|
||||||
|
{:else if t.details.kind === 'email'}
|
||||||
|
<span class="muted">
|
||||||
|
{t.details.has_inbound_secret ? 'signed (HMAC)' : 'unsigned'}
|
||||||
|
</span>
|
||||||
|
<code class="webhook-url">{emailInboundUrl(t.id)}</code>
|
||||||
{/if}
|
{/if}
|
||||||
<span class="muted small">→ {t.script_id}</span>
|
<span class="muted small">→ {t.script_id}</span>
|
||||||
</div>
|
</div>
|
||||||
@@ -939,6 +1224,159 @@
|
|||||||
</ul>
|
</ul>
|
||||||
{/if}
|
{/if}
|
||||||
</section>
|
</section>
|
||||||
|
{:else if activeTab === 'topics' && canAdmin}
|
||||||
|
<section>
|
||||||
|
<h2>Realtime topics</h2>
|
||||||
|
<p class="muted">
|
||||||
|
Pub/sub topics are <strong>internal-only</strong> by default — scripts
|
||||||
|
subscribe via triggers, browsers can't. Register a topic here and mark it
|
||||||
|
<strong>externally subscribable</strong> to let frontend clients connect over
|
||||||
|
SSE at <code>/realtime/topics/<name></code>. <code>public</code> topics
|
||||||
|
need no auth; <code>token</code> topics require a subscriber token minted by a
|
||||||
|
script via <code>pubsub::subscriber_token</code>.
|
||||||
|
</p>
|
||||||
|
|
||||||
|
<form class="create-form" onsubmit={submitCreateTopic}>
|
||||||
|
<div class="row">
|
||||||
|
<label class="grow">
|
||||||
|
<span>Topic name</span>
|
||||||
|
<input bind:value={createTopicName} required placeholder="chat-room-updates" />
|
||||||
|
</label>
|
||||||
|
</div>
|
||||||
|
<label class="checkbox-row">
|
||||||
|
<input type="checkbox" bind:checked={createTopicExternal} />
|
||||||
|
<span>Externally subscribable (allow browser SSE clients to subscribe)</span>
|
||||||
|
</label>
|
||||||
|
{#if createTopicExternal}
|
||||||
|
<fieldset class="auth-mode">
|
||||||
|
<legend>Auth mode</legend>
|
||||||
|
<label class="radio-row">
|
||||||
|
<input type="radio" value="public" bind:group={createTopicAuthMode} />
|
||||||
|
<span><strong>public</strong> — anyone with the URL can subscribe</span>
|
||||||
|
</label>
|
||||||
|
<label class="radio-row">
|
||||||
|
<input type="radio" value="token" bind:group={createTopicAuthMode} />
|
||||||
|
<span><strong>token</strong> — requires a valid subscriber token</span>
|
||||||
|
</label>
|
||||||
|
</fieldset>
|
||||||
|
{/if}
|
||||||
|
{#if createTopicError}
|
||||||
|
<div class="error">{createTopicError}</div>
|
||||||
|
{/if}
|
||||||
|
<div class="actions">
|
||||||
|
<button type="submit" disabled={creatingTopic || !createTopicName.trim()}>
|
||||||
|
{creatingTopic ? 'Creating…' : 'Register topic'}
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
</form>
|
||||||
|
|
||||||
|
{#if topics.length === 0}
|
||||||
|
<p class="muted">No registered topics in this app yet.</p>
|
||||||
|
{:else}
|
||||||
|
<ul class="list">
|
||||||
|
{#each topics as t (t.name)}
|
||||||
|
<li class="domain-row">
|
||||||
|
<div>
|
||||||
|
<code>{t.name}</code>
|
||||||
|
{#if t.external_subscribable}
|
||||||
|
<span class="badge badge-external" title="Browser clients can subscribe over SSE">
|
||||||
|
external
|
||||||
|
</span>
|
||||||
|
<span class="badge badge-auth">{t.auth_mode}</span>
|
||||||
|
{:else}
|
||||||
|
<span class="badge badge-internal" title="Internal-only: scripts subscribe via triggers">
|
||||||
|
internal
|
||||||
|
</span>
|
||||||
|
{/if}
|
||||||
|
<span class="muted small">· {shortDate(t.created_at)}</span>
|
||||||
|
</div>
|
||||||
|
<div class="topic-actions">
|
||||||
|
<button type="button" class="secondary" onclick={() => openEditTopic(t)}>
|
||||||
|
Edit
|
||||||
|
</button>
|
||||||
|
<button
|
||||||
|
type="button"
|
||||||
|
class="secondary danger"
|
||||||
|
onclick={() => (topicToRemove = t)}
|
||||||
|
>
|
||||||
|
Delete
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
</li>
|
||||||
|
{/each}
|
||||||
|
</ul>
|
||||||
|
{/if}
|
||||||
|
</section>
|
||||||
|
{:else if activeTab === 'secrets' && canAdmin}
|
||||||
|
<section>
|
||||||
|
<h2>Secrets</h2>
|
||||||
|
<p class="muted">
|
||||||
|
Encrypted per-app configuration (API keys, OAuth tokens, webhook signing
|
||||||
|
keys), available to scripts as <code>secrets::get("name")</code>. Values are
|
||||||
|
encrypted at rest with the process master key and
|
||||||
|
<strong>never leave the server</strong> — this list shows names and
|
||||||
|
last-modified times only.
|
||||||
|
</p>
|
||||||
|
|
||||||
|
<form class="create-form" onsubmit={submitCreateSecret}>
|
||||||
|
<div class="row">
|
||||||
|
<label class="grow">
|
||||||
|
<span>Name</span>
|
||||||
|
<input bind:value={createSecretName} required placeholder="stripe_key" />
|
||||||
|
</label>
|
||||||
|
</div>
|
||||||
|
<label class="grow">
|
||||||
|
<span>Value</span>
|
||||||
|
{#if showSecretValue}
|
||||||
|
<input type="text" bind:value={createSecretValue} placeholder="sk_live_…" />
|
||||||
|
{:else}
|
||||||
|
<input type="password" bind:value={createSecretValue} placeholder="sk_live_…" />
|
||||||
|
{/if}
|
||||||
|
</label>
|
||||||
|
<label class="checkbox-row">
|
||||||
|
<input type="checkbox" checked={showSecretValue} onchange={toggleShowSecretValue} />
|
||||||
|
<span>Show value</span>
|
||||||
|
</label>
|
||||||
|
{#if secretNameExists && createSecretName.trim()}
|
||||||
|
<p class="muted small">
|
||||||
|
A secret named <code>{createSecretName.trim()}</code> already exists — saving
|
||||||
|
overwrites it.
|
||||||
|
</p>
|
||||||
|
{/if}
|
||||||
|
{#if createSecretError}
|
||||||
|
<div class="error">{createSecretError}</div>
|
||||||
|
{/if}
|
||||||
|
<div class="actions">
|
||||||
|
<button type="submit" disabled={creatingSecret || !createSecretName.trim()}>
|
||||||
|
{creatingSecret ? 'Saving…' : secretNameExists ? 'Overwrite secret' : 'Save secret'}
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
</form>
|
||||||
|
|
||||||
|
{#if secrets.length === 0}
|
||||||
|
<p class="muted">No secrets in this app yet.</p>
|
||||||
|
{:else}
|
||||||
|
<ul class="list">
|
||||||
|
{#each secrets as s (s.name)}
|
||||||
|
<li class="domain-row">
|
||||||
|
<div>
|
||||||
|
<code>{s.name}</code>
|
||||||
|
<span class="muted small">· updated {shortDate(s.updated_at)}</span>
|
||||||
|
</div>
|
||||||
|
<div class="topic-actions">
|
||||||
|
<button
|
||||||
|
type="button"
|
||||||
|
class="secondary danger"
|
||||||
|
onclick={() => (secretToRemove = s)}
|
||||||
|
>
|
||||||
|
Delete
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
</li>
|
||||||
|
{/each}
|
||||||
|
</ul>
|
||||||
|
{/if}
|
||||||
|
</section>
|
||||||
{:else if activeTab === 'settings' && canAdmin}
|
{:else if activeTab === 'settings' && canAdmin}
|
||||||
<section>
|
<section>
|
||||||
<h2>Settings</h2>
|
<h2>Settings</h2>
|
||||||
@@ -1113,6 +1551,97 @@
|
|||||||
</p>
|
</p>
|
||||||
</ConfirmModal>
|
</ConfirmModal>
|
||||||
{/if}
|
{/if}
|
||||||
|
|
||||||
|
{#if topicToEdit}
|
||||||
|
<ConfirmModal
|
||||||
|
title="Edit topic “{topicToEdit.name}”"
|
||||||
|
confirmLabel="Save changes"
|
||||||
|
busyLabel="Saving…"
|
||||||
|
busy={savingTopic}
|
||||||
|
onConfirm={confirmEditTopic}
|
||||||
|
onCancel={() => (topicToEdit = null)}
|
||||||
|
>
|
||||||
|
<label class="checkbox-row">
|
||||||
|
<input type="checkbox" bind:checked={editTopicExternal} />
|
||||||
|
<span>Externally subscribable</span>
|
||||||
|
</label>
|
||||||
|
{#if editTopicExternal}
|
||||||
|
<fieldset class="auth-mode">
|
||||||
|
<legend>Auth mode</legend>
|
||||||
|
<label class="radio-row">
|
||||||
|
<input type="radio" value="public" bind:group={editTopicAuthMode} />
|
||||||
|
<span><strong>public</strong> — anyone with the URL can subscribe</span>
|
||||||
|
</label>
|
||||||
|
<label class="radio-row">
|
||||||
|
<input type="radio" value="token" bind:group={editTopicAuthMode} />
|
||||||
|
<span><strong>token</strong> — requires a valid subscriber token</span>
|
||||||
|
</label>
|
||||||
|
</fieldset>
|
||||||
|
{/if}
|
||||||
|
{#if editFlipToExternal}
|
||||||
|
<div class="warning">
|
||||||
|
Marking <strong>{topicToEdit.name}</strong> externally-subscribable means
|
||||||
|
anyone with the URL can subscribe to this topic (if auth_mode is
|
||||||
|
<code>public</code>) or anyone with a valid token can subscribe (if
|
||||||
|
auth_mode is <code>token</code>). Are you sure?
|
||||||
|
</div>
|
||||||
|
{/if}
|
||||||
|
{#if editTopicError}
|
||||||
|
<p class="modal-error">{editTopicError}</p>
|
||||||
|
{/if}
|
||||||
|
</ConfirmModal>
|
||||||
|
{/if}
|
||||||
|
|
||||||
|
{#if topicToRemove}
|
||||||
|
<ConfirmModal
|
||||||
|
title="Delete topic “{topicToRemove.name}”"
|
||||||
|
variant="danger"
|
||||||
|
confirmLabel="Delete topic"
|
||||||
|
busyLabel="Deleting…"
|
||||||
|
busy={removingTopic}
|
||||||
|
onConfirm={confirmRemoveTopic}
|
||||||
|
onCancel={() => (topicToRemove = null)}
|
||||||
|
>
|
||||||
|
<p>
|
||||||
|
Unregistering <code>{topicToRemove.name}</code> disconnects any live SSE
|
||||||
|
subscribers immediately. Scripts can still <code>publish_durable</code> to
|
||||||
|
it (internal triggers keep working) — it just won't be externally
|
||||||
|
subscribable.
|
||||||
|
</p>
|
||||||
|
</ConfirmModal>
|
||||||
|
{/if}
|
||||||
|
|
||||||
|
{#if revealConfirm}
|
||||||
|
<ConfirmModal
|
||||||
|
title="Reveal secret value?"
|
||||||
|
confirmLabel="Show value"
|
||||||
|
onConfirm={confirmRevealSecret}
|
||||||
|
onCancel={() => (revealConfirm = false)}
|
||||||
|
>
|
||||||
|
<p>
|
||||||
|
The value you type will be shown in plain text on screen. Make sure no one
|
||||||
|
is looking over your shoulder and that screen-sharing is off.
|
||||||
|
</p>
|
||||||
|
</ConfirmModal>
|
||||||
|
{/if}
|
||||||
|
|
||||||
|
{#if secretToRemove}
|
||||||
|
<ConfirmModal
|
||||||
|
title="Delete secret “{secretToRemove.name}”"
|
||||||
|
variant="danger"
|
||||||
|
confirmLabel="Delete secret"
|
||||||
|
busyLabel="Deleting…"
|
||||||
|
busy={removingSecret}
|
||||||
|
onConfirm={confirmRemoveSecret}
|
||||||
|
onCancel={() => (secretToRemove = null)}
|
||||||
|
>
|
||||||
|
<p>
|
||||||
|
Deleting <code>{secretToRemove.name}</code> is permanent. Any script calling
|
||||||
|
<code>secrets::get("{secretToRemove.name}")</code> will get <code>()</code>
|
||||||
|
until you set it again.
|
||||||
|
</p>
|
||||||
|
</ConfirmModal>
|
||||||
|
{/if}
|
||||||
{/if}
|
{/if}
|
||||||
|
|
||||||
<style>
|
<style>
|
||||||
@@ -1463,4 +1992,64 @@
|
|||||||
.small {
|
.small {
|
||||||
font-size: 0.85rem;
|
font-size: 0.85rem;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
.checkbox-row,
|
||||||
|
.radio-row {
|
||||||
|
flex-direction: row;
|
||||||
|
align-items: center;
|
||||||
|
gap: 0.5rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.checkbox-row input,
|
||||||
|
.radio-row input {
|
||||||
|
flex: none;
|
||||||
|
width: auto;
|
||||||
|
}
|
||||||
|
|
||||||
|
.auth-mode {
|
||||||
|
border: 1px solid #334155;
|
||||||
|
border-radius: 0.375rem;
|
||||||
|
padding: 0.5rem 0.75rem;
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
gap: 0.4rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.auth-mode legend {
|
||||||
|
font-size: 0.8rem;
|
||||||
|
color: #94a3b8;
|
||||||
|
padding: 0 0.3rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.topic-actions {
|
||||||
|
display: flex;
|
||||||
|
gap: 0.5rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.badge {
|
||||||
|
display: inline-block;
|
||||||
|
padding: 0 0.45rem;
|
||||||
|
margin-left: 0.4rem;
|
||||||
|
border-radius: 0.25rem;
|
||||||
|
font-size: 0.7rem;
|
||||||
|
font-weight: 600;
|
||||||
|
text-transform: uppercase;
|
||||||
|
letter-spacing: 0.05em;
|
||||||
|
line-height: 1.5;
|
||||||
|
}
|
||||||
|
|
||||||
|
.badge-external {
|
||||||
|
background: #064e3b;
|
||||||
|
color: #6ee7b7;
|
||||||
|
}
|
||||||
|
|
||||||
|
.badge-internal {
|
||||||
|
background: #334155;
|
||||||
|
color: #cbd5e1;
|
||||||
|
}
|
||||||
|
|
||||||
|
.badge-auth {
|
||||||
|
background: #1e3a5f;
|
||||||
|
color: #93c5fd;
|
||||||
|
}
|
||||||
</style>
|
</style>
|
||||||
|
|||||||
Reference in New Issue
Block a user