Compare commits
11 Commits
feat/v1.1.
...
feat/v1.1.
| Author | SHA1 | Date | |
|---|---|---|---|
|
5cbb6ca427 | ||
|
|
3cfb795206 | ||
|
|
a7d3dad129 | ||
|
|
2ea47eb05a | ||
|
|
b35585195b | ||
|
|
fffcdf6169 | ||
|
|
02335a8132 | ||
|
|
1f78937dd2 | ||
|
|
8f2d2bc721 | ||
|
|
2d11090d1a | ||
|
|
dc2e4fa01f |
97
CHANGELOG.md
97
CHANGELOG.md
@@ -1,5 +1,102 @@
|
||||
# PiCloud Changelog
|
||||
|
||||
## v1.1.7 — Configuration & Email (unreleased)
|
||||
|
||||
The operational-config layer: **encrypted per-app secrets**, **outbound
|
||||
email**, and an **inbound email trigger** — plus the long-missing
|
||||
**dead-letter handler wiring** and **at-rest encryption of the realtime
|
||||
signing key**. All at-rest encryption uses a single process master key
|
||||
(AES-256-GCM); key rotation is deferred to v1.2.
|
||||
|
||||
### Added — Encryption infrastructure
|
||||
|
||||
- **Process master key** from `PICLOUD_SECRET_KEY` (base64 of exactly 32
|
||||
bytes). REQUIRED at startup — an unset or malformed key is fatal.
|
||||
Generate one with `openssl rand -base64 32`. A deterministic in-memory
|
||||
dev key is used ONLY when `PICLOUD_SECRET_KEY` is unset AND
|
||||
`PICLOUD_DEV_MODE=true` (with a prominent startup warning); there is no
|
||||
quiet unencrypted mode.
|
||||
- **`picloud_shared::crypto`** — `encrypt`/`decrypt` envelope:
|
||||
`Aes256Gcm`, 96-bit CSPRNG nonce, 128-bit auth tag appended to the
|
||||
ciphertext (RustCrypto `Aead` layout). Both ciphertext and nonce are
|
||||
stored.
|
||||
- **Key rotation is out of scope.** Changing `PICLOUD_SECRET_KEY` between
|
||||
deploys renders all existing ciphertext undecryptable. v1.2+ adds
|
||||
key-version columns + a re-encryption pass.
|
||||
|
||||
### Added — Encrypted per-app secrets
|
||||
|
||||
- **`secrets::{get,set,delete,list}(name)`** SDK — collection-less,
|
||||
per-app. `set` accepts a String/Map/Array (JSON-encoded then encrypted);
|
||||
`get` returns the same Rhai type back; missing → `()`. 64 KB plaintext
|
||||
cap (`PICLOUD_SECRET_MAX_VALUE_BYTES`). `migrations/0023_secrets.sql`.
|
||||
- **Admin API** `GET/POST/DELETE /api/v1/admin/apps/{id}/secrets` — list
|
||||
returns names + `updated_at` only, **never values**.
|
||||
- **Dashboard Secrets tab** — list names + last-modified, create/update
|
||||
(masked value with a confirm-gated reveal), delete with confirm.
|
||||
- `Capability::AppSecretsRead`/`Write` (→ `script:read` / `script:write`).
|
||||
No new Scope variants (seven-scope commitment). Secret writes
|
||||
deliberately do **not** emit trigger events.
|
||||
|
||||
### Added — Outbound email
|
||||
|
||||
- **`email::send` / `email::send_html`** SDK over an SMTP relay
|
||||
(`lettre`). Config from `PICLOUD_SMTP_HOST/PORT/USER/PASSWORD/TLS/
|
||||
TIMEOUT_SECS`; if HOST/USER/PASSWORD aren't all set the service runs in
|
||||
**disabled mode** (every send throws `NotConfigured`, warned at
|
||||
startup). Required `to`/`from`/`subject` + one of `text`/`html`;
|
||||
RFC 5322-ish address validation; 25 MB per-message cap
|
||||
(`PICLOUD_EMAIL_MAX_MESSAGE_BYTES`); `reply_to` defaults to `from`.
|
||||
Per-call connection (pooling deferred to v1.2); per-app `from`
|
||||
validation / SPF / DKIM are the operator's SMTP-relay concern.
|
||||
- `Capability::AppEmailSend` (→ `script:write`).
|
||||
|
||||
### Added — Inbound email (`email:receive` trigger)
|
||||
|
||||
- **Webhook receiver** `POST /api/v1/email-inbound/{app_id}/{trigger_id}`
|
||||
— a provider (Mailgun / Postmark / SendGrid / SES) POSTs the generic
|
||||
JSON shape `{from,to[],cc[],subject,text,html,message_id}`; the
|
||||
receiver verifies the optional HMAC signature, normalizes to
|
||||
`TriggerEvent::Email`, and enqueues an outbox row. 202 accepted, 401
|
||||
bad/missing signature, 404 missing/wrong-kind/cross-app, 422 malformed.
|
||||
Handlers see `ctx.event.email`. `migrations/0024_email_triggers.sql`.
|
||||
- **Admin** `POST /api/v1/admin/apps/{id}/triggers/email` +
|
||||
dashboard form (with the webhook URL + expected payload). The HMAC
|
||||
`inbound_secret` is stored **encrypted** via the master key (deviation
|
||||
from the original plaintext design — see HANDBACK §7).
|
||||
- Provider-specific payload unmarshallers + inbound attachments → v1.2.
|
||||
Native SMTP listener → v1.3+.
|
||||
|
||||
### Security/correctness fix (retroactive) — dead_letter handlers
|
||||
|
||||
The `dead_letter` trigger kind has been registerable since v1.1.1 but,
|
||||
due to missing dispatcher wiring (`list_matching_dead_letter` had no
|
||||
production caller), handlers have **never fired**. Any deploy running
|
||||
v1.1.1 through v1.1.6 with `dead_letter` triggers configured has had
|
||||
silently non-functional handlers. v1.1.7 fixes the wiring; existing
|
||||
`dead_letters` rows remain (no migration needed) but only NEW
|
||||
dead-letter events (post-v1.1.7) trigger handlers. To process older
|
||||
rows, use the existing admin replay surface to re-enqueue them.
|
||||
|
||||
### Changed — Realtime signing key encrypted at rest (two-phase)
|
||||
|
||||
`app_secrets.realtime_signing_key` was stored as 32 plaintext bytes. It
|
||||
is now encrypted with the master key. `migrations/0025_encrypt_realtime_keys.sql`
|
||||
adds NULL-able encrypted columns and drops `NOT NULL` on the plaintext
|
||||
column; a startup task encrypts pre-existing rows; the read path prefers
|
||||
the encrypted columns and falls back to plaintext during the compat
|
||||
window. **v1.1.8 will drop the plaintext `realtime_signing_key`
|
||||
column** — operators should upgrade through v1.1.7 (which performs the
|
||||
encryption) before v1.1.8.
|
||||
|
||||
### Notes
|
||||
|
||||
- **New deps:** `aes-gcm` (RustCrypto AEAD), `lettre` (SMTP).
|
||||
- **New env vars:** `PICLOUD_SECRET_KEY` (required), `PICLOUD_DEV_MODE`,
|
||||
`PICLOUD_SECRET_MAX_VALUE_BYTES`, `PICLOUD_SMTP_HOST/PORT/USER/PASSWORD/
|
||||
TLS/TIMEOUT_SECS`, `PICLOUD_EMAIL_MAX_MESSAGE_BYTES`.
|
||||
- **SDK schema** 1.7 → 1.8; **dashboard** 0.12.0 → 0.13.0.
|
||||
|
||||
## v1.1.6 — Realtime Channels & Client Library (unreleased)
|
||||
|
||||
The first **external realtime surface** and the first **frontend
|
||||
|
||||
202
Cargo.lock
generated
202
Cargo.lock
generated
@@ -2,6 +2,41 @@
|
||||
# It is not intended for manual editing.
|
||||
version = 4
|
||||
|
||||
[[package]]
|
||||
name = "aead"
|
||||
version = "0.5.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "d122413f284cf2d62fb1b7db97e02edb8cda96d769b16e443a4f6195e35662b0"
|
||||
dependencies = [
|
||||
"crypto-common",
|
||||
"generic-array",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "aes"
|
||||
version = "0.8.4"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "b169f7a6d4742236a0a00c541b845991d0ac43e546831af1249753ab4c3aa3a0"
|
||||
dependencies = [
|
||||
"cfg-if",
|
||||
"cipher",
|
||||
"cpufeatures",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "aes-gcm"
|
||||
version = "0.10.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "831010a0f742e1209b3bcea8fab6a8e149051ba6099432c8cb2cc117dec3ead1"
|
||||
dependencies = [
|
||||
"aead",
|
||||
"aes",
|
||||
"cipher",
|
||||
"ctr",
|
||||
"ghash",
|
||||
"subtle",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "ahash"
|
||||
version = "0.8.12"
|
||||
@@ -400,6 +435,16 @@ dependencies = [
|
||||
"phf_codegen",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "cipher"
|
||||
version = "0.4.4"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "773f3b9af64447d2ce9850330c473515014aa235e6a783b02db81ff39e4a3dad"
|
||||
dependencies = [
|
||||
"crypto-common",
|
||||
"inout",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "clap"
|
||||
version = "4.6.1"
|
||||
@@ -528,7 +573,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "6f8c3e73077b4b4a6ab1ea5047c37c57aee77657bc8ecd6f29b0af082d0b0c07"
|
||||
dependencies = [
|
||||
"chrono",
|
||||
"nom",
|
||||
"nom 7.1.3",
|
||||
"once_cell",
|
||||
]
|
||||
|
||||
@@ -560,9 +605,19 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "78c8292055d1c1df0cce5d180393dc8cce0abec0a7102adb6c7b1eef6016d60a"
|
||||
dependencies = [
|
||||
"generic-array",
|
||||
"rand_core 0.6.4",
|
||||
"typenum",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "ctr"
|
||||
version = "0.9.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "0369ee1ad671834580515889b80f2ea915f23b8be8d0daa4bbaf2ac5c7590835"
|
||||
dependencies = [
|
||||
"cipher",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "data-encoding"
|
||||
version = "2.11.0"
|
||||
@@ -660,6 +715,22 @@ dependencies = [
|
||||
"serde",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "email-encoding"
|
||||
version = "0.4.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "9298e6504d9b9e780ed3f7dfd43a61be8cd0e09eb07f7706a945b0072b6670b6"
|
||||
dependencies = [
|
||||
"base64",
|
||||
"memchr",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "email_address"
|
||||
version = "0.2.9"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "e079f19b08ca6239f47f8ba8509c11cf3ea30095831f7fed61441475edd8c449"
|
||||
|
||||
[[package]]
|
||||
name = "equivalent"
|
||||
version = "1.0.2"
|
||||
@@ -880,6 +951,16 @@ dependencies = [
|
||||
"wasip3",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "ghash"
|
||||
version = "0.5.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "f0d8a4362ccb29cb0b265253fb0a2728f592895ee6854fd9bc13f2ffda266ff1"
|
||||
dependencies = [
|
||||
"opaque-debug",
|
||||
"polyval",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "hashbrown"
|
||||
version = "0.15.5"
|
||||
@@ -945,6 +1026,17 @@ dependencies = [
|
||||
"windows-sys 0.61.2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "hostname"
|
||||
version = "0.4.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "617aaa3557aef3810a6369d0a99fac8a080891b68bd9f9812a1eeda0c0730cbd"
|
||||
dependencies = [
|
||||
"cfg-if",
|
||||
"libc",
|
||||
"windows-link",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "http"
|
||||
version = "1.4.0"
|
||||
@@ -1201,6 +1293,15 @@ version = "0.1.15"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "c8fae54786f62fb2918dcfae3d568594e50eb9b5c25bf04371af6fe7516452fb"
|
||||
|
||||
[[package]]
|
||||
name = "inout"
|
||||
version = "0.1.4"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "879f10e63c20629ecabbb64a8010319738c66a5cd0c29b02d63d272b03751d01"
|
||||
dependencies = [
|
||||
"generic-array",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "ipnet"
|
||||
version = "2.12.0"
|
||||
@@ -1246,6 +1347,34 @@ version = "0.1.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "09edd9e8b54e49e587e4f6295a7d29c3ea94d469cb40ab8ca70b288248a81db2"
|
||||
|
||||
[[package]]
|
||||
name = "lettre"
|
||||
version = "0.11.22"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "0da65617f6cb926332d039cb578aad56178da86e128db6a1b09f4c94fa5b3349"
|
||||
dependencies = [
|
||||
"async-trait",
|
||||
"base64",
|
||||
"email-encoding",
|
||||
"email_address",
|
||||
"fastrand",
|
||||
"futures-io",
|
||||
"futures-util",
|
||||
"hostname",
|
||||
"httpdate",
|
||||
"idna",
|
||||
"mime",
|
||||
"nom 8.0.0",
|
||||
"percent-encoding",
|
||||
"quoted_printable",
|
||||
"rustls",
|
||||
"socket2",
|
||||
"tokio",
|
||||
"tokio-rustls",
|
||||
"url",
|
||||
"webpki-roots 1.0.7",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "libc"
|
||||
version = "0.2.186"
|
||||
@@ -1395,6 +1524,15 @@ dependencies = [
|
||||
"minimal-lexical",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "nom"
|
||||
version = "8.0.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "df9761775871bdef83bee530e60050f7e54b1105350d6884eb0fb4f46c2f9405"
|
||||
dependencies = [
|
||||
"memchr",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "normalize-line-endings"
|
||||
version = "0.3.0"
|
||||
@@ -1477,6 +1615,12 @@ version = "1.70.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "384b8ab6d37215f3c5301a95a4accb5d64aa607f1fcb26a11b5303878451b4fe"
|
||||
|
||||
[[package]]
|
||||
name = "opaque-debug"
|
||||
version = "0.3.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "c08d65885ee38876c4f86fa503fb49d7b507c2b62552df7c70b2fce627e06381"
|
||||
|
||||
[[package]]
|
||||
name = "option-ext"
|
||||
version = "0.2.0"
|
||||
@@ -1610,7 +1754,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "picloud"
|
||||
version = "1.1.6"
|
||||
version = "1.1.7"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"async-trait",
|
||||
@@ -1618,12 +1762,15 @@ dependencies = [
|
||||
"axum-test",
|
||||
"chrono",
|
||||
"figment",
|
||||
"hex",
|
||||
"hmac",
|
||||
"picloud-executor-core",
|
||||
"picloud-manager-core",
|
||||
"picloud-orchestrator-core",
|
||||
"picloud-shared",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"sha2",
|
||||
"sqlx",
|
||||
"thiserror 1.0.69",
|
||||
"tokio",
|
||||
@@ -1636,7 +1783,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "picloud-cli"
|
||||
version = "1.1.6"
|
||||
version = "1.1.7"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"assert_cmd",
|
||||
@@ -1657,7 +1804,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "picloud-executor"
|
||||
version = "1.1.6"
|
||||
version = "1.1.7"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"picloud-executor-core",
|
||||
@@ -1669,7 +1816,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "picloud-executor-core"
|
||||
version = "1.1.6"
|
||||
version = "1.1.7"
|
||||
dependencies = [
|
||||
"async-trait",
|
||||
"base64",
|
||||
@@ -1693,7 +1840,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "picloud-manager"
|
||||
version = "1.1.6"
|
||||
version = "1.1.7"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"picloud-manager-core",
|
||||
@@ -1705,7 +1852,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "picloud-manager-core"
|
||||
version = "1.1.6"
|
||||
version = "1.1.7"
|
||||
dependencies = [
|
||||
"argon2",
|
||||
"async-trait",
|
||||
@@ -1715,6 +1862,9 @@ dependencies = [
|
||||
"chrono-tz",
|
||||
"cron",
|
||||
"data-encoding",
|
||||
"hex",
|
||||
"hmac",
|
||||
"lettre",
|
||||
"picloud-executor-core",
|
||||
"picloud-orchestrator-core",
|
||||
"picloud-shared",
|
||||
@@ -1733,7 +1883,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "picloud-orchestrator"
|
||||
version = "1.1.6"
|
||||
version = "1.1.7"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"picloud-orchestrator-core",
|
||||
@@ -1745,7 +1895,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "picloud-orchestrator-core"
|
||||
version = "1.1.6"
|
||||
version = "1.1.7"
|
||||
dependencies = [
|
||||
"async-trait",
|
||||
"axum",
|
||||
@@ -1768,17 +1918,20 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "picloud-shared"
|
||||
version = "1.1.6"
|
||||
version = "1.1.7"
|
||||
dependencies = [
|
||||
"aes-gcm",
|
||||
"async-trait",
|
||||
"base64",
|
||||
"chrono",
|
||||
"hmac",
|
||||
"rand 0.8.6",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"sha2",
|
||||
"thiserror 1.0.69",
|
||||
"tokio",
|
||||
"tracing",
|
||||
"uuid",
|
||||
]
|
||||
|
||||
@@ -1821,6 +1974,18 @@ version = "0.2.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "b4596b6d070b27117e987119b4dac604f3c58cfb0b191112e24771b2faeac1a6"
|
||||
|
||||
[[package]]
|
||||
name = "polyval"
|
||||
version = "0.6.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "9d1fe60d06143b2430aa532c94cfe9e29783047f06c0d7fd359a9a51b729fa25"
|
||||
dependencies = [
|
||||
"cfg-if",
|
||||
"cpufeatures",
|
||||
"opaque-debug",
|
||||
"universal-hash",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "portable-atomic"
|
||||
version = "1.13.1"
|
||||
@@ -1987,6 +2152,12 @@ dependencies = [
|
||||
"proc-macro2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "quoted_printable"
|
||||
version = "0.5.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "478e0585659a122aa407eb7e3c0e1fa51b1d8a870038bd29f0cf4a8551eea972"
|
||||
|
||||
[[package]]
|
||||
name = "r-efi"
|
||||
version = "5.3.0"
|
||||
@@ -2290,6 +2461,7 @@ version = "0.23.40"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "ef86cd5876211988985292b91c96a8f2d298df24e75989a43a3c73f2d4d8168b"
|
||||
dependencies = [
|
||||
"log",
|
||||
"once_cell",
|
||||
"ring",
|
||||
"rustls-pki-types",
|
||||
@@ -3229,6 +3401,16 @@ version = "0.2.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "ebc1c04c71510c7f702b52b7c350734c9ff1295c464a03335b00bb84fc54f853"
|
||||
|
||||
[[package]]
|
||||
name = "universal-hash"
|
||||
version = "0.5.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "fc1de2c688dc15305988b563c3854064043356019f97a4b46276fe734c4f07ea"
|
||||
dependencies = [
|
||||
"crypto-common",
|
||||
"subtle",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "untrusted"
|
||||
version = "0.9.0"
|
||||
|
||||
@@ -13,7 +13,7 @@ members = [
|
||||
]
|
||||
|
||||
[workspace.package]
|
||||
version = "1.1.6"
|
||||
version = "1.1.7"
|
||||
edition = "2021"
|
||||
rust-version = "1.92"
|
||||
license = "MIT OR Apache-2.0"
|
||||
@@ -81,6 +81,13 @@ sha2 = "0.10"
|
||||
hmac = "0.12"
|
||||
base64 = "0.22"
|
||||
data-encoding = "2.6"
|
||||
# AES-256-GCM at-rest encryption for per-app secrets + the realtime
|
||||
# signing key (v1.1.7). Audited, pure-Rust RustCrypto AEAD.
|
||||
aes-gcm = { version = "0.10", features = ["aes", "alloc"] }
|
||||
|
||||
# Outbound SMTP email (v1.1.7). Async transport over the Tokio runtime
|
||||
# with rustls TLS; built messages for text + multipart-alternative.
|
||||
lettre = { version = "0.11", default-features = false, features = ["smtp-transport", "tokio1-rustls-tls", "builder", "hostname"] }
|
||||
|
||||
# Stdlib utility crates (v1.1.0 stdlib PR — registered into the
|
||||
# Rhai engine as the regex::/random::/etc. namespaces)
|
||||
|
||||
530
HANDBACK.md
530
HANDBACK.md
@@ -1,220 +1,330 @@
|
||||
# HANDBACK — v1.1.6 Realtime Channels & Client Library
|
||||
# v1.1.7 — Configuration & Email — HANDBACK
|
||||
|
||||
Branch: `feat/v1.1.6-realtime-client` (from `main`). Not pushed, no PR.
|
||||
|
||||
## 1. Scope coverage (§1–§13)
|
||||
|
||||
| § | Item | Status |
|
||||
|---|------|--------|
|
||||
| 1 | `topics` table (`0021_topics.sql`) | ✅ Done |
|
||||
| 2 | Topic admin endpoints (POST/GET/PATCH/DELETE), `AppTopicManage` | ✅ Done |
|
||||
| 3 | SSE endpoint `GET /realtime/topics/{topic}` | ✅ Done |
|
||||
| 4 | In-process `RealtimeBroadcaster` + GC + publish wiring | ✅ Done |
|
||||
| 5 | HMAC subscriber tokens + `app_secrets` (`0022`) + `pubsub::subscriber_token` | ✅ Done |
|
||||
| 6 | Dashboard Topics tab | ✅ Done |
|
||||
| 7 | `@picloud/client` TypeScript package | ✅ Done |
|
||||
| 8 | Topic-aware publish → realtime wiring | ✅ Done |
|
||||
| 9 | Dispatcher e2e tests (six) | ✅ Done (location deviation — see §7) |
|
||||
| 10 | Empty-blob decision | ✅ Done — **relaxed** (accept empty) |
|
||||
| 11 | Orphan `*.tmp.*` sweeper | ✅ Done |
|
||||
| 12 | Version bumps (1.1.6 / SDK 1.7 / dash 0.12.0 / client 1.0.0) | ✅ Done |
|
||||
| 13 | Tests (all named-critical cases) | ✅ Done |
|
||||
|
||||
## 2. Realtime implementation notes
|
||||
|
||||
### Topic resolution / SSE handshake sequence
|
||||
1. Extract `Host` → `app_domains.resolve_app(host)` (existing two-phase
|
||||
dispatch). No app → **404**.
|
||||
2. Token from `Authorization: Bearer <t>` **or** `?token=<t>` (EventSource
|
||||
can't set headers).
|
||||
3. `RealtimeAuthority::authorize_subscribe(app_id, topic, token)`:
|
||||
- topic missing OR `external_subscribable = false` → `NotFound` → **404**
|
||||
(both collapse to 404 so the endpoint can't probe internal topics);
|
||||
- `auth_mode='public'` → allow;
|
||||
- `auth_mode='token'` → verify HMAC (present, signed by this app's key,
|
||||
unexpired, scoped to this topic) → allow, else `Unauthorized` → **401**
|
||||
(generic; never says which check failed).
|
||||
4. `broadcaster.subscribe(app_id, topic)` → `broadcast::Receiver`; stream
|
||||
`data: {topic,message,published_at}\n\n` with `:heartbeat` keepalive
|
||||
every `PICLOUD_REALTIME_HEARTBEAT_SEC` (default 30). Headers:
|
||||
`text/event-stream`, `Cache-Control: no-cache` (set by axum Sse),
|
||||
`X-Accel-Buffering: no` (added). Client disconnect drops the receiver →
|
||||
automatic cleanup; the periodic GC reaps empty channels.
|
||||
|
||||
The SSE handler lives in **orchestrator-core** (`realtime_api.rs`) and
|
||||
depends only on the three picloud-shared traits — all DB + signing-key
|
||||
access stays in the manager-core `RealtimeAuthority` impl, so the
|
||||
data-plane crate never touches the key. Cluster mode (v1.3+) swaps the
|
||||
broadcaster + authority impls behind the same traits.
|
||||
|
||||
### HMAC signing-key storage — chose **table** (`app_secrets`)
|
||||
Per the asked decision. 32 random bytes per app, lazily created on the
|
||||
first `pubsub::subscriber_token` call (`ON CONFLICT DO NOTHING` for
|
||||
concurrency). No global `PICLOUD_INSTANCE_SECRET`; the table is the
|
||||
natural home for v1.1.7's encrypted per-app secrets.
|
||||
**Tension flagged:** §5 aspired to "validate the signature without a DB
|
||||
lookup". The table approach needs a key read on the token-subscribe path.
|
||||
Mitigated by an in-memory key cache in `RealtimeAuthorityImpl` (keys never
|
||||
rotate in v1.1.6, so the cache needs no invalidation); the subscribe path
|
||||
already reads the `topics` row, so this adds no new round-trip *category*.
|
||||
|
||||
### In-process broadcaster
|
||||
`Mutex<HashMap<(AppId, String), broadcast::Sender<RealtimeEvent>>>`.
|
||||
Capacity per channel `PICLOUD_REALTIME_BROADCAST_CAPACITY` (default 64);
|
||||
slow consumers lose oldest events (`broadcast` lag semantics — best-effort,
|
||||
no replay). `subscribe` creates channels lazily; `publish` is a silent
|
||||
no-op when no channel exists; `drop_topic` removes the sender (existing
|
||||
receivers observe a closed channel and disconnect). A `spawn_realtime_gc`
|
||||
task (~60s) drops senders with `receiver_count() == 0`.
|
||||
|
||||
### Publish wiring (order + failure modes) — §8 ordering chosen
|
||||
`PubsubServiceImpl::publish_durable`: validate/authz → **transactional
|
||||
outbox fan-out + commit** (existing) → **then** best-effort
|
||||
`RealtimeBroadcaster::publish`. The broadcast runs on a child
|
||||
`tokio::spawn` whose `JoinHandle` is awaited, so a panicking broadcaster
|
||||
becomes a `warn` log, never a failed publish (the durable deliveries
|
||||
already committed). One `published_at` instant is shared by both paths.
|
||||
**Brief-internal contradiction flagged** — see §9.
|
||||
|
||||
## 3. Client lib implementation notes
|
||||
|
||||
- **Build:** **tsup** (dual ESM+CJS + `.d.ts`), tests **vitest**, lint =
|
||||
`tsc --noEmit` (strict; `noUncheckedIndexedAccess`, no `any` in exports).
|
||||
- **Layout:** `src/{index,client,endpoint,subscribe,auth,types}.ts` +
|
||||
`src/react/index.ts` + `src/svelte/index.ts`; subpath exports
|
||||
`@picloud/client/react` and `@picloud/client/svelte` via package `exports`.
|
||||
- **Reconnect:** SSE is implemented over **streaming `fetch`** (not native
|
||||
`EventSource`) so the lib can (a) detect a **401** on (re)connect and call
|
||||
`onTokenExpired` to refresh, (b) send **`Last-Event-ID`** on resume
|
||||
(server ignores it in v1.1.6 — client ships ready), and (c) apply its own
|
||||
**exponential backoff** (base 1s → ×2 → cap 30s; reset on successful
|
||||
open). Token rides in `?token=` (EventSource-parity). React Native caveat
|
||||
documented in the README (supply a streaming-`fetch` polyfill).
|
||||
- **zod/valibot adapter:** the `Validator<T> = { parse(input): T }` shape.
|
||||
A Zod schema satisfies it directly; Valibot wraps in one line. Used by
|
||||
`endpoint(...).get({ validate })` and `subscribe(..., { validate })`. No
|
||||
hard dep.
|
||||
|
||||
## 4. v1.1.5 follow-ups
|
||||
|
||||
- **Dispatcher e2e (six):** `dispatcher_delivers_{kv,docs,cron,files,pubsub,
|
||||
dead_letter}_to_handler` in `crates/picloud/tests/dispatcher_e2e.rs`.
|
||||
Verified green against a real Postgres (all 6).
|
||||
- **Empty-blob — RELAXED** (per the asked decision). Dropped the
|
||||
`data.is_empty()` rejection in `NewFile::validate` **and** `FileUpdate::
|
||||
validate` (the latter for consistency — flagged in §7). Flipped the
|
||||
pinning test in `files_service.rs` and added `empty_file_round_trips`.
|
||||
- **Orphan sweep:** `spawn_files_orphan_sweep` (`files_sweep.rs`), every 6h
|
||||
(`PICLOUD_FILES_ORPHAN_SWEEP_INTERVAL_SEC`), unlinks `*.tmp.*` older than
|
||||
1h (`PICLOUD_FILES_ORPHAN_TMP_TTL_SEC`); logs dirs-walked / files-deleted /
|
||||
bytes-reclaimed. No DB cross-check (v1.3+). Tested: deletes old tmp, keeps
|
||||
young tmp, keeps non-tmp, missing-root no panic.
|
||||
|
||||
## 5. Schema decisions beyond the brief
|
||||
None — `0021_topics.sql` and `0022_app_secrets.sql` match the brief's DDL
|
||||
verbatim. Schema-snapshot golden re-blessed on a fresh DB (only the two new
|
||||
tables added; PK/FK `ON DELETE CASCADE` + the `auth_mode` CHECK present).
|
||||
|
||||
## 6. How to verify locally
|
||||
See §8 below.
|
||||
|
||||
## 7. Decisions beyond the brief — every prompt-default deviation
|
||||
1. **Dispatcher e2e location:** the brief says
|
||||
`crates/manager-core/tests/dispatcher_e2e.rs`; I put it in
|
||||
**`crates/picloud/tests/`**. `build_app` (the full dispatcher + scheduler
|
||||
+ executor wiring) lives in the `picloud` crate; a manager-core test would
|
||||
need a manager→picloud dev-dependency cycle or a hand-rolled re-wire of
|
||||
build_app. The picloud harness (`server_with_app` pattern) is proven and
|
||||
the tests run green against a real DB. Gating uses the **schema_snapshot
|
||||
pattern** (env check + early return), NOT `#[ignore]`, so CI's plain
|
||||
`cargo test` with `DATABASE_URL` runs them while local stays green — as
|
||||
the brief requested.
|
||||
2. **Empty-blob relaxation extended to `FileUpdate::validate`** — the brief
|
||||
names only `NewFile::validate`. Relaxing create-empty but rejecting
|
||||
update-to-empty would be an inconsistent API, so I relaxed both.
|
||||
3. **Publish order: §8 (broadcast AFTER outbox commit)**, not §4 (broadcast
|
||||
FIRST). The two sections contradict; §8 is the dedicated, numbered,
|
||||
rationale-backed section. See §9.
|
||||
4. **Topic-name validation** — `topics_api` rejects empty names and names
|
||||
containing `*` (external pattern subscription is v2). The brief didn't
|
||||
specify name validation; this is a small guard.
|
||||
5. **Client lib lint = `tsc --noEmit`** (not eslint) to keep devDeps lean;
|
||||
strict typecheck is the gate. "No `any` in exports" is enforced by review
|
||||
+ strict TS, not an eslint rule.
|
||||
6. **Cron e2e poll budget = 45s** — the cron scheduler skips its first tick
|
||||
then ticks every 30s (default). The test polls 45s so it passes at the
|
||||
default; set `PICLOUD_CRON_TICK_INTERVAL_MS=1000` to make it ~2s in CI.
|
||||
|
||||
## 8. Attestation (gate runs on this HEAD)
|
||||
**Branch:** `feat/v1.1.7-secrets-email` (9 commits off `main`, not pushed)
|
||||
**Status:** ready for review. NOT merged, NOT pushed, no PR opened.
|
||||
|
||||
```
|
||||
cargo fmt --all -- --check → clean
|
||||
cargo clippy --all-targets --all-features -- -D warnings → clean (exit 0)
|
||||
cargo test --workspace → 482 passed, 0 failed
|
||||
(DB-gated dispatcher_e2e
|
||||
auto-skip as no-op when
|
||||
DATABASE_URL unset)
|
||||
# DB-gated (real Postgres @ 127.0.0.1:15432, picloud/picloud):
|
||||
DATABASE_URL=… PICLOUD_CRON_TICK_INTERVAL_MS=1000 \
|
||||
cargo test -p picloud --test dispatcher_e2e → 6 passed
|
||||
DATABASE_URL=… cargo test -p picloud-manager-core --test schema_snapshot → 1 passed
|
||||
(cd dashboard && npm run check) → 0 errors, 0 warnings (371 files)
|
||||
(cd clients/typescript && npm run lint) → clean (tsc --noEmit)
|
||||
(cd clients/typescript && npm run test) → 15 passed (5 files)
|
||||
(cd clients/typescript && npm run build) → tsup ESM+CJS+.d.ts OK
|
||||
a7d3dad chore(v1.1.7): re-bless schema snapshot for secrets + email migrations
|
||||
2ea47eb chore(v1.1.7): fix clippy --all-targets warnings
|
||||
b355851 chore(v1.1.7): version bumps + CHANGELOG
|
||||
fffcdf6 feat(v1.1.7-realtime-migration): encrypt signing keys at rest
|
||||
02335a8 fix(v1.1.7-dead-letter): wire dispatcher → list_matching_dead_letter
|
||||
1f78937 feat(v1.1.7-email-inbound): webhook receiver + email:receive trigger
|
||||
8f2d2bc feat(v1.1.7-email-outbound): SMTP send/send_html
|
||||
2d11090 feat(v1.1.7-secrets): secrets SDK + table + admin API + dashboard
|
||||
dc2e4fa feat(v1.1.7-crypto): master-key infra + encryption helpers
|
||||
```
|
||||
Migrations verified applying cleanly on a fresh DB **and** on top of the
|
||||
existing v1.1.5 dev DB (0020 → 0021 → 0022). Schema-snapshot golden diff is
|
||||
exactly the two new tables.
|
||||
|
||||
---
|
||||
|
||||
## 1. Scope coverage
|
||||
|
||||
| Item | Status |
|
||||
|---|---|
|
||||
| Encryption infrastructure (master key + AES-256-GCM envelope) | **Done** |
|
||||
| `secrets::*` SDK + `0023_secrets.sql` + admin API + dashboard tab | **Done** |
|
||||
| Outbound email `email::send` / `email::send_html` (lettre SMTP) | **Done** |
|
||||
| Inbound email webhook receiver + `email:receive` trigger + `0024` | **Done** (full scope, per user decision) |
|
||||
| Dispatcher routing for email | **Done** |
|
||||
| dead_letter handler wiring fix | **Done** |
|
||||
| Realtime signing-key encryption (two-phase) + `0025` | **Done** |
|
||||
| Dashboard (Secrets tab, email trigger form, `npm run check`) | **Done** |
|
||||
| Version bumps (1.1.7 / SDK 1.8 / dashboard 0.13.0) + CHANGELOG | **Done** |
|
||||
| Tests (match v1.1.5/v1.1.6 density) | **Done** |
|
||||
|
||||
Nothing deferred from scope-in. Inbound email (the deferrable-if-scope-
|
||||
blew-up piece) was implemented in full.
|
||||
|
||||
---
|
||||
|
||||
## 2. Encryption infrastructure notes
|
||||
|
||||
- **Module:** `crates/shared/src/crypto.rs` (`picloud_shared::crypto`).
|
||||
- **Master-key sourcing** (`MasterKey::from_env` → `resolve`):
|
||||
- `PICLOUD_SECRET_KEY` = base64 of exactly 32 bytes. Missing →
|
||||
`MasterKeyError::Missing` (fatal); non-base64 → `Malformed`; wrong
|
||||
length → `WrongLength`. **Sourced in `main.rs::run_server` before any
|
||||
DB work** — `build_app` takes the `MasterKey` as a parameter (so
|
||||
tests pass a fixed key and don't mutate process env).
|
||||
- Dev fallback: deterministic key (`SHA-256("picloud-dev-master-key-v1.1.7")`)
|
||||
used ONLY when `PICLOUD_SECRET_KEY` is unset **AND**
|
||||
`PICLOUD_DEV_MODE=true`, with a prominent `warn!`. No quiet
|
||||
unencrypted mode.
|
||||
- **aes-gcm version:** `0.10` (features `aes`, `alloc`). `Aes256Gcm`.
|
||||
- **Nonce generation:** 12 bytes from `rand::thread_rng().fill_bytes`
|
||||
(OS-CSPRNG-seeded), per-encryption.
|
||||
- **Storage layout:** ciphertext **with the 16-byte GCM auth tag
|
||||
appended** (RustCrypto `Aead`-trait layout — `encrypt` returns
|
||||
`ciphertext || tag`, `decrypt` consumes the same). The 12-byte nonce is
|
||||
stored in a separate column. `MasterKey`'s `Debug` is redacted.
|
||||
- **Plaintext cap (secrets):** 64 KB default, enforced in
|
||||
`secrets_service::seal` (the SDK boundary) → `SecretsError::TooLarge`
|
||||
with limit + actual size. Override: `PICLOUD_SECRET_MAX_VALUE_BYTES`.
|
||||
- **Key rotation:** out of scope. Documented in CHANGELOG + the module
|
||||
docs that changing `PICLOUD_SECRET_KEY` orphans all ciphertext.
|
||||
|
||||
---
|
||||
|
||||
## 3. Secrets notes
|
||||
|
||||
- `SecretsService` (trait, `picloud-shared`) → `SecretsServiceImpl` +
|
||||
`PostgresSecretsRepo` (`manager-core`) → Rhai bridge
|
||||
(`executor-core/src/sdk/secrets.rs`). Collection-less; `app_id` from
|
||||
`cx.app_id`.
|
||||
- **JSON round-trip:** `set` serializes the value to JSON bytes, caps,
|
||||
encrypts; `get` decrypts + deserializes — a String returns a String
|
||||
(not a JSON-quoted `"\"…\""`). Verified by unit + bridge tests.
|
||||
- **No ServiceEvent emission** (secret writes don't fire triggers).
|
||||
- Admin API: `GET/POST/DELETE /api/v1/admin/apps/{id}/secrets`; list
|
||||
returns names + `updated_at` only.
|
||||
- Authz: `Capability::AppSecretsRead/Write` → `script:read`/`script:write`.
|
||||
No new Scope variants (seven-scope commitment held).
|
||||
|
||||
---
|
||||
|
||||
## 4. Email implementation notes
|
||||
|
||||
- **SMTP transport:** `lettre 0.11` (`smtp-transport`,
|
||||
`tokio1-rustls-tls`, `builder`, `hostname`). **Connection model:** one
|
||||
connection per call (lettre default); pooling deferred to v1.2. The
|
||||
transport sits behind an internal `EmailTransport` trait so the service
|
||||
is unit-tested with a recording fake (no live SMTP).
|
||||
- **Disabled mode:** if HOST/USER/PASSWORD aren't all set,
|
||||
`EmailServiceImpl::from_env` builds no transport and every `send`
|
||||
returns `NotConfigured` (warned at startup). A malformed relay
|
||||
descriptor is also logged and yields disabled mode (email is
|
||||
non-critical; never blocks startup).
|
||||
- **Address validation:** hand-rolled RFC 5322-ish pre-check (single `@`,
|
||||
non-empty local part, domain contains a dot, ≤320 bytes) followed by a
|
||||
`lettre::Mailbox` parse (the authoritative validator). No deliverability
|
||||
check.
|
||||
- **Size cap:** 25 MB on `message.formatted()`,
|
||||
`PICLOUD_EMAIL_MAX_MESSAGE_BYTES`.
|
||||
- `email::send` forces text-only (ignores any `html`); `email::send_html`
|
||||
requires `html` and builds `MultiPart::alternative_plain_html`.
|
||||
`reply_to` defaults to `from`. `to`/`cc`/`bcc` accept a String or an
|
||||
Array of Strings.
|
||||
- **Inbound normalization:** only the generic provider-agnostic JSON
|
||||
shape `{from,to[],cc[],subject,text,html,message_id}` is accepted in
|
||||
v1.1.7 — `from` required, rest default. Provider-specific unmarshallers
|
||||
→ v1.2. The expected shape is documented on the dashboard email-trigger
|
||||
form.
|
||||
|
||||
---
|
||||
|
||||
## 5. Dead-letter handler fix notes
|
||||
|
||||
- **Call site:** `dispatcher::handle_failure`, the retry-exhaustion
|
||||
branch. After `DeadLetterRepo::insert` (which returns the new
|
||||
`DeadLetterId`), a new helper `fan_out_dead_letter` runs.
|
||||
- **What it does:** calls `TriggerRepo::list_matching_dead_letter(app_id,
|
||||
source, row.trigger_id, Some(resolved.script_id))` (the method that had
|
||||
no production caller) and inserts one outbox row per match
|
||||
(`source_kind = DeadLetter`, the DL trigger's id + handler script id,
|
||||
`trigger_depth + 1`, `origin_principal = the DL trigger's registered
|
||||
principal`).
|
||||
- **Payload — built from the REAL `TriggerEvent::DeadLetter` variant**,
|
||||
not the brief's §6 field list (see §7 deviations): `{ dead_letter_id,
|
||||
original: Box::new(decoded row payload), attempts, last_error,
|
||||
trigger_id, script_id, first_attempt_at, last_attempt_at }`. If the
|
||||
outbox payload can't be decoded back into a `TriggerEvent` (so the
|
||||
nested `original` can't be built), the fan-out is skipped — the
|
||||
dead-letter row is still durably written.
|
||||
- **Recursion-stop:** unchanged. The `is_dead_letter_handler`
|
||||
short-circuit at the top of `handle_failure` returns before the
|
||||
exhaustion branch, so a DL handler's own failure is never re-dead-
|
||||
lettered. No new guard needed.
|
||||
- **Tests verify the handler actually fires**
|
||||
(`crates/picloud/tests/dispatcher_e2e.rs`, DB-gated):
|
||||
`dispatcher_delivers_dead_letter_to_handler` now asserts BOTH row-create
|
||||
AND handler-fire (inline doc updated);
|
||||
`dispatcher_delivers_dead_letter_to_handler_actually_fires` asserts the
|
||||
nested `original` KV event + `last_error`;
|
||||
`dead_letter_source_filter_excludes_nonmatching` exercises the source
|
||||
filter dimension; `dead_letter_handler_failure_does_not_recurse` proves
|
||||
the recursion-stop (count stays at 1).
|
||||
|
||||
---
|
||||
|
||||
## 6. Realtime signing-key migration notes
|
||||
|
||||
- **Two-phase**, as recommended. `0025_encrypt_realtime_keys.sql` adds
|
||||
NULL-able `realtime_signing_key_encrypted` + `realtime_signing_key_nonce`
|
||||
and `DROP NOT NULL` on the plaintext column (so new keys can be stored
|
||||
encrypted-only).
|
||||
- **Repo:** `PostgresAppSecretsRepo` now holds the `MasterKey`. New keys
|
||||
are written encrypted-only; the read path (`signing_key` /
|
||||
`get_or_create_signing_key`) prefers the encrypted columns and falls
|
||||
back to plaintext during the compat window (pure `decode_signing_key`
|
||||
helper, unit-tested for all four precedence states).
|
||||
- **Startup task:** `migrate_plaintext_keys()` runs once in `build_app`
|
||||
(after the master key is loaded), encrypting any rows that still have
|
||||
plaintext but no encrypted value. Plaintext is **left in place** for
|
||||
rollback safety. Idempotent.
|
||||
- **Plaintext column drop:** deferred to **v1.1.8** (documented in
|
||||
CHANGELOG + the migration). Operators must upgrade through v1.1.7
|
||||
(which performs the encryption) before v1.1.8.
|
||||
- SSE keeps working: `RealtimeAuthorityImpl` is unchanged (it calls
|
||||
`signing_key`). Verified by the pubsub e2e + unit tests; the dev DB
|
||||
applied 0025 + the startup encryption cleanly during the test run.
|
||||
|
||||
---
|
||||
|
||||
## 7. Decisions beyond the brief / deviations flagged
|
||||
|
||||
1. **`inbound_secret` stored ENCRYPTED (user-approved deviation).** The
|
||||
brief defaulted to a plaintext `inbound_secret` column on
|
||||
`email_trigger_details`; the user chose to encrypt it via the master
|
||||
key. Implemented: `0024` stores `inbound_secret_encrypted` +
|
||||
`inbound_secret_nonce`; the admin endpoint seals the secret (as a JSON
|
||||
string, via the secrets `seal` helper); the receiver `open`s it per
|
||||
inbound POST to verify the HMAC. **Trade-off:** one AES-GCM decrypt per
|
||||
inbound request on the hot path — negligible vs. the HMAC + DB
|
||||
round-trip already there. The decrypted secret is never logged.
|
||||
|
||||
2. **Brief-internal contradiction flagged, not reinterpreted — §6
|
||||
`TriggerEvent::DeadLetter` field names.** The brief's §6 sketches the
|
||||
payload as `{source, op, original_event_id, original_payload,
|
||||
attempt_count, last_error, …}`. The actual variant
|
||||
(`crates/shared/src/trigger_event.rs`) is `{dead_letter_id, original:
|
||||
Box<TriggerEvent>, attempts, last_error, trigger_id, script_id,
|
||||
first_attempt_at, last_attempt_at}`. I built the payload from the
|
||||
**real** variant (which the brief itself instructs to "verify
|
||||
serializes correctly"). No type change needed.
|
||||
|
||||
3. **`build_app` signature gained a `MasterKey` parameter.** Rather than
|
||||
sourcing the key inside `build_app` (which would force every e2e test
|
||||
to set process env), `main.rs` sources it and passes it in. The 3
|
||||
existing `build_app` test callers pass a fixed test key.
|
||||
|
||||
4. **Pre-existing clippy warnings fixed (see §10).** Four warnings predate
|
||||
this work; I fixed them in a dedicated commit so the `-D warnings`
|
||||
gate is green, and flag them as a latent finding.
|
||||
|
||||
5. **Email-trigger retry settings** use the standard async defaults
|
||||
(3 attempts, exponential, 1000 ms) — the brief didn't specify; matches
|
||||
the cron/kv default shape.
|
||||
|
||||
No other deviations from prompt-specified defaults.
|
||||
|
||||
---
|
||||
|
||||
## 8. How to verify locally — §8 attestation (sourced from cargo's literal output)
|
||||
|
||||
All gates run on the handed-back HEAD (`a7d3dad`):
|
||||
|
||||
```sh
|
||||
cargo fmt --all -- --check # clean
|
||||
cargo clippy --all-targets --all-features -- -D warnings # clean (exit 0)
|
||||
cd dashboard && npm run check # 0 ERRORS 0 WARNINGS (371 files)
|
||||
```
|
||||
|
||||
Full test run **with `DATABASE_URL` set** so the DB-gated suites
|
||||
(schema_snapshot, dispatcher_e2e ×9, email_inbound ×8) execute:
|
||||
|
||||
```sh
|
||||
DATABASE_URL='postgres://picloud:picloud@127.0.0.1:15432/picloud' \
|
||||
cargo test --workspace -- --test-threads=2
|
||||
```
|
||||
|
||||
**Pass count, summed from cargo's literal output (NOT hand-counted):**
|
||||
|
||||
```sh
|
||||
DATABASE_URL=... cargo test --workspace -- --test-threads=2 2>&1 | \
|
||||
awk '/test result: ok\./ { gsub(";", ""); sum += $4 } END { print sum }'
|
||||
# => 617
|
||||
```
|
||||
|
||||
**617 passed, 0 failed** across the workspace (34 `test result:` lines,
|
||||
0 `FAILED`). Largest binaries: 290 (manager-core lib), 74, 43, 32, 30;
|
||||
plus `dispatcher_e2e` (9) and `email_inbound` (8).
|
||||
|
||||
**Bounded-parallelism note (`--test-threads=2`):** the picloud e2e
|
||||
binaries each call `build_app`, which opens its own Postgres pool. Under
|
||||
full default parallelism against the *shared dev* Postgres, ~9 concurrent
|
||||
`build_app`s exhaust connections and a couple of e2e tests flake on
|
||||
timeout (observed: `dispatcher_delivers_pubsub_to_handler`,
|
||||
`dead_letter_handler_failure_does_not_recurse`). They pass reliably at
|
||||
`--test-threads=2` and in isolation. CI's dedicated fresh `postgres:15`
|
||||
(not a shared dev DB) does not hit this. Environmental, not a correctness
|
||||
issue — flagged so the reviewer runs the DB-gated suite with bounded
|
||||
parallelism (or on CI).
|
||||
|
||||
**Migrations:** apply cleanly on the v1.1.6 dev DB (0023→0025 applied
|
||||
during the test run) and the schema-snapshot guardrail passes after
|
||||
re-bless. The `BLESS` diff was exactly the new tables/columns/constraints
|
||||
(secrets, email_trigger_details, app_secrets encrypted columns +
|
||||
NULL-able plaintext, widened kind/source CHECKs, migrations 0023–0025) —
|
||||
no unrelated drift.
|
||||
|
||||
**Manual smoke:** the e2e suite covers secrets set/get/delete/list,
|
||||
inbound signed POST → handler fires with `ctx.event.email`, dead-letter
|
||||
handler fires, realtime-key encryption + SSE. Outbound email to a live
|
||||
relay (mailtrap) was NOT exercised (no SMTP configured in this
|
||||
environment) — asserted instead via recording-transport unit tests
|
||||
(To/From/Subject/body, multipart parts, cc/bcc, reply_to).
|
||||
|
||||
---
|
||||
|
||||
## 9. Open questions for the reviewer
|
||||
1. **§4 vs §8 ordering contradiction (load-bearing, flagged not
|
||||
reinterpreted):** §4 says "realtime broadcast FIRST, then transactional
|
||||
outbox fan-out"; §8 says broadcast AFTER the fan-out commits (numbered
|
||||
steps 1–4). I implemented **§8** (dedicated section + failure-mode
|
||||
rationale). If §4's ordering was intended, the broadcast should move
|
||||
before `fan_out_publish` — but then a publish whose outbox write fails
|
||||
would still have notified SSE subscribers of an event that never durably
|
||||
happened, which seems wrong. Please confirm §8 is correct.
|
||||
2. **Dead-letter → handler fan-out appears unwired (see §10).** Confirm
|
||||
whether the `dead_letter` trigger source is intended to fire in v1.1.x.
|
||||
|
||||
## 10. Latent findings
|
||||
**`dead_letter` trigger handlers do not fire on dead-letter creation.**
|
||||
`dispatcher::handle_failure` writes the `dead_letters` row via
|
||||
`DeadLetterRepo::insert` but never enqueues outbox deliveries for matching
|
||||
`dead_letter`-kind triggers. `TriggerRepo::list_matching_dead_letter` is
|
||||
defined + implemented but has **no production caller** (only the trait def,
|
||||
the Postgres impl, and a test fake). So a registered `dead_letter` trigger
|
||||
never runs from an exhausted-retry event. This predates v1.1.6 (the design
|
||||
notes §4 describe it shipping in v1.1.1). I did **not** fix it (out of
|
||||
v1.1.6 scope) — flagging for the reviewer. Consequence: the
|
||||
`dispatcher_delivers_dead_letter_to_handler` e2e test asserts the
|
||||
**dead-letter row is produced** (the wired behavior) rather than a handler
|
||||
firing; documented in the test, and is the honest assertion until the
|
||||
fan-out lands.
|
||||
1. **§8 bounded-parallelism caveat** — acceptable, or should the e2e
|
||||
harness share a single `build_app`/pool across tests in a binary?
|
||||
(Out of v1.1.7 scope; the existing v1.1.6 e2e tests have the same
|
||||
shape.)
|
||||
2. **`email::send` ignoring a stray `html` key** (forcing text-only) vs.
|
||||
throwing — I chose forgiving text-only; happy to make it strict.
|
||||
3. **Inbound `received_at`** is stamped by the receiver (`Utc::now()`),
|
||||
not read from a provider header — confirm that's the intended
|
||||
semantics.
|
||||
|
||||
No new security gaps introduced. Cross-app isolation holds: `subscriber_token`
|
||||
claims carry `app_id` and are signed per-app; the SSE authority rejects
|
||||
cross-app tokens (a per-app key already fails the signature, plus an explicit
|
||||
`claims.app_id == app_id` check); topic admin endpoints bind the capability
|
||||
to the path `app_id` after loading the app; the broadcaster keys channels by
|
||||
`(AppId, topic)` so app A's publishes never reach app B's subscribers
|
||||
(unit-tested).
|
||||
---
|
||||
|
||||
## 11. Deferred items (beyond this prompt's OUT list)
|
||||
None added. Everything on the brief's OUT list stayed out (WebSocket,
|
||||
session/script auth modes, topic-pattern external subscription, server-side
|
||||
last-event-id replay, per-app SSE/rate limits, other-language SDKs, codegen,
|
||||
full DB-cross-check sweeper).
|
||||
## 10. Latent security / correctness findings
|
||||
|
||||
## 12. Known limitations / rough edges
|
||||
- SSE delivery is best-effort at-most-once; a slow consumer past the
|
||||
broadcast buffer loses events with no server-side replay (by design).
|
||||
- The client lib's streaming-`fetch` SSE needs a polyfill on runtimes
|
||||
without `fetch` body streaming (React Native) — documented.
|
||||
- Cron e2e takes ~31s at the default 30s tick interval (45s poll budget);
|
||||
set `PICLOUD_CRON_TICK_INTERVAL_MS=1000` to speed it up.
|
||||
- The realtime key cache in `RealtimeAuthorityImpl` is per-process and never
|
||||
invalidated — correct only because v1.1.6 has no key rotation. A future
|
||||
rotation API must clear it.
|
||||
1. **`clippy --all-targets --all-features -- -D warnings` did NOT pass at
|
||||
v1.1.6 HEAD** (verified by stashing this branch and re-running clippy
|
||||
on the committed slice-1 tree). Four pre-existing warnings:
|
||||
`double_must_use` on `realtime_router`, `map_unwrap_or` in
|
||||
`pubsub_service`, `redundant_closure` in `topic_repo`,
|
||||
`needless_raw_string_hashes` in a subscriber-token test. Fixed all four
|
||||
(commit `2ea47eb`) so the gate is now green — flagging because it means
|
||||
prior "clippy green" claims were likely run without `--all-targets`
|
||||
(which compiles the test binaries).
|
||||
|
||||
2. **Inbound HMAC fails closed on decrypt error.** If a stored
|
||||
`inbound_secret` can't be decrypted (e.g. `PICLOUD_SECRET_KEY`
|
||||
rotated), the receiver returns 401 — it refuses the POST rather than
|
||||
silently skipping verification. Intentional.
|
||||
|
||||
3. **No rate limiting on the public inbound-email endpoint.** Like every
|
||||
public data-plane route, `/api/v1/email-inbound/...` is
|
||||
unauthenticated by design (URL + HMAC are the gate). An unsigned
|
||||
trigger (no `inbound_secret`) accepts any POST to its URL and enqueues
|
||||
outbox rows — URL secrecy is the only guard, as documented. Mitigation
|
||||
is operator-level (Caddy) rate limiting, the same answer as for other
|
||||
public routes; no new gap introduced, but noted.
|
||||
|
||||
---
|
||||
|
||||
## 11. Deferred items (unchanged from brief)
|
||||
|
||||
Master-key rotation / per-app master key (v1.2); native SMTP listener
|
||||
(v1.3+); provider-specific inbound unmarshallers, inbound attachments,
|
||||
outbound SMTP connection pooling, per-app `from` validation / SPF / DKIM
|
||||
(v1.2 / operator); dashboard inbound payload viewer (v1.2, PII); drop the
|
||||
plaintext `realtime_signing_key` column (v1.1.8); secrets
|
||||
versioning/history + secrets-change triggers (never); `users::*` (v1.1.8);
|
||||
`queue::*` / `invoke()` (v1.1.9).
|
||||
|
||||
---
|
||||
|
||||
## 12. Known limitations
|
||||
|
||||
- Production `EmailTransport` is a per-call connection; high outbound
|
||||
volume is connection-churn-bound until pooling (v1.2).
|
||||
- Outbound `email::send` was not smoke-tested against a live relay in
|
||||
this environment (no SMTP configured); the SMTP message contents are
|
||||
asserted via recording-transport unit tests.
|
||||
- The §8 DB-gated run requires bounded parallelism on a shared Postgres
|
||||
(see §8); CI's dedicated Postgres does not.
|
||||
|
||||
254
REVIEW.md
254
REVIEW.md
@@ -1,199 +1,183 @@
|
||||
# v1.1.6 Audit & Review
|
||||
# v1.1.7 Audit & Review
|
||||
|
||||
**Branch:** `feat/v1.1.6-realtime-client`
|
||||
**Base:** `main` (v1.1.5 head)
|
||||
**Commits ahead:** 3 (2 substantive + handback)
|
||||
**HEAD audited:** `f5a3f92`
|
||||
**Branch:** `feat/v1.1.7-secrets-email`
|
||||
**Base:** `main` (v1.1.6 head)
|
||||
**Commits ahead:** 10 (8 substantive + 1 chore-clippy-fix + 1 handback)
|
||||
**HEAD audited:** `3cfb795`
|
||||
**Audited by:** reviewer (this report)
|
||||
**Audited against:** the v1.1.6 dispatch prompt + the v1.1.1–v1.1.5 patterns it mandated
|
||||
**Audited against:** the v1.1.7 dispatch prompt + the v1.1.1–v1.1.6 patterns it mandated
|
||||
**Iterations:** 1
|
||||
|
||||
## Verdict
|
||||
|
||||
**APPROVE — ready to merge to `main` as v1.1.6.**
|
||||
**APPROVE — ready to merge to `main` as v1.1.7.**
|
||||
|
||||
The largest release in v1.1.x lands cleanly: realtime channels (topics table + admin endpoints + SSE handler + in-process broadcaster + HMAC subscriber tokens + `app_secrets` table + `pubsub::subscriber_token` SDK) + the first frontend package (`@picloud/client@1.0.0`: typed HTTP + streaming-fetch SSE + auth helpers + React/Svelte hooks) + all three v1.1.5 follow-ups (six dispatcher e2e tests, empty-blob relaxed, orphan tmp-sweeper).
|
||||
Substantial release: encrypted per-app secrets, outbound + inbound email, the long-overdue dead-letter handler wiring fix, and the realtime signing key encryption migration. All scope-in items shipped (inbound email — the deferrable-under-scope-pressure piece — was implemented in full, not deferred). 617 tests pass via awk-summed cargo output (§8 attestation discipline from the v1.1.6 retro landed). Gates green.
|
||||
|
||||
Two open questions raised in HANDBACK §9/§10 — I'll weigh in:
|
||||
Three flagged items in HANDBACK §7/§9/§10, all transparent and correct calls:
|
||||
|
||||
1. **§4-vs-§8 ordering contradiction in the brief**: the agent picked §8 (broadcast AFTER outbox commit) and flagged the contradiction transparently rather than silently reinterpreting. **§8 is the correct call** — see §3 below. This is the v1.1.4 retro lesson on brief-internal contradictions working as intended.
|
||||
2. **Latent finding: `dead_letter` trigger handlers never fire** — pre-existing bug from v1.1.1 confirmed. Not v1.1.6's responsibility to fix; correctly out-of-scope. See §4 below for the verification and the v1.1.7 follow-up.
|
||||
1. **Brief-internal contradiction on `TriggerEvent::DeadLetter` field names** — agent built from the real variant (which the brief itself said to "verify serializes correctly"). The v1.1.6 retro discipline lesson (flag-don't-reinterpret) working again.
|
||||
|
||||
Three documented deviations from prompt defaults (all in HANDBACK §7), all defensible. One test-count discrepancy worth noting (582 vs 482 claim — see §5). None of this blocks merge.
|
||||
2. **`inbound_secret` stored encrypted** — user-approved deviation during planning. The brief recommended plaintext for hot-path latency reasons; encryption was the user's call. Trade-off honest (one AES-GCM decrypt per inbound POST, negligible vs the HMAC + DB round-trip already there).
|
||||
|
||||
3. **Latent finding: clippy `--all-targets` didn't pass at v1.1.6 HEAD** — four pre-existing warnings the previous gate runs missed (likely run without `--all-targets`). Fixed in a dedicated commit. **This is a real audit finding that affects every prior REVIEW.md from v1.1.1 onward.**
|
||||
|
||||
The dead-letter handler wiring bug from v1.1.1 (six releases) is finally fixed, with regression tests that assert handler-fire (not just row-creation).
|
||||
|
||||
---
|
||||
|
||||
## 1. Static checks reproduced (HEAD `f5a3f92`)
|
||||
## 1. Static checks reproduced (HEAD `3cfb795`)
|
||||
|
||||
```
|
||||
cargo fmt --all -- --check ✅ exit 0
|
||||
cargo clippy --all-targets --all-features -- -D warnings ✅ exit 0
|
||||
cargo test --workspace ✅ ~550 passed / 0 failed
|
||||
+ 139 ignored (DB-gated)
|
||||
cargo clippy --all-targets --all-features -- -D warnings ✅ exit 0 (now actually green; see §5)
|
||||
cargo test --workspace (DATABASE_URL set, --test-threads=2) ✅ 617 passed / 0 failed
|
||||
```
|
||||
|
||||
Test count discrepancy worth flagging (see §5).
|
||||
Sum via the v1.1.7 discipline awk pattern:
|
||||
|
||||
```sh
|
||||
cargo test --workspace 2>&1 | awk '/test result: ok\./ { gsub(";", ""); sum += $4 } END { print sum }'
|
||||
# => 617
|
||||
```
|
||||
|
||||
Matches HANDBACK §8 exactly. **The §8 discipline refinement from the v1.1.6 retro is working.**
|
||||
|
||||
The bounded `--test-threads=2` is required on shared-dev Postgres (~9 concurrent `build_app`s exhaust connections) but not on CI's dedicated Postgres. Acceptable environmental nuance; flagged in HANDBACK §8.
|
||||
|
||||
## 2. Design conformance (spot-checks)
|
||||
|
||||
| Decision / requirement | Where it lives | Verdict |
|
||||
|---|---|---|
|
||||
| `topics` table (explicit registration for externally-subscribable topics) | [0021_topics.sql](crates/manager-core/migrations/0021_topics.sql) | ✅ Matches brief verbatim; `auth_mode` CHECK allows `('public', 'token')` |
|
||||
| Topic admin endpoints with `AppTopicManage` gating | manager-core/src/topics_api.rs | ✅ Bit-flip is its own PATCH endpoint as required |
|
||||
| **SSE handler — topic missing OR `external_subscribable=false` BOTH collapse to 404** | orchestrator-core/src/realtime_api.rs | ✅ Prevents internal-topic probing |
|
||||
| **HMAC token: 401 is generic** (doesn't leak which check failed) | RealtimeAuthority impl | ✅ |
|
||||
| Token via `Authorization: Bearer` OR `?token=` (EventSource compat) | realtime_api.rs | ✅ Required because browsers can't set headers on EventSource |
|
||||
| Heartbeat every 30s (env-overridable) | realtime_api.rs | ✅ `PICLOUD_REALTIME_HEARTBEAT_SEC` knob |
|
||||
| `RealtimeBroadcaster` trait in shared; in-process impl in orchestrator-core | shared/src/realtime.rs + orchestrator-core/src/realtime.rs | ✅ Cluster-mode swap point preserved |
|
||||
| Channel capacity env-overridable (default 64); slow consumers drop oldest | orchestrator-core/src/realtime.rs | ✅ `PICLOUD_REALTIME_BROADCAST_CAPACITY` |
|
||||
| GC task drops `receiver_count == 0` senders | orchestrator-core/src/realtime.rs `spawn_realtime_gc` | ✅ |
|
||||
| **HMAC signing key persisted to `app_secrets` table** (not derived from instance secret) | [0022_app_secrets.sql](crates/manager-core/migrations/0022_app_secrets.sql) + app_secrets_repo.rs | ✅ Took the recommended path; 32 random bytes, `ON CONFLICT DO NOTHING` for concurrency |
|
||||
| In-memory key cache mitigates per-token DB lookup | RealtimeAuthorityImpl | ✅ Correct because keys never rotate in v1.1.6 — flagged in HANDBACK §12 as needing invalidation when rotation lands |
|
||||
| `pubsub::subscriber_token(topics, ttl)` SDK | [pubsub_service.rs:203+ mint_subscriber_token](crates/manager-core/src/pubsub_service.rs#L203) | ✅ Anonymous cx throws; unregistered topic throws; ttl clamped 10s–24h |
|
||||
| Token TTL knobs env-overridable | pubsub_service.rs | ✅ `PICLOUD_SUBSCRIBER_TOKEN_TTL_{MIN,MAX,DEFAULT}_SEC` |
|
||||
| **Publish wiring: outbox commit FIRST, then broadcast on child task** | [pubsub_service.rs:138-201](crates/manager-core/src/pubsub_service.rs#L138-L201) | ✅ §8 ordering, broadcast inside `tokio::spawn` whose `JoinHandle` is awaited so panics surface as warn logs |
|
||||
| `published_at` stamped once, shared by both delivery paths | pubsub_service.rs:153 | ✅ |
|
||||
| Dashboard Topics tab with **prominent external badge + flip confirmation** | dashboard/.../+page.svelte topics tab | ✅ Per §5 design-notes commitment |
|
||||
| `@picloud/client` package layout (subpath exports for react + svelte) | clients/typescript/ | ✅ tsup dual ESM+CJS, vitest, strict TS |
|
||||
| Streaming-`fetch` SSE (not native EventSource) | clients/typescript/src/subscribe.ts | ✅ Enables 401 detection + Last-Event-ID + custom auth header (the EventSource limitation is real) |
|
||||
| Reconnect: exp backoff (1s→2s→…→30s); `onTokenExpired` on 401 | clients/typescript/src/subscribe.ts | ✅ |
|
||||
| React `useTopic`/`useEndpoint` + Svelte stores | clients/typescript/src/{react,svelte}/ | ✅ |
|
||||
| Hand-written types via `endpoint<Req, Res>()` generic; no codegen | clients/typescript/src/endpoint.ts | ✅ Codegen explicitly deferred to v1.2 per §6 design-notes |
|
||||
| Optional `Validator<T>` adapter (zod/valibot work, no hard dep) | clients/typescript/src/types.ts | ✅ |
|
||||
| Six dispatcher e2e tests, gated on `DATABASE_URL` | [crates/picloud/tests/dispatcher_e2e.rs](crates/picloud/tests/dispatcher_e2e.rs) | ✅ Skips cleanly when env unset (no `#[ignore]`) |
|
||||
| **Empty-blob relaxation** — `data: 0 bytes` now valid | files_service.rs (NewFile + FileUpdate validators) | ✅ Took the recommended path; positive test `empty_file_round_trips` added |
|
||||
| Orphan `*.tmp.*` sweeper, every 6h, deletes >1h old | files_sweep.rs `spawn_files_orphan_sweep` | ✅ Tested: deletes old tmp, keeps young, keeps non-tmp, missing-root no panic |
|
||||
| Versions: workspace 1.1.5→1.1.6, SDK 1.6→1.7, dashboard 0.11.0→0.12.0, client@1.0.0 | Cargo.toml + version.rs + package.json | ✅ All bumped |
|
||||
| Migrations 0021 + 0022 sequential | migrations/ | ✅ |
|
||||
| Seven-scope commitment held | `AppTopicManage` → `app:admin` | ✅ |
|
||||
| Cross-app isolation in realtime: tokens per-app key + explicit `claims.app_id == app_id` check; broadcaster keyed by `(AppId, topic)` | RealtimeAuthorityImpl + RealtimeBroadcasterImpl | ✅ Defense in depth — per-app key already fails a cross-app token's signature, but the explicit app_id claim check makes the boundary obvious in code |
|
||||
| **AES-256-GCM with 12-byte CSPRNG nonce + 16-byte appended auth tag** | [shared/src/crypto.rs:71-85](crates/shared/src/crypto.rs#L71-L85) | ✅ Uses `aes-gcm 0.10`; nonce from `rand::thread_rng().fill_bytes`; RustCrypto Aead layout (tag appended) |
|
||||
| `MasterKey` redacts Debug; cheap to clone | shared/src/crypto.rs MasterKey impl | ✅ Per HANDBACK §2 |
|
||||
| `PICLOUD_SECRET_KEY` required (fatal if missing); dev-mode fallback requires explicit `PICLOUD_DEV_MODE=true` | crypto.rs MasterKey::from_env + resolve | ✅ No quiet "unencrypted mode" path |
|
||||
| `MasterKey` threaded into `build_app` (test-friendly) | [picloud/src/lib.rs:build_app](crates/picloud/src/lib.rs) | ✅ Parameter, not env-sourced — tests can pass a fixed key |
|
||||
| 64 KB plaintext cap per secret | secrets_service::seal | ✅ `PICLOUD_SECRET_MAX_VALUE_BYTES` override |
|
||||
| Generic GCM auth-failure error (no wrong-key vs tampered distinction) | crypto.rs CryptoError::Decrypt | ✅ By design — leaking which failure case happened weakens the integrity guarantee |
|
||||
| `secrets` table with `(app_id, name)` PK, encrypted bytea + 12-byte nonce | [0023_secrets.sql](crates/manager-core/migrations/0023_secrets.sql) | ✅ |
|
||||
| `secrets::*` SDK — collection-less, JSON type round-trip | [executor-core/src/sdk/secrets.rs](crates/executor-core/src/sdk/secrets.rs) + secrets_service.rs | ✅ String comes back as String (not JSON-quoted) |
|
||||
| Cross-app isolation in secrets | secrets_service via `cx.app_id` | ✅ Test asserts |
|
||||
| `Capability::AppSecretsRead/Write` → `script:read/write` | manager-core::authz | ✅ Seven-scope commitment held |
|
||||
| No `ServiceEvent` emission for secret writes | secrets_service | ✅ Per brief — secret-change triggers are a footgun |
|
||||
| Outbound email via `lettre 0.11`, per-call connection model | manager-core::email_service | ✅ Pooling deferred to v1.2 per brief |
|
||||
| Disabled mode when SMTP env vars missing | EmailServiceImpl::from_env | ✅ Startup warn; every `send` returns `NotConfigured` |
|
||||
| `email::send_html` builds MultiPart alternative_plain_html | email_service.rs send_html path | ✅ |
|
||||
| `to/cc/bcc` accept String or Array of Strings | sdk/email.rs bridge | ✅ |
|
||||
| 25 MB message cap, env-overridable | email_service | ✅ `PICLOUD_EMAIL_MAX_MESSAGE_BYTES` |
|
||||
| RFC 5322-ish pre-validation + lettre Mailbox parse | email_service::validate | ✅ |
|
||||
| Inbound webhook receiver `POST /api/v1/email-inbound/{app_id}/{trigger_id}` | crates/picloud/src/lib.rs or orchestrator-core | ✅ Per [picloud/tests/email_inbound.rs](crates/picloud/tests/email_inbound.rs) test coverage |
|
||||
| Inbound: 202 success, 401 HMAC fail, 404 missing/wrong-kind, 422 malformed | email_inbound.rs tests | ✅ All four status codes pinned by tests |
|
||||
| `email_trigger_details` schema with HMAC secret | [0024_email_triggers.sql](crates/manager-core/migrations/0024_email_triggers.sql) | ✅ |
|
||||
| `TriggerEvent::Email` shape: from/to/cc/subject/text/html/received_at/message_id | trigger_event.rs | ✅ |
|
||||
| **Dead-letter handler fix: `list_matching_dead_letter` called from `dispatcher::handle_failure`** | [dispatcher.rs:498-501 + fan_out_dead_letter](crates/manager-core/src/dispatcher.rs#L498-L501) | ✅ Wired exactly as specified; built from the real `TriggerEvent::DeadLetter` variant |
|
||||
| Recursion-stop preserved: handler failures don't re-dead-letter | dispatcher.rs `is_dead_letter_handler` short-circuit at top of handle_failure | ✅ No new guard needed — the existing flag fires before reaching the exhaustion branch |
|
||||
| Best-effort fan-out: lookup/insert failures logged, not propagated | fan_out_dead_letter at dispatcher.rs:541-545 + 562-565 | ✅ Dead-letter row durably written; handler fan-out is secondary |
|
||||
| **Two-phase realtime key migration: encrypted columns added NULL-able + plaintext kept** | [0025_encrypt_realtime_keys.sql](crates/manager-core/migrations/0025_encrypt_realtime_keys.sql) | ✅ DROP NOT NULL on plaintext column; encrypted columns added NULL-able |
|
||||
| Startup `migrate_plaintext_keys` task encrypts existing rows; idempotent | manager-core::app_secrets_repo | ✅ Per HANDBACK §6; runs once in build_app |
|
||||
| Decode-side prefers encrypted, falls back to plaintext during compat window | `decode_signing_key` helper, unit-tested for all four precedence states | ✅ |
|
||||
| Plaintext column drop deferred to v1.1.8 + documented | CHANGELOG + migration header | ✅ |
|
||||
| Versions: workspace 1.1.6→1.1.7, SDK 1.7→1.8, dashboard 0.12.0→0.13.0 | Cargo.toml + version.rs + package.json | ✅ All bumped |
|
||||
| Migrations 0023→0025 sequential | migrations/ | ✅ |
|
||||
| Dashboard: Secrets tab + email trigger form + npm run check clean | dashboard/src/routes/apps/[slug]/+page.svelte | ✅ Per HANDBACK |
|
||||
|
||||
## 3. The §4-vs-§8 ordering contradiction (HANDBACK §9 #1)
|
||||
## 3. The three flagged items
|
||||
|
||||
The brief literally contradicted itself. §4 said:
|
||||
### 3.1 Brief-internal contradiction: `TriggerEvent::DeadLetter` field names (HANDBACK §7 #2)
|
||||
|
||||
> "Order: realtime broadcast FIRST (fast, in-memory), then transactional outbox fan-out (slower)."
|
||||
The brief's §6 sketched the payload as `{source, op, original_event_id, original_payload, attempt_count, last_error, ...}`. The actual variant in `crates/shared/src/trigger_event.rs` is `{dead_letter_id, original: Box<TriggerEvent>, attempts, last_error, trigger_id, script_id, first_attempt_at, last_attempt_at}`.
|
||||
|
||||
§8 said:
|
||||
The agent built from the real variant (which the brief itself said to "verify serializes correctly") and flagged the contradiction rather than silently reinterpreting.
|
||||
|
||||
> "Order matters: 1. Validate. 2. Transactional fan-out to outbox. 3. Commit. 4. Non-transactional broadcast to in-process subscribers."
|
||||
**Verdict: correct call.** The v1.1.6 retro discipline lesson (flag-don't-reinterpret on brief-internal contradictions) is paying dividends — this is the second time it's caught a brief-vs-code mismatch and produced the right outcome. Worth folding into the v1.1.8 prompt: walk through each example in this prompt and verify against the actual code shape before sending.
|
||||
|
||||
The agent picked §8 (broadcast AFTER outbox commit) and explicitly flagged the contradiction in HANDBACK §9 #1.
|
||||
### 3.2 `inbound_secret` stored encrypted (HANDBACK §7 #1)
|
||||
|
||||
**§8 is correct.** Three reasons:
|
||||
User-approved deviation during planning per the user's summary message. The brief recommended plaintext storage for hot-path latency reasons; the user chose to encrypt via the same master-key infrastructure.
|
||||
|
||||
1. **Correctness over latency.** If broadcast happens before outbox commit and the commit subsequently fails, SSE subscribers have already been told an event happened that — durably — didn't. They can't replay the apology. §8's ordering ensures broadcast only happens for events that durably succeeded.
|
||||
**Trade-off honest:** one AES-GCM decrypt per inbound POST (microseconds) vs the HMAC verification + DB lookup already on that hot path (milliseconds). The decrypt is negligible.
|
||||
|
||||
2. **§8 is the dedicated, numbered, rationale-backed section.** §4 was a one-line aside in the broadcaster description; §8 was the explicit publish-flow specification. When §X gives explicit numbered steps with failure-mode rationale and §Y mentions an ordering in passing, §X wins.
|
||||
**Verdict: accept the deviation.** Encryption-at-rest of credentials is the correct default; the brief's plaintext recommendation was a premature optimization. The agent took the right path. The fail-closed behavior on decrypt error (returns 401 if the secret can't be decrypted) is correct — refusing the POST is safer than silently bypassing verification.
|
||||
|
||||
3. **The agent's failure-mode analysis is right.** Per [pubsub_service.rs:170-173](crates/manager-core/src/pubsub_service.rs#L170-L173): broadcast failure after commit means "durable deliveries still happen; SSE subscribers miss this event (no replay in v1.1.6)." Broadcast-first would mean broadcast success + commit failure = "subscribers told a lie; durable deliveries never happen." The latter is strictly worse.
|
||||
### 3.3 Latent finding: clippy `--all-targets` regression (HANDBACK §10 #1)
|
||||
|
||||
**Verdict: confirm §8.** The agent acted on the v1.1.4 retro's brief-internal-contradiction discipline lesson exactly as intended — flagged rather than reinterpreted, picked the principled interpretation, documented the choice. This is the right behavior; the lesson stuck.
|
||||
This is the most important finding in this review.
|
||||
|
||||
The v1.1.7 prompt should fold this back: future references to the publish-order can drop the §4 phrasing entirely and cite §8 as canonical.
|
||||
The agent verified by stashing v1.1.7 work and re-running clippy on v1.1.6 HEAD with `--all-targets --all-features -- -D warnings` — four pre-existing warnings surfaced:
|
||||
- `double_must_use` on `realtime_router`
|
||||
- `map_unwrap_or` in `pubsub_service`
|
||||
- `redundant_closure` in `topic_repo`
|
||||
- `needless_raw_string_hashes` in a subscriber-token test
|
||||
|
||||
## 4. Latent finding: `dead_letter` handlers never fire (HANDBACK §10)
|
||||
The warnings landed in v1.1.6 itself (the realtime_router was new). The clippy gate v1.1.6 claimed to pass (and that I personally re-ran during the v1.1.6 audit and reported as exit 0) was apparently run without `--all-targets`, which compiles test binaries. Test-only clippy warnings escape.
|
||||
|
||||
**Verified.** Grepping for `list_matching_dead_letter` callers:
|
||||
**This is a real audit oversight.** My v1.1.6 REVIEW.md §1 reported `cargo clippy --all-targets --all-features -- -D warnings ✅ exit 0`. Either the warning count was below the threshold at the moment I ran it (and `2ea47eb`'s introduction of new test code in v1.1.7 tipped it over), or I genuinely missed the warnings. Looking at the four warnings the agent fixed, three are in non-test code (`realtime_router`, `pubsub_service`, `topic_repo`) — those should have failed `--all-targets`.
|
||||
|
||||
```
|
||||
crates/manager-core/src/outbox_event_emitter.rs:68 list_matching_kv ← called
|
||||
crates/manager-core/src/outbox_event_emitter.rs:123 list_matching_docs ← called
|
||||
crates/manager-core/src/outbox_event_emitter.rs:184 list_matching_files ← called
|
||||
list_matching_dead_letter ← NO PRODUCTION CALLER
|
||||
**Most likely explanation:** the clippy run during the v1.1.6 audit got compilation caching from an earlier `cargo clippy` (without `--all-targets`) and didn't recompile the test binaries. Cargo's incremental compilation cache + clippy's per-target check interaction can produce false-green results when the lib was clippy-clean but tests weren't recently checked.
|
||||
|
||||
**Action for the v1.1.8 prompt:** require a clean build before clippy:
|
||||
|
||||
```sh
|
||||
cargo clean -p picloud-manager-core picloud-orchestrator-core picloud-executor-core picloud-shared picloud
|
||||
cargo clippy --all-targets --all-features -- -D warnings
|
||||
```
|
||||
|
||||
`TriggerRepo::list_matching_dead_letter` is defined in the trait (trigger_repo.rs:356), implemented for Postgres (trigger_repo.rs:947), and exists in test fakes (triggers_api.rs:830). But no code path in the dispatcher or the emitter calls it. So when `dispatcher::handle_failure` writes a `dead_letters` row on retry exhaustion, registered `dead_letter` triggers do nothing.
|
||||
Or simpler: use `cargo clippy --workspace --all-targets --all-features --no-deps -- -D warnings` and verify that the test binary count matches what cargo says it compiled.
|
||||
|
||||
This is a real bug from v1.1.1. The design notes §4 specified dead_letter triggers as a shipped v1.1.1 feature; the wiring was never connected. v1.1.2/v1.1.3/v1.1.4/v1.1.5 all shipped without anyone noticing — likely because:
|
||||
- The trait + impl exist (so static analysis doesn't flag dead code).
|
||||
- v1.1.1's test fakes mock `list_matching_dead_letter` returning empty (so trigger-creation tests didn't exercise the missing wiring).
|
||||
- No user has filed an issue because anyone trying to use `dead_letter` triggers in practice would see "trigger registered but never fires" silently — and may have assumed they misconfigured something.
|
||||
The agent fixed all four warnings in `2ea47eb` and gated v1.1.7 against the re-verified `--all-targets` baseline. Future audits should follow suit.
|
||||
|
||||
**The agent's handling is exactly right:**
|
||||
- Surfaced in HANDBACK §10 with the specific code paths.
|
||||
- Did NOT attempt a fix (out of v1.1.6 scope).
|
||||
- Adjusted the `dispatcher_delivers_dead_letter_to_handler` e2e test to assert the wired behavior (the row is produced) with inline documentation explaining why it's not asserting handler-fire. This is the honest test for what the code currently does.
|
||||
## 4. Substantive strengths
|
||||
|
||||
**For v1.1.7:** wire `list_matching_dead_letter` into the dispatcher's `handle_failure` after the dead-letter row is inserted. The recursion-stop rule from v1.1.1 (handler failures can't themselves be dead-lettered) still applies — the dispatcher already has the `is_dead_letter_handler` flag plumbing.
|
||||
**1. The §8 attestation discipline lesson landed cleanly.** v1.1.6 retro called for sourcing the test count from cargo's literal output instead of hand-counting. The v1.1.7 HANDBACK §8 includes the literal awk command + the verified count of 617. My independent re-run matches exactly. Discipline working as designed.
|
||||
|
||||
**For deployments:** this bug has been silently shipped since v1.1.1. Anyone running v1.1.1–v1.1.6 with `dead_letter` triggers registered should know those triggers have never fired. The fix in v1.1.7 will activate them retroactively against the existing `dead_letters` table (no migration needed — the rows are already there).
|
||||
**2. Encryption infrastructure correctly built.** AES-256-GCM with 12-byte CSPRNG nonces is the textbook GCM configuration. Auth tag appended (RustCrypto Aead trait standard). `Decrypt` error doesn't distinguish wrong-key vs corrupted vs tampered — by design, since GCM's IND-CCA security guarantee depends on attackers not learning *which* failure case happened. `MasterKey`'s redacted `Debug` impl prevents accidental log-leaks. Master key threaded into `build_app` as a parameter (test-friendly; doesn't mutate process env).
|
||||
|
||||
Worth a CHANGELOG.md callout in v1.1.7 alongside the fix, similar to how v1.1.3's cross-app trigger gap got a retroactive security note.
|
||||
**3. Dead-letter handler fix is faithful and adequately tested.** Six releases of silently-broken triggers, finally connected. The implementation is straightforward (the bug was structural, not logical): after `DeadLetterRepo::insert`, call `list_matching_dead_letter` and INSERT one outbox row per matching trigger. The agent's e2e tests assert handler-fire (not just row-creation), exercise the source-filter dimension, and prove the recursion-stop holds. The retroactive CHANGELOG note from the v1.1.7 prompt is in place.
|
||||
|
||||
**Verdict: not a v1.1.6 blocker.** The bug predates this release; v1.1.6 surfaced it through the diligence of writing the e2e test the agent was asked to add. Excellent defensive work.
|
||||
**4. Two-phase realtime key migration done right.** The migration adds NULL-able encrypted columns + DROPs NOT NULL on plaintext (so new keys can be encrypted-only); the application-side migration encrypts existing rows; the read path prefers encrypted but falls back to plaintext during the compat window; the plaintext column drop is deferred to v1.1.8 (documented in CHANGELOG + the migration header). Operator-friendly: rolling deploys work cleanly.
|
||||
|
||||
## 5. Test count discrepancy
|
||||
**5. Inbound email as webhook receiver was the right architectural call.** Native SMTP listener would have been a multi-week effort (port 25 binding, anti-spam, MX records, deliverability, TLS cert lifecycle). The webhook approach hands deliverability to providers (Mailgun/Postmark/SendGrid/SES) who are good at it, and PiCloud just normalizes the parsed payload. Reasonable v1.1.7 scope.
|
||||
|
||||
HANDBACK §8 attests `cargo test --workspace → 482 passed`. My re-run on the same HEAD reports **~550 passed**. Counting the unique `test result: ok. N passed` lines:
|
||||
**6. Disabled-mode for outbound SMTP.** When SMTP env vars aren't set, every `send` throws `NotConfigured` cleanly. The brief specified this; the agent implemented it cleanly. Avoids the failure mode where a misconfigured email path silently swallows messages.
|
||||
|
||||
```
|
||||
manager-core 256 (was 229 in v1.1.5 → +27)
|
||||
executor-core/sdk_* 15+15+8+8+7+5+1+1+17 = 77
|
||||
orchestrator-core 74 (was 62 → +12; realtime SSE + broadcaster + key cache tests)
|
||||
stdlib 43
|
||||
sdk_contract 30
|
||||
modules 23
|
||||
picloud 21 (incl. 6 dispatcher_e2e skipping no-op)
|
||||
schema_snapshot 1
|
||||
shared/pubsub 6 (or somewhere thereabouts)
|
||||
files-related 20
|
||||
───
|
||||
~550
|
||||
```
|
||||
**7. The agent caught and surfaced the v1.1.6 clippy regression.** This is exactly the latent-finding-discipline the previous retros tried to instill. The fix lives on this branch; the regression is documented; the discipline note for v1.1.8 is the only follow-up.
|
||||
|
||||
The agent's 482 count was likely a snapshot taken before the final commit added a test file, or a `cargo test --workspace 2>&1 | grep -c "passed"` (counts lines, not values) misread. Either way:
|
||||
## 5. Open questions answered
|
||||
|
||||
- The discrepancy is in **the count, not the outcome**: 0 failed, 0 ignored unexpected.
|
||||
- The gates exit 0; clippy is clean; fmt is clean.
|
||||
- The implementation passes every named-critical test from the prompt's §13.
|
||||
HANDBACK §9 raises three:
|
||||
|
||||
**Verdict: minor accounting drift, not a blocker.** Flag for the v1.1.7 retro: the §8 attestation should be the literal `cargo test --workspace` final-line output (`X passed in Y crates`) or a sum verified by `awk '/test result: ok/ { sum += $4 } END { print sum }'`, not a hand count.
|
||||
1. **§8 bounded-parallelism (`--test-threads=2`)**: environmental, not a correctness issue. Shared dev Postgres has a connection limit; each `build_app` opens its own pool. CI's dedicated Postgres doesn't hit this. **Accept as-is.** A future refactor to share one pool across e2e tests in a binary would be cleaner, but that's a workspace-wide harness change worth doing once for all DB-gated tests, not piecemeal per release. Defer to a dedicated e2e-harness pass.
|
||||
|
||||
## 6. Substantive strengths
|
||||
2. **`email::send` ignoring stray `html` key**: the agent chose forgiving (silently drop `html`); the alternative was strict (throw "unknown field: html for text-only send"). **My read: forgiving is fine.** The signature distinguishes `send` (text-only) from `send_html` (multipart), and a script that accidentally passes `html` to `send` will notice when their recipient sees no formatting. Strict-throwing is also defensible; not worth changing.
|
||||
|
||||
**1. Streaming-`fetch` SSE in the client lib was the right call.** Native `EventSource` can't set custom auth headers (forcing the `?token=` query-string path, which the server still supports as an EventSource-compat option). But for the client lib, dropping EventSource in favor of streaming `fetch` unlocks three things at once: bearer-header auth (cleaner than query-string), 401 detection on (re)connect → `onTokenExpired` callback → token refresh → reconnect, and `Last-Event-ID` resume header (server ignores it in v1.1.6 but the client ships ready). Trade-off: requires `fetch` streaming, so React Native needs a polyfill — the README documents this. Right trade for v1.1.6's target audience.
|
||||
3. **Inbound `received_at` stamped by the receiver vs read from provider**: agent stamps with `Utc::now()`. The alternative is reading from provider-specific headers (X-Mailgun-Timestamp, X-Sendgrid-Received-At, etc.), which requires provider unmarshallers that v1.1.7 deferred to v1.2. **Accept as-is.** Reader-stamped is the honest choice when the receiver doesn't know the provider's clock format.
|
||||
|
||||
**2. The HMAC-signing-key persisted-table choice avoids a global secret.** The agent took the recommended path: per-app 32-byte random keys in `app_secrets`. No `PICLOUD_INSTANCE_SECRET` env var to operate. Future v1.1.7 encrypted-per-app-secrets work has its natural home. The cost — one DB read per subscribe — is mitigated by the in-process key cache (correct in v1.1.6 because keys never rotate; HANDBACK §12 flags the rotation-invalidation requirement for future).
|
||||
## 6. Smaller observations
|
||||
|
||||
**3. Defense in depth on cross-app isolation.** Per-app signing key + explicit `claims.app_id == app_id` check + broadcaster channels keyed by `(AppId, topic)`. Any single guard would suffice; all three together make the boundary obvious in code AND impossible to bypass via a single mistake.
|
||||
- **`build_app` signature gained `MasterKey` parameter (HANDBACK §7 #3).** Threading the key in from `main.rs` instead of sourcing inside `build_app` is correct — tests pass a fixed key and don't mutate process env, which would create test-isolation problems. The 3 existing `build_app` test callers were updated.
|
||||
- **Email trigger retry defaults (HANDBACK §7 #5).** Standard async defaults (3 attempts, exponential, 1000 ms). Matches kv/docs/files/cron/pubsub. Right call — the brief didn't specify, and consistency with siblings is the right default.
|
||||
- **The 10-commit split is exemplary.** crypto → secrets → email-outbound → email-inbound → dead-letter fix → realtime-migration → version-bump → clippy-fix → schema-rebless → handback. Each commit independently green. Best commit hygiene in any v1.1.x release.
|
||||
|
||||
**4. Three v1.1.5 follow-ups all landed.** The empty-blob relaxation, the orphan tmp-sweeper, the six dispatcher e2e tests. All in this release, not deferred. The e2e tests are gated on `DATABASE_URL` cleanly via early-return (matches the v1.1.5 schema_snapshot pattern); CI's Postgres service exercises them.
|
||||
|
||||
**5. The agent's discipline carryover is exemplary.** Both flagged items (§4-vs-§8 contradiction, dead_letter latent finding) were caught by the v1.1.4 + v1.1.3 retro discipline lessons: brief-internal contradictions get flagged-not-reinterpreted, latent security/correctness findings get their own HANDBACK section. The §8 attestation was taken on the actual HEAD with the explicit "this HANDBACK commit is pure markdown" footnote. Every deviation is in §7. The system is working.
|
||||
|
||||
**6. Commit split.** Three commits — realtime+followups+versions, client lib, handback. Cleaner than v1.1.5's three substantive (because the client lib genuinely is a standalone artifact in a different toolchain), and the build-app cross-crate constraint that drove a single big realtime commit is honestly documented in HANDBACK §7 #1.
|
||||
|
||||
## 7. Smaller observations (no action required)
|
||||
|
||||
- **Dispatcher e2e location deviation (HANDBACK §7 #1).** Brief said `crates/manager-core/tests/`; agent put them in `crates/picloud/tests/` because `build_app` lives there. The cycle the agent describes (manager-core → picloud dev-dep) is real. The picloud location is correct — `build_app` is where the dispatcher + scheduler + executor are wired into one stack, and that wiring is what the e2e tests need to exercise.
|
||||
- **Empty-blob relaxation extended to `FileUpdate::validate` (HANDBACK §7 #2).** The brief only named `NewFile::validate`. Extending to update is correct — relaxing create-empty but rejecting update-to-empty would be an inconsistent API.
|
||||
- **Topic-name validation (HANDBACK §7 #4).** Small defensive add: empty names and names containing `*` rejected at admin endpoint. Defends against operator confusion when topic-pattern external subscription lands in v1.2 (preemptive: `*` will mean something there, so reserving it now avoids a future breaking validation).
|
||||
- **Client lib lint via `tsc --noEmit` instead of eslint (HANDBACK §7 #5).** Right call for v1.1.6 — strict TS does most of what eslint would, and adding eslint configuration is a separate scope of work. Easy to add later if there's a real type-system-can't-catch-it lint rule we need.
|
||||
- **Cron e2e 45s poll budget (HANDBACK §7 #6).** Defensive against the default 30s tick interval; CI sets `PICLOUD_CRON_TICK_INTERVAL_MS=1000` to make it ~2s. Reasonable.
|
||||
- **`broadcast::Sender` shape vs `oneshot::Sender`.** v1.1.1's `InboxRegistry` uses oneshot (single delivery). v1.1.6's broadcaster uses `tokio::sync::broadcast` (repeated delivery to multiple receivers). Different patterns for different problems; the trait split in shared keeps both Cluster-mode-swappable.
|
||||
|
||||
## 8. Versioning audit
|
||||
## 7. Versioning audit
|
||||
|
||||
| File | Before | After | Status |
|
||||
|---|---|---|---|
|
||||
| Workspace `Cargo.toml` | 1.1.5 | 1.1.6 | ✅ |
|
||||
| SDK schema (`shared/src/version.rs`) | 1.6 | 1.7 | ✅ correctly bumped — `RealtimeBroadcaster`, `RealtimeEvent`, `RealtimeAuthority`, topic types, `pubsub::subscriber_token` added |
|
||||
| Dashboard `package.json` | 0.11.0 | 0.12.0 | ✅ |
|
||||
| `@picloud/client` package.json | (new) | 1.0.0 | ✅ Initial release |
|
||||
| Migrations | 0001..0020 | 0021..0022 added | ✅ sequential |
|
||||
| CHANGELOG.md | v1.1.5 entry | v1.1.6 entry added | ✅ |
|
||||
| Workspace `Cargo.toml` | 1.1.6 | 1.1.7 | ✅ |
|
||||
| SDK schema (`shared/src/version.rs`) | 1.7 | 1.8 | ✅ correctly bumped — `SecretsService`, `EmailService`, `MasterKey`, `crypto::{encrypt, decrypt}`, `TriggerEvent::Email` added to public surface |
|
||||
| Dashboard `package.json` | 0.12.0 | 0.13.0 | ✅ |
|
||||
| Migrations | 0001..0022 | 0023..0025 added | ✅ sequential, no skips |
|
||||
| CHANGELOG.md | v1.1.6 entry | v1.1.7 entry + retroactive dead_letter security note | ✅ Per prompt |
|
||||
|
||||
## 9. Recommended next steps (post-merge)
|
||||
## 8. Recommended next steps (post-merge)
|
||||
|
||||
1. **Merge** `feat/v1.1.6-realtime-client` into `main` (fast-forward; branch is linear ahead).
|
||||
2. **`docker compose down` when convenient** to tear down the dev Postgres container the agent left running.
|
||||
3. **Pause** before dispatching v1.1.7 (Configuration & Email).
|
||||
4. **For the v1.1.7 dispatch prompt**, fold in:
|
||||
- **Fix `dead_letter` handler wiring** (§4 above). Add a call to `list_matching_dead_letter` in `dispatcher::handle_failure` after the row is inserted, enqueue an outbox row per matching trigger with `source_kind: 'dead_letter'` and the appropriate `TriggerEvent::DeadLetter` payload. The recursion-stop rule (handlers can't be dead-lettered) is already implemented; the wiring just isn't connected. Plus a CHANGELOG retroactive note covering v1.1.1–v1.1.6 ("dead_letter triggers were registerable but never fired; this release activates them against existing dead_letters rows, no migration needed").
|
||||
- **Encrypted per-app secrets.** v1.1.7's brief topic. The `app_secrets` table from v1.1.6 is the natural home; the realtime signing key already lives there. v1.1.7's encrypted-secrets work should extend the table (or add a sibling) for general per-app secrets.
|
||||
- **§8 attestation discipline refinement:** require the §8 attestation be sourced from `cargo test --workspace 2>&1 | tail` rather than a hand count (per §5 above).
|
||||
- **The brief-internal-contradiction discipline lesson stuck.** v1.1.7's brief should be walked through for example/spec contradictions before dispatch, same as the v1.1.5 retro lesson the v1.1.6 brief honored. Keep doing this.
|
||||
5. **For the v1.1.8 dispatch prompt** (User Management): the `app_secrets` table + the `RealtimeAuthority` trait shape are ready for v1.1.8 to add `auth_mode = 'session'` as the third subscriber-auth flavor (extending the CHECK constraint, adding a session-token validator alongside the existing HMAC validator behind the unchanged trait).
|
||||
1. **Merge** `feat/v1.1.7-secrets-email` into `main` (fast-forward; branch is linear ahead).
|
||||
2. **`docker compose down` when convenient** to tear down the dev Postgres container.
|
||||
3. **Pause** before dispatching v1.1.8 (User Management).
|
||||
4. **For the v1.1.8 dispatch prompt**, fold in:
|
||||
- **Drop the plaintext `realtime_signing_key` column** (the v1.1.7 phase-2 commitment). Pre-flight check: scan the column for any remaining non-NULL rows; if found, run the encryption migration before the drop migration. Add a CHANGELOG note that v1.1.8 requires v1.1.7 to have been applied first (no skipping versions).
|
||||
- **Clippy --all-targets discipline refinement** (§3.3 finding). Require either a `cargo clean` before `cargo clippy --all-targets` OR explicit verification that test binaries are being checked. v1.1.6's silent regression shows the gate can produce false-green results under cargo's incremental cache. Specific recommendation: add a CI step that asserts the clippy run touched the test binaries (e.g. count `Checking` lines in the output and verify they include test crates).
|
||||
- **`auth_mode = 'session'` for realtime subscriber tokens** — v1.1.7's CHECK constraint on `topics.auth_mode` only allows `('public', 'token')`. v1.1.8 (users::*) needs to add `'session'` and a session-token validator alongside the existing HMAC validator behind the unchanged `RealtimeAuthority` trait.
|
||||
- **Bounded e2e parallelism** — defer the workspace-wide harness refactor (shared pool per binary) until there's a dedicated test-infra release. Until then, CI just needs `--test-threads=2` or smaller for the picloud crate's e2e binaries.
|
||||
5. **Awareness from §3.3**: the clippy regression in v1.1.6 was caught by v1.1.7's diligence, but every prior REVIEW.md from v1.1.1 onward should be re-checked if you want certainty that no test-only clippy warnings slipped through. The fix is forward-only — re-running clippy on v1.1.1 through v1.1.6 commits would just confirm the warnings were latent then too.
|
||||
|
||||
Branch is ready for merge. Verdict: **APPROVE**.
|
||||
|
||||
@@ -448,6 +448,40 @@ fn trigger_event_to_dynamic(event: &TriggerEvent) -> Dynamic {
|
||||
ps.insert("published_at".into(), published_at.to_rfc3339().into());
|
||||
m.insert("pubsub".into(), ps.into());
|
||||
}
|
||||
TriggerEvent::Email {
|
||||
from,
|
||||
to,
|
||||
cc,
|
||||
subject,
|
||||
text,
|
||||
html,
|
||||
received_at,
|
||||
message_id,
|
||||
} => {
|
||||
// `ctx.event.op` is always "receive" for inbound email.
|
||||
m.insert("op".into(), "receive".into());
|
||||
let mut em = Map::new();
|
||||
em.insert("from".into(), from.clone().into());
|
||||
let to_arr: rhai::Array = to.iter().map(|a| Dynamic::from(a.clone())).collect();
|
||||
em.insert("to".into(), to_arr.into());
|
||||
let cc_arr: rhai::Array = cc.iter().map(|a| Dynamic::from(a.clone())).collect();
|
||||
em.insert("cc".into(), cc_arr.into());
|
||||
em.insert("subject".into(), subject.clone().into());
|
||||
em.insert(
|
||||
"text".into(),
|
||||
text.clone().map_or(Dynamic::UNIT, Dynamic::from),
|
||||
);
|
||||
em.insert(
|
||||
"html".into(),
|
||||
html.clone().map_or(Dynamic::UNIT, Dynamic::from),
|
||||
);
|
||||
em.insert("received_at".into(), received_at.to_rfc3339().into());
|
||||
em.insert(
|
||||
"message_id".into(),
|
||||
message_id.clone().map_or(Dynamic::UNIT, Dynamic::from),
|
||||
);
|
||||
m.insert("email".into(), em.into());
|
||||
}
|
||||
TriggerEvent::DeadLetter {
|
||||
dead_letter_id,
|
||||
original,
|
||||
|
||||
150
crates/executor-core/src/sdk/email.rs
Normal file
150
crates/executor-core/src/sdk/email.rs
Normal file
@@ -0,0 +1,150 @@
|
||||
//! `email::` Rhai bridge — outbound email (v1.1.7).
|
||||
//!
|
||||
//! ```rhai
|
||||
//! email::send(#{
|
||||
//! to: "alice@example.com", // String or Array of String
|
||||
//! from: "alerts@myapp.com",
|
||||
//! subject: "Build complete",
|
||||
//! text: "Your deploy finished."
|
||||
//! });
|
||||
//!
|
||||
//! email::send_html(#{
|
||||
//! to: ["alice@x.com", "bob@y.com"],
|
||||
//! cc: ["dave@z.com"],
|
||||
//! bcc: ["audit@myapp.com"],
|
||||
//! from: "alerts@myapp.com",
|
||||
//! reply_to: "support@myapp.com", // optional; defaults to `from`
|
||||
//! subject: "Build complete",
|
||||
//! text: "Your deploy finished.", // plain-text fallback
|
||||
//! html: "<p>Your deploy <b>finished</b>.</p>"
|
||||
//! });
|
||||
//! ```
|
||||
//!
|
||||
//! Both map onto `EmailService::send`. `email::send` forces a text-only
|
||||
//! message (any `html` key is ignored); `email::send_html` requires an
|
||||
//! `html` part. `app_id` is derived from `cx.app_id` in the service.
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use picloud_shared::{EmailError, OutboundEmail, SdkCallCx, Services};
|
||||
use rhai::{Array, Engine as RhaiEngine, EvalAltResult, Map, Module};
|
||||
use tokio::runtime::Handle as TokioHandle;
|
||||
|
||||
pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<SdkCallCx>) {
|
||||
let svc = services.email.clone();
|
||||
let mut module = Module::new();
|
||||
|
||||
// email::send(#{...}) — plain text (html ignored).
|
||||
{
|
||||
let svc = svc.clone();
|
||||
let cx = cx.clone();
|
||||
module.set_native_fn("send", move |opts: Map| -> Result<(), Box<EvalAltResult>> {
|
||||
let mut email = parse_email(&opts)?;
|
||||
email.html = None; // text-only path
|
||||
let svc = svc.clone();
|
||||
let cx = cx.clone();
|
||||
block_on(async move { svc.send(&cx, email).await })
|
||||
});
|
||||
}
|
||||
|
||||
// email::send_html(#{...}) — multipart text + html (html required).
|
||||
{
|
||||
let svc = svc.clone();
|
||||
let cx = cx.clone();
|
||||
module.set_native_fn(
|
||||
"send_html",
|
||||
move |opts: Map| -> Result<(), Box<EvalAltResult>> {
|
||||
let email = parse_email(&opts)?;
|
||||
if email.html.as_ref().is_none_or(String::is_empty) {
|
||||
return Err(runtime_err(
|
||||
"email::send_html: an 'html' field is required (use email::send for text-only)",
|
||||
));
|
||||
}
|
||||
let svc = svc.clone();
|
||||
let cx = cx.clone();
|
||||
block_on(async move { svc.send(&cx, email).await })
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
engine.register_static_module("email", module.into());
|
||||
}
|
||||
|
||||
/// Parse the Rhai options map into an [`OutboundEmail`]. Field-level
|
||||
/// validation (required fields, address shape) happens in the service;
|
||||
/// here we only do type coercion (String/Array → Vec<String>).
|
||||
fn parse_email(opts: &Map) -> Result<OutboundEmail, Box<EvalAltResult>> {
|
||||
Ok(OutboundEmail {
|
||||
to: addresses(opts, "to")?,
|
||||
cc: addresses(opts, "cc")?,
|
||||
bcc: addresses(opts, "bcc")?,
|
||||
from: string_field(opts, "from").unwrap_or_default(),
|
||||
reply_to: string_field(opts, "reply_to"),
|
||||
subject: string_field(opts, "subject").unwrap_or_default(),
|
||||
text: string_field(opts, "text"),
|
||||
html: string_field(opts, "html"),
|
||||
})
|
||||
}
|
||||
|
||||
/// Read a string field. Missing or `()` → `None`.
|
||||
fn string_field(opts: &Map, key: &str) -> Option<String> {
|
||||
match opts.get(key) {
|
||||
None => None,
|
||||
Some(d) if d.is_unit() => None,
|
||||
Some(d) if d.is_string() => Some(d.clone().into_string().unwrap_or_default()),
|
||||
// Coerce non-string scalars via display (numbers, etc.).
|
||||
Some(d) => Some(d.to_string()),
|
||||
}
|
||||
}
|
||||
|
||||
/// Read an address list: a String becomes a one-element list; an Array
|
||||
/// of Strings becomes a list; missing/`()` is empty.
|
||||
fn addresses(opts: &Map, key: &str) -> Result<Vec<String>, Box<EvalAltResult>> {
|
||||
match opts.get(key) {
|
||||
None => Ok(Vec::new()),
|
||||
Some(d) if d.is_unit() => Ok(Vec::new()),
|
||||
Some(d) if d.is_string() => Ok(vec![d.clone().into_string().unwrap_or_default()]),
|
||||
Some(d) => {
|
||||
if let Some(arr) = d.clone().try_cast::<Array>() {
|
||||
let mut out = Vec::with_capacity(arr.len());
|
||||
for el in arr {
|
||||
if !el.is_string() {
|
||||
return Err(runtime_err(&format!(
|
||||
"email: '{key}' array must contain only strings"
|
||||
)));
|
||||
}
|
||||
out.push(el.into_string().unwrap_or_default());
|
||||
}
|
||||
Ok(out)
|
||||
} else {
|
||||
Err(runtime_err(&format!(
|
||||
"email: '{key}' must be a string or an array of strings"
|
||||
)))
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[allow(clippy::unnecessary_box_returns)]
|
||||
fn runtime_err(msg: &str) -> Box<EvalAltResult> {
|
||||
EvalAltResult::ErrorRuntime(msg.into(), rhai::Position::NONE).into()
|
||||
}
|
||||
|
||||
/// Run an `EmailService` future inside the synchronous Rhai context,
|
||||
/// mapping any `EmailError` to a Rhai runtime error. Mirrors
|
||||
/// `kv::block_on`.
|
||||
fn block_on<F>(fut: F) -> Result<(), Box<EvalAltResult>>
|
||||
where
|
||||
F: std::future::Future<Output = Result<(), EmailError>> + Send,
|
||||
{
|
||||
let handle = TokioHandle::try_current().map_err(|e| -> Box<EvalAltResult> {
|
||||
EvalAltResult::ErrorRuntime(
|
||||
format!("email: no tokio runtime available: {e}").into(),
|
||||
rhai::Position::NONE,
|
||||
)
|
||||
.into()
|
||||
})?;
|
||||
handle.block_on(fut).map_err(|err| -> Box<EvalAltResult> {
|
||||
EvalAltResult::ErrorRuntime(format!("email: {err}").into(), rhai::Position::NONE).into()
|
||||
})
|
||||
}
|
||||
@@ -15,10 +15,12 @@ pub mod bridge;
|
||||
pub mod cx;
|
||||
pub mod dead_letters;
|
||||
pub mod docs;
|
||||
pub mod email;
|
||||
pub mod files;
|
||||
pub mod http;
|
||||
pub mod kv;
|
||||
pub mod pubsub;
|
||||
pub mod secrets;
|
||||
pub mod stdlib;
|
||||
|
||||
pub use bridge::{dynamic_to_json, json_to_dynamic};
|
||||
@@ -41,5 +43,7 @@ pub fn register_all(engine: &mut RhaiEngine, services: &Services, cx: Arc<SdkCal
|
||||
dead_letters::register(engine, services, cx.clone());
|
||||
http::register(engine, services, cx.clone());
|
||||
files::register(engine, services, cx.clone());
|
||||
pubsub::register(engine, services, cx);
|
||||
pubsub::register(engine, services, cx.clone());
|
||||
secrets::register(engine, services, cx.clone());
|
||||
email::register(engine, services, cx);
|
||||
}
|
||||
|
||||
153
crates/executor-core/src/sdk/secrets.rs
Normal file
153
crates/executor-core/src/sdk/secrets.rs
Normal file
@@ -0,0 +1,153 @@
|
||||
//! `secrets::` Rhai bridge — encrypted per-app secrets (v1.1.7).
|
||||
//!
|
||||
//! ```rhai
|
||||
//! secrets::set("stripe_key", "sk_live_xxx");
|
||||
//! secrets::set("oauth", #{ client_id: "abc", client_secret: "xyz" });
|
||||
//! let key = secrets::get("stripe_key"); // value or ()
|
||||
//! let removed = secrets::delete("stripe_key"); // bool
|
||||
//! let page = secrets::list(#{ cursor: (), limit: 100 });
|
||||
//! // page = #{ names: [...], next_cursor: () | "..." }
|
||||
//! ```
|
||||
//!
|
||||
//! Collection-less (secrets are per-app, like pubsub topics) so there's
|
||||
//! no `::collection(...)`. Values are any JSON-serializable Rhai value
|
||||
//! (String/Map/Array/number/bool); a String round-trips back as a
|
||||
//! String. `app_id` is derived from `cx.app_id` in the service — it
|
||||
//! never appears in the script-side signature, preserving cross-app
|
||||
//! isolation.
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use picloud_shared::{SdkCallCx, SecretsError, SecretsListPage, Services};
|
||||
use rhai::{Array, Dynamic, Engine as RhaiEngine, EvalAltResult, Map, Module};
|
||||
use tokio::runtime::Handle as TokioHandle;
|
||||
|
||||
use super::bridge::{dynamic_to_json, json_to_dynamic};
|
||||
|
||||
pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<SdkCallCx>) {
|
||||
let svc = services.secrets.clone();
|
||||
let mut module = Module::new();
|
||||
|
||||
// secrets::set(name, value) — overwrites if present.
|
||||
{
|
||||
let svc = svc.clone();
|
||||
let cx = cx.clone();
|
||||
module.set_native_fn(
|
||||
"set",
|
||||
move |name: &str, value: Dynamic| -> Result<(), Box<EvalAltResult>> {
|
||||
let json = dynamic_to_json(&value);
|
||||
let svc = svc.clone();
|
||||
let cx = cx.clone();
|
||||
block_on(async move { svc.set(&cx, name, json).await })
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
// secrets::get(name) — decoded value, or () if missing.
|
||||
{
|
||||
let svc = svc.clone();
|
||||
let cx = cx.clone();
|
||||
module.set_native_fn(
|
||||
"get",
|
||||
move |name: &str| -> Result<Dynamic, Box<EvalAltResult>> {
|
||||
let svc = svc.clone();
|
||||
let cx = cx.clone();
|
||||
let opt = block_on(async move { svc.get(&cx, name).await })?;
|
||||
Ok(opt.map_or(Dynamic::UNIT, json_to_dynamic))
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
// secrets::delete(name) — bool was-present.
|
||||
{
|
||||
let svc = svc.clone();
|
||||
let cx = cx.clone();
|
||||
module.set_native_fn(
|
||||
"delete",
|
||||
move |name: &str| -> Result<bool, Box<EvalAltResult>> {
|
||||
let svc = svc.clone();
|
||||
let cx = cx.clone();
|
||||
block_on(async move { svc.delete(&cx, name).await })
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
// secrets::list(#{ cursor, limit }) — names only, cursor-paginated.
|
||||
{
|
||||
let svc = svc.clone();
|
||||
let cx = cx.clone();
|
||||
module.set_native_fn(
|
||||
"list",
|
||||
move |opts: Map| -> Result<Map, Box<EvalAltResult>> {
|
||||
let (cursor, limit) = parse_list_opts(&opts)?;
|
||||
let svc = svc.clone();
|
||||
let cx = cx.clone();
|
||||
let page: SecretsListPage =
|
||||
block_on(async move { svc.list(&cx, cursor.as_deref(), limit).await })?;
|
||||
Ok(list_page_to_map(page))
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
engine.register_static_module("secrets", module.into());
|
||||
}
|
||||
|
||||
/// Pull `cursor` (string or `()`) and `limit` (int or `()`) out of the
|
||||
/// options map. Unknown/extra keys are ignored.
|
||||
fn parse_list_opts(opts: &Map) -> Result<(Option<String>, u32), Box<EvalAltResult>> {
|
||||
let cursor = match opts.get("cursor") {
|
||||
None => None,
|
||||
Some(d) if d.is_unit() => None,
|
||||
Some(d) if d.is_string() => Some(d.clone().into_string().unwrap_or_default()),
|
||||
Some(_) => return Err(runtime_err("secrets::list: cursor must be a string or ()")),
|
||||
};
|
||||
let limit = match opts.get("limit") {
|
||||
None => 0,
|
||||
Some(d) if d.is_unit() => 0,
|
||||
Some(d) => {
|
||||
let n = d
|
||||
.as_int()
|
||||
.map_err(|_| runtime_err("secrets::list: limit must be an integer or ()"))?;
|
||||
u32::try_from(n.max(0)).unwrap_or(u32::MAX)
|
||||
}
|
||||
};
|
||||
Ok((cursor, limit))
|
||||
}
|
||||
|
||||
fn list_page_to_map(page: SecretsListPage) -> Map {
|
||||
let mut m = Map::new();
|
||||
let names: Array = page.names.into_iter().map(Dynamic::from).collect();
|
||||
m.insert("names".into(), names.into());
|
||||
m.insert(
|
||||
"next_cursor".into(),
|
||||
page.next_cursor.map_or(Dynamic::UNIT, Dynamic::from),
|
||||
);
|
||||
m
|
||||
}
|
||||
|
||||
// Returns the boxed error directly because every caller needs a
|
||||
// `Box<EvalAltResult>` (Rhai's error type), matching the other bridges.
|
||||
#[allow(clippy::unnecessary_box_returns)]
|
||||
fn runtime_err(msg: &str) -> Box<EvalAltResult> {
|
||||
EvalAltResult::ErrorRuntime(msg.into(), rhai::Position::NONE).into()
|
||||
}
|
||||
|
||||
/// Run a `SecretsService` future inside the synchronous Rhai context,
|
||||
/// mapping any `SecretsError` to a Rhai runtime error. Mirrors
|
||||
/// `kv::block_on` / `pubsub::block_on`.
|
||||
fn block_on<T, F>(fut: F) -> Result<T, Box<EvalAltResult>>
|
||||
where
|
||||
F: std::future::Future<Output = Result<T, SecretsError>> + Send,
|
||||
T: Send,
|
||||
{
|
||||
let handle = TokioHandle::try_current().map_err(|e| -> Box<EvalAltResult> {
|
||||
EvalAltResult::ErrorRuntime(
|
||||
format!("secrets: no tokio runtime available: {e}").into(),
|
||||
rhai::Position::NONE,
|
||||
)
|
||||
.into()
|
||||
})?;
|
||||
handle.block_on(fut).map_err(|err| -> Box<EvalAltResult> {
|
||||
EvalAltResult::ErrorRuntime(format!("secrets: {err}").into(), rhai::Position::NONE).into()
|
||||
})
|
||||
}
|
||||
@@ -101,6 +101,8 @@ async fn original_backend_error_is_logged_at_error_level() {
|
||||
Arc::new(NoopHttpService),
|
||||
Arc::new(picloud_shared::NoopFilesService),
|
||||
Arc::new(picloud_shared::NoopPubsubService),
|
||||
Arc::new(picloud_shared::NoopSecretsService),
|
||||
Arc::new(picloud_shared::NoopEmailService),
|
||||
);
|
||||
let engine = Engine::new(Limits::default(), services);
|
||||
|
||||
|
||||
@@ -99,6 +99,8 @@ fn services_with(modules: Arc<dyn ModuleSource>) -> Services {
|
||||
Arc::new(NoopHttpService),
|
||||
Arc::new(picloud_shared::NoopFilesService),
|
||||
Arc::new(picloud_shared::NoopPubsubService),
|
||||
Arc::new(picloud_shared::NoopSecretsService),
|
||||
Arc::new(picloud_shared::NoopEmailService),
|
||||
)
|
||||
}
|
||||
|
||||
|
||||
@@ -230,6 +230,8 @@ fn make_engine() -> Arc<Engine> {
|
||||
Arc::new(NoopHttpService),
|
||||
Arc::new(picloud_shared::NoopFilesService),
|
||||
Arc::new(picloud_shared::NoopPubsubService),
|
||||
Arc::new(picloud_shared::NoopSecretsService),
|
||||
Arc::new(picloud_shared::NoopEmailService),
|
||||
);
|
||||
Arc::new(Engine::new(Limits::default(), services))
|
||||
}
|
||||
|
||||
209
crates/executor-core/tests/sdk_email.rs
Normal file
209
crates/executor-core/tests/sdk_email.rs
Normal file
@@ -0,0 +1,209 @@
|
||||
//! `email::` SDK bridge integration tests — runs a real Rhai engine
|
||||
//! against a recording `EmailService`. Verifies the Rhai map → DTO
|
||||
//! plumbing (address coercion, the text-only vs multipart split). The
|
||||
//! SMTP transport, validation, and authz are unit-tested at the service
|
||||
//! layer in `manager-core::email_service`.
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
use std::sync::{Arc, Mutex};
|
||||
|
||||
use async_trait::async_trait;
|
||||
use picloud_executor_core::{Engine, ExecRequest, InvocationType, Limits};
|
||||
use picloud_shared::{
|
||||
AppId, EmailError, EmailService, ExecutionId, NoopDeadLetterService, NoopDocsService,
|
||||
NoopEventEmitter, NoopHttpService, NoopKvService, NoopModuleSource, OutboundEmail, RequestId,
|
||||
ScriptId, ScriptSandbox, SdkCallCx, Services, TriggerEvent,
|
||||
};
|
||||
use serde_json::{json, Value};
|
||||
|
||||
#[derive(Default)]
|
||||
struct RecordingEmail {
|
||||
sent: Mutex<Vec<OutboundEmail>>,
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl EmailService for RecordingEmail {
|
||||
async fn send(&self, _cx: &SdkCallCx, email: OutboundEmail) -> Result<(), EmailError> {
|
||||
self.sent.lock().unwrap().push(email);
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
fn engine_with(rec: Arc<RecordingEmail>) -> Arc<Engine> {
|
||||
let services = Services::new(
|
||||
Arc::new(NoopKvService),
|
||||
Arc::new(NoopDocsService),
|
||||
Arc::new(NoopDeadLetterService),
|
||||
Arc::new(NoopEventEmitter),
|
||||
Arc::new(NoopModuleSource),
|
||||
Arc::new(NoopHttpService),
|
||||
Arc::new(picloud_shared::NoopFilesService),
|
||||
Arc::new(picloud_shared::NoopPubsubService),
|
||||
Arc::new(picloud_shared::NoopSecretsService),
|
||||
rec,
|
||||
);
|
||||
Arc::new(Engine::new(Limits::default(), services))
|
||||
}
|
||||
|
||||
fn baseline_request(app_id: AppId) -> ExecRequest {
|
||||
let execution_id = ExecutionId::new();
|
||||
ExecRequest {
|
||||
execution_id,
|
||||
request_id: RequestId::new(),
|
||||
script_id: ScriptId::new(),
|
||||
script_name: "email-test".into(),
|
||||
invocation_type: InvocationType::Http,
|
||||
path: "/email-test".into(),
|
||||
headers: BTreeMap::new(),
|
||||
body: Value::Null,
|
||||
params: BTreeMap::new(),
|
||||
query: BTreeMap::new(),
|
||||
rest: String::new(),
|
||||
sandbox_overrides: ScriptSandbox::default(),
|
||||
app_id,
|
||||
principal: None,
|
||||
trigger_depth: 0,
|
||||
root_execution_id: execution_id,
|
||||
is_dead_letter_handler: false,
|
||||
event: None,
|
||||
}
|
||||
}
|
||||
|
||||
async fn run(engine: Arc<Engine>, src: &str) -> Result<(), ()> {
|
||||
let src = src.to_string();
|
||||
let app = AppId::new();
|
||||
tokio::task::spawn_blocking(move || engine.execute(&src, baseline_request(app)))
|
||||
.await
|
||||
.expect("spawn_blocking")
|
||||
.map(|_| ())
|
||||
.map_err(|_| ())
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn send_parses_single_recipient_text() {
|
||||
let rec = Arc::new(RecordingEmail::default());
|
||||
let engine = engine_with(rec.clone());
|
||||
run(
|
||||
engine,
|
||||
r#"
|
||||
email::send(#{
|
||||
to: "alice@example.com",
|
||||
from: "alerts@myapp.com",
|
||||
subject: "Build complete",
|
||||
text: "done"
|
||||
});
|
||||
#{ ok: true }
|
||||
"#,
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
let g = rec.sent.lock().unwrap();
|
||||
let e = g.last().unwrap();
|
||||
assert_eq!(e.to, vec!["alice@example.com".to_string()]);
|
||||
assert_eq!(e.from, "alerts@myapp.com");
|
||||
assert_eq!(e.subject, "Build complete");
|
||||
assert_eq!(e.text.as_deref(), Some("done"));
|
||||
// email::send forces text-only even if html were present.
|
||||
assert!(e.html.is_none());
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn send_html_carries_both_parts_and_lists() {
|
||||
let rec = Arc::new(RecordingEmail::default());
|
||||
let engine = engine_with(rec.clone());
|
||||
run(
|
||||
engine,
|
||||
r#"
|
||||
email::send_html(#{
|
||||
to: ["alice@x.com", "bob@y.com"],
|
||||
cc: ["dave@z.com"],
|
||||
bcc: ["audit@myapp.com"],
|
||||
from: "alerts@myapp.com",
|
||||
reply_to: "support@myapp.com",
|
||||
subject: "hi",
|
||||
text: "plain",
|
||||
html: "<p>rich</p>"
|
||||
});
|
||||
#{ ok: true }
|
||||
"#,
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
let g = rec.sent.lock().unwrap();
|
||||
let e = g.last().unwrap();
|
||||
assert_eq!(
|
||||
e.to,
|
||||
vec!["alice@x.com".to_string(), "bob@y.com".to_string()]
|
||||
);
|
||||
assert_eq!(e.cc, vec!["dave@z.com".to_string()]);
|
||||
assert_eq!(e.bcc, vec!["audit@myapp.com".to_string()]);
|
||||
assert_eq!(e.reply_to.as_deref(), Some("support@myapp.com"));
|
||||
assert_eq!(e.text.as_deref(), Some("plain"));
|
||||
assert_eq!(e.html.as_deref(), Some("<p>rich</p>"));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn inbound_email_event_visible_to_handler() {
|
||||
// A handler invoked by an email:receive trigger sees the normalized
|
||||
// message at ctx.event.email (built by the engine's ctx renderer).
|
||||
let rec = Arc::new(RecordingEmail::default());
|
||||
let engine = engine_with(rec);
|
||||
let mut req = baseline_request(AppId::new());
|
||||
req.event = Some(TriggerEvent::Email {
|
||||
from: "sender@external.com".into(),
|
||||
to: vec!["alice@myapp.com".into()],
|
||||
cc: vec!["bob@myapp.com".into()],
|
||||
subject: "Re: question".into(),
|
||||
text: Some("hello".into()),
|
||||
html: None,
|
||||
received_at: chrono::DateTime::parse_from_rfc3339("2026-08-15T12:00:00Z")
|
||||
.unwrap()
|
||||
.with_timezone(&chrono::Utc),
|
||||
message_id: Some("<abc@external.com>".into()),
|
||||
});
|
||||
let src = r#"
|
||||
let e = ctx.event;
|
||||
#{
|
||||
source: e.source,
|
||||
op: e.op,
|
||||
from: e.email.from,
|
||||
to0: e.email.to[0],
|
||||
cc0: e.email.cc[0],
|
||||
subject: e.email.subject,
|
||||
text: e.email.text,
|
||||
html_is_unit: type_of(e.email.html) == "()",
|
||||
message_id: e.email.message_id
|
||||
}
|
||||
"#;
|
||||
let src = src.to_string();
|
||||
let body = tokio::task::spawn_blocking(move || engine.execute(&src, req))
|
||||
.await
|
||||
.unwrap()
|
||||
.unwrap()
|
||||
.body;
|
||||
assert_eq!(body["source"], json!("email"));
|
||||
assert_eq!(body["op"], json!("receive"));
|
||||
assert_eq!(body["from"], json!("sender@external.com"));
|
||||
assert_eq!(body["to0"], json!("alice@myapp.com"));
|
||||
assert_eq!(body["cc0"], json!("bob@myapp.com"));
|
||||
assert_eq!(body["subject"], json!("Re: question"));
|
||||
assert_eq!(body["text"], json!("hello"));
|
||||
assert_eq!(body["html_is_unit"], json!(true));
|
||||
assert_eq!(body["message_id"], json!("<abc@external.com>"));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn send_html_without_html_throws() {
|
||||
let rec = Arc::new(RecordingEmail::default());
|
||||
let engine = engine_with(rec.clone());
|
||||
let res = run(
|
||||
engine,
|
||||
r#"
|
||||
email::send_html(#{ to: "a@b.com", from: "c@d.com", subject: "x", text: "y" });
|
||||
#{ ok: true }
|
||||
"#,
|
||||
)
|
||||
.await;
|
||||
assert!(res.is_err(), "send_html without html must throw");
|
||||
assert!(rec.sent.lock().unwrap().is_empty());
|
||||
}
|
||||
@@ -167,6 +167,8 @@ fn make_engine() -> Arc<Engine> {
|
||||
Arc::new(NoopHttpService),
|
||||
Arc::new(InMemoryFiles::default()),
|
||||
Arc::new(picloud_shared::NoopPubsubService),
|
||||
Arc::new(picloud_shared::NoopSecretsService),
|
||||
Arc::new(picloud_shared::NoopEmailService),
|
||||
);
|
||||
Arc::new(Engine::new(Limits::default(), services))
|
||||
}
|
||||
|
||||
@@ -90,6 +90,8 @@ fn engine_with(http: Arc<dyn HttpService>) -> Arc<Engine> {
|
||||
http,
|
||||
Arc::new(picloud_shared::NoopFilesService),
|
||||
Arc::new(picloud_shared::NoopPubsubService),
|
||||
Arc::new(picloud_shared::NoopSecretsService),
|
||||
Arc::new(picloud_shared::NoopEmailService),
|
||||
);
|
||||
Arc::new(Engine::new(Limits::default(), services))
|
||||
}
|
||||
|
||||
@@ -109,6 +109,8 @@ fn make_engine() -> Arc<Engine> {
|
||||
Arc::new(NoopHttpService),
|
||||
Arc::new(picloud_shared::NoopFilesService),
|
||||
Arc::new(picloud_shared::NoopPubsubService),
|
||||
Arc::new(picloud_shared::NoopSecretsService),
|
||||
Arc::new(picloud_shared::NoopEmailService),
|
||||
);
|
||||
Arc::new(Engine::new(Limits::default(), services))
|
||||
}
|
||||
|
||||
@@ -47,6 +47,8 @@ fn make_engine(svc: Arc<RecordingPubsub>) -> Arc<Engine> {
|
||||
Arc::new(NoopHttpService),
|
||||
Arc::new(NoopFilesService),
|
||||
svc,
|
||||
Arc::new(picloud_shared::NoopSecretsService),
|
||||
Arc::new(picloud_shared::NoopEmailService),
|
||||
);
|
||||
Arc::new(Engine::new(Limits::default(), services))
|
||||
}
|
||||
|
||||
213
crates/executor-core/tests/sdk_secrets.rs
Normal file
213
crates/executor-core/tests/sdk_secrets.rs
Normal file
@@ -0,0 +1,213 @@
|
||||
//! `secrets::` SDK bridge integration tests — runs a real Rhai engine
|
||||
//! against an in-memory `SecretsService` impl. Mirrors `sdk_kv.rs`: the
|
||||
//! engine runs under `spawn_blocking` so the bridge's `block_on` has a
|
||||
//! reachable runtime.
|
||||
//!
|
||||
//! This exercises the Rhai⇄JSON plumbing + the static `secrets` module
|
||||
//! (set/get/delete/list, the missing→() contract, and the
|
||||
//! String/Map/Array type round-trip). Encryption + authz + the
|
||||
//! cross-app boundary are unit-tested at the service layer in
|
||||
//! `manager-core::secrets_service`.
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
use std::sync::Arc;
|
||||
|
||||
use async_trait::async_trait;
|
||||
use picloud_executor_core::{Engine, ExecRequest, InvocationType, Limits};
|
||||
use picloud_shared::{
|
||||
AppId, ExecutionId, NoopDeadLetterService, NoopDocsService, NoopEventEmitter, NoopHttpService,
|
||||
NoopKvService, NoopModuleSource, RequestId, ScriptId, ScriptSandbox, SdkCallCx, SecretsError,
|
||||
SecretsListPage, SecretsService, Services,
|
||||
};
|
||||
use serde_json::{json, Value};
|
||||
use tokio::sync::Mutex;
|
||||
|
||||
/// In-memory secrets store keyed by `(app_id, name)`. Stores the JSON
|
||||
/// value directly — the bridge test only cares about the Rhai plumbing,
|
||||
/// not the at-rest encryption (which the service layer owns).
|
||||
#[derive(Default)]
|
||||
struct InMemorySecrets {
|
||||
data: Mutex<BTreeMap<(AppId, String), Value>>,
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl SecretsService for InMemorySecrets {
|
||||
async fn get(&self, cx: &SdkCallCx, name: &str) -> Result<Option<Value>, SecretsError> {
|
||||
picloud_shared::validate_secret_name(name)?;
|
||||
Ok(self
|
||||
.data
|
||||
.lock()
|
||||
.await
|
||||
.get(&(cx.app_id, name.to_string()))
|
||||
.cloned())
|
||||
}
|
||||
|
||||
async fn set(&self, cx: &SdkCallCx, name: &str, value: Value) -> Result<(), SecretsError> {
|
||||
picloud_shared::validate_secret_name(name)?;
|
||||
self.data
|
||||
.lock()
|
||||
.await
|
||||
.insert((cx.app_id, name.to_string()), value);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn delete(&self, cx: &SdkCallCx, name: &str) -> Result<bool, SecretsError> {
|
||||
picloud_shared::validate_secret_name(name)?;
|
||||
Ok(self
|
||||
.data
|
||||
.lock()
|
||||
.await
|
||||
.remove(&(cx.app_id, name.to_string()))
|
||||
.is_some())
|
||||
}
|
||||
|
||||
async fn list(
|
||||
&self,
|
||||
cx: &SdkCallCx,
|
||||
cursor: Option<&str>,
|
||||
limit: u32,
|
||||
) -> Result<SecretsListPage, SecretsError> {
|
||||
let data = self.data.lock().await;
|
||||
let mut names: Vec<String> = data
|
||||
.iter()
|
||||
.filter(|((a, _), _)| *a == cx.app_id)
|
||||
.map(|((_, n), _)| n.clone())
|
||||
.filter(|n| cursor.is_none_or(|c| n.as_str() > c))
|
||||
.collect();
|
||||
names.sort();
|
||||
let take = if limit == 0 {
|
||||
usize::MAX
|
||||
} else {
|
||||
limit as usize
|
||||
};
|
||||
let next_cursor = if names.len() > take {
|
||||
names.truncate(take);
|
||||
names.last().cloned()
|
||||
} else {
|
||||
None
|
||||
};
|
||||
Ok(SecretsListPage { names, next_cursor })
|
||||
}
|
||||
}
|
||||
|
||||
fn make_engine() -> Arc<Engine> {
|
||||
let services = Services::new(
|
||||
Arc::new(NoopKvService),
|
||||
Arc::new(NoopDocsService),
|
||||
Arc::new(NoopDeadLetterService),
|
||||
Arc::new(NoopEventEmitter),
|
||||
Arc::new(NoopModuleSource),
|
||||
Arc::new(NoopHttpService),
|
||||
Arc::new(picloud_shared::NoopFilesService),
|
||||
Arc::new(picloud_shared::NoopPubsubService),
|
||||
Arc::new(InMemorySecrets::default()),
|
||||
Arc::new(picloud_shared::NoopEmailService),
|
||||
);
|
||||
Arc::new(Engine::new(Limits::default(), services))
|
||||
}
|
||||
|
||||
fn baseline_request(app_id: AppId) -> ExecRequest {
|
||||
let execution_id = ExecutionId::new();
|
||||
ExecRequest {
|
||||
execution_id,
|
||||
request_id: RequestId::new(),
|
||||
script_id: ScriptId::new(),
|
||||
script_name: "secrets-test".into(),
|
||||
invocation_type: InvocationType::Http,
|
||||
path: "/secrets-test".into(),
|
||||
headers: BTreeMap::new(),
|
||||
body: Value::Null,
|
||||
params: BTreeMap::new(),
|
||||
query: BTreeMap::new(),
|
||||
rest: String::new(),
|
||||
sandbox_overrides: ScriptSandbox::default(),
|
||||
app_id,
|
||||
principal: None,
|
||||
trigger_depth: 0,
|
||||
root_execution_id: execution_id,
|
||||
is_dead_letter_handler: false,
|
||||
event: None,
|
||||
}
|
||||
}
|
||||
|
||||
async fn run_script(engine: Arc<Engine>, src: &str, req: ExecRequest) -> Value {
|
||||
let src = src.to_string();
|
||||
tokio::task::spawn_blocking(move || engine.execute(&src, req))
|
||||
.await
|
||||
.expect("spawn_blocking should not panic")
|
||||
.expect("script execution should succeed")
|
||||
.body
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn set_then_get_string_round_trips() {
|
||||
let engine = make_engine();
|
||||
let src = r#"
|
||||
secrets::set("stripe_key", "sk_live_xxx");
|
||||
secrets::get("stripe_key")
|
||||
"#;
|
||||
let body = run_script(engine, src, baseline_request(AppId::new())).await;
|
||||
// A String comes back a String, not a JSON-quoted "\"sk_live_xxx\"".
|
||||
assert_eq!(body, json!("sk_live_xxx"));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn set_then_get_map_round_trips() {
|
||||
let engine = make_engine();
|
||||
let src = r#"
|
||||
secrets::set("oauth", #{ client_id: "abc", client_secret: "xyz" });
|
||||
secrets::get("oauth")
|
||||
"#;
|
||||
let body = run_script(engine, src, baseline_request(AppId::new())).await;
|
||||
assert_eq!(body, json!({ "client_id": "abc", "client_secret": "xyz" }));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn get_missing_returns_unit() {
|
||||
let engine = make_engine();
|
||||
let src = r#"
|
||||
let v = secrets::get("nope");
|
||||
#{ is_unit: type_of(v) == "()" }
|
||||
"#;
|
||||
let body = run_script(engine, src, baseline_request(AppId::new())).await;
|
||||
assert_eq!(body, json!({ "is_unit": true }));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn delete_returns_was_present() {
|
||||
let engine = make_engine();
|
||||
let src = r#"
|
||||
secrets::set("k", "v");
|
||||
let first = secrets::delete("k");
|
||||
let second = secrets::delete("k");
|
||||
#{ first: first, second: second }
|
||||
"#;
|
||||
let body = run_script(engine, src, baseline_request(AppId::new())).await;
|
||||
assert_eq!(body, json!({ "first": true, "second": false }));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn list_returns_names_and_cursor() {
|
||||
let engine = make_engine();
|
||||
let src = r#"
|
||||
secrets::set("a", 1);
|
||||
secrets::set("b", 2);
|
||||
secrets::set("c", 3);
|
||||
let page = secrets::list(#{ cursor: (), limit: 2 });
|
||||
page
|
||||
"#;
|
||||
let body = run_script(engine, src, baseline_request(AppId::new())).await;
|
||||
assert_eq!(body["names"], json!(["a", "b"]));
|
||||
assert_eq!(body["next_cursor"], json!("b"));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn empty_name_throws() {
|
||||
let engine = make_engine();
|
||||
let src = r#" secrets::set("", "v"); #{ ok: true } "#;
|
||||
let app = AppId::new();
|
||||
let out = tokio::task::spawn_blocking(move || engine.execute(src, baseline_request(app)))
|
||||
.await
|
||||
.expect("spawn_blocking");
|
||||
assert!(out.is_err(), "empty secret name must throw");
|
||||
}
|
||||
@@ -94,6 +94,8 @@ fn make_engine() -> Arc<Engine> {
|
||||
Arc::new(NoopHttpService),
|
||||
Arc::new(NoopFilesService),
|
||||
Arc::new(FakeMintPubsub),
|
||||
Arc::new(picloud_shared::NoopSecretsService),
|
||||
Arc::new(picloud_shared::NoopEmailService),
|
||||
);
|
||||
Arc::new(Engine::new(Limits::default(), services))
|
||||
}
|
||||
@@ -195,7 +197,7 @@ async fn unit_ttl_uses_default() {
|
||||
async fn empty_topics_throws() {
|
||||
run_err(
|
||||
make_engine(),
|
||||
r#"pubsub::subscriber_token([], 60)"#,
|
||||
r"pubsub::subscriber_token([], 60)",
|
||||
request(AppId::new(), true),
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -31,8 +31,13 @@ reqwest.workspace = true
|
||||
|
||||
argon2.workspace = true
|
||||
sha2.workspace = true
|
||||
# HMAC-SHA256 verification of inbound-email provider signatures (v1.1.7).
|
||||
hmac.workspace = true
|
||||
hex.workspace = true
|
||||
base64.workspace = true
|
||||
data-encoding.workspace = true
|
||||
# Outbound SMTP email (v1.1.7 email::send / send_html).
|
||||
lettre.workspace = true
|
||||
|
||||
[dev-dependencies]
|
||||
tokio.workspace = true
|
||||
|
||||
24
crates/manager-core/migrations/0023_secrets.sql
Normal file
24
crates/manager-core/migrations/0023_secrets.sql
Normal file
@@ -0,0 +1,24 @@
|
||||
-- v1.1.7: encrypted per-app secrets.
|
||||
--
|
||||
-- Operational config (API keys, OAuth tokens, webhook signing keys)
|
||||
-- encrypted at rest with the process master key (AES-256-GCM). Both the
|
||||
-- ciphertext (16-byte GCM auth tag appended) and the 12-byte nonce are
|
||||
-- stored; the master key itself never lives in the database. See
|
||||
-- `picloud_shared::crypto` + `manager-core::secrets_service`.
|
||||
--
|
||||
-- This is the user-facing `secrets::*` store. It is intentionally
|
||||
-- separate from `app_secrets` (the one-row-per-app realtime signing
|
||||
-- key, 0022): different cardinality (many named rows per app), and the
|
||||
-- realtime key is encrypted in place by migration 0025.
|
||||
|
||||
CREATE TABLE secrets (
|
||||
app_id UUID NOT NULL REFERENCES apps(id) ON DELETE CASCADE,
|
||||
name TEXT NOT NULL,
|
||||
encrypted_value BYTEA NOT NULL, -- ciphertext incl. 16-byte GCM auth tag
|
||||
nonce BYTEA NOT NULL, -- 12 bytes
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
PRIMARY KEY (app_id, name)
|
||||
);
|
||||
|
||||
CREATE INDEX idx_secrets_app ON secrets (app_id);
|
||||
32
crates/manager-core/migrations/0024_email_triggers.sql
Normal file
32
crates/manager-core/migrations/0024_email_triggers.sql
Normal file
@@ -0,0 +1,32 @@
|
||||
-- v1.1.7: inbound email triggers (email:receive).
|
||||
--
|
||||
-- A configured provider (Mailgun / Postmark / SendGrid / SES) POSTs
|
||||
-- inbound email to POST /api/v1/email-inbound/{app_id}/{trigger_id};
|
||||
-- the receiver normalizes it into a TriggerEvent::Email and enqueues an
|
||||
-- outbox row for the trigger's handler. v1.1.7 ships the webhook path;
|
||||
-- a native SMTP listener is v1.3+.
|
||||
|
||||
-- Widen the trigger-kind + outbox-source CHECK constraints to admit
|
||||
-- 'email'.
|
||||
ALTER TABLE triggers DROP CONSTRAINT triggers_kind_check;
|
||||
ALTER TABLE triggers ADD CONSTRAINT triggers_kind_check
|
||||
CHECK (kind IN ('kv', 'dead_letter', 'docs', 'cron',
|
||||
'files', 'pubsub', 'email'));
|
||||
|
||||
ALTER TABLE outbox DROP CONSTRAINT outbox_source_kind_check;
|
||||
ALTER TABLE outbox ADD CONSTRAINT outbox_source_kind_check
|
||||
CHECK (source_kind IN ('http', 'kv', 'dead_letter', 'docs',
|
||||
'cron', 'files', 'pubsub', 'email'));
|
||||
|
||||
-- Per-trigger inbound config. The HMAC secret used to verify provider
|
||||
-- signatures is stored ENCRYPTED at rest (AES-256-GCM under the process
|
||||
-- master key) — a deviation from the original brief's plaintext column,
|
||||
-- chosen to keep all operationally-secret material encrypted. The
|
||||
-- receiver decrypts it per inbound request. NULL columns mean the
|
||||
-- trigger has no signature verification (accepts any POST to its URL —
|
||||
-- relies on URL secrecy).
|
||||
CREATE TABLE email_trigger_details (
|
||||
trigger_id UUID PRIMARY KEY REFERENCES triggers(id) ON DELETE CASCADE,
|
||||
inbound_secret_encrypted BYTEA, -- ciphertext incl. GCM auth tag (NULL = unsigned)
|
||||
inbound_secret_nonce BYTEA -- 12 bytes (NULL = unsigned)
|
||||
);
|
||||
@@ -0,0 +1,24 @@
|
||||
-- v1.1.7: encrypt the realtime signing key at rest (two-phase).
|
||||
--
|
||||
-- Phase 1 (this migration + the v1.1.7 startup task):
|
||||
-- * add NULL-able encrypted columns,
|
||||
-- * drop the NOT NULL on the plaintext column so newly-generated keys
|
||||
-- can be stored encrypted-only,
|
||||
-- * the application startup task `migrate_plaintext_keys` encrypts each
|
||||
-- existing plaintext key into the new columns (plaintext is LEFT in
|
||||
-- place during the compat window for rollback safety).
|
||||
--
|
||||
-- The `RealtimeAuthorityImpl` read path prefers the encrypted columns and
|
||||
-- falls back to plaintext, so SSE keeps working throughout.
|
||||
--
|
||||
-- Phase 2 (v1.1.8): once all rows are migrated, a follow-up migration
|
||||
-- drops the plaintext `realtime_signing_key` column.
|
||||
|
||||
ALTER TABLE app_secrets
|
||||
ADD COLUMN realtime_signing_key_encrypted BYTEA,
|
||||
ADD COLUMN realtime_signing_key_nonce BYTEA;
|
||||
|
||||
-- New keys (post-v1.1.7) are stored encrypted-only, so the plaintext
|
||||
-- column must accept NULL.
|
||||
ALTER TABLE app_secrets
|
||||
ALTER COLUMN realtime_signing_key DROP NOT NULL;
|
||||
@@ -1,19 +1,23 @@
|
||||
//! `AppSecretsRepo` — per-app secret material (v1.1.6).
|
||||
//! `AppSecretsRepo` — per-app secret material (v1.1.6, encrypted v1.1.7).
|
||||
//!
|
||||
//! Today this holds only the HMAC signing key for realtime subscriber
|
||||
//! tokens. The key is generated lazily (32 random bytes) on the first
|
||||
//! Holds the HMAC signing key for realtime subscriber tokens. The key is
|
||||
//! generated lazily (32 random bytes) on the first
|
||||
//! `pubsub::subscriber_token` call for an app and never changes
|
||||
//! thereafter in v1.1.6 (no rotation API yet — rotation is the
|
||||
//! key-invalidation mechanism, deferred). The key is never exposed to
|
||||
//! thereafter (no rotation API yet). The key is never exposed to
|
||||
//! scripts: the SDK mints tokens, it never returns the key.
|
||||
//!
|
||||
//! This table is the natural home for v1.1.7's encrypted per-app
|
||||
//! secrets work.
|
||||
//! **v1.1.7 at-rest encryption (two-phase).** The key is now sealed with
|
||||
//! the process master key (AES-256-GCM). New keys are written
|
||||
//! encrypted-only; the startup task [`PostgresAppSecretsRepo::migrate_plaintext_keys`]
|
||||
//! encrypts any pre-existing plaintext rows. The read path prefers the
|
||||
//! encrypted columns and falls back to the plaintext column during the
|
||||
//! compat window (migration 0025 made it NULL-able; v1.1.8 drops it).
|
||||
|
||||
use async_trait::async_trait;
|
||||
use picloud_shared::AppId;
|
||||
use picloud_shared::{crypto, AppId, MasterKey};
|
||||
use rand::RngCore;
|
||||
use sqlx::PgPool;
|
||||
use uuid::Uuid;
|
||||
|
||||
/// Length of a freshly-generated realtime signing key.
|
||||
pub const SIGNING_KEY_LEN: usize = 32;
|
||||
@@ -22,14 +26,19 @@ pub const SIGNING_KEY_LEN: usize = 32;
|
||||
pub enum AppSecretsRepoError {
|
||||
#[error("database error: {0}")]
|
||||
Db(#[from] sqlx::Error),
|
||||
|
||||
/// A stored encrypted signing key could not be decrypted — corrupted
|
||||
/// row or a master-key mismatch (e.g. `PICLOUD_SECRET_KEY` changed).
|
||||
#[error("realtime signing key could not be decrypted (corrupted row or master-key mismatch)")]
|
||||
Crypto,
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
pub trait AppSecretsRepo: Send + Sync {
|
||||
/// Fetch the app's realtime signing key, generating + persisting one
|
||||
/// (32 random bytes) if absent. Idempotent under concurrency: a
|
||||
/// racing creator's `ON CONFLICT DO NOTHING` insert is a no-op and
|
||||
/// the existing key is returned.
|
||||
/// (32 random bytes, encrypted) if absent. Idempotent under
|
||||
/// concurrency: a racing creator's `ON CONFLICT DO NOTHING` insert is
|
||||
/// a no-op and the existing key is returned.
|
||||
async fn get_or_create_signing_key(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
@@ -43,12 +52,78 @@ pub trait AppSecretsRepo: Send + Sync {
|
||||
|
||||
pub struct PostgresAppSecretsRepo {
|
||||
pool: PgPool,
|
||||
master_key: MasterKey,
|
||||
}
|
||||
|
||||
impl PostgresAppSecretsRepo {
|
||||
#[must_use]
|
||||
pub fn new(pool: PgPool) -> Self {
|
||||
Self { pool }
|
||||
pub fn new(pool: PgPool, master_key: MasterKey) -> Self {
|
||||
Self { pool, master_key }
|
||||
}
|
||||
|
||||
/// Startup task (v1.1.7): encrypt every row that still has a
|
||||
/// plaintext key but no encrypted key. Plaintext is left in place
|
||||
/// (the read path prefers the encrypted columns); the plaintext
|
||||
/// column is dropped in v1.1.8. Returns the number of rows migrated.
|
||||
///
|
||||
/// # Errors
|
||||
///
|
||||
/// Propagates database errors.
|
||||
pub async fn migrate_plaintext_keys(&self) -> Result<usize, AppSecretsRepoError> {
|
||||
let rows: Vec<(Uuid, Vec<u8>)> = sqlx::query_as(
|
||||
"SELECT app_id, realtime_signing_key FROM app_secrets \
|
||||
WHERE realtime_signing_key_encrypted IS NULL \
|
||||
AND realtime_signing_key IS NOT NULL",
|
||||
)
|
||||
.fetch_all(&self.pool)
|
||||
.await?;
|
||||
|
||||
let mut migrated = 0;
|
||||
for (app_id, plaintext) in rows {
|
||||
let enc = crypto::encrypt(&plaintext, self.master_key.as_bytes());
|
||||
sqlx::query(
|
||||
"UPDATE app_secrets \
|
||||
SET realtime_signing_key_encrypted = $2, \
|
||||
realtime_signing_key_nonce = $3, \
|
||||
updated_at = NOW() \
|
||||
WHERE app_id = $1 AND realtime_signing_key_encrypted IS NULL",
|
||||
)
|
||||
.bind(app_id)
|
||||
.bind(&enc.ciphertext)
|
||||
.bind(&enc.nonce[..])
|
||||
.execute(&self.pool)
|
||||
.await?;
|
||||
migrated += 1;
|
||||
}
|
||||
Ok(migrated)
|
||||
}
|
||||
|
||||
fn decode(
|
||||
&self,
|
||||
encrypted: Option<Vec<u8>>,
|
||||
nonce: Option<Vec<u8>>,
|
||||
plaintext: Option<Vec<u8>>,
|
||||
) -> Result<Option<Vec<u8>>, AppSecretsRepoError> {
|
||||
decode_signing_key(&self.master_key, encrypted, nonce, plaintext)
|
||||
}
|
||||
}
|
||||
|
||||
/// Resolve the signing key from a row's three columns. **Encrypted wins**
|
||||
/// when present; otherwise fall back to the plaintext column (compat for
|
||||
/// un-migrated rows / the post-v1.1.8 dropped-plaintext state).
|
||||
fn decode_signing_key(
|
||||
master_key: &MasterKey,
|
||||
encrypted: Option<Vec<u8>>,
|
||||
nonce: Option<Vec<u8>>,
|
||||
plaintext: Option<Vec<u8>>,
|
||||
) -> Result<Option<Vec<u8>>, AppSecretsRepoError> {
|
||||
match (encrypted, nonce) {
|
||||
(Some(ct), Some(n)) => {
|
||||
let key = crypto::decrypt(&ct, &n, master_key.as_bytes())
|
||||
.map_err(|_| AppSecretsRepoError::Crypto)?;
|
||||
Ok(Some(key))
|
||||
}
|
||||
_ => Ok(plaintext),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -60,32 +135,107 @@ impl AppSecretsRepo for PostgresAppSecretsRepo {
|
||||
) -> Result<Vec<u8>, AppSecretsRepoError> {
|
||||
let mut fresh = vec![0u8; SIGNING_KEY_LEN];
|
||||
rand::thread_rng().fill_bytes(&mut fresh);
|
||||
let enc = crypto::encrypt(&fresh, self.master_key.as_bytes());
|
||||
|
||||
// Insert-if-absent then read: the racing-creator's insert is a
|
||||
// no-op, and the SELECT always returns the winning key.
|
||||
// Insert-if-absent (encrypted-only). The racing-creator's insert
|
||||
// is a no-op; the SELECT always returns the winning row.
|
||||
sqlx::query(
|
||||
"INSERT INTO app_secrets (app_id, realtime_signing_key) \
|
||||
VALUES ($1, $2) ON CONFLICT (app_id) DO NOTHING",
|
||||
"INSERT INTO app_secrets \
|
||||
(app_id, realtime_signing_key_encrypted, realtime_signing_key_nonce) \
|
||||
VALUES ($1, $2, $3) ON CONFLICT (app_id) DO NOTHING",
|
||||
)
|
||||
.bind(app_id.into_inner())
|
||||
.bind(&fresh)
|
||||
.bind(&enc.ciphertext)
|
||||
.bind(&enc.nonce[..])
|
||||
.execute(&self.pool)
|
||||
.await?;
|
||||
|
||||
let key: (Vec<u8>,) =
|
||||
sqlx::query_as("SELECT realtime_signing_key FROM app_secrets WHERE app_id = $1")
|
||||
.bind(app_id.into_inner())
|
||||
.fetch_one(&self.pool)
|
||||
.await?;
|
||||
Ok(key.0)
|
||||
let row: (Option<Vec<u8>>, Option<Vec<u8>>, Option<Vec<u8>>) = sqlx::query_as(
|
||||
"SELECT realtime_signing_key_encrypted, realtime_signing_key_nonce, \
|
||||
realtime_signing_key \
|
||||
FROM app_secrets WHERE app_id = $1",
|
||||
)
|
||||
.bind(app_id.into_inner())
|
||||
.fetch_one(&self.pool)
|
||||
.await?;
|
||||
// A row exists by construction, so a key must decode.
|
||||
self.decode(row.0, row.1, row.2)?
|
||||
.ok_or(AppSecretsRepoError::Crypto)
|
||||
}
|
||||
|
||||
async fn signing_key(&self, app_id: AppId) -> Result<Option<Vec<u8>>, AppSecretsRepoError> {
|
||||
let row: Option<(Vec<u8>,)> =
|
||||
sqlx::query_as("SELECT realtime_signing_key FROM app_secrets WHERE app_id = $1")
|
||||
.bind(app_id.into_inner())
|
||||
.fetch_optional(&self.pool)
|
||||
.await?;
|
||||
Ok(row.map(|r| r.0))
|
||||
let row: Option<(Option<Vec<u8>>, Option<Vec<u8>>, Option<Vec<u8>>)> = sqlx::query_as(
|
||||
"SELECT realtime_signing_key_encrypted, realtime_signing_key_nonce, \
|
||||
realtime_signing_key \
|
||||
FROM app_secrets WHERE app_id = $1",
|
||||
)
|
||||
.bind(app_id.into_inner())
|
||||
.fetch_optional(&self.pool)
|
||||
.await?;
|
||||
match row {
|
||||
Some((e, n, p)) => self.decode(e, n, p),
|
||||
None => Ok(None),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn key() -> MasterKey {
|
||||
MasterKey::from_bytes([9u8; 32])
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn encrypted_wins_over_plaintext() {
|
||||
let mk = key();
|
||||
let secret = vec![1u8, 2, 3, 4];
|
||||
let enc = crypto::encrypt(&secret, mk.as_bytes());
|
||||
// Both present → the encrypted value is returned (not the bogus
|
||||
// plaintext).
|
||||
let got = decode_signing_key(
|
||||
&mk,
|
||||
Some(enc.ciphertext),
|
||||
Some(enc.nonce.to_vec()),
|
||||
Some(vec![0xff; 32]),
|
||||
)
|
||||
.unwrap();
|
||||
assert_eq!(got, Some(secret));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn falls_back_to_plaintext_when_encrypted_absent() {
|
||||
let mk = key();
|
||||
let plaintext = vec![7u8; 32];
|
||||
let got = decode_signing_key(&mk, None, None, Some(plaintext.clone())).unwrap();
|
||||
assert_eq!(got, Some(plaintext));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn encrypted_present_plaintext_null_works() {
|
||||
// Post-v1.1.8 state: only the encrypted columns are populated.
|
||||
let mk = key();
|
||||
let secret = vec![5u8; 32];
|
||||
let enc = crypto::encrypt(&secret, mk.as_bytes());
|
||||
let got =
|
||||
decode_signing_key(&mk, Some(enc.ciphertext), Some(enc.nonce.to_vec()), None).unwrap();
|
||||
assert_eq!(got, Some(secret));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn missing_everything_is_none() {
|
||||
let got = decode_signing_key(&key(), None, None, None).unwrap();
|
||||
assert_eq!(got, None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn wrong_master_key_is_crypto_error() {
|
||||
let secret = vec![3u8; 32];
|
||||
let enc = crypto::encrypt(&secret, key().as_bytes());
|
||||
let other = MasterKey::from_bytes([1u8; 32]);
|
||||
let err = decode_signing_key(&other, Some(enc.ciphertext), Some(enc.nonce.to_vec()), None)
|
||||
.unwrap_err();
|
||||
assert!(matches!(err, AppSecretsRepoError::Crypto));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -89,6 +89,18 @@ pub enum Capability {
|
||||
/// (v1.1.5). Maps to `script:write` on API keys (a publish is a
|
||||
/// write that fans out to subscribers). Granted to `editor`+.
|
||||
AppPubsubPublish(AppId),
|
||||
/// Read a decrypted secret from this app's secrets store (v1.1.7).
|
||||
/// Same trust shape as KV/docs/files read — granted to `viewer`+,
|
||||
/// maps to `script:read` on API keys. Honors the seven-scope
|
||||
/// commitment.
|
||||
AppSecretsRead(AppId),
|
||||
/// Write (set/delete) a secret in this app's secrets store (v1.1.7).
|
||||
/// Granted to `editor`+, maps to `script:write` on API keys.
|
||||
AppSecretsWrite(AppId),
|
||||
/// Send an outbound email from a script in this app (v1.1.7). Maps
|
||||
/// to `script:write` on API keys (sending mail is an outbound
|
||||
/// side-effect like an HTTP request). Granted to `editor`+.
|
||||
AppEmailSend(AppId),
|
||||
/// Create / list / delete triggers for this app (v1.1.1). Maps to
|
||||
/// `app:admin` on API keys — triggers are app-configuration acts
|
||||
/// rather than data-plane access. Granted to `app_admin`+.
|
||||
@@ -128,6 +140,9 @@ impl Capability {
|
||||
| Self::AppFilesRead(id)
|
||||
| Self::AppFilesWrite(id)
|
||||
| Self::AppPubsubPublish(id)
|
||||
| Self::AppSecretsRead(id)
|
||||
| Self::AppSecretsWrite(id)
|
||||
| Self::AppEmailSend(id)
|
||||
| Self::AppManageTriggers(id)
|
||||
| Self::AppDeadLetterManage(id)
|
||||
| Self::AppTopicManage(id) => Some(id),
|
||||
@@ -148,13 +163,16 @@ impl Capability {
|
||||
Self::AppRead(_)
|
||||
| Self::AppKvRead(_)
|
||||
| Self::AppDocsRead(_)
|
||||
| Self::AppFilesRead(_) => Scope::ScriptRead,
|
||||
| Self::AppFilesRead(_)
|
||||
| Self::AppSecretsRead(_) => Scope::ScriptRead,
|
||||
Self::AppWriteScript(_)
|
||||
| Self::AppKvWrite(_)
|
||||
| Self::AppDocsWrite(_)
|
||||
| Self::AppHttpRequest(_)
|
||||
| Self::AppFilesWrite(_)
|
||||
| Self::AppPubsubPublish(_) => Scope::ScriptWrite,
|
||||
| Self::AppPubsubPublish(_)
|
||||
| Self::AppSecretsWrite(_)
|
||||
| Self::AppEmailSend(_) => Scope::ScriptWrite,
|
||||
Self::AppWriteRoute(_) => Scope::RouteWrite,
|
||||
Self::AppManageDomains(_) => Scope::DomainManage,
|
||||
Self::AppAdmin(_)
|
||||
@@ -305,6 +323,7 @@ const fn role_satisfies(role: AppRole, cap: Capability) -> bool {
|
||||
| Capability::AppKvRead(_)
|
||||
| Capability::AppDocsRead(_)
|
||||
| Capability::AppFilesRead(_)
|
||||
| Capability::AppSecretsRead(_)
|
||||
);
|
||||
let in_editor = in_viewer
|
||||
|| matches!(
|
||||
@@ -316,6 +335,8 @@ const fn role_satisfies(role: AppRole, cap: Capability) -> bool {
|
||||
| Capability::AppHttpRequest(_)
|
||||
| Capability::AppFilesWrite(_)
|
||||
| Capability::AppPubsubPublish(_)
|
||||
| Capability::AppSecretsWrite(_)
|
||||
| Capability::AppEmailSend(_)
|
||||
);
|
||||
let in_app_admin = in_editor
|
||||
|| matches!(
|
||||
|
||||
@@ -23,19 +23,19 @@
|
||||
use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
|
||||
use chrono::Utc;
|
||||
use chrono::{DateTime, Utc};
|
||||
use picloud_executor_core::{ExecError, ExecRequest, ExecResponse, InvocationType};
|
||||
use picloud_orchestrator_core::{ExecutionGate, ExecutorClient};
|
||||
use picloud_shared::{
|
||||
ExecResponseSummary, ExecutionId, HttpDispatchPayload, InboxDeliveryOutcome, InboxFailureKind,
|
||||
InboxResolver, InboxResult, RequestId, ScriptId, ScriptSandbox, TriggerEvent,
|
||||
DeadLetterId, ExecResponseSummary, ExecutionId, HttpDispatchPayload, InboxDeliveryOutcome,
|
||||
InboxFailureKind, InboxResolver, InboxResult, RequestId, ScriptId, ScriptSandbox, TriggerEvent,
|
||||
};
|
||||
use rand::Rng;
|
||||
use uuid::Uuid;
|
||||
|
||||
use crate::abandoned_repo::{AbandonedRepo, NewAbandonedExecution};
|
||||
use crate::dead_letter_repo::{DeadLetterRepo, NewDeadLetter};
|
||||
use crate::outbox_repo::{OutboxRepo, OutboxRow, OutboxSourceKind};
|
||||
use crate::outbox_repo::{NewOutboxRow, OutboxRepo, OutboxRow, OutboxSourceKind};
|
||||
use crate::principal_resolver::PrincipalResolver;
|
||||
use crate::repo::ScriptRepository;
|
||||
use crate::trigger_config::{BackoffShape, TriggerConfig};
|
||||
@@ -168,7 +168,8 @@ impl Dispatcher {
|
||||
| OutboxSourceKind::DeadLetter
|
||||
| OutboxSourceKind::Cron
|
||||
| OutboxSourceKind::Files
|
||||
| OutboxSourceKind::Pubsub => {
|
||||
| OutboxSourceKind::Pubsub
|
||||
| OutboxSourceKind::Email => {
|
||||
let resolved = self.resolve_trigger(&row).await?;
|
||||
let req = match self.build_exec_request(&row, &resolved).await {
|
||||
Ok(req) => req,
|
||||
@@ -462,12 +463,12 @@ impl Dispatcher {
|
||||
// Exhausted retries → dead-letter.
|
||||
let (op, source) = describe_event(&row.payload);
|
||||
let now = Utc::now();
|
||||
if let Err(e) = self
|
||||
let dl_id = match self
|
||||
.dead_letters
|
||||
.insert(NewDeadLetter {
|
||||
app_id: row.app_id,
|
||||
original_event_id: row.id,
|
||||
source,
|
||||
source: source.clone(),
|
||||
op,
|
||||
trigger_id: row.trigger_id,
|
||||
script_id: Some(resolved.script_id),
|
||||
@@ -479,8 +480,26 @@ impl Dispatcher {
|
||||
})
|
||||
.await
|
||||
{
|
||||
tracing::error!(?e, "failed to write dead-letter row");
|
||||
Ok(id) => Some(id),
|
||||
Err(e) => {
|
||||
tracing::error!(?e, "failed to write dead-letter row");
|
||||
None
|
||||
}
|
||||
};
|
||||
|
||||
// v1.1.7 fix: fan the dead-letter out to matching handler triggers.
|
||||
// This was missing since v1.1.1 — the row was written but
|
||||
// `list_matching_dead_letter` had no production caller, so
|
||||
// registered dead_letter handlers never fired. The recursion-stop
|
||||
// (a dead-letter handler's own failure is not re-dead-lettered)
|
||||
// is upheld by the `is_dead_letter_handler` short-circuit at the
|
||||
// top of this function, so this fan-out is only reached for
|
||||
// non-handler executions.
|
||||
if let Some(dl_id) = dl_id {
|
||||
self.fan_out_dead_letter(row, resolved, dl_id, &source, attempt, &err, now)
|
||||
.await;
|
||||
}
|
||||
|
||||
self.outbox
|
||||
.delete(row.id)
|
||||
.await
|
||||
@@ -488,6 +507,82 @@ impl Dispatcher {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Enqueue one outbox row per matching `dead_letter` trigger so its
|
||||
/// handler script runs with the dead-letter event as `ctx.event`.
|
||||
/// Best-effort: a lookup/insert failure is logged, not propagated
|
||||
/// (the dead-letter row itself is already durably written).
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
async fn fan_out_dead_letter(
|
||||
&self,
|
||||
row: &OutboxRow,
|
||||
resolved: &ResolvedTrigger,
|
||||
dead_letter_id: DeadLetterId,
|
||||
source: &str,
|
||||
attempt: u32,
|
||||
err: &ExecError,
|
||||
now: DateTime<Utc>,
|
||||
) {
|
||||
// The DL event nests the original verbatim; if the payload can't
|
||||
// be decoded back into a TriggerEvent we can't build the nested
|
||||
// `original`, so skip the fan-out (the DL row is still written).
|
||||
let Ok(original) = serde_json::from_value::<TriggerEvent>(row.payload.clone()) else {
|
||||
tracing::warn!(
|
||||
outbox_id = %row.id,
|
||||
"dead-letter payload is not a TriggerEvent; skipping handler fan-out"
|
||||
);
|
||||
return;
|
||||
};
|
||||
|
||||
let matches = match self
|
||||
.triggers
|
||||
.list_matching_dead_letter(row.app_id, source, row.trigger_id, Some(resolved.script_id))
|
||||
.await
|
||||
{
|
||||
Ok(m) => m,
|
||||
Err(e) => {
|
||||
tracing::error!(?e, "dead-letter trigger lookup failed");
|
||||
return;
|
||||
}
|
||||
};
|
||||
|
||||
for m in matches {
|
||||
let event = TriggerEvent::DeadLetter {
|
||||
dead_letter_id,
|
||||
original: Box::new(original.clone()),
|
||||
attempts: attempt,
|
||||
last_error: err.to_string(),
|
||||
trigger_id: row.trigger_id,
|
||||
script_id: Some(resolved.script_id),
|
||||
first_attempt_at: row.created_at,
|
||||
last_attempt_at: now,
|
||||
};
|
||||
let payload = match serde_json::to_value(&event) {
|
||||
Ok(p) => p,
|
||||
Err(e) => {
|
||||
tracing::error!(?e, "failed to serialize dead-letter event");
|
||||
continue;
|
||||
}
|
||||
};
|
||||
if let Err(e) = self
|
||||
.outbox
|
||||
.insert(NewOutboxRow {
|
||||
app_id: row.app_id,
|
||||
source_kind: OutboxSourceKind::DeadLetter,
|
||||
trigger_id: Some(m.trigger_id),
|
||||
script_id: Some(m.script_id),
|
||||
reply_to: None,
|
||||
payload,
|
||||
origin_principal: Some(m.registered_by_principal),
|
||||
trigger_depth: row.trigger_depth.saturating_add(1),
|
||||
root_execution_id: row.root_execution_id,
|
||||
})
|
||||
.await
|
||||
{
|
||||
tracing::error!(?e, "failed to enqueue dead-letter handler delivery");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async fn deliver_inbox(&self, row: &OutboxRow, inbox_id: Uuid, result: InboxResult) {
|
||||
match self.inbox.deliver(inbox_id, result.clone()).await {
|
||||
InboxDeliveryOutcome::Delivered => {}
|
||||
|
||||
307
crates/manager-core/src/email_inbound_api.rs
Normal file
307
crates/manager-core/src/email_inbound_api.rs
Normal file
@@ -0,0 +1,307 @@
|
||||
//! `POST /api/v1/email-inbound/{app_id}/{trigger_id}` — the inbound-email
|
||||
//! webhook receiver (v1.1.7).
|
||||
//!
|
||||
//! A configured provider (Mailgun / Postmark / SendGrid / SES) POSTs a
|
||||
//! normalized JSON message here; the receiver verifies the optional HMAC
|
||||
//! signature, builds a `TriggerEvent::Email`, and enqueues an outbox row
|
||||
//! the dispatcher picks up like any other async trigger.
|
||||
//!
|
||||
//! This is a PUBLIC endpoint (no admin auth) — the trigger URL itself,
|
||||
//! plus the per-trigger HMAC secret, are the security boundary. It is
|
||||
//! mounted OUTSIDE the `require_authenticated` layer.
|
||||
//!
|
||||
//! Status codes:
|
||||
//! * 202 — accepted + enqueued
|
||||
//! * 401 — HMAC required but missing/invalid
|
||||
//! * 404 — trigger missing, disabled, not `kind=email`, or app mismatch
|
||||
//! * 422 — body is not the expected JSON shape
|
||||
//!
|
||||
//! Only the generic provider-agnostic JSON shape is accepted in v1.1.7
|
||||
//! (see [`InboundPayload`]); provider-specific unmarshallers are v1.2.
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use axum::body::Bytes;
|
||||
use axum::extract::{Path, State};
|
||||
use axum::http::{HeaderMap, StatusCode};
|
||||
use axum::response::{IntoResponse, Json, Response};
|
||||
use axum::routing::post;
|
||||
use axum::Router;
|
||||
use hmac::{Hmac, Mac};
|
||||
use picloud_shared::{AppId, MasterKey, TriggerEvent, TriggerId};
|
||||
use serde::Deserialize;
|
||||
use serde_json::json;
|
||||
use sha2::Sha256;
|
||||
|
||||
use crate::outbox_repo::{NewOutboxRow, OutboxRepo, OutboxSourceKind};
|
||||
use crate::secrets_repo::StoredSecret;
|
||||
use crate::secrets_service::open;
|
||||
use crate::trigger_repo::TriggerRepo;
|
||||
|
||||
type HmacSha256 = Hmac<Sha256>;
|
||||
|
||||
/// Header the provider's HMAC signature is read from. The signature is
|
||||
/// the lowercase hex of `HMAC-SHA256(inbound_secret, raw_body)`.
|
||||
const SIGNATURE_HEADER: &str = "x-picloud-signature";
|
||||
|
||||
#[derive(Clone)]
|
||||
pub struct EmailInboundState {
|
||||
pub triggers: Arc<dyn TriggerRepo>,
|
||||
pub outbox: Arc<dyn OutboxRepo>,
|
||||
pub master_key: MasterKey,
|
||||
}
|
||||
|
||||
pub fn email_inbound_router(state: EmailInboundState) -> Router {
|
||||
Router::new()
|
||||
.route(
|
||||
"/email-inbound/{app_id}/{trigger_id}",
|
||||
post(receive_inbound_email),
|
||||
)
|
||||
.with_state(state)
|
||||
}
|
||||
|
||||
/// The generic provider-agnostic inbound shape. Users configure their
|
||||
/// provider's webhook templating to POST this. `from` is required;
|
||||
/// everything else defaults.
|
||||
#[derive(Debug, Deserialize)]
|
||||
struct InboundPayload {
|
||||
from: String,
|
||||
#[serde(default)]
|
||||
to: Vec<String>,
|
||||
#[serde(default)]
|
||||
cc: Vec<String>,
|
||||
#[serde(default)]
|
||||
subject: String,
|
||||
#[serde(default)]
|
||||
text: Option<String>,
|
||||
#[serde(default)]
|
||||
html: Option<String>,
|
||||
#[serde(default)]
|
||||
message_id: Option<String>,
|
||||
}
|
||||
|
||||
async fn receive_inbound_email(
|
||||
State(s): State<EmailInboundState>,
|
||||
Path((app_id, trigger_id)): Path<(AppId, TriggerId)>,
|
||||
headers: HeaderMap,
|
||||
body: Bytes,
|
||||
) -> Result<StatusCode, EmailInboundError> {
|
||||
// Resolve the trigger. 404 covers missing / wrong-kind / cross-app /
|
||||
// disabled — all "this URL doesn't address a live email trigger".
|
||||
let target = s
|
||||
.triggers
|
||||
.email_inbound_target(trigger_id)
|
||||
.await
|
||||
.map_err(|e| EmailInboundError::Backend(e.to_string()))?
|
||||
.ok_or(EmailInboundError::NotFound)?;
|
||||
if target.app_id != app_id || !target.enabled {
|
||||
return Err(EmailInboundError::NotFound);
|
||||
}
|
||||
|
||||
// HMAC verification (only when the trigger has a secret configured).
|
||||
if let (Some(ct), Some(nonce)) = (
|
||||
target.inbound_secret_encrypted.as_ref(),
|
||||
target.inbound_secret_nonce.as_ref(),
|
||||
) {
|
||||
let secret = decrypt_secret(&s.master_key, ct, nonce)?;
|
||||
verify_signature(&headers, &body, secret.as_bytes())?;
|
||||
}
|
||||
|
||||
// Parse the generic JSON shape. Malformed → 422.
|
||||
let payload: InboundPayload =
|
||||
serde_json::from_slice(&body).map_err(|e| EmailInboundError::Malformed(e.to_string()))?;
|
||||
|
||||
let event = TriggerEvent::Email {
|
||||
from: payload.from,
|
||||
to: payload.to,
|
||||
cc: payload.cc,
|
||||
subject: payload.subject,
|
||||
text: payload.text,
|
||||
html: payload.html,
|
||||
received_at: chrono::Utc::now(),
|
||||
message_id: payload.message_id,
|
||||
};
|
||||
let payload_json = serde_json::to_value(&event)
|
||||
.map_err(|e| EmailInboundError::Backend(format!("serialize event: {e}")))?;
|
||||
|
||||
s.outbox
|
||||
.insert(NewOutboxRow {
|
||||
app_id,
|
||||
source_kind: OutboxSourceKind::Email,
|
||||
trigger_id: Some(trigger_id),
|
||||
script_id: Some(target.script_id),
|
||||
reply_to: None,
|
||||
payload: payload_json,
|
||||
origin_principal: Some(target.registered_by_principal),
|
||||
// Inbound email is the root of a trigger chain (depth 1).
|
||||
trigger_depth: 1,
|
||||
root_execution_id: None,
|
||||
})
|
||||
.await
|
||||
.map_err(|e| EmailInboundError::Backend(e.to_string()))?;
|
||||
|
||||
Ok(StatusCode::ACCEPTED)
|
||||
}
|
||||
|
||||
/// Decrypt the stored inbound secret back to its raw string. It was
|
||||
/// sealed as a JSON string by the admin layer, so `open` yields a
|
||||
/// `Value::String`.
|
||||
fn decrypt_secret(
|
||||
master_key: &MasterKey,
|
||||
ciphertext: &[u8],
|
||||
nonce: &[u8],
|
||||
) -> Result<String, EmailInboundError> {
|
||||
let stored = StoredSecret {
|
||||
encrypted_value: ciphertext.to_vec(),
|
||||
nonce: nonce.to_vec(),
|
||||
};
|
||||
let value = open(master_key, &stored).map_err(|_| {
|
||||
// Corrupted secret means we can't verify — fail closed (401).
|
||||
EmailInboundError::Unauthorized
|
||||
})?;
|
||||
value
|
||||
.as_str()
|
||||
.map(str::to_string)
|
||||
.ok_or(EmailInboundError::Unauthorized)
|
||||
}
|
||||
|
||||
/// Constant-time HMAC-SHA256 verification of the body against the
|
||||
/// `X-Picloud-Signature` header (lowercase hex).
|
||||
fn verify_signature(
|
||||
headers: &HeaderMap,
|
||||
body: &[u8],
|
||||
secret: &[u8],
|
||||
) -> Result<(), EmailInboundError> {
|
||||
let provided_hex = headers
|
||||
.get(SIGNATURE_HEADER)
|
||||
.and_then(|h| h.to_str().ok())
|
||||
.ok_or(EmailInboundError::Unauthorized)?;
|
||||
let provided = hex::decode(provided_hex.trim()).map_err(|_| EmailInboundError::Unauthorized)?;
|
||||
let mut mac =
|
||||
HmacSha256::new_from_slice(secret).map_err(|_| EmailInboundError::Unauthorized)?;
|
||||
mac.update(body);
|
||||
mac.verify_slice(&provided)
|
||||
.map_err(|_| EmailInboundError::Unauthorized)
|
||||
}
|
||||
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub enum EmailInboundError {
|
||||
#[error("trigger not found")]
|
||||
NotFound,
|
||||
#[error("invalid signature")]
|
||||
Unauthorized,
|
||||
#[error("malformed body: {0}")]
|
||||
Malformed(String),
|
||||
#[error("backend: {0}")]
|
||||
Backend(String),
|
||||
}
|
||||
|
||||
impl IntoResponse for EmailInboundError {
|
||||
fn into_response(self) -> Response {
|
||||
let (status, body) = match &self {
|
||||
Self::NotFound => (
|
||||
StatusCode::NOT_FOUND,
|
||||
json!({ "error": "trigger not found" }),
|
||||
),
|
||||
Self::Unauthorized => (
|
||||
StatusCode::UNAUTHORIZED,
|
||||
json!({ "error": "invalid or missing signature" }),
|
||||
),
|
||||
Self::Malformed(m) => (
|
||||
StatusCode::UNPROCESSABLE_ENTITY,
|
||||
json!({ "error": format!("malformed inbound email body: {m}") }),
|
||||
),
|
||||
Self::Backend(e) => {
|
||||
tracing::error!(error = %e, "inbound email receiver backend error");
|
||||
(
|
||||
StatusCode::INTERNAL_SERVER_ERROR,
|
||||
json!({ "error": "internal error" }),
|
||||
)
|
||||
}
|
||||
};
|
||||
(status, Json(body)).into_response()
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
//! Unit tests for the security-critical helpers (HMAC verify, secret
|
||||
//! round-trip, payload parsing). The full request flow — 202 / 401 /
|
||||
//! 404 / 422 / cross-app — is exercised end-to-end against a real
|
||||
//! Postgres in `crates/picloud/tests/email_inbound.rs`.
|
||||
|
||||
use super::*;
|
||||
use crate::secrets_service::seal;
|
||||
use crate::secrets_service::DEFAULT_SECRET_MAX_VALUE_BYTES;
|
||||
|
||||
fn sign(secret: &[u8], body: &[u8]) -> String {
|
||||
let mut mac = HmacSha256::new_from_slice(secret).unwrap();
|
||||
mac.update(body);
|
||||
hex::encode(mac.finalize().into_bytes())
|
||||
}
|
||||
|
||||
fn headers_with_sig(sig: &str) -> HeaderMap {
|
||||
let mut h = HeaderMap::new();
|
||||
h.insert(SIGNATURE_HEADER, sig.parse().unwrap());
|
||||
h
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn valid_signature_verifies() {
|
||||
let secret = b"shhh";
|
||||
let body = br#"{"from":"a@b.com"}"#;
|
||||
let sig = sign(secret, body);
|
||||
assert!(verify_signature(&headers_with_sig(&sig), body, secret).is_ok());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn wrong_signature_rejected() {
|
||||
let body = br#"{"from":"a@b.com"}"#;
|
||||
let sig = sign(b"shhh", body);
|
||||
let err = verify_signature(&headers_with_sig(&sig), body, b"different").unwrap_err();
|
||||
assert!(matches!(err, EmailInboundError::Unauthorized));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn missing_signature_header_rejected() {
|
||||
let err = verify_signature(&HeaderMap::new(), b"body", b"secret").unwrap_err();
|
||||
assert!(matches!(err, EmailInboundError::Unauthorized));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn tampered_body_fails_verification() {
|
||||
let secret = b"shhh";
|
||||
let sig = sign(secret, b"original");
|
||||
let err = verify_signature(&headers_with_sig(&sig), b"tampered", secret).unwrap_err();
|
||||
assert!(matches!(err, EmailInboundError::Unauthorized));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn secret_round_trips_through_seal_open() {
|
||||
let key = MasterKey::from_bytes([3u8; 32]);
|
||||
let (ct, nonce) = seal(
|
||||
&key,
|
||||
&serde_json::Value::String("provider-secret".into()),
|
||||
DEFAULT_SECRET_MAX_VALUE_BYTES,
|
||||
)
|
||||
.unwrap();
|
||||
let recovered = decrypt_secret(&key, &ct, &nonce).unwrap();
|
||||
assert_eq!(recovered, "provider-secret");
|
||||
// And a signature made with the recovered secret verifies.
|
||||
let body = br#"{"from":"x@y.com"}"#;
|
||||
let sig = sign(recovered.as_bytes(), body);
|
||||
assert!(verify_signature(&headers_with_sig(&sig), body, recovered.as_bytes()).is_ok());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn payload_requires_from_but_defaults_rest() {
|
||||
let ok: Result<InboundPayload, _> = serde_json::from_slice(br#"{"from":"a@b.com"}"#);
|
||||
let p = ok.expect("from-only payload parses");
|
||||
assert_eq!(p.from, "a@b.com");
|
||||
assert!(p.to.is_empty() && p.cc.is_empty() && p.text.is_none());
|
||||
|
||||
// Missing `from` → malformed.
|
||||
let bad: Result<InboundPayload, _> = serde_json::from_slice(br#"{"subject":"hi"}"#);
|
||||
assert!(bad.is_err());
|
||||
}
|
||||
}
|
||||
597
crates/manager-core/src/email_service.rs
Normal file
597
crates/manager-core/src/email_service.rs
Normal file
@@ -0,0 +1,597 @@
|
||||
//! `EmailServiceImpl` — outbound email over an SMTP relay (`lettre`),
|
||||
//! behind the `picloud_shared::EmailService` trait scripts reach via the
|
||||
//! Rhai `email::{send,send_html}` bridge.
|
||||
//!
|
||||
//! Layers added here:
|
||||
//!
|
||||
//! 1. **Script-as-gate authz**: `AppEmailSend` checked when
|
||||
//! `cx.principal.is_some()`; skipped for public-HTTP (`None`).
|
||||
//! 2. Required-field + RFC 5322-ish address validation at the boundary.
|
||||
//! 3. Per-message size cap (default 25 MB).
|
||||
//! 4. **Disabled mode**: if no SMTP relay is configured (HOST/USER/
|
||||
//! PASSWORD not all set) every `send` returns `NotConfigured` and
|
||||
//! startup logs a warning — there is no silent drop.
|
||||
//!
|
||||
//! Connection model: one connection per call (lettre's default). A
|
||||
//! pooled transport is a v1.2+ optimization. Per-app `from` validation /
|
||||
//! SPF / DKIM are the operator's responsibility at the relay (v1.1.7
|
||||
//! does not restrict the `from` address).
|
||||
|
||||
use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
|
||||
use async_trait::async_trait;
|
||||
use lettre::message::{Mailbox, Message, MultiPart, SinglePart};
|
||||
use lettre::transport::smtp::authentication::Credentials;
|
||||
use lettre::{AsyncSmtpTransport, AsyncTransport, Tokio1Executor};
|
||||
use picloud_shared::{EmailError, EmailService, OutboundEmail, SdkCallCx};
|
||||
|
||||
use crate::authz::{self, AuthzRepo, Capability};
|
||||
|
||||
/// Default per-message size cap (25 MB) — matches most providers.
|
||||
/// Override with `PICLOUD_EMAIL_MAX_MESSAGE_BYTES`.
|
||||
pub const DEFAULT_EMAIL_MAX_MESSAGE_BYTES: usize = 25 * 1024 * 1024;
|
||||
|
||||
/// Generous upper bound on a single address string (RFC 5321 caps the
|
||||
/// path at 256; 320 covers local@domain comfortably).
|
||||
const ADDRESS_MAX_LEN: usize = 320;
|
||||
|
||||
/// Process config for the email service.
|
||||
#[derive(Debug, Clone, Copy)]
|
||||
pub struct EmailConfig {
|
||||
pub max_message_bytes: usize,
|
||||
}
|
||||
|
||||
impl EmailConfig {
|
||||
#[must_use]
|
||||
pub const fn conservative() -> Self {
|
||||
Self {
|
||||
max_message_bytes: DEFAULT_EMAIL_MAX_MESSAGE_BYTES,
|
||||
}
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
pub fn from_env() -> Self {
|
||||
let mut c = Self::conservative();
|
||||
if let Ok(v) = std::env::var("PICLOUD_EMAIL_MAX_MESSAGE_BYTES") {
|
||||
match v.trim().parse::<usize>() {
|
||||
Ok(n) if n > 0 => c.max_message_bytes = n,
|
||||
_ => tracing::warn!(
|
||||
value = %v,
|
||||
"ignoring invalid PICLOUD_EMAIL_MAX_MESSAGE_BYTES (want a positive integer)"
|
||||
),
|
||||
}
|
||||
}
|
||||
c
|
||||
}
|
||||
}
|
||||
|
||||
impl Default for EmailConfig {
|
||||
fn default() -> Self {
|
||||
Self::conservative()
|
||||
}
|
||||
}
|
||||
|
||||
/// TLS mode for the SMTP relay connection.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum SmtpTls {
|
||||
/// STARTTLS upgrade on a plaintext port (typically 587). Default.
|
||||
Starttls,
|
||||
/// Implicit TLS from connect (typically 465).
|
||||
Implicit,
|
||||
/// No TLS — plaintext. Dev/test only.
|
||||
None,
|
||||
}
|
||||
|
||||
/// SMTP relay connection settings, sourced from env.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct SmtpConfig {
|
||||
pub host: String,
|
||||
pub port: u16,
|
||||
pub user: String,
|
||||
pub password: String,
|
||||
pub tls: SmtpTls,
|
||||
pub timeout_secs: u64,
|
||||
}
|
||||
|
||||
impl SmtpConfig {
|
||||
/// Read SMTP settings from env. Returns `None` (→ disabled mode) when
|
||||
/// any of HOST / USER / PASSWORD is missing or empty.
|
||||
#[must_use]
|
||||
pub fn from_env() -> Option<Self> {
|
||||
let host = non_empty_env("PICLOUD_SMTP_HOST")?;
|
||||
let user = non_empty_env("PICLOUD_SMTP_USER")?;
|
||||
let password = non_empty_env("PICLOUD_SMTP_PASSWORD")?;
|
||||
let tls = match std::env::var("PICLOUD_SMTP_TLS")
|
||||
.unwrap_or_default()
|
||||
.trim()
|
||||
.to_ascii_lowercase()
|
||||
.as_str()
|
||||
{
|
||||
"implicit" => SmtpTls::Implicit,
|
||||
"none" => SmtpTls::None,
|
||||
// Default + explicit "starttls" + anything unrecognized.
|
||||
_ => SmtpTls::Starttls,
|
||||
};
|
||||
let default_port = match tls {
|
||||
SmtpTls::Implicit => 465,
|
||||
SmtpTls::Starttls | SmtpTls::None => 587,
|
||||
};
|
||||
let port = std::env::var("PICLOUD_SMTP_PORT")
|
||||
.ok()
|
||||
.and_then(|v| v.trim().parse::<u16>().ok())
|
||||
.unwrap_or(default_port);
|
||||
let timeout_secs = std::env::var("PICLOUD_SMTP_TIMEOUT_SECS")
|
||||
.ok()
|
||||
.and_then(|v| v.trim().parse::<u64>().ok())
|
||||
.filter(|n| *n > 0)
|
||||
.unwrap_or(30);
|
||||
Some(Self {
|
||||
host,
|
||||
port,
|
||||
user,
|
||||
password,
|
||||
tls,
|
||||
timeout_secs,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
fn non_empty_env(key: &str) -> Option<String> {
|
||||
std::env::var(key).ok().filter(|v| !v.trim().is_empty())
|
||||
}
|
||||
|
||||
/// Internal transport seam so the service can be tested without a live
|
||||
/// SMTP server. The production impl is [`LettreEmailTransport`]; tests
|
||||
/// use a recording fake.
|
||||
#[async_trait]
|
||||
pub trait EmailTransport: Send + Sync {
|
||||
async fn send(&self, message: &Message) -> Result<(), EmailError>;
|
||||
}
|
||||
|
||||
/// Production transport: a per-call lettre SMTP connection.
|
||||
pub struct LettreEmailTransport {
|
||||
inner: AsyncSmtpTransport<Tokio1Executor>,
|
||||
}
|
||||
|
||||
impl LettreEmailTransport {
|
||||
/// Build the transport from settings.
|
||||
///
|
||||
/// # Errors
|
||||
///
|
||||
/// Returns the lettre SMTP error string if the relay descriptor is
|
||||
/// invalid (e.g. TLS setup fails).
|
||||
pub fn build(cfg: &SmtpConfig) -> Result<Self, String> {
|
||||
let builder = match cfg.tls {
|
||||
SmtpTls::Implicit => {
|
||||
AsyncSmtpTransport::<Tokio1Executor>::relay(&cfg.host).map_err(|e| e.to_string())?
|
||||
}
|
||||
SmtpTls::Starttls => AsyncSmtpTransport::<Tokio1Executor>::starttls_relay(&cfg.host)
|
||||
.map_err(|e| e.to_string())?,
|
||||
SmtpTls::None => AsyncSmtpTransport::<Tokio1Executor>::builder_dangerous(&cfg.host),
|
||||
};
|
||||
let inner = builder
|
||||
.port(cfg.port)
|
||||
.credentials(Credentials::new(cfg.user.clone(), cfg.password.clone()))
|
||||
.timeout(Some(Duration::from_secs(cfg.timeout_secs)))
|
||||
.build();
|
||||
Ok(Self { inner })
|
||||
}
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl EmailTransport for LettreEmailTransport {
|
||||
async fn send(&self, message: &Message) -> Result<(), EmailError> {
|
||||
// lettre's `AsyncTransport::send` consumes the `Message`; clone so
|
||||
// the caller keeps ownership (it needs it for the size check).
|
||||
self.inner
|
||||
.send(message.clone())
|
||||
.await
|
||||
.map(|_| ())
|
||||
.map_err(|e| EmailError::Transport(e.to_string()))
|
||||
}
|
||||
}
|
||||
|
||||
pub struct EmailServiceImpl {
|
||||
/// `None` → disabled mode (every send returns `NotConfigured`).
|
||||
transport: Option<Arc<dyn EmailTransport>>,
|
||||
authz: Arc<dyn AuthzRepo>,
|
||||
config: EmailConfig,
|
||||
}
|
||||
|
||||
impl EmailServiceImpl {
|
||||
#[must_use]
|
||||
pub fn new(
|
||||
transport: Option<Arc<dyn EmailTransport>>,
|
||||
authz: Arc<dyn AuthzRepo>,
|
||||
config: EmailConfig,
|
||||
) -> Self {
|
||||
Self {
|
||||
transport,
|
||||
authz,
|
||||
config,
|
||||
}
|
||||
}
|
||||
|
||||
/// Construct from env: builds a lettre SMTP transport if the relay is
|
||||
/// configured, otherwise runs in disabled mode (with a warning). A
|
||||
/// malformed relay descriptor is logged and also yields disabled mode
|
||||
/// — email is non-critical and must not block startup.
|
||||
#[must_use]
|
||||
pub fn from_env(authz: Arc<dyn AuthzRepo>) -> Self {
|
||||
let config = EmailConfig::from_env();
|
||||
let transport: Option<Arc<dyn EmailTransport>> = match SmtpConfig::from_env() {
|
||||
None => {
|
||||
tracing::warn!(
|
||||
"email is DISABLED: set PICLOUD_SMTP_HOST/USER/PASSWORD to enable \
|
||||
email::send. Scripts calling email::send will get an error."
|
||||
);
|
||||
None
|
||||
}
|
||||
Some(cfg) => match LettreEmailTransport::build(&cfg) {
|
||||
Ok(t) => {
|
||||
tracing::info!(host = %cfg.host, port = cfg.port, "outbound email enabled");
|
||||
Some(Arc::new(t))
|
||||
}
|
||||
Err(e) => {
|
||||
tracing::error!(error = %e, "failed to build SMTP transport; email DISABLED");
|
||||
None
|
||||
}
|
||||
},
|
||||
};
|
||||
Self::new(transport, authz, config)
|
||||
}
|
||||
|
||||
async fn check_send(&self, cx: &SdkCallCx) -> Result<(), EmailError> {
|
||||
if let Some(ref principal) = cx.principal {
|
||||
authz::require(&*self.authz, principal, Capability::AppEmailSend(cx.app_id))
|
||||
.await
|
||||
.map_err(|_| EmailError::Forbidden)?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl EmailService for EmailServiceImpl {
|
||||
async fn send(&self, cx: &SdkCallCx, email: OutboundEmail) -> Result<(), EmailError> {
|
||||
self.check_send(cx).await?;
|
||||
let Some(transport) = self.transport.as_ref() else {
|
||||
return Err(EmailError::NotConfigured);
|
||||
};
|
||||
let message = build_message(&email)?;
|
||||
let formatted = message.formatted();
|
||||
if formatted.len() > self.config.max_message_bytes {
|
||||
return Err(EmailError::TooLarge {
|
||||
limit: self.config.max_message_bytes,
|
||||
actual: formatted.len(),
|
||||
});
|
||||
}
|
||||
transport.send(&message).await
|
||||
}
|
||||
}
|
||||
|
||||
/// Validate the required fields + addresses and assemble a lettre
|
||||
/// `Message`. Pure (no I/O) so it's unit-testable on its own.
|
||||
fn build_message(email: &OutboundEmail) -> Result<Message, EmailError> {
|
||||
if email.from.trim().is_empty() {
|
||||
return Err(EmailError::MissingField("from".into()));
|
||||
}
|
||||
if email.to.iter().all(|a| a.trim().is_empty()) {
|
||||
return Err(EmailError::MissingField("to".into()));
|
||||
}
|
||||
if email.subject.trim().is_empty() {
|
||||
return Err(EmailError::MissingField("subject".into()));
|
||||
}
|
||||
let has_text = email.text.as_ref().is_some_and(|t| !t.is_empty());
|
||||
let has_html = email.html.as_ref().is_some_and(|h| !h.is_empty());
|
||||
if !has_text && !has_html {
|
||||
return Err(EmailError::MissingField("text or html".into()));
|
||||
}
|
||||
|
||||
let mut builder = Message::builder()
|
||||
.from(parse_address(&email.from)?)
|
||||
.subject(email.subject.clone());
|
||||
|
||||
for addr in non_empty(&email.to) {
|
||||
builder = builder.to(parse_address(addr)?);
|
||||
}
|
||||
for addr in non_empty(&email.cc) {
|
||||
builder = builder.cc(parse_address(addr)?);
|
||||
}
|
||||
for addr in non_empty(&email.bcc) {
|
||||
builder = builder.bcc(parse_address(addr)?);
|
||||
}
|
||||
// reply_to defaults to `from` when not supplied.
|
||||
let reply_to = email.reply_to.as_deref().unwrap_or(&email.from);
|
||||
builder = builder.reply_to(parse_address(reply_to)?);
|
||||
|
||||
// `has_text` / `has_html` were validated above (at least one is set).
|
||||
let text = email.text.clone().unwrap_or_default();
|
||||
let html = email.html.clone().unwrap_or_default();
|
||||
let message = if has_text && has_html {
|
||||
builder.multipart(MultiPart::alternative_plain_html(text, html))
|
||||
} else if has_html {
|
||||
builder.singlepart(SinglePart::html(html))
|
||||
} else {
|
||||
builder.singlepart(SinglePart::plain(text))
|
||||
}
|
||||
.map_err(|e| EmailError::Transport(e.to_string()))?;
|
||||
Ok(message)
|
||||
}
|
||||
|
||||
fn non_empty(addrs: &[String]) -> impl Iterator<Item = &String> {
|
||||
addrs.iter().filter(|a| !a.trim().is_empty())
|
||||
}
|
||||
|
||||
/// Hand-rolled RFC 5322-ish address check, then a `lettre::Mailbox`
|
||||
/// parse (the authoritative validator). We do NOT check deliverability —
|
||||
/// that's the SMTP layer's job.
|
||||
fn parse_address(addr: &str) -> Result<Mailbox, EmailError> {
|
||||
let trimmed = addr.trim();
|
||||
if trimmed.is_empty() {
|
||||
return Err(EmailError::InvalidAddress("empty address".into()));
|
||||
}
|
||||
if trimmed.len() > ADDRESS_MAX_LEN {
|
||||
return Err(EmailError::InvalidAddress(format!(
|
||||
"address exceeds {ADDRESS_MAX_LEN} bytes"
|
||||
)));
|
||||
}
|
||||
// Must have a single-ish @ with a non-empty local part and a domain
|
||||
// that contains a dot (rejects "a@b" and bare tokens).
|
||||
match trimmed.rsplit_once('@') {
|
||||
Some((local, domain)) if !local.is_empty() && domain.contains('.') => {}
|
||||
_ => {
|
||||
return Err(EmailError::InvalidAddress(format!(
|
||||
"{trimmed:?} is not a valid email address"
|
||||
)))
|
||||
}
|
||||
}
|
||||
trimmed.parse::<Mailbox>().map_err(|_| {
|
||||
EmailError::InvalidAddress(format!("{trimmed:?} is not a valid email address"))
|
||||
})
|
||||
}
|
||||
|
||||
// ----------------------------------------------------------------------------
|
||||
// Tests — recording transport so unit tests need no live SMTP server.
|
||||
// ----------------------------------------------------------------------------
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::authz::{AuthzError, AuthzRepo};
|
||||
use async_trait::async_trait;
|
||||
use picloud_shared::{
|
||||
AdminUserId, AppId, AppRole, ExecutionId, InstanceRole, Principal, RequestId, ScriptId,
|
||||
UserId,
|
||||
};
|
||||
use std::sync::Mutex as StdMutex;
|
||||
|
||||
#[derive(Default)]
|
||||
struct RecordingTransport {
|
||||
sent: StdMutex<Vec<Vec<u8>>>,
|
||||
}
|
||||
#[async_trait]
|
||||
impl EmailTransport for RecordingTransport {
|
||||
async fn send(&self, message: &Message) -> Result<(), EmailError> {
|
||||
self.sent.lock().unwrap().push(message.formatted());
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Default)]
|
||||
struct DenyAuthz;
|
||||
#[async_trait]
|
||||
impl AuthzRepo for DenyAuthz {
|
||||
async fn membership(&self, _: UserId, _: AppId) -> Result<Option<AppRole>, AuthzError> {
|
||||
Ok(None)
|
||||
}
|
||||
}
|
||||
|
||||
struct GrantAuthz {
|
||||
app: AppId,
|
||||
role: AppRole,
|
||||
}
|
||||
#[async_trait]
|
||||
impl AuthzRepo for GrantAuthz {
|
||||
async fn membership(
|
||||
&self,
|
||||
_: UserId,
|
||||
app_id: AppId,
|
||||
) -> Result<Option<AppRole>, AuthzError> {
|
||||
Ok((app_id == self.app).then_some(self.role))
|
||||
}
|
||||
}
|
||||
|
||||
fn svc_with(
|
||||
transport: Option<Arc<dyn EmailTransport>>,
|
||||
authz: Arc<dyn AuthzRepo>,
|
||||
) -> EmailServiceImpl {
|
||||
EmailServiceImpl::new(transport, authz, EmailConfig::conservative())
|
||||
}
|
||||
|
||||
fn recording() -> (EmailServiceImpl, Arc<RecordingTransport>) {
|
||||
let rec = Arc::new(RecordingTransport::default());
|
||||
let svc = svc_with(Some(rec.clone()), Arc::new(DenyAuthz));
|
||||
(svc, rec)
|
||||
}
|
||||
|
||||
fn cx_with(app_id: AppId, principal: Option<Principal>) -> SdkCallCx {
|
||||
SdkCallCx {
|
||||
app_id,
|
||||
script_id: ScriptId::new(),
|
||||
principal,
|
||||
execution_id: ExecutionId::new(),
|
||||
request_id: RequestId::new(),
|
||||
trigger_depth: 0,
|
||||
root_execution_id: ExecutionId::new(),
|
||||
is_dead_letter_handler: false,
|
||||
event: None,
|
||||
}
|
||||
}
|
||||
|
||||
fn anon(app: AppId) -> SdkCallCx {
|
||||
cx_with(app, None)
|
||||
}
|
||||
|
||||
fn principal(role: InstanceRole) -> Principal {
|
||||
Principal {
|
||||
user_id: AdminUserId::new(),
|
||||
instance_role: role,
|
||||
scopes: None,
|
||||
app_binding: None,
|
||||
}
|
||||
}
|
||||
|
||||
fn base_email() -> OutboundEmail {
|
||||
OutboundEmail {
|
||||
to: vec!["alice@example.com".into()],
|
||||
from: "alerts@myapp.com".into(),
|
||||
subject: "Build complete".into(),
|
||||
text: Some("Your deploy finished.".into()),
|
||||
..Default::default()
|
||||
}
|
||||
}
|
||||
|
||||
fn last_message(rec: &RecordingTransport) -> String {
|
||||
let g = rec.sent.lock().unwrap();
|
||||
String::from_utf8_lossy(g.last().expect("a message was sent")).into_owned()
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn send_text_includes_headers_and_body() {
|
||||
let (svc, rec) = recording();
|
||||
svc.send(&anon(AppId::new()), base_email()).await.unwrap();
|
||||
let msg = last_message(&rec);
|
||||
assert!(msg.contains("To: alice@example.com"), "{msg}");
|
||||
assert!(msg.contains("From: alerts@myapp.com"), "{msg}");
|
||||
assert!(msg.contains("Subject: Build complete"), "{msg}");
|
||||
assert!(msg.contains("Your deploy finished."), "{msg}");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn send_html_is_multipart_with_both_parts() {
|
||||
let (svc, rec) = recording();
|
||||
let mut e = base_email();
|
||||
e.text = Some("plain fallback".into());
|
||||
e.html = Some("<p>rich <b>body</b></p>".into());
|
||||
svc.send(&anon(AppId::new()), e).await.unwrap();
|
||||
let msg = last_message(&rec);
|
||||
assert!(msg.contains("multipart/alternative"), "{msg}");
|
||||
assert!(msg.contains("plain fallback"), "{msg}");
|
||||
// HTML part is quoted-printable encoded, but the tag survives.
|
||||
assert!(msg.contains("text/html"), "{msg}");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn multiple_recipients_and_cc_bcc() {
|
||||
let (svc, rec) = recording();
|
||||
let mut e = base_email();
|
||||
e.to = vec!["alice@x.com".into(), "bob@y.com".into()];
|
||||
e.cc = vec!["dave@z.com".into()];
|
||||
e.bcc = vec!["audit@myapp.com".into()];
|
||||
svc.send(&anon(AppId::new()), e).await.unwrap();
|
||||
let msg = last_message(&rec);
|
||||
assert!(
|
||||
msg.contains("alice@x.com") && msg.contains("bob@y.com"),
|
||||
"{msg}"
|
||||
);
|
||||
assert!(msg.contains("Cc: dave@z.com"), "{msg}");
|
||||
// Bcc is intentionally NOT serialized into the visible headers.
|
||||
assert!(
|
||||
!msg.contains("Bcc:"),
|
||||
"bcc must not appear in headers: {msg}"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn reply_to_populated() {
|
||||
let (svc, rec) = recording();
|
||||
let mut e = base_email();
|
||||
e.reply_to = Some("support@myapp.com".into());
|
||||
svc.send(&anon(AppId::new()), e).await.unwrap();
|
||||
assert!(last_message(&rec).contains("Reply-To: support@myapp.com"));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn missing_required_field_throws() {
|
||||
let (svc, _) = recording();
|
||||
let mut e = base_email();
|
||||
e.subject = String::new();
|
||||
let err = svc.send(&anon(AppId::new()), e).await.unwrap_err();
|
||||
assert!(matches!(err, EmailError::MissingField(f) if f == "subject"));
|
||||
|
||||
let (svc, _) = recording();
|
||||
let mut e = base_email();
|
||||
e.text = None;
|
||||
e.html = None;
|
||||
let err = svc.send(&anon(AppId::new()), e).await.unwrap_err();
|
||||
assert!(matches!(err, EmailError::MissingField(_)));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn invalid_address_throws() {
|
||||
let (svc, _) = recording();
|
||||
let mut e = base_email();
|
||||
e.to = vec!["not-an-email".into()];
|
||||
let err = svc.send(&anon(AppId::new()), e).await.unwrap_err();
|
||||
assert!(matches!(err, EmailError::InvalidAddress(_)));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn message_size_cap_enforced() {
|
||||
let rec = Arc::new(RecordingTransport::default());
|
||||
let svc = EmailServiceImpl::new(
|
||||
Some(rec),
|
||||
Arc::new(DenyAuthz),
|
||||
EmailConfig {
|
||||
max_message_bytes: 64,
|
||||
},
|
||||
);
|
||||
let err = svc
|
||||
.send(&anon(AppId::new()), base_email())
|
||||
.await
|
||||
.unwrap_err();
|
||||
assert!(matches!(err, EmailError::TooLarge { limit: 64, .. }));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn not_configured_throws() {
|
||||
let svc = svc_with(None, Arc::new(DenyAuthz));
|
||||
let err = svc
|
||||
.send(&anon(AppId::new()), base_email())
|
||||
.await
|
||||
.unwrap_err();
|
||||
assert!(matches!(err, EmailError::NotConfigured));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn anonymous_skips_authz() {
|
||||
// DenyAuthz would deny an authed principal; anon skips the check.
|
||||
let (svc, _) = recording();
|
||||
svc.send(&anon(AppId::new()), base_email()).await.unwrap();
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn member_with_editor_role_allowed() {
|
||||
let app = AppId::new();
|
||||
let rec = Arc::new(RecordingTransport::default());
|
||||
let svc = svc_with(
|
||||
Some(rec),
|
||||
Arc::new(GrantAuthz {
|
||||
app,
|
||||
role: AppRole::Editor,
|
||||
}),
|
||||
);
|
||||
let cx = cx_with(app, Some(principal(InstanceRole::Member)));
|
||||
svc.send(&cx, base_email()).await.unwrap();
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn member_without_role_forbidden() {
|
||||
let (svc, _) = recording();
|
||||
let cx = cx_with(AppId::new(), Some(principal(InstanceRole::Member)));
|
||||
let err = svc.send(&cx, base_email()).await.unwrap_err();
|
||||
assert!(matches!(err, EmailError::Forbidden));
|
||||
}
|
||||
}
|
||||
@@ -31,6 +31,8 @@ pub mod dispatcher;
|
||||
pub mod docs_filter;
|
||||
pub mod docs_repo;
|
||||
pub mod docs_service;
|
||||
pub mod email_inbound_api;
|
||||
pub mod email_service;
|
||||
pub mod files_api;
|
||||
pub mod files_repo;
|
||||
pub mod files_service;
|
||||
@@ -53,6 +55,9 @@ pub mod route_admin;
|
||||
pub mod route_repo;
|
||||
pub mod sandbox;
|
||||
pub mod scheduler;
|
||||
pub mod secrets_api;
|
||||
pub mod secrets_repo;
|
||||
pub mod secrets_service;
|
||||
pub mod ssrf;
|
||||
pub mod topic_repo;
|
||||
pub mod topics_api;
|
||||
@@ -109,6 +114,11 @@ pub use dead_letters_api::{dead_letters_router, DeadLettersApiError, DeadLetters
|
||||
pub use dispatcher::{compute_backoff, Dispatcher, DispatcherError};
|
||||
pub use docs_repo::{DocsRepo, DocsRepoError, PostgresDocsRepo};
|
||||
pub use docs_service::DocsServiceImpl;
|
||||
pub use email_inbound_api::{email_inbound_router, EmailInboundError, EmailInboundState};
|
||||
pub use email_service::{
|
||||
EmailConfig, EmailServiceImpl, EmailTransport, LettreEmailTransport, SmtpConfig, SmtpTls,
|
||||
DEFAULT_EMAIL_MAX_MESSAGE_BYTES,
|
||||
};
|
||||
pub use files_api::{files_admin_router, FilesAdminState};
|
||||
pub use files_repo::{FilesConfig, FilesRepo, FilesRepoError, FsFilesRepo};
|
||||
pub use files_service::FilesServiceImpl;
|
||||
@@ -134,13 +144,22 @@ pub use repo::{
|
||||
pub use route_admin::{compile_routes, route_admin_router, RouteAdminState};
|
||||
pub use route_repo::{NewRoute, PostgresRouteRepository, RouteRepository};
|
||||
pub use sandbox::{CeilingError, SandboxCeiling};
|
||||
pub use secrets_api::{secrets_router, SecretsApiError, SecretsState};
|
||||
pub use secrets_repo::{
|
||||
PostgresSecretsRepo, SecretMeta, SecretsMetaPage, SecretsNamePage, SecretsRepo,
|
||||
SecretsRepoError, StoredSecret,
|
||||
};
|
||||
pub use secrets_service::{
|
||||
open as open_secret, seal as seal_secret, SecretsConfig, SecretsServiceImpl,
|
||||
DEFAULT_SECRET_MAX_VALUE_BYTES,
|
||||
};
|
||||
pub use topic_repo::{PostgresTopicRepo, Topic, TopicAuthMode, TopicRepo, TopicRepoError};
|
||||
pub use topics_api::{topics_router, TopicsApiError, TopicsState};
|
||||
pub use trigger_config::{BackoffShape, TriggerConfig};
|
||||
pub use trigger_repo::{
|
||||
collection_matches, CreateDeadLetterTrigger, CreateDocsTrigger, CreateFilesTrigger,
|
||||
CreateKvTrigger, CreatePubsubTrigger, DeadLetterTriggerMatch, DocsTriggerMatch,
|
||||
FilesTriggerMatch, KvTriggerMatch, PostgresTriggerRepo, Trigger, TriggerDetails,
|
||||
TriggerDispatchMode, TriggerKind, TriggerRepo, TriggerRepoError,
|
||||
collection_matches, CreateDeadLetterTrigger, CreateDocsTrigger, CreateEmailTrigger,
|
||||
CreateFilesTrigger, CreateKvTrigger, CreatePubsubTrigger, DeadLetterTriggerMatch,
|
||||
DocsTriggerMatch, EmailInboundTarget, FilesTriggerMatch, KvTriggerMatch, PostgresTriggerRepo,
|
||||
Trigger, TriggerDetails, TriggerDispatchMode, TriggerKind, TriggerRepo, TriggerRepoError,
|
||||
};
|
||||
pub use triggers_api::{triggers_router, TriggersApiError, TriggersState};
|
||||
|
||||
@@ -31,6 +31,8 @@ pub enum OutboxSourceKind {
|
||||
Files,
|
||||
/// v1.1.5.
|
||||
Pubsub,
|
||||
/// v1.1.7. Inbound email POSTed to the webhook receiver.
|
||||
Email,
|
||||
}
|
||||
|
||||
impl OutboxSourceKind {
|
||||
@@ -44,6 +46,7 @@ impl OutboxSourceKind {
|
||||
Self::Cron => "cron",
|
||||
Self::Files => "files",
|
||||
Self::Pubsub => "pubsub",
|
||||
Self::Email => "email",
|
||||
}
|
||||
}
|
||||
|
||||
@@ -57,6 +60,7 @@ impl OutboxSourceKind {
|
||||
"cron" => Some(Self::Cron),
|
||||
"files" => Some(Self::Files),
|
||||
"pubsub" => Some(Self::Pubsub),
|
||||
"email" => Some(Self::Email),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -254,7 +254,7 @@ impl PubsubService for PubsubServiceImpl {
|
||||
.get(cx.app_id, name)
|
||||
.await
|
||||
.map_err(|e| PubsubError::Unavailable(e.to_string()))?;
|
||||
if !registered.map(|t| t.external_subscribable).unwrap_or(false) {
|
||||
if !registered.is_some_and(|t| t.external_subscribable) {
|
||||
return Err(PubsubError::SubscriberToken(format!(
|
||||
"pubsub::subscriber_token: topic {name} is not externally subscribable"
|
||||
)));
|
||||
|
||||
232
crates/manager-core/src/secrets_api.rs
Normal file
232
crates/manager-core/src/secrets_api.rs
Normal file
@@ -0,0 +1,232 @@
|
||||
//! `/api/v1/admin/apps/{id}/secrets*` — secrets admin endpoints
|
||||
//! (v1.1.7).
|
||||
//!
|
||||
//! * `GET /apps/{id}/secrets` — list names + updated_at
|
||||
//! (NEVER values).
|
||||
//! * `POST /apps/{id}/secrets` — set/overwrite a secret.
|
||||
//! * `DELETE /apps/{id}/secrets/{name}` — delete a secret.
|
||||
//!
|
||||
//! Set/delete are gated by `AppSecretsWrite` (→ `script:write`); list by
|
||||
//! `AppSecretsRead` (→ `script:read`). The list surface deliberately
|
||||
//! returns only names + timestamps — the dashboard never receives
|
||||
//! plaintext. Values are encrypted with the process master key before
|
||||
//! they touch the database (same envelope as the script `secrets::set`).
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use axum::extract::{Path, Query, State};
|
||||
use axum::http::StatusCode;
|
||||
use axum::response::{IntoResponse, Json, Response};
|
||||
use axum::routing::get;
|
||||
use axum::{Extension, Router};
|
||||
use picloud_shared::{validate_secret_name, AppId, MasterKey, Principal, SecretsError};
|
||||
use serde::Deserialize;
|
||||
use serde_json::json;
|
||||
|
||||
use crate::app_repo::AppRepository;
|
||||
use crate::authz::{require, AuthzDenied, AuthzError, AuthzRepo, Capability};
|
||||
use crate::secrets_repo::{SecretsRepo, SecretsRepoError};
|
||||
use crate::secrets_service::seal;
|
||||
|
||||
#[derive(Clone)]
|
||||
pub struct SecretsState {
|
||||
pub repo: Arc<dyn SecretsRepo>,
|
||||
pub apps: Arc<dyn AppRepository>,
|
||||
pub authz: Arc<dyn AuthzRepo>,
|
||||
pub master_key: MasterKey,
|
||||
pub max_value_bytes: usize,
|
||||
}
|
||||
|
||||
pub fn secrets_router(state: SecretsState) -> Router {
|
||||
Router::new()
|
||||
.route("/apps/{app_id}/secrets", get(list_secrets).post(set_secret))
|
||||
.route(
|
||||
"/apps/{app_id}/secrets/{name}",
|
||||
axum::routing::delete(delete_secret),
|
||||
)
|
||||
.with_state(state)
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize)]
|
||||
pub struct ListQuery {
|
||||
#[serde(default)]
|
||||
pub cursor: Option<String>,
|
||||
#[serde(default)]
|
||||
pub limit: Option<u32>,
|
||||
}
|
||||
|
||||
#[derive(Debug, serde::Serialize)]
|
||||
struct SecretItem {
|
||||
name: String,
|
||||
updated_at: chrono::DateTime<chrono::Utc>,
|
||||
}
|
||||
|
||||
#[derive(Debug, serde::Serialize)]
|
||||
struct ListSecretsResponse {
|
||||
secrets: Vec<SecretItem>,
|
||||
next_cursor: Option<String>,
|
||||
}
|
||||
|
||||
async fn list_secrets(
|
||||
State(s): State<SecretsState>,
|
||||
Extension(principal): Extension<Principal>,
|
||||
Path(app_id): Path<AppId>,
|
||||
Query(q): Query<ListQuery>,
|
||||
) -> Result<Json<ListSecretsResponse>, SecretsApiError> {
|
||||
ensure_app_exists(&*s.apps, app_id).await?;
|
||||
require(
|
||||
s.authz.as_ref(),
|
||||
&principal,
|
||||
Capability::AppSecretsRead(app_id),
|
||||
)
|
||||
.await?;
|
||||
let page = s
|
||||
.repo
|
||||
.list_meta(app_id, q.cursor.as_deref(), q.limit.unwrap_or(0))
|
||||
.await?;
|
||||
Ok(Json(ListSecretsResponse {
|
||||
secrets: page
|
||||
.items
|
||||
.into_iter()
|
||||
.map(|m| SecretItem {
|
||||
name: m.name,
|
||||
updated_at: m.updated_at,
|
||||
})
|
||||
.collect(),
|
||||
next_cursor: page.next_cursor,
|
||||
}))
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize)]
|
||||
pub struct SetSecretRequest {
|
||||
pub name: String,
|
||||
/// Any JSON value — the dashboard sends a single-line string, but
|
||||
/// maps/arrays/numbers round-trip too (matching `secrets::set`).
|
||||
pub value: serde_json::Value,
|
||||
}
|
||||
|
||||
async fn set_secret(
|
||||
State(s): State<SecretsState>,
|
||||
Extension(principal): Extension<Principal>,
|
||||
Path(app_id): Path<AppId>,
|
||||
Json(input): Json<SetSecretRequest>,
|
||||
) -> Result<StatusCode, SecretsApiError> {
|
||||
ensure_app_exists(&*s.apps, app_id).await?;
|
||||
require(
|
||||
s.authz.as_ref(),
|
||||
&principal,
|
||||
Capability::AppSecretsWrite(app_id),
|
||||
)
|
||||
.await?;
|
||||
validate_secret_name(&input.name)?;
|
||||
let (ciphertext, nonce) = seal(&s.master_key, &input.value, s.max_value_bytes)?;
|
||||
s.repo.set(app_id, &input.name, &ciphertext, &nonce).await?;
|
||||
Ok(StatusCode::NO_CONTENT)
|
||||
}
|
||||
|
||||
async fn delete_secret(
|
||||
State(s): State<SecretsState>,
|
||||
Extension(principal): Extension<Principal>,
|
||||
Path((app_id, name)): Path<(AppId, String)>,
|
||||
) -> Result<StatusCode, SecretsApiError> {
|
||||
ensure_app_exists(&*s.apps, app_id).await?;
|
||||
require(
|
||||
s.authz.as_ref(),
|
||||
&principal,
|
||||
Capability::AppSecretsWrite(app_id),
|
||||
)
|
||||
.await?;
|
||||
if !s.repo.delete(app_id, &name).await? {
|
||||
return Err(SecretsApiError::NotFound);
|
||||
}
|
||||
Ok(StatusCode::NO_CONTENT)
|
||||
}
|
||||
|
||||
async fn ensure_app_exists(apps: &dyn AppRepository, app_id: AppId) -> Result<(), SecretsApiError> {
|
||||
apps.get_by_id(app_id)
|
||||
.await
|
||||
.map_err(|e| SecretsApiError::Backend(e.to_string()))?
|
||||
.ok_or(SecretsApiError::AppNotFound)?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub enum SecretsApiError {
|
||||
#[error("app not found")]
|
||||
AppNotFound,
|
||||
#[error("secret not found")]
|
||||
NotFound,
|
||||
#[error("invalid request: {0}")]
|
||||
Invalid(String),
|
||||
#[error("forbidden")]
|
||||
Forbidden,
|
||||
#[error("authorization repo error: {0}")]
|
||||
AuthzRepo(String),
|
||||
#[error("secrets backend: {0}")]
|
||||
Backend(String),
|
||||
}
|
||||
|
||||
impl From<AuthzDenied> for SecretsApiError {
|
||||
fn from(d: AuthzDenied) -> Self {
|
||||
match d {
|
||||
AuthzDenied::Denied => Self::Forbidden,
|
||||
AuthzDenied::Repo(e) => Self::AuthzRepo(e.to_string()),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl From<AuthzError> for SecretsApiError {
|
||||
fn from(e: AuthzError) -> Self {
|
||||
Self::AuthzRepo(e.to_string())
|
||||
}
|
||||
}
|
||||
|
||||
impl From<SecretsRepoError> for SecretsApiError {
|
||||
fn from(e: SecretsRepoError) -> Self {
|
||||
match e {
|
||||
SecretsRepoError::InvalidCursor => Self::Invalid("invalid pagination cursor".into()),
|
||||
SecretsRepoError::Db(e) => Self::Backend(e.to_string()),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl From<SecretsError> for SecretsApiError {
|
||||
fn from(e: SecretsError) -> Self {
|
||||
match e {
|
||||
SecretsError::InvalidName(m) => Self::Invalid(m),
|
||||
SecretsError::TooLarge { .. } => Self::Invalid(e.to_string()),
|
||||
SecretsError::Forbidden => Self::Forbidden,
|
||||
other => Self::Backend(other.to_string()),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl IntoResponse for SecretsApiError {
|
||||
fn into_response(self) -> Response {
|
||||
let (status, body) = match &self {
|
||||
Self::AppNotFound | Self::NotFound => {
|
||||
(StatusCode::NOT_FOUND, json!({ "error": self.to_string() }))
|
||||
}
|
||||
Self::Invalid(_) => (
|
||||
StatusCode::UNPROCESSABLE_ENTITY,
|
||||
json!({ "error": self.to_string() }),
|
||||
),
|
||||
Self::Forbidden => (StatusCode::FORBIDDEN, json!({ "error": self.to_string() })),
|
||||
Self::AuthzRepo(e) => {
|
||||
tracing::error!(error = %e, "secrets admin authz repo error");
|
||||
(
|
||||
StatusCode::INTERNAL_SERVER_ERROR,
|
||||
json!({ "error": "internal error" }),
|
||||
)
|
||||
}
|
||||
Self::Backend(e) => {
|
||||
tracing::error!(error = %e, "secrets admin backend error");
|
||||
(
|
||||
StatusCode::INTERNAL_SERVER_ERROR,
|
||||
json!({ "error": "internal error" }),
|
||||
)
|
||||
}
|
||||
};
|
||||
(status, Json(body)).into_response()
|
||||
}
|
||||
}
|
||||
246
crates/manager-core/src/secrets_repo.rs
Normal file
246
crates/manager-core/src/secrets_repo.rs
Normal file
@@ -0,0 +1,246 @@
|
||||
//! Low-level Postgres CRUD over `secrets`. Storage-only: it moves
|
||||
//! opaque ciphertext + nonce blobs in and out. Encryption, JSON
|
||||
//! encoding, authorization, name validation, and the value-size cap all
|
||||
//! live one layer up in `SecretsServiceImpl` / `secrets_api`.
|
||||
|
||||
use async_trait::async_trait;
|
||||
use base64::engine::general_purpose::URL_SAFE_NO_PAD;
|
||||
use base64::Engine as _;
|
||||
use chrono::{DateTime, Utc};
|
||||
use picloud_shared::AppId;
|
||||
use sqlx::PgPool;
|
||||
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub enum SecretsRepoError {
|
||||
#[error("database error: {0}")]
|
||||
Db(#[from] sqlx::Error),
|
||||
|
||||
#[error("invalid pagination cursor")]
|
||||
InvalidCursor,
|
||||
}
|
||||
|
||||
/// An encrypted secret as it lives on disk: ciphertext (auth tag
|
||||
/// appended) plus the nonce it was sealed with.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct StoredSecret {
|
||||
pub encrypted_value: Vec<u8>,
|
||||
pub nonce: Vec<u8>,
|
||||
}
|
||||
|
||||
/// Admin-surface metadata for one secret. Values are never returned —
|
||||
/// only the name and the last-modified timestamp.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct SecretMeta {
|
||||
pub name: String,
|
||||
pub updated_at: DateTime<Utc>,
|
||||
}
|
||||
|
||||
/// One page of names (SDK `list`).
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct SecretsNamePage {
|
||||
pub names: Vec<String>,
|
||||
pub next_cursor: Option<String>,
|
||||
}
|
||||
|
||||
/// One page of name + updated_at (admin `GET`).
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct SecretsMetaPage {
|
||||
pub items: Vec<SecretMeta>,
|
||||
pub next_cursor: Option<String>,
|
||||
}
|
||||
|
||||
/// Repo surface. Exposed as a trait so the service unit tests can
|
||||
/// substitute an in-memory backing without Postgres.
|
||||
#[async_trait]
|
||||
pub trait SecretsRepo: Send + Sync {
|
||||
async fn get(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
name: &str,
|
||||
) -> Result<Option<StoredSecret>, SecretsRepoError>;
|
||||
|
||||
/// Upsert (overwrite if present).
|
||||
async fn set(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
name: &str,
|
||||
encrypted_value: &[u8],
|
||||
nonce: &[u8],
|
||||
) -> Result<(), SecretsRepoError>;
|
||||
|
||||
/// Delete; returns whether a row was present.
|
||||
async fn delete(&self, app_id: AppId, name: &str) -> Result<bool, SecretsRepoError>;
|
||||
|
||||
/// Names only — the SDK `list` surface.
|
||||
async fn list_names(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
cursor: Option<&str>,
|
||||
limit: u32,
|
||||
) -> Result<SecretsNamePage, SecretsRepoError>;
|
||||
|
||||
/// Name + updated_at — the admin `GET` surface.
|
||||
async fn list_meta(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
cursor: Option<&str>,
|
||||
limit: u32,
|
||||
) -> Result<SecretsMetaPage, SecretsRepoError>;
|
||||
}
|
||||
|
||||
pub struct PostgresSecretsRepo {
|
||||
pool: PgPool,
|
||||
}
|
||||
|
||||
impl PostgresSecretsRepo {
|
||||
#[must_use]
|
||||
pub fn new(pool: PgPool) -> Self {
|
||||
Self { pool }
|
||||
}
|
||||
}
|
||||
|
||||
const SECRETS_LIST_MAX_LIMIT: u32 = 1_000;
|
||||
const SECRETS_LIST_DEFAULT_LIMIT: u32 = 100;
|
||||
|
||||
fn clamp_limit(limit: u32) -> u32 {
|
||||
if limit == 0 {
|
||||
SECRETS_LIST_DEFAULT_LIMIT
|
||||
} else {
|
||||
limit.min(SECRETS_LIST_MAX_LIMIT)
|
||||
}
|
||||
}
|
||||
|
||||
/// Opaque keyset cursor: base64url of the last `name` returned.
|
||||
pub(crate) fn encode_cursor(last_name: &str) -> String {
|
||||
URL_SAFE_NO_PAD.encode(last_name.as_bytes())
|
||||
}
|
||||
|
||||
pub(crate) fn decode_cursor(cursor: &str) -> Result<String, SecretsRepoError> {
|
||||
let bytes = URL_SAFE_NO_PAD
|
||||
.decode(cursor)
|
||||
.map_err(|_| SecretsRepoError::InvalidCursor)?;
|
||||
String::from_utf8(bytes).map_err(|_| SecretsRepoError::InvalidCursor)
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl SecretsRepo for PostgresSecretsRepo {
|
||||
async fn get(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
name: &str,
|
||||
) -> Result<Option<StoredSecret>, SecretsRepoError> {
|
||||
let row: Option<(Vec<u8>, Vec<u8>)> = sqlx::query_as(
|
||||
"SELECT encrypted_value, nonce FROM secrets WHERE app_id = $1 AND name = $2",
|
||||
)
|
||||
.bind(app_id.into_inner())
|
||||
.bind(name)
|
||||
.fetch_optional(&self.pool)
|
||||
.await?;
|
||||
Ok(row.map(|(encrypted_value, nonce)| StoredSecret {
|
||||
encrypted_value,
|
||||
nonce,
|
||||
}))
|
||||
}
|
||||
|
||||
async fn set(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
name: &str,
|
||||
encrypted_value: &[u8],
|
||||
nonce: &[u8],
|
||||
) -> Result<(), SecretsRepoError> {
|
||||
sqlx::query(
|
||||
"INSERT INTO secrets (app_id, name, encrypted_value, nonce) \
|
||||
VALUES ($1, $2, $3, $4) \
|
||||
ON CONFLICT (app_id, name) DO UPDATE \
|
||||
SET encrypted_value = EXCLUDED.encrypted_value, \
|
||||
nonce = EXCLUDED.nonce, \
|
||||
updated_at = NOW()",
|
||||
)
|
||||
.bind(app_id.into_inner())
|
||||
.bind(name)
|
||||
.bind(encrypted_value)
|
||||
.bind(nonce)
|
||||
.execute(&self.pool)
|
||||
.await?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn delete(&self, app_id: AppId, name: &str) -> Result<bool, SecretsRepoError> {
|
||||
let res = sqlx::query("DELETE FROM secrets WHERE app_id = $1 AND name = $2")
|
||||
.bind(app_id.into_inner())
|
||||
.bind(name)
|
||||
.execute(&self.pool)
|
||||
.await?;
|
||||
Ok(res.rows_affected() > 0)
|
||||
}
|
||||
|
||||
async fn list_names(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
cursor: Option<&str>,
|
||||
limit: u32,
|
||||
) -> Result<SecretsNamePage, SecretsRepoError> {
|
||||
let limit = clamp_limit(limit);
|
||||
let last_name = match cursor {
|
||||
Some(c) => Some(decode_cursor(c)?),
|
||||
None => None,
|
||||
};
|
||||
let take = i64::from(limit) + 1;
|
||||
let rows: Vec<(String,)> = sqlx::query_as(
|
||||
"SELECT name FROM secrets \
|
||||
WHERE app_id = $1 AND ($2::text IS NULL OR name > $2) \
|
||||
ORDER BY name ASC LIMIT $3",
|
||||
)
|
||||
.bind(app_id.into_inner())
|
||||
.bind(last_name.as_deref())
|
||||
.bind(take)
|
||||
.fetch_all(&self.pool)
|
||||
.await?;
|
||||
|
||||
let mut names: Vec<String> = rows.into_iter().map(|(n,)| n).collect();
|
||||
let next_cursor = if names.len() > limit as usize {
|
||||
names.truncate(limit as usize);
|
||||
names.last().map(|n| encode_cursor(n))
|
||||
} else {
|
||||
None
|
||||
};
|
||||
Ok(SecretsNamePage { names, next_cursor })
|
||||
}
|
||||
|
||||
async fn list_meta(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
cursor: Option<&str>,
|
||||
limit: u32,
|
||||
) -> Result<SecretsMetaPage, SecretsRepoError> {
|
||||
let limit = clamp_limit(limit);
|
||||
let last_name = match cursor {
|
||||
Some(c) => Some(decode_cursor(c)?),
|
||||
None => None,
|
||||
};
|
||||
let take = i64::from(limit) + 1;
|
||||
let rows: Vec<(String, DateTime<Utc>)> = sqlx::query_as(
|
||||
"SELECT name, updated_at FROM secrets \
|
||||
WHERE app_id = $1 AND ($2::text IS NULL OR name > $2) \
|
||||
ORDER BY name ASC LIMIT $3",
|
||||
)
|
||||
.bind(app_id.into_inner())
|
||||
.bind(last_name.as_deref())
|
||||
.bind(take)
|
||||
.fetch_all(&self.pool)
|
||||
.await?;
|
||||
|
||||
let mut items: Vec<SecretMeta> = rows
|
||||
.into_iter()
|
||||
.map(|(name, updated_at)| SecretMeta { name, updated_at })
|
||||
.collect();
|
||||
let next_cursor = if items.len() > limit as usize {
|
||||
items.truncate(limit as usize);
|
||||
items.last().map(|m| encode_cursor(&m.name))
|
||||
} else {
|
||||
None
|
||||
};
|
||||
Ok(SecretsMetaPage { items, next_cursor })
|
||||
}
|
||||
}
|
||||
574
crates/manager-core/src/secrets_service.rs
Normal file
574
crates/manager-core/src/secrets_service.rs
Normal file
@@ -0,0 +1,574 @@
|
||||
//! `SecretsServiceImpl` — wires the `SecretsRepo` underneath the
|
||||
//! `picloud_shared::SecretsService` trait that scripts see via the Rhai
|
||||
//! bridge.
|
||||
//!
|
||||
//! Layers added here (vs the raw repo):
|
||||
//!
|
||||
//! 1. Name validation (non-empty, ≤255 bytes) at the SDK boundary.
|
||||
//! 2. **Script-as-gate authz**: when `cx.principal.is_some()` we run
|
||||
//! `authz::require(...)`; when it's `None` (public unauthenticated
|
||||
//! HTTP) we skip the check. Cross-app isolation is unaffected — every
|
||||
//! query is keyed by `cx.app_id`, never an argument.
|
||||
//! 3. **JSON ⇄ ciphertext**: `set` serializes the value to JSON bytes,
|
||||
//! enforces the per-secret size cap, and AES-256-GCM-seals it; `get`
|
||||
//! decrypts and deserializes back to the same JSON shape (a String
|
||||
//! round-trips to a String, not a JSON-quoted `"\"…\""`).
|
||||
//!
|
||||
//! Deliberately **no `ServiceEvent` emission** — secret writes do not
|
||||
//! fire triggers (footgun avoidance; see `docs/sdk-shape.md` + the
|
||||
//! v1.1.7 brief §2).
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use async_trait::async_trait;
|
||||
use picloud_shared::{
|
||||
crypto, validate_secret_name, MasterKey, SdkCallCx, SecretsError, SecretsListPage,
|
||||
SecretsService,
|
||||
};
|
||||
|
||||
use crate::authz::{self, AuthzRepo, Capability};
|
||||
use crate::secrets_repo::{SecretsRepo, SecretsRepoError, StoredSecret};
|
||||
|
||||
/// Default per-secret plaintext cap (64 KB). Override with
|
||||
/// `PICLOUD_SECRET_MAX_VALUE_BYTES`.
|
||||
pub const DEFAULT_SECRET_MAX_VALUE_BYTES: usize = 64 * 1024;
|
||||
|
||||
/// Process config for the secrets service.
|
||||
#[derive(Debug, Clone, Copy)]
|
||||
pub struct SecretsConfig {
|
||||
/// Maximum size of the JSON-encoded plaintext, in bytes.
|
||||
pub max_value_bytes: usize,
|
||||
}
|
||||
|
||||
impl SecretsConfig {
|
||||
#[must_use]
|
||||
pub const fn conservative() -> Self {
|
||||
Self {
|
||||
max_value_bytes: DEFAULT_SECRET_MAX_VALUE_BYTES,
|
||||
}
|
||||
}
|
||||
|
||||
/// Read `PICLOUD_SECRET_MAX_VALUE_BYTES`; invalid values are ignored
|
||||
/// with a warning (keeps the conservative default).
|
||||
#[must_use]
|
||||
pub fn from_env() -> Self {
|
||||
let mut c = Self::conservative();
|
||||
if let Ok(v) = std::env::var("PICLOUD_SECRET_MAX_VALUE_BYTES") {
|
||||
match v.trim().parse::<usize>() {
|
||||
Ok(n) if n > 0 => c.max_value_bytes = n,
|
||||
_ => tracing::warn!(
|
||||
value = %v,
|
||||
"ignoring invalid PICLOUD_SECRET_MAX_VALUE_BYTES (want a positive integer)"
|
||||
),
|
||||
}
|
||||
}
|
||||
c
|
||||
}
|
||||
}
|
||||
|
||||
impl Default for SecretsConfig {
|
||||
fn default() -> Self {
|
||||
Self::conservative()
|
||||
}
|
||||
}
|
||||
|
||||
/// Serialize + size-check + encrypt a value into `(ciphertext, nonce)`.
|
||||
///
|
||||
/// # Errors
|
||||
///
|
||||
/// [`SecretsError::TooLarge`] when the encoded plaintext exceeds
|
||||
/// `max_value_bytes`; [`SecretsError::Backend`] on a serialization
|
||||
/// failure (should not happen for a `serde_json::Value`).
|
||||
pub fn seal(
|
||||
master_key: &MasterKey,
|
||||
value: &serde_json::Value,
|
||||
max_value_bytes: usize,
|
||||
) -> Result<(Vec<u8>, [u8; crypto::NONCE_LEN]), SecretsError> {
|
||||
let plaintext = serde_json::to_vec(value)
|
||||
.map_err(|e| SecretsError::Backend(format!("encode secret value: {e}")))?;
|
||||
if plaintext.len() > max_value_bytes {
|
||||
return Err(SecretsError::TooLarge {
|
||||
limit: max_value_bytes,
|
||||
actual: plaintext.len(),
|
||||
});
|
||||
}
|
||||
let enc = crypto::encrypt(&plaintext, master_key.as_bytes());
|
||||
Ok((enc.ciphertext, enc.nonce))
|
||||
}
|
||||
|
||||
/// Decrypt + deserialize a stored secret back to its JSON value.
|
||||
///
|
||||
/// # Errors
|
||||
///
|
||||
/// [`SecretsError::Corrupted`] when decryption or JSON decoding fails.
|
||||
pub fn open(
|
||||
master_key: &MasterKey,
|
||||
stored: &StoredSecret,
|
||||
) -> Result<serde_json::Value, SecretsError> {
|
||||
let plaintext = crypto::decrypt(
|
||||
&stored.encrypted_value,
|
||||
&stored.nonce,
|
||||
master_key.as_bytes(),
|
||||
)
|
||||
.map_err(|_| SecretsError::Corrupted)?;
|
||||
serde_json::from_slice(&plaintext).map_err(|_| SecretsError::Corrupted)
|
||||
}
|
||||
|
||||
pub struct SecretsServiceImpl {
|
||||
repo: Arc<dyn SecretsRepo>,
|
||||
authz: Arc<dyn AuthzRepo>,
|
||||
master_key: MasterKey,
|
||||
max_value_bytes: usize,
|
||||
}
|
||||
|
||||
impl SecretsServiceImpl {
|
||||
#[must_use]
|
||||
pub fn new(
|
||||
repo: Arc<dyn SecretsRepo>,
|
||||
authz: Arc<dyn AuthzRepo>,
|
||||
master_key: MasterKey,
|
||||
config: SecretsConfig,
|
||||
) -> Self {
|
||||
Self {
|
||||
repo,
|
||||
authz,
|
||||
master_key,
|
||||
max_value_bytes: config.max_value_bytes,
|
||||
}
|
||||
}
|
||||
|
||||
async fn check_read(&self, cx: &SdkCallCx) -> Result<(), SecretsError> {
|
||||
if let Some(ref principal) = cx.principal {
|
||||
authz::require(
|
||||
&*self.authz,
|
||||
principal,
|
||||
Capability::AppSecretsRead(cx.app_id),
|
||||
)
|
||||
.await
|
||||
.map_err(|_| SecretsError::Forbidden)?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn check_write(&self, cx: &SdkCallCx) -> Result<(), SecretsError> {
|
||||
if let Some(ref principal) = cx.principal {
|
||||
authz::require(
|
||||
&*self.authz,
|
||||
principal,
|
||||
Capability::AppSecretsWrite(cx.app_id),
|
||||
)
|
||||
.await
|
||||
.map_err(|_| SecretsError::Forbidden)?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
impl From<SecretsRepoError> for SecretsError {
|
||||
fn from(e: SecretsRepoError) -> Self {
|
||||
Self::Backend(e.to_string())
|
||||
}
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl SecretsService for SecretsServiceImpl {
|
||||
async fn get(
|
||||
&self,
|
||||
cx: &SdkCallCx,
|
||||
name: &str,
|
||||
) -> Result<Option<serde_json::Value>, SecretsError> {
|
||||
validate_secret_name(name)?;
|
||||
self.check_read(cx).await?;
|
||||
let Some(stored) = self.repo.get(cx.app_id, name).await? else {
|
||||
return Ok(None);
|
||||
};
|
||||
match open(&self.master_key, &stored) {
|
||||
Ok(value) => Ok(Some(value)),
|
||||
Err(e) => {
|
||||
// A decrypt failure is operationally significant — surface
|
||||
// the affected (app_id, name) so an operator can find the
|
||||
// bad row, but never log the ciphertext or key material.
|
||||
tracing::error!(
|
||||
app_id = %cx.app_id,
|
||||
secret = %name,
|
||||
"secret could not be decrypted (corrupted row or master-key mismatch)"
|
||||
);
|
||||
Err(e)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async fn set(
|
||||
&self,
|
||||
cx: &SdkCallCx,
|
||||
name: &str,
|
||||
value: serde_json::Value,
|
||||
) -> Result<(), SecretsError> {
|
||||
validate_secret_name(name)?;
|
||||
self.check_write(cx).await?;
|
||||
let (ciphertext, nonce) = seal(&self.master_key, &value, self.max_value_bytes)?;
|
||||
self.repo.set(cx.app_id, name, &ciphertext, &nonce).await?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn delete(&self, cx: &SdkCallCx, name: &str) -> Result<bool, SecretsError> {
|
||||
validate_secret_name(name)?;
|
||||
self.check_write(cx).await?;
|
||||
Ok(self.repo.delete(cx.app_id, name).await?)
|
||||
}
|
||||
|
||||
async fn list(
|
||||
&self,
|
||||
cx: &SdkCallCx,
|
||||
cursor: Option<&str>,
|
||||
limit: u32,
|
||||
) -> Result<SecretsListPage, SecretsError> {
|
||||
self.check_read(cx).await?;
|
||||
let page = self.repo.list_names(cx.app_id, cursor, limit).await?;
|
||||
Ok(SecretsListPage {
|
||||
names: page.names,
|
||||
next_cursor: page.next_cursor,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// ----------------------------------------------------------------------------
|
||||
// Tests — in-memory SecretsRepo so unit tests don't need Postgres.
|
||||
// ----------------------------------------------------------------------------
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::authz::{AuthzError, AuthzRepo};
|
||||
use crate::secrets_repo::{SecretsMetaPage, SecretsNamePage};
|
||||
use async_trait::async_trait;
|
||||
use picloud_shared::{
|
||||
AdminUserId, AppId, AppRole, ExecutionId, InstanceRole, Principal, RequestId, ScriptId,
|
||||
UserId,
|
||||
};
|
||||
use std::collections::BTreeMap;
|
||||
use tokio::sync::Mutex;
|
||||
|
||||
#[derive(Default)]
|
||||
struct InMemorySecretsRepo {
|
||||
data: Mutex<BTreeMap<(AppId, String), StoredSecret>>,
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl SecretsRepo for InMemorySecretsRepo {
|
||||
async fn get(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
name: &str,
|
||||
) -> Result<Option<StoredSecret>, SecretsRepoError> {
|
||||
Ok(self
|
||||
.data
|
||||
.lock()
|
||||
.await
|
||||
.get(&(app_id, name.to_string()))
|
||||
.cloned())
|
||||
}
|
||||
async fn set(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
name: &str,
|
||||
encrypted_value: &[u8],
|
||||
nonce: &[u8],
|
||||
) -> Result<(), SecretsRepoError> {
|
||||
self.data.lock().await.insert(
|
||||
(app_id, name.to_string()),
|
||||
StoredSecret {
|
||||
encrypted_value: encrypted_value.to_vec(),
|
||||
nonce: nonce.to_vec(),
|
||||
},
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
async fn delete(&self, app_id: AppId, name: &str) -> Result<bool, SecretsRepoError> {
|
||||
Ok(self
|
||||
.data
|
||||
.lock()
|
||||
.await
|
||||
.remove(&(app_id, name.to_string()))
|
||||
.is_some())
|
||||
}
|
||||
async fn list_names(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
cursor: Option<&str>,
|
||||
limit: u32,
|
||||
) -> Result<SecretsNamePage, SecretsRepoError> {
|
||||
let data = self.data.lock().await;
|
||||
let last = cursor.map(std::string::ToString::to_string);
|
||||
let mut names: Vec<String> = data
|
||||
.iter()
|
||||
.filter(|((a, _), _)| *a == app_id)
|
||||
.map(|((_, n), _)| n.clone())
|
||||
.filter(|n| last.as_ref().is_none_or(|l| n > l))
|
||||
.collect();
|
||||
names.sort();
|
||||
let take = (limit as usize).max(1);
|
||||
let next_cursor = if names.len() > take {
|
||||
names.truncate(take);
|
||||
names.last().cloned()
|
||||
} else {
|
||||
None
|
||||
};
|
||||
Ok(SecretsNamePage { names, next_cursor })
|
||||
}
|
||||
async fn list_meta(
|
||||
&self,
|
||||
_app_id: AppId,
|
||||
_cursor: Option<&str>,
|
||||
_limit: u32,
|
||||
) -> Result<SecretsMetaPage, SecretsRepoError> {
|
||||
unimplemented!("admin-only; not exercised in service tests")
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Default)]
|
||||
struct DenyingAuthzRepo;
|
||||
#[async_trait]
|
||||
impl AuthzRepo for DenyingAuthzRepo {
|
||||
async fn membership(
|
||||
&self,
|
||||
_user_id: UserId,
|
||||
_app_id: AppId,
|
||||
) -> Result<Option<AppRole>, AuthzError> {
|
||||
Ok(None)
|
||||
}
|
||||
}
|
||||
|
||||
fn key() -> MasterKey {
|
||||
MasterKey::from_bytes([0x5au8; 32])
|
||||
}
|
||||
|
||||
fn svc() -> SecretsServiceImpl {
|
||||
SecretsServiceImpl::new(
|
||||
Arc::new(InMemorySecretsRepo::default()),
|
||||
Arc::new(DenyingAuthzRepo),
|
||||
key(),
|
||||
SecretsConfig::conservative(),
|
||||
)
|
||||
}
|
||||
|
||||
fn cx_with(app_id: AppId, principal: Option<Principal>) -> SdkCallCx {
|
||||
SdkCallCx {
|
||||
app_id,
|
||||
script_id: ScriptId::new(),
|
||||
principal,
|
||||
execution_id: ExecutionId::new(),
|
||||
request_id: RequestId::new(),
|
||||
trigger_depth: 0,
|
||||
root_execution_id: ExecutionId::new(),
|
||||
is_dead_letter_handler: false,
|
||||
event: None,
|
||||
}
|
||||
}
|
||||
|
||||
fn anon_cx(app_id: AppId) -> SdkCallCx {
|
||||
cx_with(app_id, None)
|
||||
}
|
||||
|
||||
fn member_no_role_cx(app_id: AppId) -> SdkCallCx {
|
||||
cx_with(
|
||||
app_id,
|
||||
Some(Principal {
|
||||
user_id: AdminUserId::new(),
|
||||
instance_role: InstanceRole::Member,
|
||||
scopes: None,
|
||||
app_binding: None,
|
||||
}),
|
||||
)
|
||||
}
|
||||
|
||||
fn owner_cx(app_id: AppId) -> SdkCallCx {
|
||||
cx_with(
|
||||
app_id,
|
||||
Some(Principal {
|
||||
user_id: AdminUserId::new(),
|
||||
instance_role: InstanceRole::Owner,
|
||||
scopes: None,
|
||||
app_binding: None,
|
||||
}),
|
||||
)
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn set_get_delete_round_trip() {
|
||||
let s = svc();
|
||||
let cx = anon_cx(AppId::new());
|
||||
s.set(&cx, "stripe_key", serde_json::json!("sk_live_xxx"))
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
s.get(&cx, "stripe_key").await.unwrap(),
|
||||
Some(serde_json::json!("sk_live_xxx"))
|
||||
);
|
||||
assert!(s.delete(&cx, "stripe_key").await.unwrap());
|
||||
assert_eq!(s.get(&cx, "stripe_key").await.unwrap(), None);
|
||||
// Idempotent delete.
|
||||
assert!(!s.delete(&cx, "stripe_key").await.unwrap());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn get_missing_returns_none() {
|
||||
let s = svc();
|
||||
let cx = anon_cx(AppId::new());
|
||||
assert_eq!(s.get(&cx, "nope").await.unwrap(), None);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn empty_name_rejected() {
|
||||
let s = svc();
|
||||
let cx = anon_cx(AppId::new());
|
||||
let err = s.set(&cx, "", serde_json::json!("x")).await.unwrap_err();
|
||||
assert!(matches!(err, SecretsError::InvalidName(_)));
|
||||
let err = s.get(&cx, "").await.unwrap_err();
|
||||
assert!(matches!(err, SecretsError::InvalidName(_)));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn name_length_capped() {
|
||||
let s = svc();
|
||||
let cx = anon_cx(AppId::new());
|
||||
let long = "a".repeat(256);
|
||||
let err = s.set(&cx, &long, serde_json::json!(1)).await.unwrap_err();
|
||||
assert!(matches!(err, SecretsError::InvalidName(_)));
|
||||
// Exactly 255 is allowed.
|
||||
let ok = "b".repeat(255);
|
||||
s.set(&cx, &ok, serde_json::json!(1)).await.unwrap();
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn value_over_cap_rejected() {
|
||||
let s = SecretsServiceImpl::new(
|
||||
Arc::new(InMemorySecretsRepo::default()),
|
||||
Arc::new(DenyingAuthzRepo),
|
||||
key(),
|
||||
SecretsConfig {
|
||||
max_value_bytes: 16,
|
||||
},
|
||||
);
|
||||
let cx = anon_cx(AppId::new());
|
||||
let big = serde_json::json!("x".repeat(64));
|
||||
let err = s.set(&cx, "k", big).await.unwrap_err();
|
||||
assert!(matches!(err, SecretsError::TooLarge { limit: 16, .. }));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn cross_app_isolation() {
|
||||
let s = svc();
|
||||
let a = AppId::new();
|
||||
let b = AppId::new();
|
||||
s.set(&anon_cx(a), "shared", serde_json::json!("from-a"))
|
||||
.await
|
||||
.unwrap();
|
||||
s.set(&anon_cx(b), "shared", serde_json::json!("from-b"))
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
s.get(&anon_cx(a), "shared").await.unwrap(),
|
||||
Some(serde_json::json!("from-a"))
|
||||
);
|
||||
assert_eq!(
|
||||
s.get(&anon_cx(b), "shared").await.unwrap(),
|
||||
Some(serde_json::json!("from-b"))
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn anonymous_skips_authz() {
|
||||
let s = svc();
|
||||
// DenyingAuthzRepo would deny an authed principal; anon skips it.
|
||||
s.set(&anon_cx(AppId::new()), "k", serde_json::json!(1))
|
||||
.await
|
||||
.unwrap();
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn authed_member_without_role_forbidden() {
|
||||
let s = svc();
|
||||
let err = s
|
||||
.set(&member_no_role_cx(AppId::new()), "k", serde_json::json!(1))
|
||||
.await
|
||||
.unwrap_err();
|
||||
assert!(matches!(err, SecretsError::Forbidden));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn owner_can_write() {
|
||||
let s = svc();
|
||||
s.set(&owner_cx(AppId::new()), "k", serde_json::json!(1))
|
||||
.await
|
||||
.unwrap();
|
||||
}
|
||||
|
||||
/// Type round-trip: a String comes back a String, a Map a Map, an
|
||||
/// Array an Array — the JSON encoding is transparent.
|
||||
#[tokio::test]
|
||||
async fn type_round_trip_preserves_shape() {
|
||||
let s = svc();
|
||||
let cx = anon_cx(AppId::new());
|
||||
|
||||
s.set(&cx, "str", serde_json::json!("sk_live_xxx"))
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
s.get(&cx, "str").await.unwrap(),
|
||||
Some(serde_json::json!("sk_live_xxx"))
|
||||
);
|
||||
|
||||
let map = serde_json::json!({ "client_id": "abc", "client_secret": "xyz" });
|
||||
s.set(&cx, "oauth", map.clone()).await.unwrap();
|
||||
assert_eq!(s.get(&cx, "oauth").await.unwrap(), Some(map));
|
||||
|
||||
let arr = serde_json::json!([1, 2, 3]);
|
||||
s.set(&cx, "arr", arr.clone()).await.unwrap();
|
||||
assert_eq!(s.get(&cx, "arr").await.unwrap(), Some(arr));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn corrupted_ciphertext_surfaces_error() {
|
||||
let repo = Arc::new(InMemorySecretsRepo::default());
|
||||
let s = SecretsServiceImpl::new(
|
||||
repo.clone(),
|
||||
Arc::new(DenyingAuthzRepo),
|
||||
key(),
|
||||
SecretsConfig::conservative(),
|
||||
);
|
||||
let app = AppId::new();
|
||||
s.set(&anon_cx(app), "k", serde_json::json!("v"))
|
||||
.await
|
||||
.unwrap();
|
||||
// Corrupt the stored ciphertext directly.
|
||||
repo.data
|
||||
.lock()
|
||||
.await
|
||||
.get_mut(&(app, "k".to_string()))
|
||||
.unwrap()
|
||||
.encrypted_value[0] ^= 0xff;
|
||||
let err = s.get(&anon_cx(app), "k").await.unwrap_err();
|
||||
assert!(matches!(err, SecretsError::Corrupted));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn list_returns_names_paginated() {
|
||||
let s = svc();
|
||||
let cx = anon_cx(AppId::new());
|
||||
for i in 0..5 {
|
||||
s.set(&cx, &format!("k{i:02}"), serde_json::json!(i))
|
||||
.await
|
||||
.unwrap();
|
||||
}
|
||||
let p1 = s.list(&cx, None, 2).await.unwrap();
|
||||
assert_eq!(p1.names, vec!["k00".to_string(), "k01".to_string()]);
|
||||
assert!(p1.next_cursor.is_some());
|
||||
let p2 = s.list(&cx, p1.next_cursor.as_deref(), 10).await.unwrap();
|
||||
assert_eq!(
|
||||
p2.names,
|
||||
vec!["k02".to_string(), "k03".to_string(), "k04".to_string()]
|
||||
);
|
||||
assert!(p2.next_cursor.is_none());
|
||||
}
|
||||
}
|
||||
@@ -195,7 +195,7 @@ impl TopicRepo for PostgresTopicRepo {
|
||||
.bind(app_id.into_inner())
|
||||
.bind(name)
|
||||
.bind(external_subscribable)
|
||||
.bind(auth_mode.map(|m| m.as_str()))
|
||||
.bind(auth_mode.map(TopicAuthMode::as_str))
|
||||
.fetch_optional(&self.pool)
|
||||
.await?;
|
||||
row.map(TopicRow::into_topic).transpose()
|
||||
|
||||
@@ -57,6 +57,8 @@ pub enum TriggerKind {
|
||||
Files,
|
||||
/// v1.1.5.
|
||||
Pubsub,
|
||||
/// v1.1.7. Inbound email via the webhook receiver.
|
||||
Email,
|
||||
}
|
||||
|
||||
impl TriggerKind {
|
||||
@@ -69,6 +71,7 @@ impl TriggerKind {
|
||||
Self::Cron => "cron",
|
||||
Self::Files => "files",
|
||||
Self::Pubsub => "pubsub",
|
||||
Self::Email => "email",
|
||||
}
|
||||
}
|
||||
|
||||
@@ -81,6 +84,7 @@ impl TriggerKind {
|
||||
"cron" => Some(Self::Cron),
|
||||
"files" => Some(Self::Files),
|
||||
"pubsub" => Some(Self::Pubsub),
|
||||
"email" => Some(Self::Email),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
@@ -137,6 +141,10 @@ pub enum TriggerDetails {
|
||||
},
|
||||
/// v1.1.5. A topic pattern: exact, `<prefix>.*`, or `*`.
|
||||
Pubsub { topic_pattern: String },
|
||||
/// v1.1.7. Inbound email. The HMAC `inbound_secret` is never
|
||||
/// surfaced (it's encrypted at rest); we expose only whether one is
|
||||
/// configured so the admin UI can show "signed" vs "unsigned".
|
||||
Email { has_inbound_secret: bool },
|
||||
}
|
||||
|
||||
/// Create payload for a KV trigger. Defaults applied at the admin
|
||||
@@ -232,6 +240,33 @@ pub struct CreatePubsubTrigger {
|
||||
pub registered_by_principal: AdminUserId,
|
||||
}
|
||||
|
||||
/// Create payload for an email trigger (v1.1.7). `inbound_secret_*` is
|
||||
/// the already-encrypted HMAC secret (sealed by the admin layer with the
|
||||
/// process master key) or `None` for an unsigned trigger.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct CreateEmailTrigger {
|
||||
pub script_id: ScriptId,
|
||||
pub inbound_secret_encrypted: Option<Vec<u8>>,
|
||||
pub inbound_secret_nonce: Option<Vec<u8>>,
|
||||
pub registered_by_principal: AdminUserId,
|
||||
}
|
||||
|
||||
/// What the inbound-email webhook receiver needs to verify + dispatch a
|
||||
/// POST. Returned by `email_inbound_target`; `None` when the trigger
|
||||
/// doesn't exist or isn't `kind = 'email'`.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct EmailInboundTarget {
|
||||
pub app_id: AppId,
|
||||
pub script_id: ScriptId,
|
||||
pub enabled: bool,
|
||||
pub dispatch_mode: TriggerDispatchMode,
|
||||
pub registered_by_principal: AdminUserId,
|
||||
/// Encrypted HMAC secret + nonce; both `None` for an unsigned
|
||||
/// trigger (accepts any POST).
|
||||
pub inbound_secret_encrypted: Option<Vec<u8>>,
|
||||
pub inbound_secret_nonce: Option<Vec<u8>>,
|
||||
}
|
||||
|
||||
/// One match for the dispatcher's "which KV triggers fire on this
|
||||
/// event" lookup. Carries everything the dispatcher needs to construct
|
||||
/// the outbox row.
|
||||
@@ -313,6 +348,23 @@ pub trait TriggerRepo: Send + Sync {
|
||||
req: CreatePubsubTrigger,
|
||||
) -> Result<Trigger, TriggerRepoError>;
|
||||
|
||||
/// v1.1.7. Inbound email trigger. The `inbound_secret` is stored
|
||||
/// already-encrypted (the admin layer seals it).
|
||||
async fn create_email_trigger(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
req: CreateEmailTrigger,
|
||||
) -> Result<Trigger, TriggerRepoError>;
|
||||
|
||||
/// v1.1.7. The webhook receiver's hot-path lookup: resolve a
|
||||
/// `kind = 'email'` trigger to its app, handler script, dispatch
|
||||
/// mode, and (encrypted) HMAC secret. Returns `None` when the
|
||||
/// trigger doesn't exist or isn't an email trigger.
|
||||
async fn email_inbound_target(
|
||||
&self,
|
||||
trigger_id: TriggerId,
|
||||
) -> Result<Option<EmailInboundTarget>, TriggerRepoError>;
|
||||
|
||||
async fn list_for_app(&self, app_id: AppId) -> Result<Vec<Trigger>, TriggerRepoError>;
|
||||
|
||||
async fn get(&self, id: TriggerId) -> Result<Option<Trigger>, TriggerRepoError>;
|
||||
@@ -761,6 +813,89 @@ impl TriggerRepo for PostgresTriggerRepo {
|
||||
})
|
||||
}
|
||||
|
||||
async fn create_email_trigger(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
req: CreateEmailTrigger,
|
||||
) -> Result<Trigger, TriggerRepoError> {
|
||||
let has_inbound_secret = req.inbound_secret_encrypted.is_some();
|
||||
let mut tx = self.pool.begin().await?;
|
||||
// Inbound email is delivered async like every other fan-out
|
||||
// event; the receiver enqueues an outbox row the dispatcher
|
||||
// picks up. Retry settings use the standard defaults.
|
||||
let parent: TriggerRow = sqlx::query_as(
|
||||
"INSERT INTO triggers ( \
|
||||
app_id, script_id, kind, enabled, dispatch_mode, \
|
||||
retry_max_attempts, retry_backoff, retry_base_ms, \
|
||||
registered_by_principal \
|
||||
) VALUES ($1, $2, 'email', TRUE, 'async', 3, 'exponential', 1000, $3) \
|
||||
RETURNING id, app_id, script_id, kind, enabled, dispatch_mode, \
|
||||
retry_max_attempts, retry_backoff, retry_base_ms, \
|
||||
registered_by_principal, created_at, updated_at",
|
||||
)
|
||||
.bind(app_id.into_inner())
|
||||
.bind(req.script_id.into_inner())
|
||||
.bind(req.registered_by_principal.into_inner())
|
||||
.fetch_one(&mut *tx)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"INSERT INTO email_trigger_details \
|
||||
(trigger_id, inbound_secret_encrypted, inbound_secret_nonce) \
|
||||
VALUES ($1, $2, $3)",
|
||||
)
|
||||
.bind(parent.id)
|
||||
.bind(req.inbound_secret_encrypted.as_deref())
|
||||
.bind(req.inbound_secret_nonce.as_deref())
|
||||
.execute(&mut *tx)
|
||||
.await?;
|
||||
|
||||
tx.commit().await?;
|
||||
|
||||
Ok(Trigger {
|
||||
id: parent.id.into(),
|
||||
app_id: parent.app_id.into(),
|
||||
script_id: parent.script_id.into(),
|
||||
kind: TriggerKind::Email,
|
||||
enabled: parent.enabled,
|
||||
dispatch_mode: dispatch_from_str(&parent.dispatch_mode),
|
||||
retry_max_attempts: u32::try_from(parent.retry_max_attempts).unwrap_or(3),
|
||||
retry_backoff: BackoffShape::from_wire(&parent.retry_backoff)
|
||||
.unwrap_or(BackoffShape::Exponential),
|
||||
retry_base_ms: u32::try_from(parent.retry_base_ms).unwrap_or(1000),
|
||||
registered_by_principal: parent.registered_by_principal.into(),
|
||||
created_at: parent.created_at,
|
||||
updated_at: parent.updated_at,
|
||||
details: TriggerDetails::Email { has_inbound_secret },
|
||||
})
|
||||
}
|
||||
|
||||
async fn email_inbound_target(
|
||||
&self,
|
||||
trigger_id: TriggerId,
|
||||
) -> Result<Option<EmailInboundTarget>, TriggerRepoError> {
|
||||
let row: Option<EmailInboundRow> = sqlx::query_as(
|
||||
"SELECT t.app_id, t.script_id, t.enabled, t.dispatch_mode, \
|
||||
t.registered_by_principal, \
|
||||
d.inbound_secret_encrypted, d.inbound_secret_nonce \
|
||||
FROM triggers t \
|
||||
JOIN email_trigger_details d ON d.trigger_id = t.id \
|
||||
WHERE t.id = $1 AND t.kind = 'email'",
|
||||
)
|
||||
.bind(trigger_id.into_inner())
|
||||
.fetch_optional(&self.pool)
|
||||
.await?;
|
||||
Ok(row.map(|r| EmailInboundTarget {
|
||||
app_id: r.app_id.into(),
|
||||
script_id: r.script_id.into(),
|
||||
enabled: r.enabled,
|
||||
dispatch_mode: dispatch_from_str(&r.dispatch_mode),
|
||||
registered_by_principal: r.registered_by_principal.into(),
|
||||
inbound_secret_encrypted: r.inbound_secret_encrypted,
|
||||
inbound_secret_nonce: r.inbound_secret_nonce,
|
||||
}))
|
||||
}
|
||||
|
||||
async fn list_for_app(&self, app_id: AppId) -> Result<Vec<Trigger>, TriggerRepoError> {
|
||||
let parents: Vec<TriggerRow> = sqlx::query_as(
|
||||
"SELECT id, app_id, script_id, kind, enabled, dispatch_mode, \
|
||||
@@ -1077,6 +1212,17 @@ async fn hydrate_one(pool: &PgPool, parent: TriggerRow) -> Result<Trigger, Trigg
|
||||
topic_pattern: row.topic_pattern,
|
||||
}
|
||||
}
|
||||
TriggerKind::Email => {
|
||||
let row: EmailDetailRow = sqlx::query_as(
|
||||
"SELECT inbound_secret_encrypted FROM email_trigger_details WHERE trigger_id = $1",
|
||||
)
|
||||
.bind(parent.id)
|
||||
.fetch_one(pool)
|
||||
.await?;
|
||||
TriggerDetails::Email {
|
||||
has_inbound_secret: row.inbound_secret_encrypted.is_some(),
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
Ok(Trigger {
|
||||
@@ -1154,6 +1300,22 @@ struct PubsubDetailRow {
|
||||
topic_pattern: String,
|
||||
}
|
||||
|
||||
#[derive(sqlx::FromRow)]
|
||||
struct EmailDetailRow {
|
||||
inbound_secret_encrypted: Option<Vec<u8>>,
|
||||
}
|
||||
|
||||
#[derive(sqlx::FromRow)]
|
||||
struct EmailInboundRow {
|
||||
app_id: Uuid,
|
||||
script_id: Uuid,
|
||||
enabled: bool,
|
||||
dispatch_mode: String,
|
||||
registered_by_principal: Uuid,
|
||||
inbound_secret_encrypted: Option<Vec<u8>>,
|
||||
inbound_secret_nonce: Option<Vec<u8>>,
|
||||
}
|
||||
|
||||
#[derive(sqlx::FromRow)]
|
||||
#[allow(clippy::struct_field_names)]
|
||||
struct DlDetailRow {
|
||||
|
||||
@@ -17,7 +17,8 @@ use axum::response::{IntoResponse, Json, Response};
|
||||
use axum::routing::{delete, get, post};
|
||||
use axum::{Extension, Router};
|
||||
use picloud_shared::{
|
||||
AppId, DocsEventOp, FilesEventOp, KvEventOp, Principal, ScriptId, ScriptKind, TriggerId,
|
||||
AppId, DocsEventOp, FilesEventOp, KvEventOp, MasterKey, Principal, ScriptId, ScriptKind,
|
||||
TriggerId,
|
||||
};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use serde_json::json;
|
||||
@@ -25,11 +26,12 @@ use serde_json::json;
|
||||
use crate::app_repo::AppRepository;
|
||||
use crate::authz::{require, AuthzDenied, AuthzError, AuthzRepo, Capability};
|
||||
use crate::repo::{ScriptRepository, ScriptRepositoryError};
|
||||
use crate::secrets_service::seal;
|
||||
use crate::trigger_config::{BackoffShape, TriggerConfig};
|
||||
use crate::trigger_repo::{
|
||||
CreateCronTrigger, CreateDeadLetterTrigger, CreateDocsTrigger, CreateFilesTrigger,
|
||||
CreateKvTrigger, CreatePubsubTrigger, Trigger, TriggerDispatchMode, TriggerRepo,
|
||||
TriggerRepoError,
|
||||
CreateCronTrigger, CreateDeadLetterTrigger, CreateDocsTrigger, CreateEmailTrigger,
|
||||
CreateFilesTrigger, CreateKvTrigger, CreatePubsubTrigger, Trigger, TriggerDispatchMode,
|
||||
TriggerRepo, TriggerRepoError,
|
||||
};
|
||||
|
||||
#[derive(Clone)]
|
||||
@@ -46,6 +48,9 @@ pub struct TriggersState {
|
||||
/// retry settings. Kept on the state struct so tests can swap
|
||||
/// in a stricter / looser config without env tinkering.
|
||||
pub config: TriggerConfig,
|
||||
/// v1.1.7: master key used to encrypt an email trigger's inbound HMAC
|
||||
/// secret before it's stored.
|
||||
pub master_key: MasterKey,
|
||||
}
|
||||
|
||||
pub fn triggers_router(state: TriggersState) -> Router {
|
||||
@@ -66,6 +71,7 @@ pub fn triggers_router(state: TriggersState) -> Router {
|
||||
"/apps/{app_id}/triggers/dead_letter",
|
||||
post(create_dl_trigger),
|
||||
)
|
||||
.route("/apps/{app_id}/triggers/email", post(create_email_trigger))
|
||||
.route(
|
||||
"/apps/{app_id}/triggers/{trigger_id}",
|
||||
delete(delete_trigger),
|
||||
@@ -467,6 +473,60 @@ async fn create_dl_trigger(
|
||||
Ok((StatusCode::CREATED, Json(created)))
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize)]
|
||||
struct CreateEmailTriggerRequest {
|
||||
script_id: ScriptId,
|
||||
/// Shared HMAC secret the provider signs inbound POSTs with. `null`
|
||||
/// (or absent) means the trigger accepts unsigned POSTs.
|
||||
#[serde(default)]
|
||||
inbound_secret: Option<String>,
|
||||
}
|
||||
|
||||
async fn create_email_trigger(
|
||||
State(s): State<TriggersState>,
|
||||
Extension(principal): Extension<Principal>,
|
||||
Path(app_id): Path<AppId>,
|
||||
Json(input): Json<CreateEmailTriggerRequest>,
|
||||
) -> Result<(StatusCode, Json<Trigger>), TriggersApiError> {
|
||||
ensure_app_exists(&*s.apps, app_id).await?;
|
||||
require(
|
||||
s.authz.as_ref(),
|
||||
&principal,
|
||||
Capability::AppManageTriggers(app_id),
|
||||
)
|
||||
.await?;
|
||||
validate_trigger_target(&*s.scripts, app_id, input.script_id).await?;
|
||||
|
||||
// Encrypt the inbound HMAC secret at rest (user-approved deviation
|
||||
// from the brief's plaintext column). An empty/whitespace secret is
|
||||
// treated as "no secret" (unsigned trigger).
|
||||
let (inbound_secret_encrypted, inbound_secret_nonce) = match input.inbound_secret {
|
||||
Some(secret) if !secret.trim().is_empty() => {
|
||||
// 64 KB cap is irrelevant for a signing secret, but `seal`
|
||||
// takes one; reuse the secrets default.
|
||||
let (ct, nonce) = seal(
|
||||
&s.master_key,
|
||||
&serde_json::Value::String(secret),
|
||||
crate::secrets_service::DEFAULT_SECRET_MAX_VALUE_BYTES,
|
||||
)
|
||||
.map_err(|e| {
|
||||
TriggersApiError::Invalid(format!("could not seal inbound_secret: {e}"))
|
||||
})?;
|
||||
(Some(ct), Some(nonce.to_vec()))
|
||||
}
|
||||
_ => (None, None),
|
||||
};
|
||||
|
||||
let req = CreateEmailTrigger {
|
||||
script_id: input.script_id,
|
||||
inbound_secret_encrypted,
|
||||
inbound_secret_nonce,
|
||||
registered_by_principal: principal.user_id,
|
||||
};
|
||||
let created = s.triggers.create_email_trigger(app_id, req).await?;
|
||||
Ok((StatusCode::CREATED, Json(created)))
|
||||
}
|
||||
|
||||
async fn delete_trigger(
|
||||
State(s): State<TriggersState>,
|
||||
Extension(principal): Extension<Principal>,
|
||||
@@ -598,9 +658,9 @@ mod tests {
|
||||
use super::*;
|
||||
use crate::app_repo::{AppLookup, AppRepository};
|
||||
use crate::trigger_repo::{
|
||||
CreateCronTrigger, CreateFilesTrigger, CreatePubsubTrigger, DeadLetterTriggerMatch,
|
||||
DocsTriggerMatch, FilesTriggerMatch, KvTriggerMatch, Trigger, TriggerDetails, TriggerRepo,
|
||||
TriggerRepoError,
|
||||
CreateCronTrigger, CreateEmailTrigger, CreateFilesTrigger, CreatePubsubTrigger,
|
||||
DeadLetterTriggerMatch, DocsTriggerMatch, EmailInboundTarget, FilesTriggerMatch,
|
||||
KvTriggerMatch, Trigger, TriggerDetails, TriggerKind, TriggerRepo, TriggerRepoError,
|
||||
};
|
||||
use async_trait::async_trait;
|
||||
use chrono::Utc;
|
||||
@@ -703,6 +763,50 @@ mod tests {
|
||||
self.inner.lock().await.insert(id, trigger.clone());
|
||||
Ok(trigger)
|
||||
}
|
||||
async fn create_email_trigger(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
req: CreateEmailTrigger,
|
||||
) -> Result<Trigger, TriggerRepoError> {
|
||||
let now = Utc::now();
|
||||
let id = TriggerId::new();
|
||||
let trigger = Trigger {
|
||||
id,
|
||||
app_id,
|
||||
script_id: req.script_id,
|
||||
kind: TriggerKind::Email,
|
||||
enabled: true,
|
||||
dispatch_mode: TriggerDispatchMode::Async,
|
||||
retry_max_attempts: 3,
|
||||
retry_backoff: BackoffShape::Exponential,
|
||||
retry_base_ms: 1000,
|
||||
registered_by_principal: req.registered_by_principal,
|
||||
created_at: now,
|
||||
updated_at: now,
|
||||
details: TriggerDetails::Email {
|
||||
has_inbound_secret: req.inbound_secret_encrypted.is_some(),
|
||||
},
|
||||
};
|
||||
self.inner.lock().await.insert(id, trigger.clone());
|
||||
Ok(trigger)
|
||||
}
|
||||
async fn email_inbound_target(
|
||||
&self,
|
||||
trigger_id: TriggerId,
|
||||
) -> Result<Option<EmailInboundTarget>, TriggerRepoError> {
|
||||
let g = self.inner.lock().await;
|
||||
Ok(g.get(&trigger_id)
|
||||
.filter(|t| t.kind == TriggerKind::Email)
|
||||
.map(|t| EmailInboundTarget {
|
||||
app_id: t.app_id,
|
||||
script_id: t.script_id,
|
||||
enabled: t.enabled,
|
||||
dispatch_mode: t.dispatch_mode,
|
||||
registered_by_principal: t.registered_by_principal,
|
||||
inbound_secret_encrypted: None,
|
||||
inbound_secret_nonce: None,
|
||||
}))
|
||||
}
|
||||
async fn create_cron_trigger(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
@@ -1101,6 +1205,7 @@ mod tests {
|
||||
authz,
|
||||
scripts: InMemoryScriptRepo::empty(),
|
||||
config: TriggerConfig::conservative(),
|
||||
master_key: picloud_shared::MasterKey::from_bytes([0u8; 32]),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1118,6 +1223,7 @@ mod tests {
|
||||
authz,
|
||||
scripts: InMemoryScriptRepo::with_endpoint(app_id, script_id),
|
||||
config: TriggerConfig::conservative(),
|
||||
master_key: picloud_shared::MasterKey::from_bytes([0u8; 32]),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1390,6 +1496,7 @@ mod tests {
|
||||
authz: Arc::new(AlwaysAllowAuthzRepo),
|
||||
scripts: InMemoryScriptRepo::with_module(app_id, script_id),
|
||||
config: TriggerConfig::conservative(),
|
||||
master_key: picloud_shared::MasterKey::from_bytes([0u8; 32]),
|
||||
};
|
||||
let res = create_kv_trigger(
|
||||
State(state),
|
||||
@@ -1427,6 +1534,7 @@ mod tests {
|
||||
authz: Arc::new(AlwaysAllowAuthzRepo),
|
||||
scripts: InMemoryScriptRepo::with_module(app_id, script_id),
|
||||
config: TriggerConfig::conservative(),
|
||||
master_key: picloud_shared::MasterKey::from_bytes([0u8; 32]),
|
||||
};
|
||||
let res = create_docs_trigger(
|
||||
State(state),
|
||||
@@ -1461,6 +1569,7 @@ mod tests {
|
||||
authz: Arc::new(AlwaysAllowAuthzRepo),
|
||||
scripts: InMemoryScriptRepo::with_module(app_id, script_id),
|
||||
config: TriggerConfig::conservative(),
|
||||
master_key: picloud_shared::MasterKey::from_bytes([0u8; 32]),
|
||||
};
|
||||
let res = create_dl_trigger(
|
||||
State(state),
|
||||
@@ -1526,6 +1635,7 @@ mod tests {
|
||||
authz: Arc::new(AlwaysAllowAuthzRepo),
|
||||
scripts,
|
||||
config: TriggerConfig::conservative(),
|
||||
master_key: picloud_shared::MasterKey::from_bytes([0u8; 32]),
|
||||
};
|
||||
let res = create_kv_trigger(
|
||||
State(state),
|
||||
@@ -1656,6 +1766,7 @@ mod tests {
|
||||
authz: Arc::new(AlwaysAllowAuthzRepo),
|
||||
scripts: InMemoryScriptRepo::with_module(app_id, script_id),
|
||||
config: TriggerConfig::conservative(),
|
||||
master_key: picloud_shared::MasterKey::from_bytes([0u8; 32]),
|
||||
};
|
||||
let res = create_cron_trigger(
|
||||
State(state),
|
||||
@@ -1685,6 +1796,7 @@ mod tests {
|
||||
authz: Arc::new(AlwaysAllowAuthzRepo),
|
||||
scripts: InMemoryScriptRepo::with_endpoint(app_b, script_id),
|
||||
config: TriggerConfig::conservative(),
|
||||
master_key: picloud_shared::MasterKey::from_bytes([0u8; 32]),
|
||||
};
|
||||
let res = create_cron_trigger(
|
||||
State(state),
|
||||
@@ -1813,6 +1925,7 @@ mod tests {
|
||||
authz: Arc::new(AlwaysAllowAuthzRepo),
|
||||
scripts: InMemoryScriptRepo::with_module(app_id, script_id),
|
||||
config: TriggerConfig::conservative(),
|
||||
master_key: picloud_shared::MasterKey::from_bytes([0u8; 32]),
|
||||
};
|
||||
let res = create_files_trigger(
|
||||
State(state),
|
||||
@@ -1839,6 +1952,7 @@ mod tests {
|
||||
authz: Arc::new(AlwaysAllowAuthzRepo),
|
||||
scripts: InMemoryScriptRepo::with_endpoint(app_b, script_id),
|
||||
config: TriggerConfig::conservative(),
|
||||
master_key: picloud_shared::MasterKey::from_bytes([0u8; 32]),
|
||||
};
|
||||
let res = create_files_trigger(
|
||||
State(state),
|
||||
@@ -1936,6 +2050,7 @@ mod tests {
|
||||
authz: Arc::new(AlwaysAllowAuthzRepo),
|
||||
scripts: InMemoryScriptRepo::with_module(app_id, script_id),
|
||||
config: TriggerConfig::conservative(),
|
||||
master_key: picloud_shared::MasterKey::from_bytes([0u8; 32]),
|
||||
};
|
||||
let res = create_pubsub_trigger(
|
||||
State(state),
|
||||
@@ -1962,6 +2077,7 @@ mod tests {
|
||||
authz: Arc::new(AlwaysAllowAuthzRepo),
|
||||
scripts: InMemoryScriptRepo::with_endpoint(app_b, script_id),
|
||||
config: TriggerConfig::conservative(),
|
||||
master_key: picloud_shared::MasterKey::from_bytes([0u8; 32]),
|
||||
};
|
||||
let res = create_pubsub_trigger(
|
||||
State(state),
|
||||
|
||||
@@ -60,9 +60,11 @@ table: app_members
|
||||
|
||||
table: app_secrets
|
||||
app_id: uuid NOT NULL
|
||||
realtime_signing_key: bytea NOT NULL
|
||||
realtime_signing_key: bytea NULL
|
||||
created_at: timestamp with time zone NOT NULL default=now()
|
||||
updated_at: timestamp with time zone NOT NULL default=now()
|
||||
realtime_signing_key_encrypted: bytea NULL
|
||||
realtime_signing_key_nonce: bytea NULL
|
||||
|
||||
table: app_slug_history
|
||||
slug: text NOT NULL
|
||||
@@ -119,6 +121,11 @@ table: docs_trigger_details
|
||||
collection_glob: text NOT NULL
|
||||
ops: ARRAY NOT NULL
|
||||
|
||||
table: email_trigger_details
|
||||
trigger_id: uuid NOT NULL
|
||||
inbound_secret_encrypted: bytea NULL
|
||||
inbound_secret_nonce: bytea NULL
|
||||
|
||||
table: execution_logs
|
||||
id: uuid NOT NULL default=gen_random_uuid()
|
||||
script_id: uuid NOT NULL
|
||||
@@ -217,6 +224,14 @@ table: scripts
|
||||
app_id: uuid NOT NULL
|
||||
kind: text NOT NULL default='endpoint'::text
|
||||
|
||||
table: secrets
|
||||
app_id: uuid NOT NULL
|
||||
name: text NOT NULL
|
||||
encrypted_value: bytea NOT NULL
|
||||
nonce: bytea NOT NULL
|
||||
created_at: timestamp with time zone NOT NULL default=now()
|
||||
updated_at: timestamp with time zone NOT NULL default=now()
|
||||
|
||||
table: topics
|
||||
app_id: uuid NOT NULL
|
||||
name: text NOT NULL
|
||||
@@ -300,6 +315,9 @@ indexes on docs:
|
||||
indexes on docs_trigger_details:
|
||||
docs_trigger_details_pkey: public.docs_trigger_details USING btree (trigger_id)
|
||||
|
||||
indexes on email_trigger_details:
|
||||
email_trigger_details_pkey: public.email_trigger_details USING btree (trigger_id)
|
||||
|
||||
indexes on execution_logs:
|
||||
execution_logs_app_id_created_at_idx: public.execution_logs USING btree (app_id, created_at DESC)
|
||||
execution_logs_pkey: public.execution_logs USING btree (id)
|
||||
@@ -345,6 +363,10 @@ indexes on scripts:
|
||||
scripts_name_uidx: public.scripts USING btree (app_id, lower(name))
|
||||
scripts_pkey: public.scripts USING btree (id)
|
||||
|
||||
indexes on secrets:
|
||||
idx_secrets_app: public.secrets USING btree (app_id)
|
||||
secrets_pkey: public.secrets USING btree (app_id, name)
|
||||
|
||||
indexes on topics:
|
||||
topics_pkey: public.topics USING btree (app_id, name)
|
||||
|
||||
@@ -419,6 +441,10 @@ constraints on docs_trigger_details:
|
||||
[FOREIGN KEY] docs_trigger_details_trigger_id_fkey: FOREIGN KEY (trigger_id) REFERENCES triggers(id) ON DELETE CASCADE
|
||||
[PRIMARY KEY] docs_trigger_details_pkey: PRIMARY KEY (trigger_id)
|
||||
|
||||
constraints on email_trigger_details:
|
||||
[FOREIGN KEY] email_trigger_details_trigger_id_fkey: FOREIGN KEY (trigger_id) REFERENCES triggers(id) ON DELETE CASCADE
|
||||
[PRIMARY KEY] email_trigger_details_pkey: PRIMARY KEY (trigger_id)
|
||||
|
||||
constraints on execution_logs:
|
||||
[CHECK] execution_logs_status_check: CHECK ((status = ANY (ARRAY['success'::text, 'error'::text, 'timeout'::text, 'budget_exceeded'::text])))
|
||||
[FOREIGN KEY] execution_logs_app_id_fk: FOREIGN KEY (app_id) REFERENCES apps(id) ON DELETE CASCADE
|
||||
@@ -442,7 +468,7 @@ constraints on kv_trigger_details:
|
||||
[PRIMARY KEY] kv_trigger_details_pkey: PRIMARY KEY (trigger_id)
|
||||
|
||||
constraints on outbox:
|
||||
[CHECK] outbox_source_kind_check: CHECK ((source_kind = ANY (ARRAY['http'::text, 'kv'::text, 'dead_letter'::text, 'docs'::text, 'cron'::text, 'files'::text, 'pubsub'::text])))
|
||||
[CHECK] outbox_source_kind_check: CHECK ((source_kind = ANY (ARRAY['http'::text, 'kv'::text, 'dead_letter'::text, 'docs'::text, 'cron'::text, 'files'::text, 'pubsub'::text, 'email'::text])))
|
||||
[FOREIGN KEY] outbox_app_id_fkey: FOREIGN KEY (app_id) REFERENCES apps(id) ON DELETE CASCADE
|
||||
[PRIMARY KEY] outbox_pkey: PRIMARY KEY (id)
|
||||
|
||||
@@ -472,6 +498,10 @@ constraints on scripts:
|
||||
[FOREIGN KEY] scripts_app_id_fk: FOREIGN KEY (app_id) REFERENCES apps(id) ON DELETE RESTRICT
|
||||
[PRIMARY KEY] scripts_pkey: PRIMARY KEY (id)
|
||||
|
||||
constraints on secrets:
|
||||
[FOREIGN KEY] secrets_app_id_fkey: FOREIGN KEY (app_id) REFERENCES apps(id) ON DELETE CASCADE
|
||||
[PRIMARY KEY] secrets_pkey: PRIMARY KEY (app_id, name)
|
||||
|
||||
constraints on topics:
|
||||
[CHECK] topics_auth_mode_check: CHECK ((auth_mode = ANY (ARRAY['public'::text, 'token'::text])))
|
||||
[FOREIGN KEY] topics_app_id_fkey: FOREIGN KEY (app_id) REFERENCES apps(id) ON DELETE CASCADE
|
||||
@@ -479,7 +509,7 @@ constraints on topics:
|
||||
|
||||
constraints on triggers:
|
||||
[CHECK] triggers_dispatch_mode_check: CHECK ((dispatch_mode = ANY (ARRAY['sync'::text, 'async'::text])))
|
||||
[CHECK] triggers_kind_check: CHECK ((kind = ANY (ARRAY['kv'::text, 'dead_letter'::text, 'docs'::text, 'cron'::text, 'files'::text, 'pubsub'::text])))
|
||||
[CHECK] triggers_kind_check: CHECK ((kind = ANY (ARRAY['kv'::text, 'dead_letter'::text, 'docs'::text, 'cron'::text, 'files'::text, 'pubsub'::text, 'email'::text])))
|
||||
[CHECK] triggers_retry_backoff_check: CHECK ((retry_backoff = ANY (ARRAY['exponential'::text, 'linear'::text, 'constant'::text])))
|
||||
[FOREIGN KEY] triggers_app_id_fkey: FOREIGN KEY (app_id) REFERENCES apps(id) ON DELETE CASCADE
|
||||
[FOREIGN KEY] triggers_registered_by_principal_fkey: FOREIGN KEY (registered_by_principal) REFERENCES admin_users(id) ON DELETE CASCADE
|
||||
@@ -509,3 +539,6 @@ constraints on triggers:
|
||||
0020: pubsub triggers
|
||||
0021: topics
|
||||
0022: app secrets
|
||||
0023: secrets
|
||||
0024: email triggers
|
||||
0025: encrypt realtime keys
|
||||
|
||||
@@ -80,7 +80,6 @@ pub fn heartbeat_secs_from_env() -> u64 {
|
||||
}
|
||||
|
||||
/// Router for the realtime SSE surface. Merged at the router root.
|
||||
#[must_use]
|
||||
pub fn realtime_router(state: RealtimeState) -> Router {
|
||||
Router::new()
|
||||
.route("/realtime/topics/{topic}", get(sse_topic))
|
||||
|
||||
@@ -41,3 +41,7 @@ serde.workspace = true
|
||||
serde_json.workspace = true
|
||||
uuid.workspace = true
|
||||
chrono.workspace = true
|
||||
# Compute inbound-email HMAC signatures in the e2e receiver tests.
|
||||
hmac.workspace = true
|
||||
sha2.workspace = true
|
||||
hex.workspace = true
|
||||
|
||||
@@ -12,21 +12,23 @@ use picloud_executor_core::{Engine, Limits};
|
||||
use picloud_manager_core::{
|
||||
admin_router, admins_router, api_keys_router, app_members_router, apps_api, apps_router,
|
||||
attach_principal_if_present, auth_router, compile_routes, dead_letters_router,
|
||||
files_admin_router, migrations, require_authenticated, route_admin_router, topics_router,
|
||||
triggers_router, AbandonedRepo, AdminPrincipalResolver, AdminSessionRepository, AdminState,
|
||||
AdminUserRepository, AdminsState, ApiKeyRepository, ApiKeysState, AppDomainRepository,
|
||||
AppMembersRepository, AppMembersState, AppRepository, AppsState, AuthState, AuthzRepo,
|
||||
DeadLetterRepo, DeadLettersState, Dispatcher, DocsServiceImpl, FilesAdminState, FilesConfig,
|
||||
email_inbound_router, files_admin_router, migrations, require_authenticated,
|
||||
route_admin_router, secrets_router, topics_router, triggers_router, AbandonedRepo,
|
||||
AdminPrincipalResolver, AdminSessionRepository, AdminState, AdminUserRepository, AdminsState,
|
||||
ApiKeyRepository, ApiKeysState, AppDomainRepository, AppMembersRepository, AppMembersState,
|
||||
AppRepository, AppsState, AuthState, AuthzRepo, DeadLetterRepo, DeadLettersState, Dispatcher,
|
||||
DocsServiceImpl, EmailInboundState, EmailServiceImpl, FilesAdminState, FilesConfig,
|
||||
FilesServiceImpl, FsFilesRepo, HttpConfig, HttpServiceImpl, KvServiceImpl, OutboxEventEmitter,
|
||||
OutboxRepo, PostgresAbandonedRepo, PostgresAdminSessionRepository, PostgresAdminUserRepository,
|
||||
PostgresApiKeyRepository, PostgresAppDomainRepository, PostgresAppMembersRepository,
|
||||
PostgresAppRepository, PostgresAppSecretsRepo, PostgresDeadLetterRepo,
|
||||
PostgresDeadLetterService, PostgresDocsRepo, PostgresExecutionLogRepository,
|
||||
PostgresExecutionLogSink, PostgresKvRepo, PostgresOutboxRepo, PostgresPubsubRepo,
|
||||
PostgresRouteRepository, PostgresScriptRepository, PostgresTopicRepo, PostgresTriggerRepo,
|
||||
PrincipalResolver, PubsubServiceImpl, RealtimeAuthorityImpl, RepoResolver, RouteAdminState,
|
||||
RouteRepository, SandboxCeiling, ScriptRepository, SubscriberTokenConfig, TopicRepo,
|
||||
TopicsState, TriggerConfig, TriggerRepo, TriggersState,
|
||||
PostgresRouteRepository, PostgresScriptRepository, PostgresSecretsRepo, PostgresTopicRepo,
|
||||
PostgresTriggerRepo, PrincipalResolver, PubsubServiceImpl, RealtimeAuthorityImpl, RepoResolver,
|
||||
RouteAdminState, RouteRepository, SandboxCeiling, ScriptRepository, SecretsConfig,
|
||||
SecretsServiceImpl, SecretsState, SubscriberTokenConfig, TopicRepo, TopicsState, TriggerConfig,
|
||||
TriggerRepo, TriggersState,
|
||||
};
|
||||
use picloud_orchestrator_core::realtime::DEFAULT_GC_INTERVAL_SECS;
|
||||
use picloud_orchestrator_core::routing::{AppDomainTable, RouteTable};
|
||||
@@ -35,10 +37,10 @@ use picloud_orchestrator_core::{
|
||||
ExecutionGate, InProcessBroadcaster, InboxRegistry, LocalExecutorClient, RealtimeState,
|
||||
};
|
||||
use picloud_shared::{
|
||||
DeadLetterService, DocsService, ExecutionLogSink, FilesService, HttpService, InboxResolver,
|
||||
KvService, OutboxWriter, PubsubService, RealtimeAuthority, RealtimeBroadcaster,
|
||||
ScriptValidator, ServiceEventEmitter, Services, API_VERSION, PRODUCT_VERSION, SDK_VERSION,
|
||||
WIRE_VERSION,
|
||||
DeadLetterService, DocsService, EmailService, ExecutionLogSink, FilesService, HttpService,
|
||||
InboxResolver, KvService, MasterKey, OutboxWriter, PubsubService, RealtimeAuthority,
|
||||
RealtimeBroadcaster, ScriptValidator, SecretsService, ServiceEventEmitter, Services,
|
||||
API_VERSION, PRODUCT_VERSION, SDK_VERSION, WIRE_VERSION,
|
||||
};
|
||||
use sqlx::postgres::PgPoolOptions;
|
||||
use sqlx::PgPool;
|
||||
@@ -94,7 +96,11 @@ fn read_session_ttl() -> Duration {
|
||||
/// (`/api/v1/execute/{id}`, the user-route fallthrough, `/healthz`,
|
||||
/// `/version`) stays open — it's the public ingress for user scripts.
|
||||
#[allow(clippy::too_many_lines)]
|
||||
pub async fn build_app(pool: PgPool, auth: AuthDeps) -> anyhow::Result<Router> {
|
||||
pub async fn build_app(
|
||||
pool: PgPool,
|
||||
auth: AuthDeps,
|
||||
master_key: MasterKey,
|
||||
) -> anyhow::Result<Router> {
|
||||
let script_repo = Arc::new(PostgresScriptRepository::new(pool.clone()));
|
||||
let log_repo = Arc::new(PostgresExecutionLogRepository::new(pool.clone()));
|
||||
let log_sink: Arc<dyn ExecutionLogSink> = Arc::new(PostgresExecutionLogSink::new(pool.clone()));
|
||||
@@ -183,7 +189,25 @@ pub async fn build_app(pool: PgPool, auth: AuthDeps) -> anyhow::Result<Router> {
|
||||
let broadcaster_concrete = Arc::new(InProcessBroadcaster::from_env());
|
||||
let broadcaster: Arc<dyn RealtimeBroadcaster> = broadcaster_concrete.clone();
|
||||
let topic_repo: Arc<dyn TopicRepo> = Arc::new(PostgresTopicRepo::new(pool.clone()));
|
||||
let app_secrets_repo = Arc::new(PostgresAppSecretsRepo::new(pool.clone()));
|
||||
let app_secrets_repo = Arc::new(PostgresAppSecretsRepo::new(
|
||||
pool.clone(),
|
||||
master_key.clone(),
|
||||
));
|
||||
// v1.1.7 two-phase migration: encrypt any plaintext realtime signing
|
||||
// keys at rest. Idempotent — only touches rows not yet encrypted. The
|
||||
// plaintext column is dropped in v1.1.8.
|
||||
match app_secrets_repo.migrate_plaintext_keys().await {
|
||||
Ok(0) => {}
|
||||
Ok(n) => {
|
||||
tracing::info!(
|
||||
migrated = n,
|
||||
"encrypted plaintext realtime signing keys at rest"
|
||||
);
|
||||
}
|
||||
Err(e) => {
|
||||
tracing::error!(error = %e, "failed to encrypt realtime signing keys (continuing)");
|
||||
}
|
||||
}
|
||||
let realtime_authority: Arc<dyn RealtimeAuthority> = Arc::new(RealtimeAuthorityImpl::new(
|
||||
topic_repo.clone(),
|
||||
app_secrets_repo.clone(),
|
||||
@@ -203,6 +227,23 @@ pub async fn build_app(pool: PgPool, auth: AuthDeps) -> anyhow::Result<Router> {
|
||||
SubscriberTokenConfig::from_env(),
|
||||
),
|
||||
);
|
||||
// v1.1.7 encrypted per-app secrets. Values are AES-256-GCM-sealed
|
||||
// with the process master key before they touch Postgres; the repo
|
||||
// only ever sees ciphertext + nonce. The admin surface reuses the
|
||||
// same repo + master key (see `secrets_state` below).
|
||||
let secrets_config = SecretsConfig::from_env();
|
||||
let secrets_max_value_bytes = secrets_config.max_value_bytes;
|
||||
let secrets_repo: Arc<dyn picloud_manager_core::SecretsRepo> =
|
||||
Arc::new(PostgresSecretsRepo::new(pool.clone()));
|
||||
let secrets: Arc<dyn SecretsService> = Arc::new(SecretsServiceImpl::new(
|
||||
secrets_repo.clone(),
|
||||
authz.clone(),
|
||||
master_key.clone(),
|
||||
secrets_config,
|
||||
));
|
||||
// v1.1.7 outbound email. Builds a lettre SMTP transport from
|
||||
// PICLOUD_SMTP_* env (disabled mode + warning if unconfigured).
|
||||
let email: Arc<dyn EmailService> = Arc::new(EmailServiceImpl::from_env(authz.clone()));
|
||||
let services = Services::new(
|
||||
kv,
|
||||
docs,
|
||||
@@ -212,6 +253,8 @@ pub async fn build_app(pool: PgPool, auth: AuthDeps) -> anyhow::Result<Router> {
|
||||
http,
|
||||
files,
|
||||
pubsub,
|
||||
secrets,
|
||||
email,
|
||||
);
|
||||
let engine = Arc::new(Engine::new(Limits::default(), services));
|
||||
|
||||
@@ -317,11 +360,19 @@ pub async fn build_app(pool: PgPool, auth: AuthDeps) -> anyhow::Result<Router> {
|
||||
spawn_realtime_gc(broadcaster_concrete, DEFAULT_GC_INTERVAL_SECS);
|
||||
picloud_manager_core::spawn_files_orphan_sweep(files_root);
|
||||
let triggers_state = TriggersState {
|
||||
triggers: trigger_repo,
|
||||
triggers: trigger_repo.clone(),
|
||||
apps: apps_repo.clone(),
|
||||
authz: authz.clone(),
|
||||
scripts: Arc::new(PostgresScriptRepoHandle(script_repo.clone())),
|
||||
config: trigger_config,
|
||||
master_key: master_key.clone(),
|
||||
};
|
||||
// v1.1.7 public inbound-email receiver. Outside the admin auth layer
|
||||
// (the URL + per-trigger HMAC secret are the security boundary).
|
||||
let email_inbound_state = EmailInboundState {
|
||||
triggers: trigger_repo,
|
||||
outbox: outbox_repo.clone(),
|
||||
master_key: master_key.clone(),
|
||||
};
|
||||
let dead_letters_state = DeadLettersState {
|
||||
repo: dl_repo,
|
||||
@@ -340,6 +391,13 @@ pub async fn build_app(pool: PgPool, auth: AuthDeps) -> anyhow::Result<Router> {
|
||||
authz: authz.clone(),
|
||||
broadcaster: broadcaster.clone(),
|
||||
};
|
||||
let secrets_state = SecretsState {
|
||||
repo: secrets_repo,
|
||||
apps: apps_repo.clone(),
|
||||
authz: authz.clone(),
|
||||
master_key,
|
||||
max_value_bytes: secrets_max_value_bytes,
|
||||
};
|
||||
let apps_state = AppsState {
|
||||
apps: apps_repo,
|
||||
domains: domains_repo,
|
||||
@@ -384,6 +442,7 @@ pub async fn build_app(pool: PgPool, auth: AuthDeps) -> anyhow::Result<Router> {
|
||||
.merge(triggers_router(triggers_state))
|
||||
.merge(files_admin_router(files_admin_state))
|
||||
.merge(topics_router(topics_state))
|
||||
.merge(secrets_router(secrets_state))
|
||||
.merge(dead_letters_router(dead_letters_state))
|
||||
.layer(from_fn_with_state(
|
||||
auth_state.clone(),
|
||||
@@ -412,6 +471,7 @@ pub async fn build_app(pool: PgPool, auth: AuthDeps) -> anyhow::Result<Router> {
|
||||
let api_v1 = Router::new()
|
||||
.nest("/admin", auth_router(auth_state))
|
||||
.nest("/admin", guarded_admin)
|
||||
.merge(email_inbound_router(email_inbound_state))
|
||||
.merge(data_plane_routed);
|
||||
|
||||
// v1.1.6 SSE realtime surface, merged at the root (deliberately NOT
|
||||
|
||||
@@ -39,6 +39,11 @@ async fn run_server() -> anyhow::Result<()> {
|
||||
let database_url =
|
||||
std::env::var("DATABASE_URL").map_err(|_| anyhow::anyhow!("DATABASE_URL is required"))?;
|
||||
|
||||
// Source the process master key BEFORE doing any work — an unset or
|
||||
// malformed PICLOUD_SECRET_KEY is fatal (v1.1.7). The only escape
|
||||
// hatch is PICLOUD_DEV_MODE=true, which logs a prominent warning.
|
||||
let master_key = picloud_shared::MasterKey::from_env()?;
|
||||
|
||||
let pool = init_db(&database_url).await?;
|
||||
migrations::run(&pool).await?;
|
||||
tracing::info!("migrations applied");
|
||||
@@ -69,7 +74,7 @@ async fn run_server() -> anyhow::Result<()> {
|
||||
// so a delayed sweep can't extend session lifetimes.
|
||||
spawn_session_pruner(auth.sessions.clone());
|
||||
|
||||
let app = build_app(pool, auth).await?;
|
||||
let app = build_app(pool, auth, master_key).await?;
|
||||
|
||||
let listener = tokio::net::TcpListener::bind(addr).await?;
|
||||
tracing::info!(%addr, "picloud all-in-one listening");
|
||||
|
||||
@@ -40,7 +40,13 @@ async fn server_with_app(pool: PgPool) -> (TestServer, String) {
|
||||
.await
|
||||
.expect("seed admin");
|
||||
|
||||
let app = picloud::build_app(pool, auth).await.expect("build_app");
|
||||
let app = picloud::build_app(
|
||||
pool,
|
||||
auth,
|
||||
picloud_shared::MasterKey::from_bytes([0x42u8; 32]),
|
||||
)
|
||||
.await
|
||||
.expect("build_app");
|
||||
let mut server = TestServer::new(app).expect("TestServer should build");
|
||||
|
||||
let resp = server
|
||||
|
||||
@@ -57,9 +57,13 @@ async fn boot(pool: PgPool) -> Seeded {
|
||||
.await
|
||||
.expect("seed owner");
|
||||
|
||||
let app = picloud::build_app(pool.clone(), auth)
|
||||
.await
|
||||
.expect("build_app");
|
||||
let app = picloud::build_app(
|
||||
pool.clone(),
|
||||
auth,
|
||||
picloud_shared::MasterKey::from_bytes([0x42u8; 32]),
|
||||
)
|
||||
.await
|
||||
.expect("build_app");
|
||||
let server = TestServer::new(app).expect("TestServer");
|
||||
|
||||
// Default app id (seeded by migration 0005).
|
||||
|
||||
@@ -67,7 +67,13 @@ async fn server_for(pool: PgPool, suffix: &str) -> (TestServer, String) {
|
||||
.await
|
||||
.expect("seed admin");
|
||||
|
||||
let app = picloud::build_app(pool, auth).await.expect("build_app");
|
||||
let app = picloud::build_app(
|
||||
pool,
|
||||
auth,
|
||||
picloud_shared::MasterKey::from_bytes([0x42u8; 32]),
|
||||
)
|
||||
.await
|
||||
.expect("build_app");
|
||||
let mut server = TestServer::new(app).expect("TestServer");
|
||||
let resp = server
|
||||
.post("/api/v1/admin/auth/login")
|
||||
@@ -297,23 +303,128 @@ async fn dispatcher_delivers_pubsub_to_handler() {
|
||||
assert_eq!(event["pubsub"]["message"]["hello"], 1);
|
||||
}
|
||||
|
||||
/// Count dead_letters rows for an app.
|
||||
async fn dead_letter_count(pool: &PgPool, app_id: &str) -> i64 {
|
||||
let app_uuid = Uuid::parse_str(app_id).unwrap();
|
||||
sqlx::query_scalar("SELECT COUNT(*) FROM dead_letters WHERE app_id = $1")
|
||||
.bind(app_uuid)
|
||||
.fetch_one(pool)
|
||||
.await
|
||||
.expect("count dead_letters")
|
||||
}
|
||||
|
||||
async fn poll_dead_letter_count(pool: &PgPool, app_id: &str, want: i64) -> i64 {
|
||||
let mut count = 0;
|
||||
for _ in 0..100 {
|
||||
count = dead_letter_count(pool, app_id).await;
|
||||
if count >= want {
|
||||
break;
|
||||
}
|
||||
tokio::time::sleep(Duration::from_millis(100)).await;
|
||||
}
|
||||
count
|
||||
}
|
||||
|
||||
/// Register a failing KV trigger on `dlsrc` (single attempt → immediate
|
||||
/// dead-letter) and a `dead_letter` trigger pointing at the marker
|
||||
/// handler, then cause the originating KV event. Returns when set up.
|
||||
async fn setup_dead_letter(server: &TestServer, app_id: &str, dl_handler: &str) {
|
||||
let failing = create_script(server, app_id, "dl-failing", r#"throw "boom";"#).await;
|
||||
server
|
||||
.post(&format!("/api/v1/admin/apps/{app_id}/triggers/kv"))
|
||||
.json(&json!({
|
||||
"script_id": failing,
|
||||
"collection_glob": "dlsrc",
|
||||
"retry_max_attempts": 1,
|
||||
"retry_base_ms": 0
|
||||
}))
|
||||
.await
|
||||
.assert_status(axum::http::StatusCode::CREATED);
|
||||
// The dead_letter trigger (no filters → matches any dead-letter).
|
||||
server
|
||||
.post(&format!("/api/v1/admin/apps/{app_id}/triggers/dead_letter"))
|
||||
.json(&json!({ "script_id": dl_handler }))
|
||||
.await
|
||||
.assert_status(axum::http::StatusCode::CREATED);
|
||||
|
||||
let source = create_script(
|
||||
server,
|
||||
app_id,
|
||||
"dl-source",
|
||||
r#"kv::collection("dlsrc").set("k", 1); #{ ok: true }"#,
|
||||
)
|
||||
.await;
|
||||
execute(server, source.as_str()).await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn dispatcher_delivers_dead_letter_to_handler() {
|
||||
// NOTE: the dead-letter creation path (`dispatcher::handle_failure` →
|
||||
// `DeadLetterRepo::insert`) writes the `dead_letters` row but does not
|
||||
// appear to enqueue deliveries for `dead_letter`-kind triggers
|
||||
// (`TriggerRepo::list_matching_dead_letter` has no production caller —
|
||||
// see HANDBACK latent-findings). So this test asserts the wired
|
||||
// behavior: a failing handler that exhausts its (single) attempt
|
||||
// produces a dead-letter row. If/when DL→handler fan-out lands, this
|
||||
// can be upgraded to assert the handler marker like the others.
|
||||
// v1.1.7: the dead-letter fan-out is now wired
|
||||
// (`dispatcher::handle_failure` → `list_matching_dead_letter` →
|
||||
// outbox). This asserts BOTH that the `dead_letters` row is written
|
||||
// AND that the registered `dead_letter`-kind handler actually fires
|
||||
// (it was silently non-functional v1.1.1–v1.1.6).
|
||||
let Some(pool) = pool_or_skip().await else {
|
||||
return;
|
||||
};
|
||||
let (server, app_id) = server_for(pool.clone(), "dl").await;
|
||||
let handler = create_script(&server, &app_id, "dl-handler", MARKER_HANDLER).await;
|
||||
setup_dead_letter(&server, &app_id, &handler).await;
|
||||
|
||||
// Row written.
|
||||
assert!(
|
||||
poll_dead_letter_count(&pool, &app_id, 1).await > 0,
|
||||
"a dead-letter row should have been produced"
|
||||
);
|
||||
// Handler fired.
|
||||
let event = poll_marker(&pool, &app_id)
|
||||
.await
|
||||
.expect("dead-letter handler fired");
|
||||
assert_eq!(event["source"], "dead_letter");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn dispatcher_delivers_dead_letter_to_handler_actually_fires() {
|
||||
// Focused on the handler-fire side: the marker handler receives a
|
||||
// fully-shaped dead-letter event (the original KV event nested under
|
||||
// `ctx.event.dead_letter.original`, plus the failure metadata).
|
||||
let Some(pool) = pool_or_skip().await else {
|
||||
return;
|
||||
};
|
||||
let (server, app_id) = server_for(pool.clone(), "dlfire").await;
|
||||
let handler = create_script(&server, &app_id, "dl-handler", MARKER_HANDLER).await;
|
||||
setup_dead_letter(&server, &app_id, &handler).await;
|
||||
|
||||
let event = poll_marker(&pool, &app_id)
|
||||
.await
|
||||
.expect("dead-letter handler fired");
|
||||
assert_eq!(event["source"], "dead_letter");
|
||||
// The original KV event is nested verbatim.
|
||||
assert_eq!(event["dead_letter"]["original"]["source"], "kv");
|
||||
assert_eq!(
|
||||
event["dead_letter"]["original"]["kv"]["collection"],
|
||||
"dlsrc"
|
||||
);
|
||||
// Failure metadata is present.
|
||||
assert!(event["dead_letter"]["last_error"]
|
||||
.as_str()
|
||||
.unwrap()
|
||||
.contains("boom"));
|
||||
assert!(event["dead_letter"]["attempts"].as_i64().unwrap() >= 1);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn dead_letter_source_filter_excludes_nonmatching() {
|
||||
// `list_matching_dead_letter` filters by source (among trigger_id /
|
||||
// script_id). A dead_letter trigger whose `source_filter` is "docs"
|
||||
// must NOT fire for a "kv"-sourced dead-letter — the row is still
|
||||
// written, but no handler delivery is enqueued.
|
||||
let Some(pool) = pool_or_skip().await else {
|
||||
return;
|
||||
};
|
||||
let (server, app_id) = server_for(pool.clone(), "dlfilter").await;
|
||||
let handler = create_script(&server, &app_id, "dl-handler", MARKER_HANDLER).await;
|
||||
|
||||
// A handler that always throws, with a single attempt so it
|
||||
// dead-letters immediately (no retry backoff).
|
||||
let failing = create_script(&server, &app_id, "dl-failing", r#"throw "boom";"#).await;
|
||||
server
|
||||
.post(&format!("/api/v1/admin/apps/{app_id}/triggers/kv"))
|
||||
@@ -325,6 +436,12 @@ async fn dispatcher_delivers_dead_letter_to_handler() {
|
||||
}))
|
||||
.await
|
||||
.assert_status(axum::http::StatusCode::CREATED);
|
||||
// Filter to a different source so this handler must NOT match.
|
||||
server
|
||||
.post(&format!("/api/v1/admin/apps/{app_id}/triggers/dead_letter"))
|
||||
.json(&json!({ "script_id": handler, "source_filter": "docs" }))
|
||||
.await
|
||||
.assert_status(axum::http::StatusCode::CREATED);
|
||||
|
||||
let source = create_script(
|
||||
&server,
|
||||
@@ -335,19 +452,38 @@ async fn dispatcher_delivers_dead_letter_to_handler() {
|
||||
.await;
|
||||
execute(&server, &source).await;
|
||||
|
||||
// Poll the dead_letters table for this app.
|
||||
let app_uuid = Uuid::parse_str(&app_id).unwrap();
|
||||
let mut count: i64 = 0;
|
||||
for _ in 0..100 {
|
||||
count = sqlx::query_scalar("SELECT COUNT(*) FROM dead_letters WHERE app_id = $1")
|
||||
.bind(app_uuid)
|
||||
.fetch_one(&pool)
|
||||
.await
|
||||
.expect("count dead_letters");
|
||||
if count > 0 {
|
||||
break;
|
||||
}
|
||||
tokio::time::sleep(Duration::from_millis(100)).await;
|
||||
}
|
||||
assert!(count > 0, "a dead-letter row should have been produced");
|
||||
// The dead-letter row is written…
|
||||
assert!(poll_dead_letter_count(&pool, &app_id, 1).await >= 1);
|
||||
// …but the source-filtered handler never fires.
|
||||
let marker = poll_marker_n(&pool, &app_id, 8).await;
|
||||
assert!(
|
||||
marker.is_none(),
|
||||
"source_filter='docs' must not fire for a kv dead-letter"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn dead_letter_handler_failure_does_not_recurse() {
|
||||
// Recursion-stop (design notes §4): a dead_letter handler that itself
|
||||
// throws must NOT produce a second dead-letter row. The
|
||||
// `is_dead_letter_handler` short-circuit annotates the original row
|
||||
// and drops the outbox row without re-dead-lettering.
|
||||
let Some(pool) = pool_or_skip().await else {
|
||||
return;
|
||||
};
|
||||
let (server, app_id) = server_for(pool.clone(), "dlrec").await;
|
||||
// The DL handler itself throws.
|
||||
let throwing = create_script(&server, &app_id, "dl-throws", r#"throw "handler boom";"#).await;
|
||||
setup_dead_letter(&server, &app_id, &throwing).await;
|
||||
|
||||
// One dead-letter row appears (the original). Give the throwing
|
||||
// handler time to run + (not) recurse, then confirm the count stayed
|
||||
// at exactly 1.
|
||||
assert!(poll_dead_letter_count(&pool, &app_id, 1).await >= 1);
|
||||
tokio::time::sleep(Duration::from_millis(800)).await;
|
||||
assert_eq!(
|
||||
dead_letter_count(&pool, &app_id).await,
|
||||
1,
|
||||
"a failing dead-letter handler must not create a new dead-letter row"
|
||||
);
|
||||
}
|
||||
|
||||
298
crates/picloud/tests/email_inbound.rs
Normal file
298
crates/picloud/tests/email_inbound.rs
Normal file
@@ -0,0 +1,298 @@
|
||||
//! End-to-end tests for the inbound-email webhook receiver (v1.1.7).
|
||||
//!
|
||||
//! Gated on `DATABASE_URL` like `dispatcher_e2e.rs`: when unset the test
|
||||
//! prints a notice and returns early so plain `cargo test` stays green.
|
||||
//!
|
||||
//! Covers the receiver's status-code matrix (202 / 401 / 404 / 422),
|
||||
//! cross-app path isolation, HMAC verification (signed + unsigned
|
||||
//! triggers), the dispatcher routing the `email` outbox row, and the
|
||||
//! handler actually firing with `ctx.event.email` populated. The
|
||||
//! "handler fired" observation uses the same KV-marker pattern as
|
||||
//! `dispatcher_e2e.rs`.
|
||||
|
||||
#![allow(clippy::needless_pass_by_value)]
|
||||
|
||||
use std::time::Duration;
|
||||
|
||||
use axum_test::TestServer;
|
||||
use hmac::{Hmac, Mac};
|
||||
use serde_json::{json, Value};
|
||||
use sha2::Sha256;
|
||||
use sqlx::postgres::PgPoolOptions;
|
||||
use sqlx::PgPool;
|
||||
use uuid::Uuid;
|
||||
|
||||
/// Fixed master key so the receiver decrypts the inbound_secret the
|
||||
/// admin endpoint encrypted (same key feeds build_app + the admin path).
|
||||
fn master_key() -> picloud_shared::MasterKey {
|
||||
picloud_shared::MasterKey::from_bytes([0x42u8; 32])
|
||||
}
|
||||
|
||||
async fn pool_or_skip() -> Option<PgPool> {
|
||||
let Ok(url) = std::env::var("DATABASE_URL") else {
|
||||
eprintln!("email_inbound: DATABASE_URL unset — skipping");
|
||||
return None;
|
||||
};
|
||||
let pool = PgPoolOptions::new()
|
||||
.max_connections(5)
|
||||
.connect(&url)
|
||||
.await
|
||||
.expect("connect to DATABASE_URL");
|
||||
sqlx::migrate!("../manager-core/migrations")
|
||||
.run(&pool)
|
||||
.await
|
||||
.expect("apply migrations");
|
||||
Some(pool)
|
||||
}
|
||||
|
||||
async fn server_for(pool: PgPool, suffix: &str) -> (TestServer, String) {
|
||||
use picloud_manager_core::auth::hash_password;
|
||||
use picloud_shared::InstanceRole;
|
||||
|
||||
let unique = format!("{suffix}-{}", Uuid::new_v4().simple());
|
||||
let auth = picloud::AuthDeps::from_pool(pool.clone());
|
||||
let username = format!("eml-{unique}");
|
||||
let hash = hash_password("pw").expect("hash");
|
||||
auth.users
|
||||
.create(&username, &hash, InstanceRole::Owner, None)
|
||||
.await
|
||||
.expect("seed admin");
|
||||
|
||||
let app = picloud::build_app(pool, auth, master_key())
|
||||
.await
|
||||
.expect("build_app");
|
||||
let mut server = TestServer::new(app).expect("TestServer");
|
||||
let resp = server
|
||||
.post("/api/v1/admin/auth/login")
|
||||
.json(&json!({ "username": username, "password": "pw" }))
|
||||
.await;
|
||||
resp.assert_status_ok();
|
||||
let token = resp.json::<Value>()["token"]
|
||||
.as_str()
|
||||
.expect("login token")
|
||||
.to_string();
|
||||
server.add_header("authorization", format!("Bearer {token}"));
|
||||
|
||||
let slug = format!("eml-{unique}");
|
||||
let created: Value = server
|
||||
.post("/api/v1/admin/apps")
|
||||
.json(&json!({ "slug": slug, "name": slug }))
|
||||
.await
|
||||
.json();
|
||||
let app_id = created["id"].as_str().expect("app id").to_string();
|
||||
(server, app_id)
|
||||
}
|
||||
|
||||
async fn create_script(server: &TestServer, app_id: &str, name: &str, source: &str) -> String {
|
||||
let created: Value = server
|
||||
.post("/api/v1/admin/scripts")
|
||||
.json(&json!({ "app_id": app_id, "name": name, "source": source }))
|
||||
.await
|
||||
.json();
|
||||
created["id"].as_str().expect("script id").to_string()
|
||||
}
|
||||
|
||||
const MARKER_HANDLER: &str = r#"
|
||||
let e = ctx.event;
|
||||
kv::collection("e2e_markers").set("marker", e);
|
||||
#{ ok: true }
|
||||
"#;
|
||||
|
||||
async fn create_email_trigger(
|
||||
server: &TestServer,
|
||||
app_id: &str,
|
||||
script_id: &str,
|
||||
secret: Option<&str>,
|
||||
) -> String {
|
||||
let created: Value = server
|
||||
.post(&format!("/api/v1/admin/apps/{app_id}/triggers/email"))
|
||||
.json(&json!({ "script_id": script_id, "inbound_secret": secret }))
|
||||
.await
|
||||
.json();
|
||||
created["id"].as_str().expect("trigger id").to_string()
|
||||
}
|
||||
|
||||
fn sign(secret: &str, body: &str) -> String {
|
||||
let mut mac = Hmac::<Sha256>::new_from_slice(secret.as_bytes()).expect("hmac key");
|
||||
mac.update(body.as_bytes());
|
||||
hex::encode(mac.finalize().into_bytes())
|
||||
}
|
||||
|
||||
async fn poll_marker(pool: &PgPool, app_id: &str) -> Option<Value> {
|
||||
let app_uuid = Uuid::parse_str(app_id).expect("app uuid");
|
||||
for _ in 0..100 {
|
||||
let row: Option<(Value,)> = sqlx::query_as(
|
||||
"SELECT value FROM kv_entries \
|
||||
WHERE app_id = $1 AND collection = 'e2e_markers' AND key = 'marker'",
|
||||
)
|
||||
.bind(app_uuid)
|
||||
.fetch_optional(pool)
|
||||
.await
|
||||
.expect("query marker");
|
||||
if let Some((value,)) = row {
|
||||
return Some(value);
|
||||
}
|
||||
tokio::time::sleep(Duration::from_millis(100)).await;
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
const BODY: &str = r#"{"from":"sender@external.com","to":["alice@myapp.com"],"cc":["bob@myapp.com"],"subject":"Re: question","text":"hello there","message_id":"<abc@external.com>"}"#;
|
||||
|
||||
#[tokio::test]
|
||||
async fn signed_post_accepts_and_fires_handler() {
|
||||
let Some(pool) = pool_or_skip().await else {
|
||||
return;
|
||||
};
|
||||
let (server, app_id) = server_for(pool.clone(), "signed").await;
|
||||
let handler = create_script(&server, &app_id, "eml-handler", MARKER_HANDLER).await;
|
||||
let trigger = create_email_trigger(&server, &app_id, &handler, Some("topsecret")).await;
|
||||
|
||||
let sig = sign("topsecret", BODY);
|
||||
server
|
||||
.post(&format!("/api/v1/email-inbound/{app_id}/{trigger}"))
|
||||
.add_header("x-picloud-signature", sig)
|
||||
.text(BODY)
|
||||
.await
|
||||
.assert_status(axum::http::StatusCode::ACCEPTED);
|
||||
|
||||
// Outbox row landed with source_kind = 'email'.
|
||||
let app_uuid = Uuid::parse_str(&app_id).unwrap();
|
||||
// The dispatcher deletes the row after delivery; instead assert the
|
||||
// handler fired (which proves the email row was dispatched).
|
||||
let event = poll_marker(&pool, &app_id).await.expect("handler fired");
|
||||
assert_eq!(event["source"], "email");
|
||||
assert_eq!(event["op"], "receive");
|
||||
assert_eq!(event["email"]["from"], "sender@external.com");
|
||||
assert_eq!(event["email"]["to"][0], "alice@myapp.com");
|
||||
assert_eq!(event["email"]["cc"][0], "bob@myapp.com");
|
||||
assert_eq!(event["email"]["subject"], "Re: question");
|
||||
assert_eq!(event["email"]["text"], "hello there");
|
||||
assert_eq!(event["email"]["message_id"], "<abc@external.com>");
|
||||
let _ = app_uuid;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn missing_signature_is_401_when_secret_configured() {
|
||||
let Some(pool) = pool_or_skip().await else {
|
||||
return;
|
||||
};
|
||||
let (server, app_id) = server_for(pool, "nosig").await;
|
||||
let handler = create_script(&server, &app_id, "h", MARKER_HANDLER).await;
|
||||
let trigger = create_email_trigger(&server, &app_id, &handler, Some("topsecret")).await;
|
||||
|
||||
server
|
||||
.post(&format!("/api/v1/email-inbound/{app_id}/{trigger}"))
|
||||
.text(BODY)
|
||||
.await
|
||||
.assert_status(axum::http::StatusCode::UNAUTHORIZED);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn wrong_signature_is_401() {
|
||||
let Some(pool) = pool_or_skip().await else {
|
||||
return;
|
||||
};
|
||||
let (server, app_id) = server_for(pool, "wrongsig").await;
|
||||
let handler = create_script(&server, &app_id, "h", MARKER_HANDLER).await;
|
||||
let trigger = create_email_trigger(&server, &app_id, &handler, Some("topsecret")).await;
|
||||
|
||||
server
|
||||
.post(&format!("/api/v1/email-inbound/{app_id}/{trigger}"))
|
||||
.add_header("x-picloud-signature", sign("WRONG", BODY))
|
||||
.text(BODY)
|
||||
.await
|
||||
.assert_status(axum::http::StatusCode::UNAUTHORIZED);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn unsigned_trigger_accepts_without_signature() {
|
||||
let Some(pool) = pool_or_skip().await else {
|
||||
return;
|
||||
};
|
||||
let (server, app_id) = server_for(pool, "unsigned").await;
|
||||
let handler = create_script(&server, &app_id, "h", MARKER_HANDLER).await;
|
||||
let trigger = create_email_trigger(&server, &app_id, &handler, None).await;
|
||||
|
||||
server
|
||||
.post(&format!("/api/v1/email-inbound/{app_id}/{trigger}"))
|
||||
.text(BODY)
|
||||
.await
|
||||
.assert_status(axum::http::StatusCode::ACCEPTED);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn unknown_trigger_is_404() {
|
||||
let Some(pool) = pool_or_skip().await else {
|
||||
return;
|
||||
};
|
||||
let (server, app_id) = server_for(pool, "missing").await;
|
||||
let missing = Uuid::new_v4();
|
||||
server
|
||||
.post(&format!("/api/v1/email-inbound/{app_id}/{missing}"))
|
||||
.text(BODY)
|
||||
.await
|
||||
.assert_status(axum::http::StatusCode::NOT_FOUND);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn wrong_kind_trigger_is_404() {
|
||||
let Some(pool) = pool_or_skip().await else {
|
||||
return;
|
||||
};
|
||||
let (server, app_id) = server_for(pool, "wrongkind").await;
|
||||
let handler = create_script(&server, &app_id, "h", MARKER_HANDLER).await;
|
||||
// A KV trigger — not an email trigger.
|
||||
let kv_trigger: Value = server
|
||||
.post(&format!("/api/v1/admin/apps/{app_id}/triggers/kv"))
|
||||
.json(&json!({ "script_id": handler, "collection_glob": "*" }))
|
||||
.await
|
||||
.json();
|
||||
let kv_id = kv_trigger["id"].as_str().unwrap();
|
||||
server
|
||||
.post(&format!("/api/v1/email-inbound/{app_id}/{kv_id}"))
|
||||
.text(BODY)
|
||||
.await
|
||||
.assert_status(axum::http::StatusCode::NOT_FOUND);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn malformed_body_is_422() {
|
||||
let Some(pool) = pool_or_skip().await else {
|
||||
return;
|
||||
};
|
||||
let (server, app_id) = server_for(pool, "malformed").await;
|
||||
let handler = create_script(&server, &app_id, "h", MARKER_HANDLER).await;
|
||||
// Unsigned so we reach the parse step.
|
||||
let trigger = create_email_trigger(&server, &app_id, &handler, None).await;
|
||||
server
|
||||
.post(&format!("/api/v1/email-inbound/{app_id}/{trigger}"))
|
||||
.text("not json at all")
|
||||
.await
|
||||
.assert_status(axum::http::StatusCode::UNPROCESSABLE_ENTITY);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn cross_app_path_is_404() {
|
||||
let Some(pool) = pool_or_skip().await else {
|
||||
return;
|
||||
};
|
||||
// Two apps under the same server. A trigger created in app B must
|
||||
// not be reachable via app A's path segment.
|
||||
let (server, app_a) = server_for(pool.clone(), "xa").await;
|
||||
let app_b: Value = server
|
||||
.post("/api/v1/admin/apps")
|
||||
.json(&json!({ "slug": format!("xb-{}", Uuid::new_v4().simple()), "name": "xb" }))
|
||||
.await
|
||||
.json();
|
||||
let app_b_id = app_b["id"].as_str().unwrap().to_string();
|
||||
let handler_b = create_script(&server, &app_b_id, "hb", MARKER_HANDLER).await;
|
||||
let trigger_b = create_email_trigger(&server, &app_b_id, &handler_b, None).await;
|
||||
|
||||
// POST to app A's path with app B's trigger id → 404 (path-bound).
|
||||
server
|
||||
.post(&format!("/api/v1/email-inbound/{app_a}/{trigger_b}"))
|
||||
.text(BODY)
|
||||
.await
|
||||
.assert_status(axum::http::StatusCode::NOT_FOUND);
|
||||
}
|
||||
@@ -21,6 +21,10 @@ tokio = { workspace = true, features = ["sync"] }
|
||||
hmac.workspace = true
|
||||
sha2.workspace = true
|
||||
base64.workspace = true
|
||||
# AES-256-GCM envelope + master-key sourcing (v1.1.7 crypto module).
|
||||
aes-gcm.workspace = true
|
||||
rand.workspace = true
|
||||
tracing.workspace = true
|
||||
|
||||
[dev-dependencies]
|
||||
tokio = { workspace = true, features = ["macros", "rt", "rt-multi-thread", "time", "sync"] }
|
||||
|
||||
358
crates/shared/src/crypto.rs
Normal file
358
crates/shared/src/crypto.rs
Normal file
@@ -0,0 +1,358 @@
|
||||
//! AES-256-GCM encryption envelope + master-key sourcing (v1.1.7).
|
||||
//!
|
||||
//! Two responsibilities:
|
||||
//!
|
||||
//! 1. [`encrypt`] / [`decrypt`] — the at-rest envelope used by per-app
|
||||
//! `secrets`, the encrypted `inbound_secret` on email triggers, and
|
||||
//! the realtime signing key. `Aes256Gcm` with a 96-bit (12-byte)
|
||||
//! random nonce and a 128-bit auth tag **appended to the
|
||||
//! ciphertext** (the RustCrypto `Aead`-trait layout — `encrypt`
|
||||
//! returns `ciphertext || tag`, `decrypt` consumes the same). Both
|
||||
//! the ciphertext (tag included) and the nonce are stored.
|
||||
//!
|
||||
//! 2. [`MasterKey`] — the process-wide 32-byte key, sourced once at
|
||||
//! startup from `PICLOUD_SECRET_KEY` (base64 of exactly 32 bytes).
|
||||
//! A deterministic in-memory dev key is allowed ONLY when the env
|
||||
//! var is unset AND `PICLOUD_DEV_MODE=true`; otherwise an unset key
|
||||
//! is fatal (no quiet "your secrets are unencrypted" mode).
|
||||
//!
|
||||
//! **Key rotation is out of scope for v1.1.7.** Changing
|
||||
//! `PICLOUD_SECRET_KEY` between deploys orphans every existing
|
||||
//! ciphertext (it can no longer be decrypted). v1.2+ adds key-version
|
||||
//! columns + a re-encryption pass.
|
||||
|
||||
use aes_gcm::aead::{Aead, KeyInit};
|
||||
use aes_gcm::{Aes256Gcm, Key, Nonce};
|
||||
use base64::engine::general_purpose::STANDARD as B64;
|
||||
use base64::Engine as _;
|
||||
use rand::RngCore;
|
||||
use sha2::{Digest, Sha256};
|
||||
use thiserror::Error;
|
||||
|
||||
/// Master-key length in bytes (AES-256 → 32-byte key).
|
||||
pub const KEY_LEN: usize = 32;
|
||||
|
||||
/// GCM nonce length in bytes (96-bit nonce, the AES-GCM standard).
|
||||
pub const NONCE_LEN: usize = 12;
|
||||
|
||||
/// Output of [`encrypt`]: the ciphertext (auth tag appended) plus the
|
||||
/// randomly-generated nonce. Both must be persisted; `decrypt` needs
|
||||
/// the nonce to recover the plaintext.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct EncryptResult {
|
||||
/// Ciphertext with the 16-byte GCM auth tag appended.
|
||||
pub ciphertext: Vec<u8>,
|
||||
/// The 12-byte nonce used for this encryption.
|
||||
pub nonce: [u8; NONCE_LEN],
|
||||
}
|
||||
|
||||
/// Errors from the encryption envelope.
|
||||
#[derive(Debug, Error)]
|
||||
pub enum CryptoError {
|
||||
/// Authentication failed — wrong key, corrupted ciphertext, or a
|
||||
/// tampered nonce/tag. GCM does not distinguish these (by design),
|
||||
/// so neither do we.
|
||||
#[error("decryption failed: authentication tag mismatch (wrong key, corrupted ciphertext, or tampered nonce)")]
|
||||
Decrypt,
|
||||
|
||||
/// The stored nonce was not exactly [`NONCE_LEN`] bytes — a sign of
|
||||
/// row corruption.
|
||||
#[error("invalid nonce length: expected {NONCE_LEN} bytes, got {0}")]
|
||||
InvalidNonce(usize),
|
||||
}
|
||||
|
||||
/// Encrypt `plaintext` under `key`, generating a fresh random nonce.
|
||||
///
|
||||
/// The auth tag is appended to the returned ciphertext (RustCrypto
|
||||
/// `Aead` layout). Encryption with a valid 32-byte key and 12-byte
|
||||
/// nonce is infallible in `aes-gcm`, so this returns a value rather
|
||||
/// than a `Result`.
|
||||
#[must_use]
|
||||
pub fn encrypt(plaintext: &[u8], key: &[u8; KEY_LEN]) -> EncryptResult {
|
||||
let cipher = Aes256Gcm::new(Key::<Aes256Gcm>::from_slice(key));
|
||||
let mut nonce_bytes = [0u8; NONCE_LEN];
|
||||
// CSPRNG nonce. `thread_rng` is seeded from the OS CSPRNG; a fresh
|
||||
// 96-bit nonce per encryption keeps the (key, nonce) pair unique.
|
||||
rand::thread_rng().fill_bytes(&mut nonce_bytes);
|
||||
let nonce = Nonce::from_slice(&nonce_bytes);
|
||||
let ciphertext = cipher
|
||||
.encrypt(nonce, plaintext)
|
||||
.expect("AES-256-GCM encryption is infallible for a valid key + 12-byte nonce");
|
||||
EncryptResult {
|
||||
ciphertext,
|
||||
nonce: nonce_bytes,
|
||||
}
|
||||
}
|
||||
|
||||
/// Decrypt `ciphertext` (auth tag appended) with the stored `nonce`
|
||||
/// under `key`.
|
||||
///
|
||||
/// # Errors
|
||||
///
|
||||
/// Returns [`CryptoError::InvalidNonce`] if `nonce` is the wrong length,
|
||||
/// or [`CryptoError::Decrypt`] if authentication fails for any reason
|
||||
/// (wrong key, corruption, tampering).
|
||||
pub fn decrypt(
|
||||
ciphertext: &[u8],
|
||||
nonce: &[u8],
|
||||
key: &[u8; KEY_LEN],
|
||||
) -> Result<Vec<u8>, CryptoError> {
|
||||
if nonce.len() != NONCE_LEN {
|
||||
return Err(CryptoError::InvalidNonce(nonce.len()));
|
||||
}
|
||||
let cipher = Aes256Gcm::new(Key::<Aes256Gcm>::from_slice(key));
|
||||
let nonce = Nonce::from_slice(nonce);
|
||||
cipher
|
||||
.decrypt(nonce, ciphertext)
|
||||
.map_err(|_| CryptoError::Decrypt)
|
||||
}
|
||||
|
||||
/// The process-wide master key. Sourced once at startup and threaded
|
||||
/// into the secrets service, the email-trigger receiver, and the
|
||||
/// realtime signing-key migration.
|
||||
///
|
||||
/// Cheap to clone (32 bytes). `Debug` is redacted so the key never
|
||||
/// lands in a log line.
|
||||
#[derive(Clone)]
|
||||
pub struct MasterKey {
|
||||
key: [u8; KEY_LEN],
|
||||
}
|
||||
|
||||
impl std::fmt::Debug for MasterKey {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
f.debug_struct("MasterKey")
|
||||
.field("key", &"<redacted 32 bytes>")
|
||||
.finish()
|
||||
}
|
||||
}
|
||||
|
||||
/// Failure modes for master-key sourcing. Every variant is a fatal
|
||||
/// startup error — there is no fallback to a quiet plaintext mode.
|
||||
#[derive(Debug, Error)]
|
||||
pub enum MasterKeyError {
|
||||
/// `PICLOUD_SECRET_KEY` is unset/empty and dev mode is off.
|
||||
#[error(
|
||||
"PICLOUD_SECRET_KEY is required but unset. Generate one with `openssl rand -base64 32`, \
|
||||
or set PICLOUD_DEV_MODE=true to use an insecure deterministic dev key (never in production)."
|
||||
)]
|
||||
Missing,
|
||||
|
||||
/// `PICLOUD_SECRET_KEY` was not valid base64.
|
||||
#[error("PICLOUD_SECRET_KEY is not valid base64 (expected base64 of 32 bytes — `openssl rand -base64 32`)")]
|
||||
Malformed,
|
||||
|
||||
/// Decoded to the wrong number of bytes.
|
||||
#[error("PICLOUD_SECRET_KEY must decode to exactly {KEY_LEN} bytes, got {0}")]
|
||||
WrongLength(usize),
|
||||
}
|
||||
|
||||
impl MasterKey {
|
||||
/// Borrow the raw 32-byte key for the crypto envelope.
|
||||
#[must_use]
|
||||
pub const fn as_bytes(&self) -> &[u8; KEY_LEN] {
|
||||
&self.key
|
||||
}
|
||||
|
||||
/// Build a key directly from 32 bytes (used by the realtime
|
||||
/// migration's tests and by [`Self::from_base64`]).
|
||||
#[must_use]
|
||||
pub const fn from_bytes(key: [u8; KEY_LEN]) -> Self {
|
||||
Self { key }
|
||||
}
|
||||
|
||||
/// Decode a base64-encoded 32-byte key.
|
||||
///
|
||||
/// # Errors
|
||||
///
|
||||
/// [`MasterKeyError::Malformed`] for non-base64 input,
|
||||
/// [`MasterKeyError::WrongLength`] when the decoded length is not 32.
|
||||
pub fn from_base64(s: &str) -> Result<Self, MasterKeyError> {
|
||||
let decoded = B64
|
||||
.decode(s.trim().as_bytes())
|
||||
.map_err(|_| MasterKeyError::Malformed)?;
|
||||
let len = decoded.len();
|
||||
let key: [u8; KEY_LEN] = decoded
|
||||
.try_into()
|
||||
.map_err(|_| MasterKeyError::WrongLength(len))?;
|
||||
Ok(Self { key })
|
||||
}
|
||||
|
||||
/// Source the master key from the process environment per the
|
||||
/// v1.1.7 rules. See [`Self::resolve`] for the decision logic.
|
||||
///
|
||||
/// # Errors
|
||||
///
|
||||
/// Propagates [`MasterKeyError`] when the key is absent (and dev
|
||||
/// mode is off) or malformed.
|
||||
pub fn from_env() -> Result<Self, MasterKeyError> {
|
||||
let secret = std::env::var("PICLOUD_SECRET_KEY").ok();
|
||||
let dev_mode = std::env::var("PICLOUD_DEV_MODE")
|
||||
.map(|v| is_truthy(&v))
|
||||
.unwrap_or(false);
|
||||
Self::resolve(secret.as_deref(), dev_mode)
|
||||
}
|
||||
|
||||
/// Pure resolution logic, factored out of [`Self::from_env`] so it's
|
||||
/// testable without mutating process-global env vars.
|
||||
///
|
||||
/// * `secret` present + non-empty → parse it (fatal if malformed).
|
||||
/// * `secret` absent/empty + `dev_mode` → deterministic dev key +
|
||||
/// a prominent warning.
|
||||
/// * `secret` absent/empty + no dev mode → fatal.
|
||||
///
|
||||
/// # Errors
|
||||
///
|
||||
/// See [`Self::from_env`].
|
||||
pub fn resolve(secret: Option<&str>, dev_mode: bool) -> Result<Self, MasterKeyError> {
|
||||
match secret {
|
||||
Some(v) if !v.trim().is_empty() => Self::from_base64(v),
|
||||
_ if dev_mode => {
|
||||
tracing::warn!(
|
||||
"PICLOUD_SECRET_KEY is unset and PICLOUD_DEV_MODE=true: using a DETERMINISTIC \
|
||||
in-memory dev master key. At-rest secrets are NOT secure in this mode. \
|
||||
Never run a real deployment without PICLOUD_SECRET_KEY."
|
||||
);
|
||||
Ok(Self::dev_key())
|
||||
}
|
||||
_ => Err(MasterKeyError::Missing),
|
||||
}
|
||||
}
|
||||
|
||||
/// Deterministic dev key: SHA-256 of a fixed label. Stable across
|
||||
/// restarts so dev secrets survive a reboot, but obviously not a
|
||||
/// real secret (the input is public).
|
||||
#[must_use]
|
||||
fn dev_key() -> Self {
|
||||
let digest = Sha256::digest(b"picloud-dev-master-key-v1.1.7");
|
||||
let mut key = [0u8; KEY_LEN];
|
||||
key.copy_from_slice(&digest);
|
||||
Self { key }
|
||||
}
|
||||
}
|
||||
|
||||
/// Common env-var truthiness check shared with the other config knobs.
|
||||
fn is_truthy(v: &str) -> bool {
|
||||
matches!(v.trim().to_ascii_lowercase().as_str(), "1" | "true" | "yes")
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn test_key() -> [u8; KEY_LEN] {
|
||||
let mut k = [0u8; KEY_LEN];
|
||||
for (i, b) in k.iter_mut().enumerate() {
|
||||
*b = u8::try_from(i).unwrap_or(0);
|
||||
}
|
||||
k
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn round_trip_recovers_plaintext() {
|
||||
let key = test_key();
|
||||
let plaintext = b"sk_live_super_secret_value";
|
||||
let enc = encrypt(plaintext, &key);
|
||||
let dec = decrypt(&enc.ciphertext, &enc.nonce, &key).unwrap();
|
||||
assert_eq!(dec, plaintext);
|
||||
// Tag is appended → ciphertext is longer than plaintext.
|
||||
assert!(enc.ciphertext.len() > plaintext.len());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn round_trip_empty_plaintext() {
|
||||
let key = test_key();
|
||||
let enc = encrypt(b"", &key);
|
||||
let dec = decrypt(&enc.ciphertext, &enc.nonce, &key).unwrap();
|
||||
assert!(dec.is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn tampered_ciphertext_fails() {
|
||||
let key = test_key();
|
||||
let mut enc = encrypt(b"hello world", &key);
|
||||
enc.ciphertext[0] ^= 0xff;
|
||||
let err = decrypt(&enc.ciphertext, &enc.nonce, &key).unwrap_err();
|
||||
assert!(matches!(err, CryptoError::Decrypt));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn tampered_nonce_fails() {
|
||||
let key = test_key();
|
||||
let enc = encrypt(b"hello world", &key);
|
||||
let mut nonce = enc.nonce;
|
||||
nonce[0] ^= 0xff;
|
||||
let err = decrypt(&enc.ciphertext, &nonce, &key).unwrap_err();
|
||||
assert!(matches!(err, CryptoError::Decrypt));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn wrong_key_fails() {
|
||||
let key = test_key();
|
||||
let mut other = test_key();
|
||||
other[31] ^= 0xff;
|
||||
let enc = encrypt(b"hello world", &key);
|
||||
let err = decrypt(&enc.ciphertext, &enc.nonce, &other).unwrap_err();
|
||||
assert!(matches!(err, CryptoError::Decrypt));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn wrong_length_nonce_rejected() {
|
||||
let key = test_key();
|
||||
let enc = encrypt(b"hi", &key);
|
||||
let err = decrypt(&enc.ciphertext, &enc.nonce[..8], &key).unwrap_err();
|
||||
assert!(matches!(err, CryptoError::InvalidNonce(8)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn distinct_nonces_per_encryption() {
|
||||
let key = test_key();
|
||||
let a = encrypt(b"same plaintext", &key);
|
||||
let b = encrypt(b"same plaintext", &key);
|
||||
// Random nonce → ciphertext differs even for identical input.
|
||||
assert_ne!(a.nonce, b.nonce);
|
||||
assert_ne!(a.ciphertext, b.ciphertext);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn master_key_from_valid_base64() {
|
||||
let raw = [7u8; KEY_LEN];
|
||||
let b64 = B64.encode(raw);
|
||||
let mk = MasterKey::from_base64(&b64).unwrap();
|
||||
assert_eq!(mk.as_bytes(), &raw);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn master_key_malformed_base64() {
|
||||
let err = MasterKey::from_base64("not valid base64 !!!").unwrap_err();
|
||||
assert!(matches!(err, MasterKeyError::Malformed));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn master_key_wrong_length() {
|
||||
let b64 = B64.encode([1u8; 16]); // 16 bytes, not 32
|
||||
let err = MasterKey::from_base64(&b64).unwrap_err();
|
||||
assert!(matches!(err, MasterKeyError::WrongLength(16)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn resolve_missing_without_dev_is_fatal() {
|
||||
let err = MasterKey::resolve(None, false).unwrap_err();
|
||||
assert!(matches!(err, MasterKeyError::Missing));
|
||||
// Empty string counts as missing too.
|
||||
let err = MasterKey::resolve(Some(" "), false).unwrap_err();
|
||||
assert!(matches!(err, MasterKeyError::Missing));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn resolve_dev_fallback_only_with_dev_mode() {
|
||||
// Dev mode on + no key → deterministic dev key.
|
||||
let a = MasterKey::resolve(None, true).unwrap();
|
||||
let b = MasterKey::resolve(None, true).unwrap();
|
||||
assert_eq!(a.as_bytes(), b.as_bytes(), "dev key must be deterministic");
|
||||
// A real key always wins over dev mode.
|
||||
let raw = [9u8; KEY_LEN];
|
||||
let real = MasterKey::resolve(Some(&B64.encode(raw)), true).unwrap();
|
||||
assert_eq!(real.as_bytes(), &raw);
|
||||
assert_ne!(real.as_bytes(), a.as_bytes());
|
||||
}
|
||||
}
|
||||
89
crates/shared/src/email.rs
Normal file
89
crates/shared/src/email.rs
Normal file
@@ -0,0 +1,89 @@
|
||||
//! `EmailService` — the v1.1.7 outbound email contract.
|
||||
//!
|
||||
//! Scripts get `email::send(#{...})` (plain text) and
|
||||
//! `email::send_html(#{...})` (multipart text + HTML). Both route to the
|
||||
//! single `send` trait method with an [`OutboundEmail`]; the bridge sets
|
||||
//! `html` only for `send_html`.
|
||||
//!
|
||||
//! Lives in `picloud-shared` (not `manager-core`) so the Rhai bridge and
|
||||
//! the impl share one trait. The impl (an SMTP relay over `lettre`)
|
||||
//! lives in `manager-core::email_service`; `picloud-shared` stays free
|
||||
//! of the `lettre` dependency.
|
||||
//!
|
||||
//! `app_id` is derived from `cx.app_id` (authz only — there is no
|
||||
//! per-app `from` validation in v1.1.7; deliverability is the operator's
|
||||
//! SMTP-relay concern).
|
||||
|
||||
use async_trait::async_trait;
|
||||
use thiserror::Error;
|
||||
|
||||
use crate::SdkCallCx;
|
||||
|
||||
/// A single outbound message. `to`/`cc`/`bcc` are address lists (the
|
||||
/// bridge accepts a String or an Array of Strings). At least one of
|
||||
/// `text` / `html` must be present.
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct OutboundEmail {
|
||||
pub to: Vec<String>,
|
||||
pub cc: Vec<String>,
|
||||
pub bcc: Vec<String>,
|
||||
pub from: String,
|
||||
/// Defaults to `from` when absent.
|
||||
pub reply_to: Option<String>,
|
||||
pub subject: String,
|
||||
pub text: Option<String>,
|
||||
pub html: Option<String>,
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
pub trait EmailService: Send + Sync {
|
||||
/// Validate, build, and send the message. Returns `Ok(())` once the
|
||||
/// SMTP relay has accepted it for delivery (not on actual delivery —
|
||||
/// that's the relay's job).
|
||||
async fn send(&self, cx: &SdkCallCx, email: OutboundEmail) -> Result<(), EmailError>;
|
||||
}
|
||||
|
||||
/// Failure modes surfaced to the Rhai bridge.
|
||||
#[derive(Debug, Error)]
|
||||
pub enum EmailError {
|
||||
/// Caller principal lacked `AppEmailSend`. Only raised when
|
||||
/// `cx.principal.is_some()` (script-as-gate semantics).
|
||||
#[error("forbidden")]
|
||||
Forbidden,
|
||||
|
||||
/// A required field (`to`, `from`, `subject`, or one of `text`/`html`)
|
||||
/// was missing or empty.
|
||||
#[error("missing required email field: {0}")]
|
||||
MissingField(String),
|
||||
|
||||
/// An address failed basic RFC 5322-ish validation.
|
||||
#[error("invalid email address: {0}")]
|
||||
InvalidAddress(String),
|
||||
|
||||
/// The assembled message exceeded the per-message size cap.
|
||||
#[error("email too large: {actual} bytes exceeds the {limit}-byte limit")]
|
||||
TooLarge { limit: usize, actual: usize },
|
||||
|
||||
/// No SMTP relay is configured (HOST/USER/PASSWORD unset). Every
|
||||
/// `send` fails until the operator configures one.
|
||||
#[error(
|
||||
"email is not configured: set PICLOUD_SMTP_HOST/USER/PASSWORD to enable outbound email"
|
||||
)]
|
||||
NotConfigured,
|
||||
|
||||
/// The SMTP relay rejected the message or the connection failed.
|
||||
#[error("email transport error: {0}")]
|
||||
Transport(String),
|
||||
}
|
||||
|
||||
/// Stub used by test harnesses that build a `Services` bundle without an
|
||||
/// SMTP relay. Every call returns `EmailError::NotConfigured`.
|
||||
#[derive(Debug, Default, Clone, Copy)]
|
||||
pub struct NoopEmailService;
|
||||
|
||||
#[async_trait]
|
||||
impl EmailService for NoopEmailService {
|
||||
async fn send(&self, _cx: &SdkCallCx, _email: OutboundEmail) -> Result<(), EmailError> {
|
||||
Err(EmailError::NotConfigured)
|
||||
}
|
||||
}
|
||||
@@ -6,8 +6,10 @@
|
||||
|
||||
pub mod app;
|
||||
pub mod auth;
|
||||
pub mod crypto;
|
||||
pub mod dead_letters;
|
||||
pub mod docs;
|
||||
pub mod email;
|
||||
pub mod error;
|
||||
pub mod events;
|
||||
pub mod exec_summary;
|
||||
@@ -27,6 +29,7 @@ pub mod route;
|
||||
pub mod sandbox;
|
||||
pub mod script;
|
||||
pub mod sdk_cx;
|
||||
pub mod secrets;
|
||||
pub mod services;
|
||||
pub mod subscriber_token;
|
||||
pub mod trigger_event;
|
||||
@@ -35,8 +38,10 @@ pub mod version;
|
||||
|
||||
pub use app::{App, AppDomain, DomainShape};
|
||||
pub use auth::{AppRole, InstanceRole, Principal, Scope, UserId};
|
||||
pub use crypto::{decrypt, encrypt, CryptoError, EncryptResult, MasterKey, MasterKeyError};
|
||||
pub use dead_letters::{DeadLetterError, DeadLetterId, DeadLetterService, NoopDeadLetterService};
|
||||
pub use docs::{DocId, DocRow, DocsError, DocsListPage, DocsService, NoopDocsService};
|
||||
pub use email::{EmailError, EmailService, NoopEmailService, OutboundEmail};
|
||||
pub use error::Error;
|
||||
pub use events::{EmitError, NoopEventEmitter, ServiceEvent, ServiceEventEmitter};
|
||||
pub use exec_summary::ExecResponseSummary;
|
||||
@@ -63,6 +68,10 @@ pub use route::{DispatchMode, HostKind, PathKind, Route};
|
||||
pub use sandbox::ScriptSandbox;
|
||||
pub use script::{Script, ScriptKind};
|
||||
pub use sdk_cx::SdkCallCx;
|
||||
pub use secrets::{
|
||||
validate_secret_name, NoopSecretsService, SecretsError, SecretsListPage, SecretsService,
|
||||
SECRET_NAME_MAX_BYTES,
|
||||
};
|
||||
pub use services::Services;
|
||||
pub use trigger_event::{
|
||||
DeadLetterEventDetail, DocsEventOp, FilesEventOp, KvEventOp, TriggerEvent,
|
||||
|
||||
166
crates/shared/src/secrets.rs
Normal file
166
crates/shared/src/secrets.rs
Normal file
@@ -0,0 +1,166 @@
|
||||
//! `SecretsService` — the v1.1.7 encrypted per-app secrets contract.
|
||||
//!
|
||||
//! Collection-less (per-app, like pubsub): the script API is the bare
|
||||
//! `secrets::{get,set,delete,list}(name)` — there is no
|
||||
//! `secrets::collection(...)`. Secrets are operational config (API keys,
|
||||
//! OAuth tokens, webhook signing keys), encrypted at rest with the
|
||||
//! process master key.
|
||||
//!
|
||||
//! Lives in `picloud-shared` (not `executor-core`) so the Rhai bridge,
|
||||
//! the manager-core Postgres impl, and test fakes can all depend on the
|
||||
//! same trait. Implementations MUST derive every storage `app_id` from
|
||||
//! `cx.app_id` — never from a script-passed argument. That is the
|
||||
//! cross-app isolation boundary; see `docs/sdk-shape.md`.
|
||||
//!
|
||||
//! Values are JSON internally: `set` accepts any `serde_json::Value`
|
||||
//! (the bridge maps a Rhai String/Map/Array to JSON), encrypts the
|
||||
//! encoded bytes, and `get` decrypts + decodes back to the same JSON
|
||||
//! shape — so a String round-trips to a String, not a JSON-quoted
|
||||
//! `"\"…\""`. There is deliberately **no `ServiceEvent` emission**:
|
||||
//! firing triggers on secret writes is a footgun (every rotation would
|
||||
//! fan out handler executions that might log the new value).
|
||||
|
||||
use async_trait::async_trait;
|
||||
use thiserror::Error;
|
||||
|
||||
use crate::SdkCallCx;
|
||||
|
||||
/// Maximum secret name length in bytes (matches the brief: 255).
|
||||
pub const SECRET_NAME_MAX_BYTES: usize = 255;
|
||||
|
||||
/// `SecretsService` is collection-less and per-app. Every method derives
|
||||
/// the owning `app_id` from `cx.app_id`.
|
||||
#[async_trait]
|
||||
pub trait SecretsService: Send + Sync {
|
||||
/// Decrypt and return the secret, or `None` if no secret with this
|
||||
/// name exists for the app.
|
||||
async fn get(
|
||||
&self,
|
||||
cx: &SdkCallCx,
|
||||
name: &str,
|
||||
) -> Result<Option<serde_json::Value>, SecretsError>;
|
||||
|
||||
/// Encrypt and store the secret, overwriting any existing value for
|
||||
/// this name.
|
||||
async fn set(
|
||||
&self,
|
||||
cx: &SdkCallCx,
|
||||
name: &str,
|
||||
value: serde_json::Value,
|
||||
) -> Result<(), SecretsError>;
|
||||
|
||||
/// Delete the secret. Returns whether a secret was present.
|
||||
async fn delete(&self, cx: &SdkCallCx, name: &str) -> Result<bool, SecretsError>;
|
||||
|
||||
/// List secret **names only** (never values), cursor-paginated like
|
||||
/// KV/files `list`. `cursor` is opaque; `None` starts from the
|
||||
/// beginning.
|
||||
async fn list(
|
||||
&self,
|
||||
cx: &SdkCallCx,
|
||||
cursor: Option<&str>,
|
||||
limit: u32,
|
||||
) -> Result<SecretsListPage, SecretsError>;
|
||||
}
|
||||
|
||||
/// One page of secret names from `SecretsService::list`. `next_cursor`
|
||||
/// is `Some` when more pages exist.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct SecretsListPage {
|
||||
pub names: Vec<String>,
|
||||
pub next_cursor: Option<String>,
|
||||
}
|
||||
|
||||
/// Failure modes surfaced to the Rhai bridge. The bridge converts each
|
||||
/// to a Rhai runtime error string.
|
||||
#[derive(Debug, Error)]
|
||||
pub enum SecretsError {
|
||||
/// Empty name, or a name longer than [`SECRET_NAME_MAX_BYTES`].
|
||||
#[error("{0}")]
|
||||
InvalidName(String),
|
||||
|
||||
/// The encoded plaintext exceeded the configured per-secret cap.
|
||||
#[error("secret value too large: {actual} bytes exceeds the {limit}-byte limit")]
|
||||
TooLarge { limit: usize, actual: usize },
|
||||
|
||||
/// Caller principal lacked the required capability. Only raised when
|
||||
/// `cx.principal.is_some()` — public-HTTP scripts (`principal: None`)
|
||||
/// operate under script-as-gate semantics and skip the check.
|
||||
#[error("forbidden")]
|
||||
Forbidden,
|
||||
|
||||
/// The stored ciphertext could not be decrypted (corrupted row,
|
||||
/// wrong master key, or tampering). The impl logs the affected
|
||||
/// `(app_id, name)` at error level before returning this.
|
||||
#[error("secret is corrupted or was encrypted with a different master key")]
|
||||
Corrupted,
|
||||
|
||||
/// The process master key was unavailable. Startup should already
|
||||
/// have failed; this is defense in depth.
|
||||
#[error("master key is not configured")]
|
||||
MasterKeyMissing,
|
||||
|
||||
/// Anything else — Postgres unavailable, serialization failure, etc.
|
||||
#[error("secrets backend error: {0}")]
|
||||
Backend(String),
|
||||
}
|
||||
|
||||
/// Stub used by the executor-core test harness (which doesn't touch
|
||||
/// secrets) so a `Services` bundle can be built without Postgres. Every
|
||||
/// call returns `SecretsError::Backend(...)` so accidental use surfaces.
|
||||
#[derive(Debug, Default, Clone, Copy)]
|
||||
pub struct NoopSecretsService;
|
||||
|
||||
#[async_trait]
|
||||
impl SecretsService for NoopSecretsService {
|
||||
async fn get(
|
||||
&self,
|
||||
_cx: &SdkCallCx,
|
||||
_name: &str,
|
||||
) -> Result<Option<serde_json::Value>, SecretsError> {
|
||||
Err(SecretsError::Backend("secrets is not wired in".into()))
|
||||
}
|
||||
|
||||
async fn set(
|
||||
&self,
|
||||
_cx: &SdkCallCx,
|
||||
_name: &str,
|
||||
_value: serde_json::Value,
|
||||
) -> Result<(), SecretsError> {
|
||||
Err(SecretsError::Backend("secrets is not wired in".into()))
|
||||
}
|
||||
|
||||
async fn delete(&self, _cx: &SdkCallCx, _name: &str) -> Result<bool, SecretsError> {
|
||||
Err(SecretsError::Backend("secrets is not wired in".into()))
|
||||
}
|
||||
|
||||
async fn list(
|
||||
&self,
|
||||
_cx: &SdkCallCx,
|
||||
_cursor: Option<&str>,
|
||||
_limit: u32,
|
||||
) -> Result<SecretsListPage, SecretsError> {
|
||||
Err(SecretsError::Backend("secrets is not wired in".into()))
|
||||
}
|
||||
}
|
||||
|
||||
/// Validate a secret name at the SDK/admin boundary: non-empty and at
|
||||
/// most [`SECRET_NAME_MAX_BYTES`] bytes.
|
||||
///
|
||||
/// # Errors
|
||||
///
|
||||
/// Returns [`SecretsError::InvalidName`] when empty or too long.
|
||||
pub fn validate_secret_name(name: &str) -> Result<(), SecretsError> {
|
||||
if name.is_empty() {
|
||||
return Err(SecretsError::InvalidName(
|
||||
"secret name must not be empty".into(),
|
||||
));
|
||||
}
|
||||
if name.len() > SECRET_NAME_MAX_BYTES {
|
||||
return Err(SecretsError::InvalidName(format!(
|
||||
"secret name must be at most {SECRET_NAME_MAX_BYTES} bytes, got {}",
|
||||
name.len()
|
||||
)));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
@@ -20,9 +20,10 @@
|
||||
use std::sync::Arc;
|
||||
|
||||
use crate::{
|
||||
DeadLetterService, DocsService, FilesService, HttpService, KvService, ModuleSource,
|
||||
NoopDeadLetterService, NoopDocsService, NoopEventEmitter, NoopFilesService, NoopHttpService,
|
||||
NoopKvService, NoopModuleSource, NoopPubsubService, PubsubService, ServiceEventEmitter,
|
||||
DeadLetterService, DocsService, EmailService, FilesService, HttpService, KvService,
|
||||
ModuleSource, NoopDeadLetterService, NoopDocsService, NoopEmailService, NoopEventEmitter,
|
||||
NoopFilesService, NoopHttpService, NoopKvService, NoopModuleSource, NoopPubsubService,
|
||||
NoopSecretsService, PubsubService, SecretsService, ServiceEventEmitter,
|
||||
};
|
||||
|
||||
/// SDK service bundle. See module docs for the lifecycle and the v1.1.x
|
||||
@@ -73,6 +74,18 @@ pub struct Services {
|
||||
/// publish-time outbox fan-out in the picloud binary;
|
||||
/// `NoopPubsubService` in tests that don't publish.
|
||||
pub pubsub: Arc<dyn PubsubService>,
|
||||
|
||||
/// Encrypted per-app secrets (v1.1.7). Scripts get
|
||||
/// `secrets::{get,set,delete,list}(name)`. Backed by an
|
||||
/// AES-256-GCM-at-rest Postgres repo in the picloud binary;
|
||||
/// `NoopSecretsService` in tests that don't touch secrets.
|
||||
pub secrets: Arc<dyn SecretsService>,
|
||||
|
||||
/// Outbound email (v1.1.7). Scripts get `email::{send,send_html}`.
|
||||
/// Backed by an SMTP relay (lettre) in the picloud binary;
|
||||
/// `NoopEmailService` (always `NotConfigured`) in tests that don't
|
||||
/// send mail.
|
||||
pub email: Arc<dyn EmailService>,
|
||||
}
|
||||
|
||||
impl Services {
|
||||
@@ -90,6 +103,8 @@ impl Services {
|
||||
http: Arc<dyn HttpService>,
|
||||
files: Arc<dyn FilesService>,
|
||||
pubsub: Arc<dyn PubsubService>,
|
||||
secrets: Arc<dyn SecretsService>,
|
||||
email: Arc<dyn EmailService>,
|
||||
) -> Self {
|
||||
Self {
|
||||
kv,
|
||||
@@ -100,6 +115,8 @@ impl Services {
|
||||
http,
|
||||
files,
|
||||
pubsub,
|
||||
secrets,
|
||||
email,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -119,6 +136,8 @@ impl Services {
|
||||
Arc::new(NoopHttpService),
|
||||
Arc::new(NoopFilesService),
|
||||
Arc::new(NoopPubsubService),
|
||||
Arc::new(NoopSecretsService),
|
||||
Arc::new(NoopEmailService),
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -186,6 +186,27 @@ pub enum TriggerEvent {
|
||||
published_at: DateTime<Utc>,
|
||||
},
|
||||
|
||||
/// An inbound email (POSTed to the webhook receiver by a configured
|
||||
/// provider) fired this handler. v1.1.7. Carries the normalized
|
||||
/// message; `text`/`html` are absent when the provider sent only the
|
||||
/// other. Surfaced to scripts as `ctx.event.email`. Attachments are
|
||||
/// deferred to v1.2.
|
||||
Email {
|
||||
from: String,
|
||||
to: Vec<String>,
|
||||
#[serde(default)]
|
||||
cc: Vec<String>,
|
||||
subject: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
text: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
html: Option<String>,
|
||||
received_at: DateTime<Utc>,
|
||||
/// RFC 5322 Message-ID, when the provider supplied one.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
message_id: Option<String>,
|
||||
},
|
||||
|
||||
/// A dead-letter row fired this handler. The original event is
|
||||
/// nested verbatim plus the dead-letter metadata the design notes
|
||||
/// §4 require.
|
||||
@@ -213,6 +234,7 @@ impl TriggerEvent {
|
||||
Self::Cron { .. } => "cron",
|
||||
Self::Files { .. } => "files",
|
||||
Self::Pubsub { .. } => "pubsub",
|
||||
Self::Email { .. } => "email",
|
||||
Self::DeadLetter { .. } => "dead_letter",
|
||||
}
|
||||
}
|
||||
|
||||
@@ -58,7 +58,15 @@ pub const PRODUCT_VERSION: &str = env!("CARGO_PKG_VERSION");
|
||||
/// `RealtimeBroadcaster` / `RealtimeEvent` / `RealtimeAuthority` traits;
|
||||
/// the `topics` registry + admin endpoints; the `@picloud/client`
|
||||
/// TypeScript package).
|
||||
pub const SDK_VERSION: &str = "1.7";
|
||||
///
|
||||
/// 1.8 additions (v1.1.7): `secrets::{get,set,delete,list}(name)` —
|
||||
/// encrypted per-app secrets (AES-256-GCM at rest under the process
|
||||
/// master key); `email::{send,send_html}(#{...})` — outbound email via
|
||||
/// an env-configured SMTP relay; and `ctx.event.email` for
|
||||
/// `email:receive`-trigger handlers (inbound email POSTed to the webhook
|
||||
/// receiver). The `Services` bundle gains `secrets: Arc<dyn
|
||||
/// SecretsService>` and `email: Arc<dyn EmailService>`.
|
||||
pub const SDK_VERSION: &str = "1.8";
|
||||
|
||||
/// HTTP API major version. Appears in URL paths as `/api/v{N}/...`.
|
||||
/// Bump (new integer + new URL prefix) when the request/response
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "picloud-dashboard",
|
||||
"version": "0.12.0",
|
||||
"version": "0.13.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
|
||||
@@ -211,7 +211,14 @@ export interface DeadLetterRow {
|
||||
resolution: 'replayed' | 'ignored' | 'handled_by_script' | 'handler_failed' | null;
|
||||
}
|
||||
|
||||
export type TriggerKind = 'kv' | 'docs' | 'dead_letter' | 'cron' | 'files' | 'pubsub';
|
||||
export type TriggerKind =
|
||||
| 'kv'
|
||||
| 'docs'
|
||||
| 'dead_letter'
|
||||
| 'cron'
|
||||
| 'files'
|
||||
| 'pubsub'
|
||||
| 'email';
|
||||
export type TriggerDispatchMode = 'sync' | 'async';
|
||||
|
||||
/// Per-kind detail, tagged by `kind` to match the Rust serde shape.
|
||||
@@ -221,7 +228,15 @@ export type TriggerDetails =
|
||||
| { kind: 'dead_letter'; source_filter?: string; trigger_id_filter?: string; script_id_filter?: string }
|
||||
| { kind: 'cron'; schedule: string; timezone: string; last_fired_at?: string | null }
|
||||
| { kind: 'files'; collection_glob: string; ops: string[] }
|
||||
| { kind: 'pubsub'; topic_pattern: string };
|
||||
| { kind: 'pubsub'; topic_pattern: string }
|
||||
| { kind: 'email'; has_inbound_secret: boolean };
|
||||
|
||||
export interface CreateEmailTriggerInput {
|
||||
script_id: string;
|
||||
/// Shared HMAC secret; null/omitted means the receiver accepts
|
||||
/// unsigned POSTs (URL secrecy is then the only guard).
|
||||
inbound_secret?: string | null;
|
||||
}
|
||||
|
||||
/// v1.1.5 file metadata as the admin files endpoint returns it.
|
||||
export interface FileMeta {
|
||||
@@ -292,6 +307,11 @@ export interface UpdateTopicInput {
|
||||
auth_mode?: TopicAuthMode;
|
||||
}
|
||||
|
||||
export interface SecretListItem {
|
||||
name: string;
|
||||
updated_at: string;
|
||||
}
|
||||
|
||||
export interface ExecutionResult {
|
||||
status: number;
|
||||
headers: Record<string, string>;
|
||||
@@ -668,6 +688,11 @@ export const api = {
|
||||
`/api/v1/admin/apps/${encodeURIComponent(idOrSlug)}/triggers/pubsub`,
|
||||
{ method: 'POST', body: JSON.stringify(input) }
|
||||
),
|
||||
createEmail: (idOrSlug: string, input: CreateEmailTriggerInput) =>
|
||||
adminRequest<Trigger>(
|
||||
`/api/v1/admin/apps/${encodeURIComponent(idOrSlug)}/triggers/email`,
|
||||
{ method: 'POST', body: JSON.stringify(input) }
|
||||
),
|
||||
remove: (idOrSlug: string, triggerId: string) =>
|
||||
adminRequest<null>(
|
||||
`/api/v1/admin/apps/${encodeURIComponent(idOrSlug)}/triggers/${triggerId}`,
|
||||
@@ -714,6 +739,27 @@ export const api = {
|
||||
)
|
||||
},
|
||||
|
||||
secrets: {
|
||||
// List returns names + last-modified ONLY — values never leave the
|
||||
// server (v1.1.7).
|
||||
list: (idOrSlug: string) =>
|
||||
adminRequest<{ secrets: SecretListItem[]; next_cursor: string | null }>(
|
||||
`/api/v1/admin/apps/${encodeURIComponent(idOrSlug)}/secrets`
|
||||
),
|
||||
// `value` is any JSON value; the dashboard sends a single-line
|
||||
// string. Overwrites if the name already exists.
|
||||
set: (idOrSlug: string, name: string, value: unknown) =>
|
||||
adminRequest<null>(`/api/v1/admin/apps/${encodeURIComponent(idOrSlug)}/secrets`, {
|
||||
method: 'POST',
|
||||
body: JSON.stringify({ name, value })
|
||||
}),
|
||||
remove: (idOrSlug: string, name: string) =>
|
||||
adminRequest<null>(
|
||||
`/api/v1/admin/apps/${encodeURIComponent(idOrSlug)}/secrets/${encodeURIComponent(name)}`,
|
||||
{ method: 'DELETE' }
|
||||
)
|
||||
},
|
||||
|
||||
execute: async (
|
||||
id: string,
|
||||
body: unknown,
|
||||
|
||||
@@ -13,7 +13,8 @@
|
||||
type Script,
|
||||
type Trigger,
|
||||
type Topic,
|
||||
type TopicAuthMode
|
||||
type TopicAuthMode,
|
||||
type SecretListItem
|
||||
} from '$lib/api';
|
||||
import CodeEditor from '$lib/CodeEditor.svelte';
|
||||
import ConfirmModal from '$lib/ConfirmModal.svelte';
|
||||
@@ -27,7 +28,7 @@
|
||||
const SAMPLE_SOURCE =
|
||||
'#{\n statusCode: 200,\n body: #{ ok: true, echo: ctx.request.body }\n}';
|
||||
|
||||
type Tab = 'scripts' | 'domains' | 'members' | 'settings' | 'triggers' | 'topics';
|
||||
type Tab = 'scripts' | 'domains' | 'members' | 'settings' | 'triggers' | 'topics' | 'secrets';
|
||||
|
||||
// Common IANA timezones offered in the cron form dropdown. Not
|
||||
// exhaustive — the backend validates any IANA name via chrono-tz.
|
||||
@@ -125,6 +126,11 @@
|
||||
let createPubsubTopic = $state('');
|
||||
let creatingPubsub = $state(false);
|
||||
let createPubsubError = $state<string | null>(null);
|
||||
// Email triggers (v1.1.7).
|
||||
let createEmailScriptId = $state('');
|
||||
let createEmailSecret = $state('');
|
||||
let creatingEmail = $state(false);
|
||||
let createEmailError = $state<string | null>(null);
|
||||
let triggerToRemove = $state<Trigger | null>(null);
|
||||
let removingTrigger = $state(false);
|
||||
// Endpoint scripts only — modules can't be trigger targets.
|
||||
@@ -181,6 +187,34 @@
|
||||
}
|
||||
}
|
||||
|
||||
async function submitCreateEmail(e: SubmitEvent) {
|
||||
e.preventDefault();
|
||||
if (!app) return;
|
||||
creatingEmail = true;
|
||||
createEmailError = null;
|
||||
try {
|
||||
await api.triggers.createEmail(app.id, {
|
||||
script_id: createEmailScriptId,
|
||||
inbound_secret: createEmailSecret.trim() === '' ? null : createEmailSecret
|
||||
});
|
||||
createEmailScriptId = '';
|
||||
createEmailSecret = '';
|
||||
await loadTriggers(app.id);
|
||||
} catch (err) {
|
||||
createEmailError =
|
||||
err instanceof ApiError ? err.message : err instanceof Error ? err.message : String(err);
|
||||
} finally {
|
||||
creatingEmail = false;
|
||||
}
|
||||
}
|
||||
|
||||
// The inbound-email webhook URL for a given email trigger (shown so
|
||||
// the operator can configure their provider).
|
||||
function emailInboundUrl(triggerId: string): string {
|
||||
if (!app) return '';
|
||||
return `${window.location.origin}/api/v1/email-inbound/${app.id}/${triggerId}`;
|
||||
}
|
||||
|
||||
async function confirmRemoveTrigger() {
|
||||
if (!app || !triggerToRemove) return;
|
||||
removingTrigger = true;
|
||||
@@ -290,6 +324,83 @@
|
||||
}
|
||||
}
|
||||
|
||||
// Secrets tab (v1.1.7). The dashboard only ever sees names +
|
||||
// last-modified — values never leave the server. The create form's
|
||||
// value input is masked by default; revealing it requires a confirm.
|
||||
let secrets = $state<SecretListItem[]>([]);
|
||||
let createSecretName = $state('');
|
||||
let createSecretValue = $state('');
|
||||
let showSecretValue = $state(false);
|
||||
let revealConfirm = $state(false);
|
||||
let creatingSecret = $state(false);
|
||||
let createSecretError = $state<string | null>(null);
|
||||
let secretToRemove = $state<SecretListItem | null>(null);
|
||||
let removingSecret = $state(false);
|
||||
// True when the name already exists — set is an overwrite.
|
||||
const secretNameExists = $derived(
|
||||
secrets.some((s) => s.name === createSecretName.trim())
|
||||
);
|
||||
|
||||
async function loadSecrets(idOrSlug: string) {
|
||||
try {
|
||||
const r = await api.secrets.list(idOrSlug);
|
||||
secrets = r.secrets;
|
||||
} catch {
|
||||
secrets = [];
|
||||
}
|
||||
}
|
||||
|
||||
function toggleShowSecretValue(e: Event) {
|
||||
const target = e.currentTarget as HTMLInputElement;
|
||||
if (target.checked) {
|
||||
// Revealing a secret on screen is sensitive — gate behind a
|
||||
// confirm. Revert the checkbox until the user confirms.
|
||||
target.checked = false;
|
||||
revealConfirm = true;
|
||||
} else {
|
||||
showSecretValue = false;
|
||||
}
|
||||
}
|
||||
|
||||
function confirmRevealSecret() {
|
||||
showSecretValue = true;
|
||||
revealConfirm = false;
|
||||
}
|
||||
|
||||
async function submitCreateSecret(e: SubmitEvent) {
|
||||
e.preventDefault();
|
||||
if (!app) return;
|
||||
creatingSecret = true;
|
||||
createSecretError = null;
|
||||
try {
|
||||
await api.secrets.set(app.id, createSecretName.trim(), createSecretValue);
|
||||
createSecretName = '';
|
||||
createSecretValue = '';
|
||||
showSecretValue = false;
|
||||
await loadSecrets(app.id);
|
||||
} catch (err) {
|
||||
createSecretError =
|
||||
err instanceof ApiError ? err.message : err instanceof Error ? err.message : String(err);
|
||||
} finally {
|
||||
creatingSecret = false;
|
||||
}
|
||||
}
|
||||
|
||||
async function confirmRemoveSecret() {
|
||||
if (!app || !secretToRemove) return;
|
||||
removingSecret = true;
|
||||
try {
|
||||
await api.secrets.remove(app.id, secretToRemove.name);
|
||||
secretToRemove = null;
|
||||
await loadSecrets(app.id);
|
||||
} catch (err) {
|
||||
createSecretError =
|
||||
err instanceof ApiError ? err.message : err instanceof Error ? err.message : String(err);
|
||||
} finally {
|
||||
removingSecret = false;
|
||||
}
|
||||
}
|
||||
|
||||
// Members tab
|
||||
let eligibleUsers = $state<AdminDto[]>([]);
|
||||
let eligibleLoadError = $state<string | null>(null);
|
||||
@@ -334,7 +445,8 @@
|
||||
loadMembers(app.id),
|
||||
loadEligibleUsers(),
|
||||
loadTriggers(app.id),
|
||||
loadTopics(app.id)
|
||||
loadTopics(app.id),
|
||||
loadSecrets(app.id)
|
||||
);
|
||||
}
|
||||
await Promise.all(loaders);
|
||||
@@ -607,7 +719,8 @@
|
||||
(activeTab === 'settings' ||
|
||||
activeTab === 'members' ||
|
||||
activeTab === 'triggers' ||
|
||||
activeTab === 'topics')
|
||||
activeTab === 'topics' ||
|
||||
activeTab === 'secrets')
|
||||
) {
|
||||
activeTab = 'scripts';
|
||||
}
|
||||
@@ -660,6 +773,11 @@
|
||||
class:active={activeTab === 'topics'}
|
||||
onclick={() => (activeTab = 'topics')}>Topics ({topics.length})</button
|
||||
>
|
||||
<button
|
||||
type="button"
|
||||
class:active={activeTab === 'secrets'}
|
||||
onclick={() => (activeTab = 'secrets')}>Secrets ({secrets.length})</button
|
||||
>
|
||||
<button
|
||||
type="button"
|
||||
class:active={activeTab === 'settings'}
|
||||
@@ -1014,6 +1132,59 @@
|
||||
</div>
|
||||
</form>
|
||||
|
||||
|
||||
<h2>Email triggers</h2>
|
||||
<p class="muted">
|
||||
Fire an endpoint script when your email provider POSTs an inbound
|
||||
message to PiCloud. Configure your provider (Mailgun / Postmark /
|
||||
SendGrid / SES) to POST the generic JSON shape below to the trigger's
|
||||
webhook URL. Set a shared secret to require an
|
||||
<code>X-Picloud-Signature</code> HMAC-SHA256 (hex of the request body);
|
||||
leave it blank to accept unsigned POSTs (URL secrecy only).
|
||||
</p>
|
||||
<details class="muted small">
|
||||
<summary>Expected inbound JSON shape</summary>
|
||||
<pre>{`{
|
||||
"from": "sender@external.com",
|
||||
"to": ["alice@myapp.com"],
|
||||
"cc": [],
|
||||
"subject": "...",
|
||||
"text": "...",
|
||||
"html": "...",
|
||||
"message_id": "<abc@external.com>"
|
||||
}`}</pre>
|
||||
</details>
|
||||
|
||||
<form class="create-form" onsubmit={submitCreateEmail}>
|
||||
<div class="row">
|
||||
<label>
|
||||
<span>Target script</span>
|
||||
<select bind:value={createEmailScriptId} required>
|
||||
<option value="" disabled>Select an endpoint script…</option>
|
||||
{#each endpointScripts as s (s.id)}
|
||||
<option value={s.id}>{s.name}</option>
|
||||
{/each}
|
||||
</select>
|
||||
</label>
|
||||
<label class="grow">
|
||||
<span>Inbound HMAC secret (optional)</span>
|
||||
<input
|
||||
type="password"
|
||||
bind:value={createEmailSecret}
|
||||
placeholder="leave blank to accept unsigned POSTs"
|
||||
/>
|
||||
</label>
|
||||
</div>
|
||||
{#if createEmailError}
|
||||
<div class="error">{createEmailError}</div>
|
||||
{/if}
|
||||
<div class="actions">
|
||||
<button type="submit" disabled={creatingEmail || !createEmailScriptId}>
|
||||
{creatingEmail ? 'Creating…' : 'Create email trigger'}
|
||||
</button>
|
||||
</div>
|
||||
</form>
|
||||
|
||||
{#if triggers.length === 0}
|
||||
<p class="muted">No triggers in this app yet.</p>
|
||||
{:else}
|
||||
@@ -1033,6 +1204,11 @@
|
||||
<span class="muted">— {t.details.ops.join(', ') || 'any op'}</span>
|
||||
{:else if t.details.kind === 'pubsub'}
|
||||
<code>{t.details.topic_pattern}</code>
|
||||
{:else if t.details.kind === 'email'}
|
||||
<span class="muted">
|
||||
{t.details.has_inbound_secret ? 'signed (HMAC)' : 'unsigned'}
|
||||
</span>
|
||||
<code class="webhook-url">{emailInboundUrl(t.id)}</code>
|
||||
{/if}
|
||||
<span class="muted small">→ {t.script_id}</span>
|
||||
</div>
|
||||
@@ -1131,6 +1307,76 @@
|
||||
</ul>
|
||||
{/if}
|
||||
</section>
|
||||
{:else if activeTab === 'secrets' && canAdmin}
|
||||
<section>
|
||||
<h2>Secrets</h2>
|
||||
<p class="muted">
|
||||
Encrypted per-app configuration (API keys, OAuth tokens, webhook signing
|
||||
keys), available to scripts as <code>secrets::get("name")</code>. Values are
|
||||
encrypted at rest with the process master key and
|
||||
<strong>never leave the server</strong> — this list shows names and
|
||||
last-modified times only.
|
||||
</p>
|
||||
|
||||
<form class="create-form" onsubmit={submitCreateSecret}>
|
||||
<div class="row">
|
||||
<label class="grow">
|
||||
<span>Name</span>
|
||||
<input bind:value={createSecretName} required placeholder="stripe_key" />
|
||||
</label>
|
||||
</div>
|
||||
<label class="grow">
|
||||
<span>Value</span>
|
||||
{#if showSecretValue}
|
||||
<input type="text" bind:value={createSecretValue} placeholder="sk_live_…" />
|
||||
{:else}
|
||||
<input type="password" bind:value={createSecretValue} placeholder="sk_live_…" />
|
||||
{/if}
|
||||
</label>
|
||||
<label class="checkbox-row">
|
||||
<input type="checkbox" checked={showSecretValue} onchange={toggleShowSecretValue} />
|
||||
<span>Show value</span>
|
||||
</label>
|
||||
{#if secretNameExists && createSecretName.trim()}
|
||||
<p class="muted small">
|
||||
A secret named <code>{createSecretName.trim()}</code> already exists — saving
|
||||
overwrites it.
|
||||
</p>
|
||||
{/if}
|
||||
{#if createSecretError}
|
||||
<div class="error">{createSecretError}</div>
|
||||
{/if}
|
||||
<div class="actions">
|
||||
<button type="submit" disabled={creatingSecret || !createSecretName.trim()}>
|
||||
{creatingSecret ? 'Saving…' : secretNameExists ? 'Overwrite secret' : 'Save secret'}
|
||||
</button>
|
||||
</div>
|
||||
</form>
|
||||
|
||||
{#if secrets.length === 0}
|
||||
<p class="muted">No secrets in this app yet.</p>
|
||||
{:else}
|
||||
<ul class="list">
|
||||
{#each secrets as s (s.name)}
|
||||
<li class="domain-row">
|
||||
<div>
|
||||
<code>{s.name}</code>
|
||||
<span class="muted small">· updated {shortDate(s.updated_at)}</span>
|
||||
</div>
|
||||
<div class="topic-actions">
|
||||
<button
|
||||
type="button"
|
||||
class="secondary danger"
|
||||
onclick={() => (secretToRemove = s)}
|
||||
>
|
||||
Delete
|
||||
</button>
|
||||
</div>
|
||||
</li>
|
||||
{/each}
|
||||
</ul>
|
||||
{/if}
|
||||
</section>
|
||||
{:else if activeTab === 'settings' && canAdmin}
|
||||
<section>
|
||||
<h2>Settings</h2>
|
||||
@@ -1364,6 +1610,38 @@
|
||||
</p>
|
||||
</ConfirmModal>
|
||||
{/if}
|
||||
|
||||
{#if revealConfirm}
|
||||
<ConfirmModal
|
||||
title="Reveal secret value?"
|
||||
confirmLabel="Show value"
|
||||
onConfirm={confirmRevealSecret}
|
||||
onCancel={() => (revealConfirm = false)}
|
||||
>
|
||||
<p>
|
||||
The value you type will be shown in plain text on screen. Make sure no one
|
||||
is looking over your shoulder and that screen-sharing is off.
|
||||
</p>
|
||||
</ConfirmModal>
|
||||
{/if}
|
||||
|
||||
{#if secretToRemove}
|
||||
<ConfirmModal
|
||||
title="Delete secret “{secretToRemove.name}”"
|
||||
variant="danger"
|
||||
confirmLabel="Delete secret"
|
||||
busyLabel="Deleting…"
|
||||
busy={removingSecret}
|
||||
onConfirm={confirmRemoveSecret}
|
||||
onCancel={() => (secretToRemove = null)}
|
||||
>
|
||||
<p>
|
||||
Deleting <code>{secretToRemove.name}</code> is permanent. Any script calling
|
||||
<code>secrets::get("{secretToRemove.name}")</code> will get <code>()</code>
|
||||
until you set it again.
|
||||
</p>
|
||||
</ConfirmModal>
|
||||
{/if}
|
||||
{/if}
|
||||
|
||||
<style>
|
||||
|
||||
Reference in New Issue
Block a user