- workspace 1.1.6 → 1.1.7 - SDK schema 1.7 → 1.8 (SecretsService, EmailService, TriggerEvent::Email) - dashboard 0.12.0 → 0.13.0 - CHANGELOG entry: secrets, outbound email, inbound email, retroactive dead_letter fix note, realtime-key encryption migration (+ v1.1.8 drop) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
635 lines
32 KiB
Markdown
635 lines
32 KiB
Markdown
# PiCloud Changelog
|
|
|
|
## v1.1.7 — Configuration & Email (unreleased)
|
|
|
|
The operational-config layer: **encrypted per-app secrets**, **outbound
|
|
email**, and an **inbound email trigger** — plus the long-missing
|
|
**dead-letter handler wiring** and **at-rest encryption of the realtime
|
|
signing key**. All at-rest encryption uses a single process master key
|
|
(AES-256-GCM); key rotation is deferred to v1.2.
|
|
|
|
### Added — Encryption infrastructure
|
|
|
|
- **Process master key** from `PICLOUD_SECRET_KEY` (base64 of exactly 32
|
|
bytes). REQUIRED at startup — an unset or malformed key is fatal.
|
|
Generate one with `openssl rand -base64 32`. A deterministic in-memory
|
|
dev key is used ONLY when `PICLOUD_SECRET_KEY` is unset AND
|
|
`PICLOUD_DEV_MODE=true` (with a prominent startup warning); there is no
|
|
quiet unencrypted mode.
|
|
- **`picloud_shared::crypto`** — `encrypt`/`decrypt` envelope:
|
|
`Aes256Gcm`, 96-bit CSPRNG nonce, 128-bit auth tag appended to the
|
|
ciphertext (RustCrypto `Aead` layout). Both ciphertext and nonce are
|
|
stored.
|
|
- **Key rotation is out of scope.** Changing `PICLOUD_SECRET_KEY` between
|
|
deploys renders all existing ciphertext undecryptable. v1.2+ adds
|
|
key-version columns + a re-encryption pass.
|
|
|
|
### Added — Encrypted per-app secrets
|
|
|
|
- **`secrets::{get,set,delete,list}(name)`** SDK — collection-less,
|
|
per-app. `set` accepts a String/Map/Array (JSON-encoded then encrypted);
|
|
`get` returns the same Rhai type back; missing → `()`. 64 KB plaintext
|
|
cap (`PICLOUD_SECRET_MAX_VALUE_BYTES`). `migrations/0023_secrets.sql`.
|
|
- **Admin API** `GET/POST/DELETE /api/v1/admin/apps/{id}/secrets` — list
|
|
returns names + `updated_at` only, **never values**.
|
|
- **Dashboard Secrets tab** — list names + last-modified, create/update
|
|
(masked value with a confirm-gated reveal), delete with confirm.
|
|
- `Capability::AppSecretsRead`/`Write` (→ `script:read` / `script:write`).
|
|
No new Scope variants (seven-scope commitment). Secret writes
|
|
deliberately do **not** emit trigger events.
|
|
|
|
### Added — Outbound email
|
|
|
|
- **`email::send` / `email::send_html`** SDK over an SMTP relay
|
|
(`lettre`). Config from `PICLOUD_SMTP_HOST/PORT/USER/PASSWORD/TLS/
|
|
TIMEOUT_SECS`; if HOST/USER/PASSWORD aren't all set the service runs in
|
|
**disabled mode** (every send throws `NotConfigured`, warned at
|
|
startup). Required `to`/`from`/`subject` + one of `text`/`html`;
|
|
RFC 5322-ish address validation; 25 MB per-message cap
|
|
(`PICLOUD_EMAIL_MAX_MESSAGE_BYTES`); `reply_to` defaults to `from`.
|
|
Per-call connection (pooling deferred to v1.2); per-app `from`
|
|
validation / SPF / DKIM are the operator's SMTP-relay concern.
|
|
- `Capability::AppEmailSend` (→ `script:write`).
|
|
|
|
### Added — Inbound email (`email:receive` trigger)
|
|
|
|
- **Webhook receiver** `POST /api/v1/email-inbound/{app_id}/{trigger_id}`
|
|
— a provider (Mailgun / Postmark / SendGrid / SES) POSTs the generic
|
|
JSON shape `{from,to[],cc[],subject,text,html,message_id}`; the
|
|
receiver verifies the optional HMAC signature, normalizes to
|
|
`TriggerEvent::Email`, and enqueues an outbox row. 202 accepted, 401
|
|
bad/missing signature, 404 missing/wrong-kind/cross-app, 422 malformed.
|
|
Handlers see `ctx.event.email`. `migrations/0024_email_triggers.sql`.
|
|
- **Admin** `POST /api/v1/admin/apps/{id}/triggers/email` +
|
|
dashboard form (with the webhook URL + expected payload). The HMAC
|
|
`inbound_secret` is stored **encrypted** via the master key (deviation
|
|
from the original plaintext design — see HANDBACK §7).
|
|
- Provider-specific payload unmarshallers + inbound attachments → v1.2.
|
|
Native SMTP listener → v1.3+.
|
|
|
|
### Security/correctness fix (retroactive) — dead_letter handlers
|
|
|
|
The `dead_letter` trigger kind has been registerable since v1.1.1 but,
|
|
due to missing dispatcher wiring (`list_matching_dead_letter` had no
|
|
production caller), handlers have **never fired**. Any deploy running
|
|
v1.1.1 through v1.1.6 with `dead_letter` triggers configured has had
|
|
silently non-functional handlers. v1.1.7 fixes the wiring; existing
|
|
`dead_letters` rows remain (no migration needed) but only NEW
|
|
dead-letter events (post-v1.1.7) trigger handlers. To process older
|
|
rows, use the existing admin replay surface to re-enqueue them.
|
|
|
|
### Changed — Realtime signing key encrypted at rest (two-phase)
|
|
|
|
`app_secrets.realtime_signing_key` was stored as 32 plaintext bytes. It
|
|
is now encrypted with the master key. `migrations/0025_encrypt_realtime_keys.sql`
|
|
adds NULL-able encrypted columns and drops `NOT NULL` on the plaintext
|
|
column; a startup task encrypts pre-existing rows; the read path prefers
|
|
the encrypted columns and falls back to plaintext during the compat
|
|
window. **v1.1.8 will drop the plaintext `realtime_signing_key`
|
|
column** — operators should upgrade through v1.1.7 (which performs the
|
|
encryption) before v1.1.8.
|
|
|
|
### Notes
|
|
|
|
- **New deps:** `aes-gcm` (RustCrypto AEAD), `lettre` (SMTP).
|
|
- **New env vars:** `PICLOUD_SECRET_KEY` (required), `PICLOUD_DEV_MODE`,
|
|
`PICLOUD_SECRET_MAX_VALUE_BYTES`, `PICLOUD_SMTP_HOST/PORT/USER/PASSWORD/
|
|
TLS/TIMEOUT_SECS`, `PICLOUD_EMAIL_MAX_MESSAGE_BYTES`.
|
|
- **SDK schema** 1.7 → 1.8; **dashboard** 0.12.0 → 0.13.0.
|
|
|
|
## v1.1.6 — Realtime Channels & Client Library (unreleased)
|
|
|
|
The first **external realtime surface** and the first **frontend
|
|
library**, co-shipped per the §5/§6 design-notes decisions. Browser
|
|
clients can subscribe over SSE to per-app pub/sub topics that have been
|
|
explicitly externalized; everything else stays internal-only. The
|
|
`@picloud/client` TypeScript package wraps typed HTTP, SSE, auth, and
|
|
React/Svelte hooks. Plus three v1.1.5 follow-ups.
|
|
|
|
### Added — Realtime
|
|
|
|
- **`topics` registry** (`migrations/0021_topics.sql`) — pub/sub topics
|
|
are internal-only by default; a `topics` row with
|
|
`external_subscribable = true` opts one into external SSE subscription.
|
|
`auth_mode` is `'public'` or `'token'`.
|
|
- **Topic admin endpoints** under `/api/v1/admin/apps/{id}/topics` —
|
|
`POST` (register), `GET` (list), `PATCH /{name}` (flip
|
|
external/auth_mode — its own audited surface), `DELETE /{name}`
|
|
(unregister + disconnect live subscribers). Gated by the new
|
|
`Capability::AppTopicManage` → `app:admin` scope (no new scope; the
|
|
seven-scope commitment holds).
|
|
- **SSE endpoint `GET /realtime/topics/{topic}`** — data-plane surface
|
|
(deliberately not under `/api/`). Resolves `Host` → app, authorizes
|
|
via the `RealtimeAuthority` (404 for missing/internal topics, 401 for
|
|
bad/absent tokens), then streams `data: {topic,message,published_at}`
|
|
events with a configurable heartbeat (`PICLOUD_REALTIME_HEARTBEAT_SEC`,
|
|
default 30). Token via `Authorization: Bearer` or `?token=`.
|
|
- **`RealtimeBroadcaster` + `RealtimeEvent` + `RealtimeAuthority`**
|
|
traits (`picloud-shared`); in-process `InProcessBroadcaster`
|
|
(`tokio::sync::broadcast`, per-channel capacity
|
|
`PICLOUD_REALTIME_BROADCAST_CAPACITY` default 64, periodic empty-channel
|
|
GC) and the DB-backed `RealtimeAuthorityImpl` (orchestrator-core /
|
|
manager-core respectively). The publish path now also fans out to
|
|
in-process SSE subscribers, best-effort, after the durable outbox
|
|
fan-out commits — a broadcast failure never fails the publish.
|
|
- **`pubsub::subscriber_token(topics, ttl)`** Rhai SDK (SDK schema
|
|
1.6 → 1.7) — mints an HMAC-SHA256 subscriber token (URL-safe
|
|
`payload.signature`) scoped to externally-subscribable topics.
|
|
Requires an authenticated principal + the pub/sub publish capability.
|
|
TTL clamped to `[10s, 24h]` (default 1h), env-overridable via
|
|
`PICLOUD_SUBSCRIBER_TOKEN_TTL_{MIN,MAX,DEFAULT}_SEC`. Per-app signing
|
|
keys persist in the new `app_secrets` table
|
|
(`migrations/0022_app_secrets.sql`), created lazily on first mint. No
|
|
per-token revocation (rotation invalidates wholesale; short TTL is the
|
|
safety mechanism).
|
|
- **Dashboard Topics tab** — register/list/edit/delete topics with a
|
|
prominent external/internal badge, auth-mode radio (conditional on
|
|
external), and a confirmation when flipping a topic external.
|
|
|
|
### Added — `@picloud/client` (TypeScript, v1.0.0)
|
|
|
|
- New top-level package `clients/typescript/` (tsup dual ESM+CJS +
|
|
`.d.ts`, vitest). Typed HTTP via `endpoint<Req,Res>(path).get()/.post()`
|
|
with auth-token injection and structured errors; SSE `subscribe(topic,
|
|
cb, {token, onTokenExpired})` with exponential-backoff reconnect,
|
|
401 token-refresh, and `Last-Event-ID` resume; `auth.login/logout/token`
|
|
over dev-defined endpoints; React (`useTopic`/`useEndpoint` +
|
|
`PicloudProvider`) and Svelte (`topicStore`/`endpointStore`) subpath
|
|
exports. Optional zod/valibot runtime validation via a `{ parse }`
|
|
adapter (no hard dep). Hybrid model: no direct service access from the
|
|
browser.
|
|
|
|
### Changed / Fixed — v1.1.5 follow-ups
|
|
|
|
- **Empty blobs accepted** — `NewFile::validate` / `FileUpdate::validate`
|
|
no longer reject zero-length `data`; empty files are a valid stored
|
|
state (sentinels, placeholders). Non-breaking.
|
|
- **Orphan `*.tmp.*` sweeper** — a startup tokio task
|
|
(`spawn_files_orphan_sweep`) walks the files root every
|
|
`PICLOUD_FILES_ORPHAN_SWEEP_INTERVAL_SEC` (default 6h) and unlinks temp
|
|
blobs older than `PICLOUD_FILES_ORPHAN_TMP_TTL_SEC` (default 1h). No DB
|
|
cross-check (that full reconciler is v1.3+).
|
|
- **Dispatcher end-to-end tests** — `crates/picloud/tests/dispatcher_e2e.rs`,
|
|
one per trigger kind (kv/docs/cron/files/pubsub/dead_letter),
|
|
DATABASE_URL-gated (skip cleanly when unset).
|
|
|
|
### Notes
|
|
|
|
- New deps: `hmac` (token signing, picloud-shared), `tokio-stream` (SSE
|
|
body stream, orchestrator-core).
|
|
- New env vars: `PICLOUD_REALTIME_HEARTBEAT_SEC`,
|
|
`PICLOUD_REALTIME_BROADCAST_CAPACITY`,
|
|
`PICLOUD_SUBSCRIBER_TOKEN_TTL_{MIN,MAX,DEFAULT}_SEC`,
|
|
`PICLOUD_FILES_ORPHAN_SWEEP_INTERVAL_SEC`,
|
|
`PICLOUD_FILES_ORPHAN_TMP_TTL_SEC`.
|
|
|
|
## v1.1.5 — Files & Pub/Sub (unreleased)
|
|
|
|
Two stateful services + two trigger kinds. **`files::*`** is
|
|
filesystem-backed blob storage (atomic writes, path-sharded layout,
|
|
single-pass SHA-256 with checksum-verified reads); the metadata row
|
|
lives in Postgres, the bytes on disk. **`pubsub::publish_durable`** is
|
|
durable pub/sub through the universal outbox, fanning out one delivery
|
|
row per matching subscriber **at publish time** inside a single
|
|
transaction. Both ride the v1.1.1 trigger framework as the fifth and
|
|
sixth concrete kinds via the established Layout-E extension pattern.
|
|
|
|
### Added
|
|
|
|
- **`files::collection(name).{create,head,get,update,delete,list}`** —
|
|
blob storage SDK. `create`/`update` take a Rhai `Blob`; `get` returns
|
|
a `Blob` (or `()` if missing); `head`/`list` return metadata maps
|
|
(`id, name, content_type, size, checksum, created_at, updated_at`).
|
|
`create`/`update`/`delete` throw on failure; `get`/`head` return `()`
|
|
for a missing file; `delete` returns a was-present bool. Missing
|
|
required field on `create` throws naming the field.
|
|
- **Atomic writes** — temp file → fsync → rename → fsync parent dir →
|
|
DB row, so a crash never leaves a readable half-written file. SHA-256
|
|
is computed in a single pass during the write; `get` re-verifies it
|
|
and surfaces `FilesError::Corrupted` (logged with the path, never
|
|
auto-deleted) on a mismatch. Shard dirs are created `0o700`.
|
|
- **`files:*` trigger kind** — `ctx.event.files` carries the metadata
|
|
only (never the bytes; a handler that wants them calls
|
|
`files::collection(c).get(id)`). `prev` is `()` on create, the prior
|
|
metadata on update, the deleted metadata on delete.
|
|
- **`pubsub::publish_durable(topic, message)`** — durable publish.
|
|
Message is any JSON-serializable Rhai value; Blobs encode as base64
|
|
(at any nesting depth). No matching subscriber → the publish succeeds
|
|
silently with zero outbox rows.
|
|
- **`pubsub:*` trigger kind** — topic patterns are exact, `<prefix>.*`,
|
|
or `*`; mid-pattern wildcards are rejected at trigger creation.
|
|
`ctx.event.pubsub` carries `topic`, `message`, `published_at`.
|
|
- **`FilesService` + `PubsubService` traits** (`picloud-shared`) +
|
|
`FsFilesRepo`/`FilesServiceImpl` and `PostgresPubsubRepo`/
|
|
`PubsubServiceImpl` (manager-core). Wired into the `Services` bundle
|
|
as `files` and `pubsub`.
|
|
- **Capabilities** `AppFilesRead`/`AppFilesWrite` → `script:read`/
|
|
`script:write`, `AppPubsubPublish` → `script:write`. No new `Scope`
|
|
variant — the seven-scope commitment holds. Script-as-gate: skipped
|
|
when the script runs unauthenticated.
|
|
- **Admin files API** (`GET`/`DELETE /apps/{id}/files`) + dashboard
|
|
Files view per app; **Pub/Sub trigger form** on the Triggers tab.
|
|
- **CI** — first `.github/workflows/ci.yml` (Postgres service, fmt +
|
|
clippy + `cargo test --workspace`); the schema-snapshot guardrail now
|
|
runs instead of being `#[ignore]`'d.
|
|
|
|
### Changed
|
|
|
|
- Workspace version: 1.1.4 → 1.1.5
|
|
- Rhai SDK version: 1.5 → 1.6
|
|
- Dashboard version: 0.10.0 → 0.11.0
|
|
- `schema_snapshot` test: no longer `#[ignore]`'d — runs against
|
|
`DATABASE_URL` when set, skips cleanly when absent.
|
|
|
|
### Migrations
|
|
|
|
- 0018_files.sql — `files` metadata table (bytes live on disk).
|
|
- 0019_files_triggers.sql — widen kind/source_kind CHECKs + add
|
|
`files_trigger_details`.
|
|
- 0020_pubsub_triggers.sql — widen kind/source_kind CHECKs + add
|
|
`pubsub_trigger_details` + partial index.
|
|
|
|
### New environment variables
|
|
|
|
- `PICLOUD_FILES_ROOT` (default `./data`)
|
|
- `PICLOUD_FILES_MAX_FILE_SIZE_BYTES` (default 100 MB)
|
|
|
|
## v1.1.4 — Outbound HTTP & Cron triggers (unreleased)
|
|
|
|
Two surfaces. **`http::*`** lets Rhai scripts make outbound HTTP
|
|
requests (Slack webhooks, Stripe, third-party REST) fronted by an SSRF
|
|
deny-list applied to the *resolved IP* (DNS-rebinding defense), with
|
|
scheme/port restrictions, request/response body caps, and a layered
|
|
timeout. **Cron triggers** add the fourth concrete kind on the v1.1.1
|
|
trigger framework: a scheduler task enqueues due triggers into the same
|
|
universal outbox the dispatcher already drains.
|
|
|
|
### Added
|
|
|
|
- **`http::{get,post,put,patch,delete,head,post_form,request}`** — outbound
|
|
HTTP SDK. Body and options are separate positional args
|
|
(`verb(url, body, opts)`); `opts` is
|
|
`{headers, timeout_ms, follow_redirects, max_redirects}` (unknown keys
|
|
throw). Body dispatch by type: Map/Array → JSON, String → text/plain,
|
|
`()` → none. Response is `#{ status, headers, body, body_raw }` with
|
|
`body` auto-parsed when the response is `application/json`. Non-2xx
|
|
does NOT throw (fetch-style); network/timeout/SSRF/size errors throw
|
|
with an `"http: …"` prefix.
|
|
- **SSRF deny-list** — applied to the resolved IP via a custom reqwest
|
|
`dns_resolver` (so it covers every redirect hop and defeats DNS
|
|
rebinding), plus a literal-IP check at URL-parse time. Blocks
|
|
loopback, RFC1918 private, link-local (incl. `169.254.169.254`),
|
|
carrier-grade NAT, multicast, reserved, IPv6 ULA/link-local/loopback,
|
|
and IPv4-mapped IPv6 (re-checked against the embedded v4 address).
|
|
The script-visible error carries a CIDR-category reason, never the IP.
|
|
`PICLOUD_HTTP_ALLOW_PRIVATE=true` disables it (dev-only; logs a startup
|
|
warning).
|
|
- **`HttpService` trait** (`picloud-shared`) + `HttpServiceImpl`
|
|
(manager-core, reqwest-backed). Wired into the `Services` bundle as
|
|
`http: Arc<dyn HttpService>`.
|
|
- **`Capability::AppHttpRequest(AppId)`** — maps to the existing
|
|
`script:write` scope (any outbound request can exfiltrate data, so the
|
|
conservative write mapping is used). No new `Scope` variant — the
|
|
seven-scope commitment holds. Script-as-gate: skipped when the script
|
|
runs unauthenticated.
|
|
- **Cron triggers** — `POST /api/v1/admin/apps/{id}/triggers/cron`
|
|
(`script_id`, `schedule`, `timezone`, optional retry overrides).
|
|
6-field cron expressions (with seconds) validated by the `cron` crate;
|
|
IANA timezones validated by `chrono-tz`. A scheduler task
|
|
(`spawn_cron_scheduler`, poll cadence `PICLOUD_CRON_TICK_INTERVAL_MS`,
|
|
default 30s) enqueues due triggers into the outbox; the existing
|
|
dispatcher delivers them. Catch-up policy: a trigger that missed N
|
|
windows fires exactly **once** on the next tick, not N times.
|
|
- **`ctx.event.cron`** — `{ schedule, timezone, scheduled_at, fired_at }`
|
|
for cron-trigger handlers (`ctx.event.source == "cron"`,
|
|
`ctx.event.op == "tick"`).
|
|
- **Dashboard Triggers tab** — admin-gated cron trigger create form
|
|
(target endpoint script, schedule, timezone dropdown) + triggers list
|
|
showing schedule / timezone / last-fired.
|
|
|
|
### Changed
|
|
|
|
- **Workspace version**: `1.1.3` → `1.1.4`.
|
|
- **Rhai SDK version**: `1.4` → `1.5` (additive — `http::*` SDK +
|
|
`ctx.event.cron`). The `Services` bundle constructor becomes
|
|
`Services::new(kv, docs, dead_letters, events, modules, http)`.
|
|
- **Dashboard version**: `0.9.0` → `0.10.0`.
|
|
- **`SdkCallCx`** — gains a `script_id` field (audit attribution + the
|
|
default outbound `User-Agent`, `picloud/<version> (script:<id>)`).
|
|
- **Rhai pin tightened** — workspace dep `rhai = "1.19"` → `rhai = "=1.24"`
|
|
so future bumps of the non-semver-stable `internals` surface are
|
|
deliberate.
|
|
- **Module backend errors redacted** — `PicloudModuleResolver` now
|
|
surfaces a stable generic (`"module backend unavailable; check server
|
|
logs"`) to scripts and logs the original at error level, instead of
|
|
leaking the backend error verbatim (see v1.1.3 follow-up).
|
|
|
|
### Migrations
|
|
|
|
- `0017_cron_triggers.sql` — widens `triggers.kind` and
|
|
`outbox.source_kind` CHECK constraints to include `'cron'`; adds
|
|
`cron_trigger_details (trigger_id, schedule, timezone, last_fired_at)`
|
|
with a `last_fired_at` index. Additive — applies cleanly on a fresh DB
|
|
and on top of the v1.1.3 schema.
|
|
|
|
### New environment variables
|
|
|
|
- `PICLOUD_HTTP_ALLOW_PRIVATE` (default false; dev-only) — disable the
|
|
SSRF deny-list.
|
|
- `PICLOUD_HTTP_MAX_REQUEST_BODY_BYTES` / `PICLOUD_HTTP_MAX_RESPONSE_BODY_BYTES`
|
|
(default 10 MB each).
|
|
- `PICLOUD_CRON_TICK_INTERVAL_MS` (default 30000) — cron scheduler poll
|
|
cadence (floored at 1s).
|
|
|
|
## v1.1.3 — Modules (unreleased)
|
|
|
|
Real per-app Rhai module system. Scripts can `import "<name>" as
|
|
<alias>;` other scripts in the same app as reusable libraries. The
|
|
v1.0 placeholder `DummyModuleResolver` is replaced by a per-call
|
|
`PicloudModuleResolver` that loads `kind = 'module'` scripts via a
|
|
new `ModuleSource` trait, compiles them into Rhai modules, caches
|
|
the compiled output, and enforces cross-app isolation, circular-
|
|
import detection, and an import-depth limit. Two LRU AST caches
|
|
(top-level script + per-module compiled module) eliminate the
|
|
per-invocation compile cost; both invalidate on `updated_at` change.
|
|
|
|
### Added
|
|
|
|
- **`scripts.kind` column** — `'endpoint' | 'module'`, default
|
|
`'endpoint'`. Endpoints handle HTTP routes / trigger events;
|
|
modules are libraries imported by other scripts. The dashboard
|
|
scripts list + script detail page surface the distinction as a
|
|
colored badge.
|
|
- **`script_imports` dep-graph table** — populated at script save-
|
|
time from the literal-path `import "<name>"` declarations in the
|
|
source. FK-CASCADE on both columns. No admin surface in v1.1.3
|
|
(drives a v1.2+ "Used by" dashboard panel and v1.3+ cluster-mode
|
|
eager invalidation).
|
|
- **`ModuleSource` trait** — `lookup(&SdkCallCx, name)`. Postgres
|
|
impl `PostgresModuleSource` in manager-core. `app_id` derived from
|
|
`cx.app_id` (cross-app isolation boundary, mirrors KV / docs).
|
|
- **`PicloudModuleResolver`** — implements `rhai::ModuleResolver`.
|
|
Per-call instance owns `Arc<SdkCallCx>`, the in-progress imports
|
|
stack, the depth counter. Bridges sync `resolve()` to async
|
|
`lookup()` via `Handle::block_on` (safe under the executor's
|
|
`spawn_blocking` wrap). Replaces `DummyModuleResolver` at line 139
|
|
of `executor-core::engine::build_engine`.
|
|
- **Module-shape validation** — `kind = 'module'` source must contain
|
|
only `fn` declarations, `const` declarations, and `import`
|
|
statements at top level (no executable expressions). Walks
|
|
`ast.statements()` via `rhai/internals`. Admin endpoint is the
|
|
primary gate; the resolver re-runs the check at load time for
|
|
defense in depth against DB-direct inserts.
|
|
- **Per-module compiled-Module cache** — `LruCache<(AppId, name),
|
|
(updated_at, Arc<rhai::Module>)>` owned by `Engine`. Invalidated
|
|
lazily on `updated_at` mismatch. Size via
|
|
`PICLOUD_MODULE_CACHE_SIZE` (default 512).
|
|
- **Top-level script AST cache** — `LruCache<ScriptId, (updated_at,
|
|
Arc<rhai::AST>)>` owned by `LocalExecutorClient`. Same staleness
|
|
semantics. Size via `PICLOUD_SCRIPT_CACHE_SIZE` (default 256).
|
|
- **`ScriptIdentity` + `ExecutorClient::execute_with_identity`** —
|
|
new method on the trait; default impl forwards to `execute` so
|
|
`RemoteExecutorClient` (and future transports) keep working.
|
|
`LocalExecutorClient` overrides it to consult the script cache and
|
|
pass the resulting `Arc<rhai::AST>` to `Engine::execute_ast`.
|
|
- **`Engine::execute_ast`** — companion to `execute` that takes a
|
|
pre-compiled AST so callers (the orchestrator) can reuse one
|
|
compile across many invocations.
|
|
- **Import depth limit** — `Limits::module_import_depth_max`
|
|
(default 8). Not script-overridable.
|
|
- **Reserved module names** — module-kind scripts cannot be named
|
|
`log`, `regex`, `random`, `time`, `json`, `base64`, `hex`, `url`,
|
|
`kv`, `docs`, `dead_letters`, `http`, `files`, `pubsub`, `secrets`,
|
|
`email`, `users`, `queue`. Defense against author confusion with
|
|
stdlib namespaces.
|
|
|
|
### Changed
|
|
|
|
- **Workspace version**: `1.1.2` → `1.1.3`.
|
|
- **Rhai SDK version**: `1.3` → `1.4` (additive — every v1.3 script
|
|
still runs unchanged; new surface: `import "<name>" as <alias>;`
|
|
for endpoint scripts that consume modules in the same app).
|
|
- **Dashboard version**: `0.8.0` → `0.9.0`. Adds kind dropdown on
|
|
script create + kind badges on the scripts list and detail page.
|
|
- **`Services` bundle** — grows a `modules: Arc<dyn ModuleSource>`
|
|
field. Constructor signature becomes
|
|
`Services::new(kv, docs, dead_letters, events, modules)`.
|
|
- **`ScriptValidator` trait** — `validate` now returns
|
|
`ValidatedScript { imports: Vec<String> }` so the repo can write
|
|
dep-graph edges in the same transaction as the script row. New
|
|
`validate_module` method enforces module-shape rules.
|
|
- **Trigger creation tightening** — `POST /api/v1/admin/apps/{id}/triggers/{kv,docs,dead_letter}`
|
|
now load the target script and reject when (1) it doesn't exist,
|
|
(2) it belongs to a different app (latent v1.1.1/v1.1.2 gap —
|
|
closed in v1.1.3), or (3) it is `kind = 'module'`.
|
|
- **Route creation** — `POST /api/v1/admin/scripts/{id}/routes`
|
|
returns 400 when the target script is `kind = 'module'`.
|
|
|
|
### Security fix
|
|
|
|
- **Cross-app trigger target (CVE-class: broken access control).** In
|
|
v1.1.1 and v1.1.2, `POST /api/v1/admin/apps/{id}/triggers/{kv,docs,dead_letter}`
|
|
validated only that the caller could manage triggers on `{id}` — it
|
|
did **not** verify that the target `script_id` belonged to that same
|
|
app. A member with trigger-management rights on app A could therefore
|
|
register a trigger in A pointing at a script owned by app B, causing
|
|
B's script to execute on A's events (a cross-app isolation break).
|
|
v1.1.3 closes this: every trigger-create handler now loads the target
|
|
script and rejects it unless `script.app_id == path app_id` (and it is
|
|
not a module). **Upgrade recommendation:** anyone running a pre-v1.1.3
|
|
multi-tenant deploy should upgrade and audit existing `triggers` rows
|
|
for any whose `script_id` resolves to a script in a different `app_id`.
|
|
|
|
### Migrations
|
|
|
|
- `0015_scripts_kind.sql` — adds `scripts.kind` with CHECK
|
|
`IN ('endpoint','module')`, composite index `(app_id, kind)`, and
|
|
a module-name shape CHECK (`^[a-zA-Z_][a-zA-Z0-9_]{0,63}$`).
|
|
- `0016_script_imports.sql` — adds the dep-graph table with FK
|
|
CASCADE on both columns, PK `(importer, imported)`, and a
|
|
reverse-edge index on `imported_script_id`.
|
|
|
|
### Downgrade caveats
|
|
|
|
Rolling back v1.1.3 → v1.1.2 with module-kind scripts present
|
|
strands them (no `kind` column means everything looks like an
|
|
endpoint; modules will then succeed as route targets and immediately
|
|
fail to execute meaningfully). Migration `0016_script_imports.sql`
|
|
is safe to drop (the table is auxiliary). `0015_scripts_kind.sql`
|
|
must be reversed by `DROP COLUMN kind` only after manually re-homing
|
|
or deleting module-kind rows.
|
|
|
|
## v1.1.2 — Documents (unreleased)
|
|
|
|
`docs::*` SDK — schemaless JSONB document storage with a first-cut
|
|
query DSL — plus `docs:*` triggers as the second concrete kind on the
|
|
v1.1.1 triggers framework. Sets the precedent for the v1.2 query DSL
|
|
expansion and `dead_letters::list`.
|
|
|
|
### Added
|
|
|
|
- **Docs store** — `docs` table keyed `(app_id, collection, id)` with
|
|
JSONB values and a GIN-on-`jsonb_path_ops` index. Rhai SDK exposes
|
|
the handle pattern:
|
|
`docs::collection(name).{create,get,find,find_one,update,delete,list}`.
|
|
Cursor-style pagination on `list`. Cross-app isolation enforced via
|
|
`cx.app_id` (never script-passed). Document envelope shape returned
|
|
by reads: `#{ id, data: #{...}, created_at, updated_at }` — explicit
|
|
metadata + user-data separation (sets precedent for v1.2
|
|
`dead_letters::list`).
|
|
- **Query DSL (v1.1.2 subset)** — implicit equality at top level
|
|
(`#{ tier: "gold" }`), operator-object form
|
|
(`#{ created_at: #{ "$gt": "..." } }`), dotted field paths up to 5
|
|
levels (`"user.email"`), and operators `$eq`/`$ne`/`$gt`/`$gte`/
|
|
`$lt`/`$lte`/`$in`. Filter modifiers `$sort` (single field) and
|
|
`$limit`. Unsupported operators (`$or`, `$regex`, etc.) reject with
|
|
a clear v1.2-pointer error.
|
|
- **Docs triggers (`docs:*`)** — `docs_trigger_details` table mirrors
|
|
`kv_trigger_details`. Admin endpoint
|
|
`POST /api/v1/admin/apps/{id}/triggers/docs` accepts the same DTO
|
|
shape as the KV endpoint with `ops` of `DocsEventOp` (create /
|
|
update / delete). Dispatcher routes `OutboxSourceKind::Docs` through
|
|
the same generic path as KV + dead-letter.
|
|
- **`ctx.event.docs.prev_data`** — change-data-capture surface for
|
|
docs trigger handlers. `prev_data` carries the document state prior
|
|
to the mutation (`None` for create), letting handlers see what
|
|
changed. The repo reads the old row in the same SQL statement as
|
|
the write so the trigger event has the prior value.
|
|
- **`Capability::AppDocsRead(AppId)`** + `AppDocsWrite(AppId)` —
|
|
granted to Viewer / Editor respectively in the per-app role table.
|
|
Same trust shape as KV's `AppKvRead` / `AppKvWrite`.
|
|
|
|
### Changed
|
|
|
|
- **Workspace version**: `1.1.1` → `1.1.2`.
|
|
- **Rhai SDK version**: `1.2` → `1.3` (additive — every v1.2 script
|
|
still runs unchanged; new surfaces: `docs::collection(name).{...}`,
|
|
`ctx.event.docs` for triggered handlers).
|
|
- **Dashboard version**: `0.7.0` → `0.8.0`. Workspace alignment; no
|
|
docs-specific UI in v1.1.2 (the dashboard's Rhai-mode hints don't
|
|
list KV completions either — focused UX pass is a separate task).
|
|
- **`Services` bundle** — grows a `docs: Arc<dyn DocsService>` field.
|
|
Constructor signature becomes
|
|
`Services::new(kv, docs, dead_letters, events)`.
|
|
- **Scope mapping**: API keys with `script:read` scope can call
|
|
`docs::find` / `get` / `list`; `script:write` can call
|
|
`docs::create` / `update` / `delete`. Same trust shape as KV —
|
|
honors the seven-scope commitment from v1.1.0.
|
|
|
|
### Migrations
|
|
|
|
- `0013_docs.sql` — `docs` table + per-`(app_id, collection)` index +
|
|
GIN-on-`jsonb_path_ops` index.
|
|
- `0014_docs_triggers.sql` — extends `triggers.kind` and
|
|
`outbox.source_kind` CHECK constraints to include `'docs'`; adds
|
|
`docs_trigger_details` table.
|
|
|
|
### Downgrade caveats
|
|
|
|
Rolling a deployment back from v1.1.2 → v1.1.1 with `docs`-source
|
|
outbox rows still queued will cause the v1.1.1 dispatcher to fail
|
|
deserialising `TriggerEvent::Docs` (`#[serde(tag = "source")]`
|
|
rejects unknown variants). Drain or delete
|
|
`outbox WHERE source_kind = 'docs'` before downgrading. Trunk-only
|
|
deployments don't hit this.
|
|
|
|
### Known limitations
|
|
|
|
- Text-lex comparison for `$gt` / `$gte` / `$lt` / `$lte` is
|
|
incorrect for unpadded numbers crossing digit-count boundaries
|
|
(`'10' < '9'` is TRUE under any text collation). Workaround:
|
|
zero-pad numeric strings. v1.2's advanced query expansion adds
|
|
numeric-aware operators.
|
|
- Concurrent `update()`s on the same doc may both emit the
|
|
pre-update `prev_data` (last-writer-wins). Inherited from KV's
|
|
`set` pattern; documented for forensic-trace use cases.
|
|
- v1.1.2 has no partial-update DSL — scripts that want partial
|
|
update do `get + modify + update`. Planned for v1.2.
|
|
|
|
## v1.1.1 — Storage & Events (unreleased)
|
|
|
|
The triggers framework — KV store + universal outbox + dispatcher +
|
|
NATS-style sync HTTP + per-route async dispatch + dead-letter
|
|
handling + dashboard surface. Every subsequent v1.1.x service module
|
|
(docs, files, pubsub, …) hangs off the dispatcher built here.
|
|
|
|
### Added
|
|
|
|
- **KV store** — `kv_entries` table keyed `(app_id, collection, key)`
|
|
with JSONB values. Rhai SDK exposes the handle pattern:
|
|
`kv::collection(name).{get,set,has,delete,list}`. Cursor-style
|
|
pagination with opaque base64 cursors. Cross-app isolation
|
|
enforced via `cx.app_id` (never script-passed).
|
|
- **Triggers framework (Layout E)** — parent `triggers` table +
|
|
per-kind detail tables (`kv_trigger_details`,
|
|
`dead_letter_trigger_details`). Trigger CRUD admin endpoints
|
|
(`/api/v1/admin/apps/{id}/triggers/{kv,dead_letter}`) +
|
|
`Capability::AppManageTriggers(AppId)`.
|
|
- **Universal outbox + dispatcher** — single tokio task that polls
|
|
the outbox via `FOR UPDATE SKIP LOCKED`, routes due rows to the
|
|
executor through the shared `ExecutionGate`. Retry with
|
|
exponential backoff + ±jitter; on exhaustion, dead-letter.
|
|
- **NATS-style sync HTTP via outbox** — `InboxRegistry` (in-process
|
|
oneshot map) lets the orchestrator await dispatcher delivery on
|
|
every sync HTTP request. Cluster mode (v1.3+) swaps this for
|
|
`LISTEN/NOTIFY` behind the same `InboxResolver` trait.
|
|
- **`dispatch_mode: async` on routes** — `POST` to a route with
|
|
`dispatch_mode = 'async'` returns `202 Accepted` immediately;
|
|
the script runs via the dispatcher (with retries / dead-letter).
|
|
- **Dead-letter handling** — separate `dead_letters` table per
|
|
design notes §4. `dead_letters::{replay,resolve}` Rhai SDK +
|
|
admin endpoints + `Capability::AppDeadLetterManage(AppId)`.
|
|
Recursion-stop rule: dead-letter handler failures annotate the
|
|
original row as `resolution = 'handler_failed'` and never produce
|
|
a new dead-letter or retry.
|
|
- **Dashboard surface for dead letters** — unresolved-count red
|
|
badge on the apps list + per-app page; per-app dead-letters list
|
|
view at `/admin/apps/{slug}/dead-letters` with Replay + Mark
|
|
resolved per-row actions and expandable payload detail.
|
|
- **`abandoned_executions` table** — forensic row written by the
|
|
dispatcher when it tries to resolve an inbox the orchestrator
|
|
already abandoned (timed out). Counter metric path reserved.
|
|
- **Trigger-depth limit** — `cx.trigger_depth > max_trigger_depth`
|
|
(default 8) skips execution + logs; does NOT dead-letter
|
|
(depth-exceeded means "you built a loop").
|
|
- **GC sweepers** — weekly retention sweeps for `dead_letters`
|
|
(30 days) and `abandoned_executions` (7 days), both with
|
|
`FOR UPDATE SKIP LOCKED` for cluster-mode safety.
|
|
- **Env-overridable trigger config** — `TriggerConfig::from_env`
|
|
reads `PICLOUD_MAX_TRIGGER_DEPTH`, `PICLOUD_TRIGGER_RETRY_*`,
|
|
`PICLOUD_DEAD_LETTER_RETENTION_DAYS`,
|
|
`PICLOUD_ABANDONED_EXECUTIONS_RETENTION_DAYS`.
|
|
|
|
### Changed
|
|
|
|
- **Workspace version**: `1.1.0` → `1.1.1`.
|
|
- **Rhai SDK version**: `1.1` → `1.2` (additive — every v1.1 script
|
|
still runs unchanged; new surfaces: `kv::*`, `dead_letters::*`,
|
|
`ctx.event` for triggered handlers).
|
|
- **Dashboard version**: `0.6.0` → `0.7.0` for the dead-letters UI.
|
|
- **`Services` bundle** — replaces v1.1.0's no-arg `Services::new()`
|
|
with explicit `Services::new(kv, dead_letters, events)`. Tests
|
|
use `Services::default()` for an all-noop bundle.
|
|
- **`SdkCallCx`** grows `is_dead_letter_handler: bool` and
|
|
`event: Option<TriggerEvent>` fields.
|
|
- **`ExecRequest`** mirrors the new `SdkCallCx` fields and grows
|
|
`event` for serializable trigger payload transport.
|
|
- **Routes table** grows `dispatch_mode TEXT NOT NULL DEFAULT 'sync'`
|
|
(CHECK in {sync, async}).
|
|
- **Schema version**: 6 → 12 (migrations 0007 through 0012).
|
|
|
|
### Migrations
|
|
|
|
- `0007_kv.sql` — `kv_entries` table + index
|
|
- `0008_triggers.sql` — `triggers` + `kv_trigger_details` +
|
|
`dead_letter_trigger_details`
|
|
- `0009_outbox.sql` — universal `outbox` table + due-row partial index
|
|
- `0010_dead_letters.sql` — `dead_letters` table + unresolved partial
|
|
index + GC index
|
|
- `0011_abandoned_executions.sql` — forensic table + GC index
|
|
- `0012_routes_dispatch_mode.sql` — `routes.dispatch_mode` column
|
|
|
|
## v1.1.0 — Foundation & Standard Library
|
|
|
|
See `docs/v1.1.x-design-notes.md` §7 for the full v1.1.x roadmap.
|