PiCloud/CHANGELOG.md

# PiCloud Changelog

## v1.1.4 — Outbound HTTP & Cron triggers (unreleased)

Two surfaces. **`http::*`** lets Rhai scripts make outbound HTTP
requests (Slack webhooks, Stripe, third-party REST) fronted by an SSRF
deny-list applied to the *resolved IP* (DNS-rebinding defense), with
scheme/port restrictions, request/response body caps, and a layered
timeout. **Cron triggers** add the fourth concrete kind on the v1.1.1
trigger framework: a scheduler task enqueues due triggers into the same
universal outbox the dispatcher already drains.

### Added

- **`http::{get,post,put,patch,delete,head,post_form,request}`** — outbound
  HTTP SDK. Body and options are separate positional args
  (`verb(url, body, opts)`); `opts` is
  `{headers, timeout_ms, follow_redirects, max_redirects}` (unknown keys
  throw). Body dispatch by type: Map/Array → JSON, String → text/plain,
  `()` → none. Response is `#{ status, headers, body, body_raw }` with
  `body` auto-parsed when the response is `application/json`. Non-2xx
  does NOT throw (fetch-style); network/timeout/SSRF/size errors throw
  with an `"http: …"` prefix.
- **SSRF deny-list** — applied to the resolved IP via a custom reqwest
  `dns_resolver` (so it covers every redirect hop and defeats DNS
  rebinding), plus a literal-IP check at URL-parse time. Blocks
  loopback, RFC1918 private, link-local (incl. `169.254.169.254`),
  carrier-grade NAT, multicast, reserved, IPv6 ULA/link-local/loopback,
  and IPv4-mapped IPv6 (re-checked against the embedded v4 address).
  The script-visible error carries a CIDR-category reason, never the IP.
  `PICLOUD_HTTP_ALLOW_PRIVATE=true` disables it (dev-only; logs a startup
  warning).
- **`HttpService` trait** (`picloud-shared`) + `HttpServiceImpl`
  (manager-core, reqwest-backed). Wired into the `Services` bundle as
  `http: Arc<dyn HttpService>`.
- **`Capability::AppHttpRequest(AppId)`** — maps to the existing
  `script:write` scope (any outbound request can exfiltrate data, so the
  conservative write mapping is used). No new `Scope` variant — the
  seven-scope commitment holds. Script-as-gate: skipped when the script
  runs unauthenticated.
- **Cron triggers** — `POST /api/v1/admin/apps/{id}/triggers/cron`
  (`script_id`, `schedule`, `timezone`, optional retry overrides).
  6-field cron expressions (with seconds) validated by the `cron` crate;
  IANA timezones validated by `chrono-tz`. A scheduler task
  (`spawn_cron_scheduler`, poll cadence `PICLOUD_CRON_TICK_INTERVAL_MS`,
  default 30s) enqueues due triggers into the outbox; the existing
  dispatcher delivers them. Catch-up policy: a trigger that missed N
  windows fires exactly **once** on the next tick, not N times.
- **`ctx.event.cron`** — `{ schedule, timezone, scheduled_at, fired_at }`
  for cron-trigger handlers (`ctx.event.source == "cron"`,
  `ctx.event.op == "tick"`).
- **Dashboard Triggers tab** — admin-gated cron trigger create form
  (target endpoint script, schedule, timezone dropdown) + triggers list
  showing schedule / timezone / last-fired.

### Changed

- **Workspace version**: `1.1.3` → `1.1.4`.
- **Rhai SDK version**: `1.4` → `1.5` (additive — `http::*` SDK +
  `ctx.event.cron`). The `Services` bundle constructor becomes
  `Services::new(kv, docs, dead_letters, events, modules, http)`.
- **Dashboard version**: `0.9.0` → `0.10.0`.
- **`SdkCallCx`** — gains a `script_id` field (audit attribution + the
  default outbound `User-Agent`, `picloud/<version> (script:<id>)`).
- **Rhai pin tightened** — workspace dep `rhai = "1.19"` → `rhai = "=1.24"`
  so future bumps of the non-semver-stable `internals` surface are
  deliberate.
- **Module backend errors redacted** — `PicloudModuleResolver` now
  surfaces a stable generic (`"module backend unavailable; check server
  logs"`) to scripts and logs the original at error level, instead of
  leaking the backend error verbatim (see v1.1.3 follow-up).

### Migrations

- `0017_cron_triggers.sql` — widens `triggers.kind` and
  `outbox.source_kind` CHECK constraints to include `'cron'`; adds
  `cron_trigger_details (trigger_id, schedule, timezone, last_fired_at)`
  with a `last_fired_at` index. Additive — applies cleanly on a fresh DB
  and on top of the v1.1.3 schema.

### New environment variables

- `PICLOUD_HTTP_ALLOW_PRIVATE` (default false; dev-only) — disable the
  SSRF deny-list.
- `PICLOUD_HTTP_MAX_REQUEST_BODY_BYTES` / `PICLOUD_HTTP_MAX_RESPONSE_BODY_BYTES`
  (default 10 MB each).
- `PICLOUD_CRON_TICK_INTERVAL_MS` (default 30000) — cron scheduler poll
  cadence (floored at 1s).

## v1.1.3 — Modules (unreleased)

Real per-app Rhai module system. Scripts can `import "<name>" as
<alias>;` other scripts in the same app as reusable libraries. The
v1.0 placeholder `DummyModuleResolver` is replaced by a per-call
`PicloudModuleResolver` that loads `kind = 'module'` scripts via a
new `ModuleSource` trait, compiles them into Rhai modules, caches
the compiled output, and enforces cross-app isolation, circular-
import detection, and an import-depth limit. Two LRU AST caches
(top-level script + per-module compiled module) eliminate the
per-invocation compile cost; both invalidate on `updated_at` change.

### Added

- **`scripts.kind` column** — `'endpoint' | 'module'`, default
  `'endpoint'`. Endpoints handle HTTP routes / trigger events;
  modules are libraries imported by other scripts. The dashboard
  scripts list + script detail page surface the distinction as a
  colored badge.
- **`script_imports` dep-graph table** — populated at script save-
  time from the literal-path `import "<name>"` declarations in the
  source. FK-CASCADE on both columns. No admin surface in v1.1.3
  (drives a v1.2+ "Used by" dashboard panel and v1.3+ cluster-mode
  eager invalidation).
- **`ModuleSource` trait** — `lookup(&SdkCallCx, name)`. Postgres
  impl `PostgresModuleSource` in manager-core. `app_id` derived from
  `cx.app_id` (cross-app isolation boundary, mirrors KV / docs).
- **`PicloudModuleResolver`** — implements `rhai::ModuleResolver`.
  Per-call instance owns `Arc<SdkCallCx>`, the in-progress imports
  stack, the depth counter. Bridges sync `resolve()` to async
  `lookup()` via `Handle::block_on` (safe under the executor's
  `spawn_blocking` wrap). Replaces `DummyModuleResolver` at line 139
  of `executor-core::engine::build_engine`.
- **Module-shape validation** — `kind = 'module'` source must contain
  only `fn` declarations, `const` declarations, and `import`
  statements at top level (no executable expressions). Walks
  `ast.statements()` via `rhai/internals`. Admin endpoint is the
  primary gate; the resolver re-runs the check at load time for
  defense in depth against DB-direct inserts.
- **Per-module compiled-Module cache** — `LruCache<(AppId, name),
  (updated_at, Arc<rhai::Module>)>` owned by `Engine`. Invalidated
  lazily on `updated_at` mismatch. Size via
  `PICLOUD_MODULE_CACHE_SIZE` (default 512).
- **Top-level script AST cache** — `LruCache<ScriptId, (updated_at,
  Arc<rhai::AST>)>` owned by `LocalExecutorClient`. Same staleness
  semantics. Size via `PICLOUD_SCRIPT_CACHE_SIZE` (default 256).
- **`ScriptIdentity` + `ExecutorClient::execute_with_identity`** —
  new method on the trait; default impl forwards to `execute` so
  `RemoteExecutorClient` (and future transports) keep working.
  `LocalExecutorClient` overrides it to consult the script cache and
  pass the resulting `Arc<rhai::AST>` to `Engine::execute_ast`.
- **`Engine::execute_ast`** — companion to `execute` that takes a
  pre-compiled AST so callers (the orchestrator) can reuse one
  compile across many invocations.
- **Import depth limit** — `Limits::module_import_depth_max`
  (default 8). Not script-overridable.
- **Reserved module names** — module-kind scripts cannot be named
  `log`, `regex`, `random`, `time`, `json`, `base64`, `hex`, `url`,
  `kv`, `docs`, `dead_letters`, `http`, `files`, `pubsub`, `secrets`,
  `email`, `users`, `queue`. Defense against author confusion with
  stdlib namespaces.

### Changed

- **Workspace version**: `1.1.2` → `1.1.3`.
- **Rhai SDK version**: `1.3` → `1.4` (additive — every v1.3 script
  still runs unchanged; new surface: `import "<name>" as <alias>;`
  for endpoint scripts that consume modules in the same app).
- **Dashboard version**: `0.8.0` → `0.9.0`. Adds kind dropdown on
  script create + kind badges on the scripts list and detail page.
- **`Services` bundle** — grows a `modules: Arc<dyn ModuleSource>`
  field. Constructor signature becomes
  `Services::new(kv, docs, dead_letters, events, modules)`.
- **`ScriptValidator` trait** — `validate` now returns
  `ValidatedScript { imports: Vec<String> }` so the repo can write
  dep-graph edges in the same transaction as the script row. New
  `validate_module` method enforces module-shape rules.
- **Trigger creation tightening** — `POST /api/v1/admin/apps/{id}/triggers/{kv,docs,dead_letter}`
  now load the target script and reject when (1) it doesn't exist,
  (2) it belongs to a different app (latent v1.1.1/v1.1.2 gap —
  closed in v1.1.3), or (3) it is `kind = 'module'`.
- **Route creation** — `POST /api/v1/admin/scripts/{id}/routes`
  returns 400 when the target script is `kind = 'module'`.

### Security fix

- **Cross-app trigger target (CVE-class: broken access control).** In
  v1.1.1 and v1.1.2, `POST /api/v1/admin/apps/{id}/triggers/{kv,docs,dead_letter}`
  validated only that the caller could manage triggers on `{id}` — it
  did **not** verify that the target `script_id` belonged to that same
  app. A member with trigger-management rights on app A could therefore
  register a trigger in A pointing at a script owned by app B, causing
  B's script to execute on A's events (a cross-app isolation break).
  v1.1.3 closes this: every trigger-create handler now loads the target
  script and rejects it unless `script.app_id == path app_id` (and it is
  not a module). **Upgrade recommendation:** anyone running a pre-v1.1.3
  multi-tenant deploy should upgrade and audit existing `triggers` rows
  for any whose `script_id` resolves to a script in a different `app_id`.

### Migrations

- `0015_scripts_kind.sql` — adds `scripts.kind` with CHECK
  `IN ('endpoint','module')`, composite index `(app_id, kind)`, and
  a module-name shape CHECK (`^[a-zA-Z_][a-zA-Z0-9_]{0,63}$`).
- `0016_script_imports.sql` — adds the dep-graph table with FK
  CASCADE on both columns, PK `(importer, imported)`, and a
  reverse-edge index on `imported_script_id`.

### Downgrade caveats

Rolling back v1.1.3 → v1.1.2 with module-kind scripts present
strands them (no `kind` column means everything looks like an
endpoint; modules will then succeed as route targets and immediately
fail to execute meaningfully). Migration `0016_script_imports.sql`
is safe to drop (the table is auxiliary). `0015_scripts_kind.sql`
must be reversed by `DROP COLUMN kind` only after manually re-homing
or deleting module-kind rows.

## v1.1.2 — Documents (unreleased)

`docs::*` SDK — schemaless JSONB document storage with a first-cut
query DSL — plus `docs:*` triggers as the second concrete kind on the
v1.1.1 triggers framework. Sets the precedent for the v1.2 query DSL
expansion and `dead_letters::list`.

### Added

- **Docs store** — `docs` table keyed `(app_id, collection, id)` with
  JSONB values and a GIN-on-`jsonb_path_ops` index. Rhai SDK exposes
  the handle pattern:
  `docs::collection(name).{create,get,find,find_one,update,delete,list}`.
  Cursor-style pagination on `list`. Cross-app isolation enforced via
  `cx.app_id` (never script-passed). Document envelope shape returned
  by reads: `#{ id, data: #{...}, created_at, updated_at }` — explicit
  metadata + user-data separation (sets precedent for v1.2
  `dead_letters::list`).
- **Query DSL (v1.1.2 subset)** — implicit equality at top level
  (`#{ tier: "gold" }`), operator-object form
  (`#{ created_at: #{ "$gt": "..." } }`), dotted field paths up to 5
  levels (`"user.email"`), and operators `$eq`/`$ne`/`$gt`/`$gte`/
  `$lt`/`$lte`/`$in`. Filter modifiers `$sort` (single field) and
  `$limit`. Unsupported operators (`$or`, `$regex`, etc.) reject with
  a clear v1.2-pointer error.
- **Docs triggers (`docs:*`)** — `docs_trigger_details` table mirrors
  `kv_trigger_details`. Admin endpoint
  `POST /api/v1/admin/apps/{id}/triggers/docs` accepts the same DTO
  shape as the KV endpoint with `ops` of `DocsEventOp` (create /
  update / delete). Dispatcher routes `OutboxSourceKind::Docs` through
  the same generic path as KV + dead-letter.
- **`ctx.event.docs.prev_data`** — change-data-capture surface for
  docs trigger handlers. `prev_data` carries the document state prior
  to the mutation (`None` for create), letting handlers see what
  changed. The repo reads the old row in the same SQL statement as
  the write so the trigger event has the prior value.
- **`Capability::AppDocsRead(AppId)`** + `AppDocsWrite(AppId)` —
  granted to Viewer / Editor respectively in the per-app role table.
  Same trust shape as KV's `AppKvRead` / `AppKvWrite`.

### Changed

- **Workspace version**: `1.1.1` → `1.1.2`.
- **Rhai SDK version**: `1.2` → `1.3` (additive — every v1.2 script
  still runs unchanged; new surfaces: `docs::collection(name).{...}`,
  `ctx.event.docs` for triggered handlers).
- **Dashboard version**: `0.7.0` → `0.8.0`. Workspace alignment; no
  docs-specific UI in v1.1.2 (the dashboard's Rhai-mode hints don't
  list KV completions either — focused UX pass is a separate task).
- **`Services` bundle** — grows a `docs: Arc<dyn DocsService>` field.
  Constructor signature becomes
  `Services::new(kv, docs, dead_letters, events)`.
- **Scope mapping**: API keys with `script:read` scope can call
  `docs::find` / `get` / `list`; `script:write` can call
  `docs::create` / `update` / `delete`. Same trust shape as KV —
  honors the seven-scope commitment from v1.1.0.

### Migrations

- `0013_docs.sql` — `docs` table + per-`(app_id, collection)` index +
  GIN-on-`jsonb_path_ops` index.
- `0014_docs_triggers.sql` — extends `triggers.kind` and
  `outbox.source_kind` CHECK constraints to include `'docs'`; adds
  `docs_trigger_details` table.

### Downgrade caveats

Rolling a deployment back from v1.1.2 → v1.1.1 with `docs`-source
outbox rows still queued will cause the v1.1.1 dispatcher to fail
deserialising `TriggerEvent::Docs` (`#[serde(tag = "source")]`
rejects unknown variants). Drain or delete
`outbox WHERE source_kind = 'docs'` before downgrading. Trunk-only
deployments don't hit this.

### Known limitations

- Text-lex comparison for `$gt` / `$gte` / `$lt` / `$lte` is
  incorrect for unpadded numbers crossing digit-count boundaries
  (`'10' < '9'` is TRUE under any text collation). Workaround:
  zero-pad numeric strings. v1.2's advanced query expansion adds
  numeric-aware operators.
- Concurrent `update()`s on the same doc may both emit the
  pre-update `prev_data` (last-writer-wins). Inherited from KV's
  `set` pattern; documented for forensic-trace use cases.
- v1.1.2 has no partial-update DSL — scripts that want partial
  update do `get + modify + update`. Planned for v1.2.

## v1.1.1 — Storage & Events (unreleased)

The triggers framework — KV store + universal outbox + dispatcher +
NATS-style sync HTTP + per-route async dispatch + dead-letter
handling + dashboard surface. Every subsequent v1.1.x service module
(docs, files, pubsub, …) hangs off the dispatcher built here.

### Added

- **KV store** — `kv_entries` table keyed `(app_id, collection, key)`
  with JSONB values. Rhai SDK exposes the handle pattern:
  `kv::collection(name).{get,set,has,delete,list}`. Cursor-style
  pagination with opaque base64 cursors. Cross-app isolation
  enforced via `cx.app_id` (never script-passed).
- **Triggers framework (Layout E)** — parent `triggers` table +
  per-kind detail tables (`kv_trigger_details`,
  `dead_letter_trigger_details`). Trigger CRUD admin endpoints
  (`/api/v1/admin/apps/{id}/triggers/{kv,dead_letter}`) +
  `Capability::AppManageTriggers(AppId)`.
- **Universal outbox + dispatcher** — single tokio task that polls
  the outbox via `FOR UPDATE SKIP LOCKED`, routes due rows to the
  executor through the shared `ExecutionGate`. Retry with
  exponential backoff + ±jitter; on exhaustion, dead-letter.
- **NATS-style sync HTTP via outbox** — `InboxRegistry` (in-process
  oneshot map) lets the orchestrator await dispatcher delivery on
  every sync HTTP request. Cluster mode (v1.3+) swaps this for
  `LISTEN/NOTIFY` behind the same `InboxResolver` trait.
- **`dispatch_mode: async` on routes** — `POST` to a route with
  `dispatch_mode = 'async'` returns `202 Accepted` immediately;
  the script runs via the dispatcher (with retries / dead-letter).
- **Dead-letter handling** — separate `dead_letters` table per
  design notes §4. `dead_letters::{replay,resolve}` Rhai SDK +
  admin endpoints + `Capability::AppDeadLetterManage(AppId)`.
  Recursion-stop rule: dead-letter handler failures annotate the
  original row as `resolution = 'handler_failed'` and never produce
  a new dead-letter or retry.
- **Dashboard surface for dead letters** — unresolved-count red
  badge on the apps list + per-app page; per-app dead-letters list
  view at `/admin/apps/{slug}/dead-letters` with Replay + Mark
  resolved per-row actions and expandable payload detail.
- **`abandoned_executions` table** — forensic row written by the
  dispatcher when it tries to resolve an inbox the orchestrator
  already abandoned (timed out). Counter metric path reserved.
- **Trigger-depth limit** — `cx.trigger_depth > max_trigger_depth`
  (default 8) skips execution + logs; does NOT dead-letter
  (depth-exceeded means "you built a loop").
- **GC sweepers** — weekly retention sweeps for `dead_letters`
  (30 days) and `abandoned_executions` (7 days), both with
  `FOR UPDATE SKIP LOCKED` for cluster-mode safety.
- **Env-overridable trigger config** — `TriggerConfig::from_env`
  reads `PICLOUD_MAX_TRIGGER_DEPTH`, `PICLOUD_TRIGGER_RETRY_*`,
  `PICLOUD_DEAD_LETTER_RETENTION_DAYS`,
  `PICLOUD_ABANDONED_EXECUTIONS_RETENTION_DAYS`.

### Changed

- **Workspace version**: `1.1.0` → `1.1.1`.
- **Rhai SDK version**: `1.1` → `1.2` (additive — every v1.1 script
  still runs unchanged; new surfaces: `kv::*`, `dead_letters::*`,
  `ctx.event` for triggered handlers).
- **Dashboard version**: `0.6.0` → `0.7.0` for the dead-letters UI.
- **`Services` bundle** — replaces v1.1.0's no-arg `Services::new()`
  with explicit `Services::new(kv, dead_letters, events)`. Tests
  use `Services::default()` for an all-noop bundle.
- **`SdkCallCx`** grows `is_dead_letter_handler: bool` and
  `event: Option<TriggerEvent>` fields.
- **`ExecRequest`** mirrors the new `SdkCallCx` fields and grows
  `event` for serializable trigger payload transport.
- **Routes table** grows `dispatch_mode TEXT NOT NULL DEFAULT 'sync'`
  (CHECK in {sync, async}).
- **Schema version**: 6 → 12 (migrations 0007 through 0012).

### Migrations

- `0007_kv.sql` — `kv_entries` table + index
- `0008_triggers.sql` — `triggers` + `kv_trigger_details` +
  `dead_letter_trigger_details`
- `0009_outbox.sql` — universal `outbox` table + due-row partial index
- `0010_dead_letters.sql` — `dead_letters` table + unresolved partial
  index + GC index
- `0011_abandoned_executions.sql` — forensic table + GC index
- `0012_routes_dispatch_mode.sql` — `routes.dispatch_mode` column

## v1.1.0 — Foundation & Standard Library

See `docs/v1.1.x-design-notes.md` §7 for the full v1.1.x roadmap.