PiCloud/CHANGELOG.md

# PiCloud Changelog

## v1.1.3 — Modules (unreleased)

Real per-app Rhai module system. Scripts can `import "<name>" as
<alias>;` other scripts in the same app as reusable libraries. The
v1.0 placeholder `DummyModuleResolver` is replaced by a per-call
`PicloudModuleResolver` that loads `kind = 'module'` scripts via a
new `ModuleSource` trait, compiles them into Rhai modules, caches
the compiled output, and enforces cross-app isolation, circular-
import detection, and an import-depth limit. Two LRU AST caches
(top-level script + per-module compiled module) eliminate the
per-invocation compile cost; both invalidate on `updated_at` change.

### Added

- **`scripts.kind` column** — `'endpoint' | 'module'`, default
  `'endpoint'`. Endpoints handle HTTP routes / trigger events;
  modules are libraries imported by other scripts. The dashboard
  scripts list + script detail page surface the distinction as a
  colored badge.
- **`script_imports` dep-graph table** — populated at script save-
  time from the literal-path `import "<name>"` declarations in the
  source. FK-CASCADE on both columns. No admin surface in v1.1.3
  (drives a v1.2+ "Used by" dashboard panel and v1.3+ cluster-mode
  eager invalidation).
- **`ModuleSource` trait** — `lookup(&SdkCallCx, name)`. Postgres
  impl `PostgresModuleSource` in manager-core. `app_id` derived from
  `cx.app_id` (cross-app isolation boundary, mirrors KV / docs).
- **`PicloudModuleResolver`** — implements `rhai::ModuleResolver`.
  Per-call instance owns `Arc<SdkCallCx>`, the in-progress imports
  stack, the depth counter. Bridges sync `resolve()` to async
  `lookup()` via `Handle::block_on` (safe under the executor's
  `spawn_blocking` wrap). Replaces `DummyModuleResolver` at line 139
  of `executor-core::engine::build_engine`.
- **Module-shape validation** — `kind = 'module'` source must contain
  only `fn` declarations, `const` declarations, and `import`
  statements at top level (no executable expressions). Walks
  `ast.statements()` via `rhai/internals`. Admin endpoint is the
  primary gate; the resolver re-runs the check at load time for
  defense in depth against DB-direct inserts.
- **Per-module compiled-Module cache** — `LruCache<(AppId, name),
  (updated_at, Arc<rhai::Module>)>` owned by `Engine`. Invalidated
  lazily on `updated_at` mismatch. Size via
  `PICLOUD_MODULE_CACHE_SIZE` (default 512).
- **Top-level script AST cache** — `LruCache<ScriptId, (updated_at,
  Arc<rhai::AST>)>` owned by `LocalExecutorClient`. Same staleness
  semantics. Size via `PICLOUD_SCRIPT_CACHE_SIZE` (default 256).
- **`ScriptIdentity` + `ExecutorClient::execute_with_identity`** —
  new method on the trait; default impl forwards to `execute` so
  `RemoteExecutorClient` (and future transports) keep working.
  `LocalExecutorClient` overrides it to consult the script cache and
  pass the resulting `Arc<rhai::AST>` to `Engine::execute_ast`.
- **`Engine::execute_ast`** — companion to `execute` that takes a
  pre-compiled AST so callers (the orchestrator) can reuse one
  compile across many invocations.
- **Import depth limit** — `Limits::module_import_depth_max`
  (default 8). Not script-overridable.
- **Reserved module names** — module-kind scripts cannot be named
  `log`, `regex`, `random`, `time`, `json`, `base64`, `hex`, `url`,
  `kv`, `docs`, `dead_letters`, `http`, `files`, `pubsub`, `secrets`,
  `email`, `users`, `queue`. Defense against author confusion with
  stdlib namespaces.

### Changed

- **Workspace version**: `1.1.2` → `1.1.3`.
- **Rhai SDK version**: `1.3` → `1.4` (additive — every v1.3 script
  still runs unchanged; new surface: `import "<name>" as <alias>;`
  for endpoint scripts that consume modules in the same app).
- **Dashboard version**: `0.8.0` → `0.9.0`. Adds kind dropdown on
  script create + kind badges on the scripts list and detail page.
- **`Services` bundle** — grows a `modules: Arc<dyn ModuleSource>`
  field. Constructor signature becomes
  `Services::new(kv, docs, dead_letters, events, modules)`.
- **`ScriptValidator` trait** — `validate` now returns
  `ValidatedScript { imports: Vec<String> }` so the repo can write
  dep-graph edges in the same transaction as the script row. New
  `validate_module` method enforces module-shape rules.
- **Trigger creation tightening** — `POST /api/v1/admin/apps/{id}/triggers/{kv,docs,dead_letter}`
  now load the target script and reject when (1) it doesn't exist,
  (2) it belongs to a different app (latent v1.1.1/v1.1.2 gap —
  closed in v1.1.3), or (3) it is `kind = 'module'`.
- **Route creation** — `POST /api/v1/admin/scripts/{id}/routes`
  returns 400 when the target script is `kind = 'module'`.

### Migrations

- `0015_scripts_kind.sql` — adds `scripts.kind` with CHECK
  `IN ('endpoint','module')`, composite index `(app_id, kind)`, and
  a module-name shape CHECK (`^[a-zA-Z_][a-zA-Z0-9_]{0,63}$`).
- `0016_script_imports.sql` — adds the dep-graph table with FK
  CASCADE on both columns, PK `(importer, imported)`, and a
  reverse-edge index on `imported_script_id`.

### Downgrade caveats

Rolling back v1.1.3 → v1.1.2 with module-kind scripts present
strands them (no `kind` column means everything looks like an
endpoint; modules will then succeed as route targets and immediately
fail to execute meaningfully). Migration `0016_script_imports.sql`
is safe to drop (the table is auxiliary). `0015_scripts_kind.sql`
must be reversed by `DROP COLUMN kind` only after manually re-homing
or deleting module-kind rows.

## v1.1.2 — Documents (unreleased)

`docs::*` SDK — schemaless JSONB document storage with a first-cut
query DSL — plus `docs:*` triggers as the second concrete kind on the
v1.1.1 triggers framework. Sets the precedent for the v1.2 query DSL
expansion and `dead_letters::list`.

### Added

- **Docs store** — `docs` table keyed `(app_id, collection, id)` with
  JSONB values and a GIN-on-`jsonb_path_ops` index. Rhai SDK exposes
  the handle pattern:
  `docs::collection(name).{create,get,find,find_one,update,delete,list}`.
  Cursor-style pagination on `list`. Cross-app isolation enforced via
  `cx.app_id` (never script-passed). Document envelope shape returned
  by reads: `#{ id, data: #{...}, created_at, updated_at }` — explicit
  metadata + user-data separation (sets precedent for v1.2
  `dead_letters::list`).
- **Query DSL (v1.1.2 subset)** — implicit equality at top level
  (`#{ tier: "gold" }`), operator-object form
  (`#{ created_at: #{ "$gt": "..." } }`), dotted field paths up to 5
  levels (`"user.email"`), and operators `$eq`/`$ne`/`$gt`/`$gte`/
  `$lt`/`$lte`/`$in`. Filter modifiers `$sort` (single field) and
  `$limit`. Unsupported operators (`$or`, `$regex`, etc.) reject with
  a clear v1.2-pointer error.
- **Docs triggers (`docs:*`)** — `docs_trigger_details` table mirrors
  `kv_trigger_details`. Admin endpoint
  `POST /api/v1/admin/apps/{id}/triggers/docs` accepts the same DTO
  shape as the KV endpoint with `ops` of `DocsEventOp` (create /
  update / delete). Dispatcher routes `OutboxSourceKind::Docs` through
  the same generic path as KV + dead-letter.
- **`ctx.event.docs.prev_data`** — change-data-capture surface for
  docs trigger handlers. `prev_data` carries the document state prior
  to the mutation (`None` for create), letting handlers see what
  changed. The repo reads the old row in the same SQL statement as
  the write so the trigger event has the prior value.
- **`Capability::AppDocsRead(AppId)`** + `AppDocsWrite(AppId)` —
  granted to Viewer / Editor respectively in the per-app role table.
  Same trust shape as KV's `AppKvRead` / `AppKvWrite`.

### Changed

- **Workspace version**: `1.1.1` → `1.1.2`.
- **Rhai SDK version**: `1.2` → `1.3` (additive — every v1.2 script
  still runs unchanged; new surfaces: `docs::collection(name).{...}`,
  `ctx.event.docs` for triggered handlers).
- **Dashboard version**: `0.7.0` → `0.8.0`. Workspace alignment; no
  docs-specific UI in v1.1.2 (the dashboard's Rhai-mode hints don't
  list KV completions either — focused UX pass is a separate task).
- **`Services` bundle** — grows a `docs: Arc<dyn DocsService>` field.
  Constructor signature becomes
  `Services::new(kv, docs, dead_letters, events)`.
- **Scope mapping**: API keys with `script:read` scope can call
  `docs::find` / `get` / `list`; `script:write` can call
  `docs::create` / `update` / `delete`. Same trust shape as KV —
  honors the seven-scope commitment from v1.1.0.

### Migrations

- `0013_docs.sql` — `docs` table + per-`(app_id, collection)` index +
  GIN-on-`jsonb_path_ops` index.
- `0014_docs_triggers.sql` — extends `triggers.kind` and
  `outbox.source_kind` CHECK constraints to include `'docs'`; adds
  `docs_trigger_details` table.

### Downgrade caveats

Rolling a deployment back from v1.1.2 → v1.1.1 with `docs`-source
outbox rows still queued will cause the v1.1.1 dispatcher to fail
deserialising `TriggerEvent::Docs` (`#[serde(tag = "source")]`
rejects unknown variants). Drain or delete
`outbox WHERE source_kind = 'docs'` before downgrading. Trunk-only
deployments don't hit this.

### Known limitations

- Text-lex comparison for `$gt` / `$gte` / `$lt` / `$lte` is
  incorrect for unpadded numbers crossing digit-count boundaries
  (`'10' < '9'` is TRUE under any text collation). Workaround:
  zero-pad numeric strings. v1.2's advanced query expansion adds
  numeric-aware operators.
- Concurrent `update()`s on the same doc may both emit the
  pre-update `prev_data` (last-writer-wins). Inherited from KV's
  `set` pattern; documented for forensic-trace use cases.
- v1.1.2 has no partial-update DSL — scripts that want partial
  update do `get + modify + update`. Planned for v1.2.

## v1.1.1 — Storage & Events (unreleased)

The triggers framework — KV store + universal outbox + dispatcher +
NATS-style sync HTTP + per-route async dispatch + dead-letter
handling + dashboard surface. Every subsequent v1.1.x service module
(docs, files, pubsub, …) hangs off the dispatcher built here.

### Added

- **KV store** — `kv_entries` table keyed `(app_id, collection, key)`
  with JSONB values. Rhai SDK exposes the handle pattern:
  `kv::collection(name).{get,set,has,delete,list}`. Cursor-style
  pagination with opaque base64 cursors. Cross-app isolation
  enforced via `cx.app_id` (never script-passed).
- **Triggers framework (Layout E)** — parent `triggers` table +
  per-kind detail tables (`kv_trigger_details`,
  `dead_letter_trigger_details`). Trigger CRUD admin endpoints
  (`/api/v1/admin/apps/{id}/triggers/{kv,dead_letter}`) +
  `Capability::AppManageTriggers(AppId)`.
- **Universal outbox + dispatcher** — single tokio task that polls
  the outbox via `FOR UPDATE SKIP LOCKED`, routes due rows to the
  executor through the shared `ExecutionGate`. Retry with
  exponential backoff + ±jitter; on exhaustion, dead-letter.
- **NATS-style sync HTTP via outbox** — `InboxRegistry` (in-process
  oneshot map) lets the orchestrator await dispatcher delivery on
  every sync HTTP request. Cluster mode (v1.3+) swaps this for
  `LISTEN/NOTIFY` behind the same `InboxResolver` trait.
- **`dispatch_mode: async` on routes** — `POST` to a route with
  `dispatch_mode = 'async'` returns `202 Accepted` immediately;
  the script runs via the dispatcher (with retries / dead-letter).
- **Dead-letter handling** — separate `dead_letters` table per
  design notes §4. `dead_letters::{replay,resolve}` Rhai SDK +
  admin endpoints + `Capability::AppDeadLetterManage(AppId)`.
  Recursion-stop rule: dead-letter handler failures annotate the
  original row as `resolution = 'handler_failed'` and never produce
  a new dead-letter or retry.
- **Dashboard surface for dead letters** — unresolved-count red
  badge on the apps list + per-app page; per-app dead-letters list
  view at `/admin/apps/{slug}/dead-letters` with Replay + Mark
  resolved per-row actions and expandable payload detail.
- **`abandoned_executions` table** — forensic row written by the
  dispatcher when it tries to resolve an inbox the orchestrator
  already abandoned (timed out). Counter metric path reserved.
- **Trigger-depth limit** — `cx.trigger_depth > max_trigger_depth`
  (default 8) skips execution + logs; does NOT dead-letter
  (depth-exceeded means "you built a loop").
- **GC sweepers** — weekly retention sweeps for `dead_letters`
  (30 days) and `abandoned_executions` (7 days), both with
  `FOR UPDATE SKIP LOCKED` for cluster-mode safety.
- **Env-overridable trigger config** — `TriggerConfig::from_env`
  reads `PICLOUD_MAX_TRIGGER_DEPTH`, `PICLOUD_TRIGGER_RETRY_*`,
  `PICLOUD_DEAD_LETTER_RETENTION_DAYS`,
  `PICLOUD_ABANDONED_EXECUTIONS_RETENTION_DAYS`.

### Changed

- **Workspace version**: `1.1.0` → `1.1.1`.
- **Rhai SDK version**: `1.1` → `1.2` (additive — every v1.1 script
  still runs unchanged; new surfaces: `kv::*`, `dead_letters::*`,
  `ctx.event` for triggered handlers).
- **Dashboard version**: `0.6.0` → `0.7.0` for the dead-letters UI.
- **`Services` bundle** — replaces v1.1.0's no-arg `Services::new()`
  with explicit `Services::new(kv, dead_letters, events)`. Tests
  use `Services::default()` for an all-noop bundle.
- **`SdkCallCx`** grows `is_dead_letter_handler: bool` and
  `event: Option<TriggerEvent>` fields.
- **`ExecRequest`** mirrors the new `SdkCallCx` fields and grows
  `event` for serializable trigger payload transport.
- **Routes table** grows `dispatch_mode TEXT NOT NULL DEFAULT 'sync'`
  (CHECK in {sync, async}).
- **Schema version**: 6 → 12 (migrations 0007 through 0012).

### Migrations

- `0007_kv.sql` — `kv_entries` table + index
- `0008_triggers.sql` — `triggers` + `kv_trigger_details` +
  `dead_letter_trigger_details`
- `0009_outbox.sql` — universal `outbox` table + due-row partial index
- `0010_dead_letters.sql` — `dead_letters` table + unresolved partial
  index + GC index
- `0011_abandoned_executions.sql` — forensic table + GC index
- `0012_routes_dispatch_mode.sql` — `routes.dispatch_mode` column

## v1.1.0 — Foundation & Standard Library

See `docs/v1.1.x-design-notes.md` §7 for the full v1.1.x roadmap.