Commit Graph

6 Commits

Author SHA1 Message Date
MechaCat02
6e132b6ee0 feat(v1.1.5): files SDK + files:* triggers
Filesystem-backed blob storage as the fifth concrete trigger kind.

- `files::collection(c).{create,head,get,update,delete,list}` Rhai SDK
  (blob in/out; metadata maps; missing-field throws naming the field).
- `FilesService` trait in picloud-shared; `FsFilesRepo` (atomic
  write: temp→fsync→rename→fsync-dir→DB; single-pass SHA-256;
  checksum-verified reads → Corrupted) + `FilesServiceImpl` in
  manager-core. Metadata in Postgres (0018), bytes on disk under
  PICLOUD_FILES_ROOT with 0o700 shard dirs.
- `files:*` trigger kind via the Layout-E pattern (0019: widen both
  CHECKs + files_trigger_details), TriggerEvent::Files (metadata only,
  no bytes), emit_files fan-out, dispatcher arm, admin endpoint
  POST /triggers/files (reuses validate_trigger_target).
- AppFilesRead/AppFilesWrite capabilities → script:read/script:write
  (seven-scope commitment held). AppPubsubPublish reserved for v1.1.6.
- Admin files API (list + delete) + dashboard Files view per app.

Cross-app isolation keyed on cx.app_id at every layer. ~45 new tests
(service in-memory, fs tempdir, bridge integration). No DB required
for the suite. publish_ephemeral and the orphan sweep stay deferred.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-03 21:18:17 +02:00
MechaCat02
10b5f655d5 feat(v1.1.4): outbound HTTP SDK + cron triggers
HTTP (`http::*`):
- `HttpService` trait (picloud-shared) + reqwest-backed `HttpServiceImpl`
  (manager-core), wired into the `Services` bundle.
- SSRF deny-list applied to the resolved IP via a custom reqwest
  `dns_resolver` (covers every redirect hop + defeats DNS rebinding) plus
  a literal-IP check at URL-parse time. Scheme/port restrictions, request
  + response body caps (stream-with-cap), layered timeout. Error reason is
  a CIDR category, never the IP. `PICLOUD_HTTP_ALLOW_PRIVATE` dev override
  (logs a startup warning).
- Rhai bridge with three-arg split `verb(url, body, opts)` (resolves the
  brief's body-vs-opts contradiction; unknown opt keys throw). Body
  dispatch by type; response `#{status,headers,body,body_raw}` with JSON
  auto-parse; non-2xx does not throw.
- `Capability::AppHttpRequest` → existing `script:write` scope (no new
  Scope variant). `SdkCallCx` gains `script_id` (attribution + User-Agent).

Cron triggers (4th trigger kind):
- Migration 0017 widens the kind/source_kind CHECKs and adds
  `cron_trigger_details`. `cron`/`chrono-tz` parse + validate 6-field
  schedules and IANA timezones.
- `spawn_cron_scheduler` polls due triggers and enqueues to the universal
  outbox; the dispatcher delivers them (one-line match-arm extension).
  Catch-up fires exactly once per trigger per tick, not once per missed
  window. `ctx.event.cron` for handlers.
- `POST /api/v1/admin/apps/{id}/triggers/cron` reuses the v1.1.3
  cross-app + kind!=module target check.
- Dashboard: admin-gated Triggers tab (cron create form + list).

Follow-ups: redact module backend errors at the resolver boundary (log
original at error level); pin `rhai = "=1.24"`; CHANGELOG incl. retroactive
v1.1.3 cross-app-trigger security note. Version bumps: workspace 1.1.4,
SDK 1.5, dashboard 0.10.0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-03 20:23:18 +02:00
MechaCat02
66b41bb978 feat(v1.1.3-modules): top-level script AST cache in LocalExecutorClient
- New `ScriptIdentity { script_id, updated_at }` DTO.
- `ExecutorClient` trait gains an `execute_with_identity` method;
  default impl forwards to `execute` so `RemoteExecutorClient` (and
  cluster-mode transports later) keep working without bespoke caching.
- `LocalExecutorClient` overrides `execute_with_identity` to consult
  an `LruCache<ScriptId, CachedScript>`. Cache hit only when the
  cached entry's `updated_at` matches the caller's identity; mismatch
  triggers a fresh `Engine::compile`. `Engine::execute_ast(&Arc<AST>, req)`
  is called inside `spawn_blocking` exactly as `execute` does today.
- Cache size from `PICLOUD_SCRIPT_CACHE_SIZE` (default 256).
- Orchestrator's HTTP data-plane path and the dispatcher both switch
  to `execute_with_identity`. `ResolvedTrigger` carries
  `script_updated_at` for the dispatcher's identity construction.

Workspace builds; full test suite (~440 tests) green.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-02 22:23:11 +02:00
MechaCat02
ef5930910b feat(v1.1.2-docs): triggers framework + dispatcher + emitter extended for docs
The docs trigger kind hangs off the same Layout-E shape that v1.1.1
established for KV: a parent triggers row + a docs_trigger_details
row (collection_glob TEXT + ops TEXT[]) with the empty-array =
any-op semantic preserved.

- trigger_repo.rs adds TriggerKind::Docs + TriggerDetails::Docs +
  CreateDocsTrigger + DocsTriggerMatch + PostgresTriggerRepo
  implementations of create_docs_trigger and list_matching_docs.
  list_matching_docs mirrors KV's Rust-side filter (does NOT push
  ops membership into SQL — that would exclude empty-ops rows).
- outbox_repo.rs adds OutboxSourceKind::Docs to the enum + wire form.
- dispatcher.rs's generic Kv | DeadLetter routing arm extends to
  Kv | DeadLetter | Docs. No kind-specific logic needed — the
  resolve_trigger + build_exec_request path is already abstract.
- outbox_event_emitter.rs gains a "docs" arm in the emit match plus
  emit_docs which builds TriggerEvent::Docs (carrying data +
  prev_data) and fans out across matching triggers.
- triggers_api.rs adds CreateDocsTriggerRequest + create_docs_trigger
  + the POST /api/v1/admin/apps/{id}/triggers/docs route, all
  guarded by Capability::AppManageTriggers (same as KV).

3 new triggers_api unit tests covering happy path, empty-glob
rejection, and capability denial. All existing trigger-related
tests still pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-02 19:55:27 +02:00
MechaCat02
77b2cb58bb feat(v1.1.1-routes): outbox-routed sync HTTP + dispatch_mode=async
Routes gain `dispatch_mode TEXT NOT NULL DEFAULT 'sync'` (migration
0012). Existing routes default to sync so the migration is
non-breaking. `DispatchMode` enum lands in `picloud-shared`.

The user-routes orchestrator handler now branches:
- `dispatch_mode = async` → write outbox row with `reply_to = None`,
  return `202 Accepted` + `{accepted_at, execution_id}`. Dispatcher
  fires the script in the background; retries / dead-letters via
  the framework from commit 5.
- `dispatch_mode = sync` → register an inbox channel
  (`tokio::sync::oneshot`), write outbox row with `reply_to =
  inbox_id`, `.await` on the receiver with a timeout =
  script.timeout_seconds + 2s buffer. Dispatcher hands the result
  back; orchestrator maps `InboxResult` into the HTTP response per
  the design-notes §3 status-code table (422/502/503/504/507/500).

`InboxRegistry` (orchestrator-core/src/inbox.rs) is the in-process
implementation of `InboxResolver`. Lock-free HashMap of pending
oneshot senders keyed by `inbox_id`. Tests cover register/deliver
round-trip, unknown-id is abandoned, dropped-receiver is abandoned,
explicit cancel. Cluster mode (v1.3+) swaps this for
LISTEN/NOTIFY-keyed lookup behind the same trait.

`OutboxWriter` trait lives in `picloud-shared` so orchestrator-core
can write to the outbox without depending on manager-core (which
would invert the dependency arrow). `PostgresOutboxRepo` implements
both `OutboxRepo` (dispatcher surface) and `OutboxWriter`
(orchestrator surface); the picloud binary clones the same concrete
Arc into both trait views.

The dispatcher's HTTP arm (commit 5 had a stub) now decodes the
`HttpDispatchPayload` off the outbox row, looks up the script,
synthesizes an `ExecRequest`, and runs it through the executor.
Outcome routing reuses the same path as KV triggers — sync HTTP
flows through the inbox, async dispatch gets dropped after
success (or DL'd on exhaustion).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-01 22:12:55 +02:00
MechaCat02
6a2971ac70 feat(v1.1.1-dispatcher): dispatcher loop + retry + depth limit + outbox emitter
`OutboxEventEmitter` replaces `NoopEventEmitter` in the picloud
binary's `Services` bundle. KV mutations now fan out to the outbox
via `TriggerRepo::list_matching_kv` — one row per matching trigger,
carrying the serialized `TriggerEvent` payload + the matching
trigger's retry policy.

`Dispatcher` is the single tokio task that polls the outbox every
100ms, claims due rows via FOR UPDATE SKIP LOCKED (with a batch cap),
and routes each to the executor. Shares the `ExecutionGate` with
sync HTTP per design notes §2 — gate saturation reschedules the
row instead of dropping it.

Outcome handling matches design notes §3 and §4:
- reply_to.is_some() (sync HTTP): never retry. Deliver via
  `InboxResolver`; if the receiver was dropped, write an
  `abandoned_executions` row.
- is_dead_letter_handler == true: never retry, never DL. On
  failure, annotate the original DL row with
  `resolution = 'handler_failed'`. Stops the recursion that would
  otherwise re-fire a broken handler script.
- Otherwise async: bump attempt_count, reschedule with exponential
  backoff + ±jitter; once max_attempts is reached, write a
  `dead_letters` row and drop from outbox.
- Trigger-depth limit: `cx.trigger_depth > max_trigger_depth` skips
  execution entirely (log + future metric), NEVER dead-letters.
  Loops are not retried via the DL chain — they're terminated.

`InboxResolver` trait lands in `picloud-shared` with a
`NoopInboxResolver` bootstrap that flags every delivery as
`Abandoned`. Commit 6 replaces the noop with the real
in-process registry in `orchestrator-core`.

`AdminPrincipalResolver` builds a `Principal` from a trigger's
`registered_by_principal` user id so the dispatched script executes
as the trigger registrant (design notes §4).

Unit tests cover backoff math (exponential/linear/constant) +
jitter range + ExecError → InboxFailureKind classification + the
status-code table mapping. Integration tests for the full
dispatcher loop need a real Postgres + executor; reviewer runs them
via the manual smoke flow in the plan / HANDBACK.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-01 22:01:42 +02:00