Compare commits
3 Commits
feat/v1.1.
...
feat/v1.1.
| Author | SHA1 | Date | |
|---|---|---|---|
|
03d03ea6e7 | ||
|
|
6080fc67f6 | ||
|
|
10b5f655d5 |
102
CHANGELOG.md
102
CHANGELOG.md
@@ -1,5 +1,92 @@
|
||||
# PiCloud Changelog
|
||||
|
||||
## v1.1.4 — Outbound HTTP & Cron triggers (unreleased)
|
||||
|
||||
Two surfaces. **`http::*`** lets Rhai scripts make outbound HTTP
|
||||
requests (Slack webhooks, Stripe, third-party REST) fronted by an SSRF
|
||||
deny-list applied to the *resolved IP* (DNS-rebinding defense), with
|
||||
scheme/port restrictions, request/response body caps, and a layered
|
||||
timeout. **Cron triggers** add the fourth concrete kind on the v1.1.1
|
||||
trigger framework: a scheduler task enqueues due triggers into the same
|
||||
universal outbox the dispatcher already drains.
|
||||
|
||||
### Added
|
||||
|
||||
- **`http::{get,post,put,patch,delete,head,post_form,request}`** — outbound
|
||||
HTTP SDK. Body and options are separate positional args
|
||||
(`verb(url, body, opts)`); `opts` is
|
||||
`{headers, timeout_ms, follow_redirects, max_redirects}` (unknown keys
|
||||
throw). Body dispatch by type: Map/Array → JSON, String → text/plain,
|
||||
`()` → none. Response is `#{ status, headers, body, body_raw }` with
|
||||
`body` auto-parsed when the response is `application/json`. Non-2xx
|
||||
does NOT throw (fetch-style); network/timeout/SSRF/size errors throw
|
||||
with an `"http: …"` prefix.
|
||||
- **SSRF deny-list** — applied to the resolved IP via a custom reqwest
|
||||
`dns_resolver` (so it covers every redirect hop and defeats DNS
|
||||
rebinding), plus a literal-IP check at URL-parse time. Blocks
|
||||
loopback, RFC1918 private, link-local (incl. `169.254.169.254`),
|
||||
carrier-grade NAT, multicast, reserved, IPv6 ULA/link-local/loopback,
|
||||
and IPv4-mapped IPv6 (re-checked against the embedded v4 address).
|
||||
The script-visible error carries a CIDR-category reason, never the IP.
|
||||
`PICLOUD_HTTP_ALLOW_PRIVATE=true` disables it (dev-only; logs a startup
|
||||
warning).
|
||||
- **`HttpService` trait** (`picloud-shared`) + `HttpServiceImpl`
|
||||
(manager-core, reqwest-backed). Wired into the `Services` bundle as
|
||||
`http: Arc<dyn HttpService>`.
|
||||
- **`Capability::AppHttpRequest(AppId)`** — maps to the existing
|
||||
`script:write` scope (any outbound request can exfiltrate data, so the
|
||||
conservative write mapping is used). No new `Scope` variant — the
|
||||
seven-scope commitment holds. Script-as-gate: skipped when the script
|
||||
runs unauthenticated.
|
||||
- **Cron triggers** — `POST /api/v1/admin/apps/{id}/triggers/cron`
|
||||
(`script_id`, `schedule`, `timezone`, optional retry overrides).
|
||||
6-field cron expressions (with seconds) validated by the `cron` crate;
|
||||
IANA timezones validated by `chrono-tz`. A scheduler task
|
||||
(`spawn_cron_scheduler`, poll cadence `PICLOUD_CRON_TICK_INTERVAL_MS`,
|
||||
default 30s) enqueues due triggers into the outbox; the existing
|
||||
dispatcher delivers them. Catch-up policy: a trigger that missed N
|
||||
windows fires exactly **once** on the next tick, not N times.
|
||||
- **`ctx.event.cron`** — `{ schedule, timezone, scheduled_at, fired_at }`
|
||||
for cron-trigger handlers (`ctx.event.source == "cron"`,
|
||||
`ctx.event.op == "tick"`).
|
||||
- **Dashboard Triggers tab** — admin-gated cron trigger create form
|
||||
(target endpoint script, schedule, timezone dropdown) + triggers list
|
||||
showing schedule / timezone / last-fired.
|
||||
|
||||
### Changed
|
||||
|
||||
- **Workspace version**: `1.1.3` → `1.1.4`.
|
||||
- **Rhai SDK version**: `1.4` → `1.5` (additive — `http::*` SDK +
|
||||
`ctx.event.cron`). The `Services` bundle constructor becomes
|
||||
`Services::new(kv, docs, dead_letters, events, modules, http)`.
|
||||
- **Dashboard version**: `0.9.0` → `0.10.0`.
|
||||
- **`SdkCallCx`** — gains a `script_id` field (audit attribution + the
|
||||
default outbound `User-Agent`, `picloud/<version> (script:<id>)`).
|
||||
- **Rhai pin tightened** — workspace dep `rhai = "1.19"` → `rhai = "=1.24"`
|
||||
so future bumps of the non-semver-stable `internals` surface are
|
||||
deliberate.
|
||||
- **Module backend errors redacted** — `PicloudModuleResolver` now
|
||||
surfaces a stable generic (`"module backend unavailable; check server
|
||||
logs"`) to scripts and logs the original at error level, instead of
|
||||
leaking the backend error verbatim (see v1.1.3 follow-up).
|
||||
|
||||
### Migrations
|
||||
|
||||
- `0017_cron_triggers.sql` — widens `triggers.kind` and
|
||||
`outbox.source_kind` CHECK constraints to include `'cron'`; adds
|
||||
`cron_trigger_details (trigger_id, schedule, timezone, last_fired_at)`
|
||||
with a `last_fired_at` index. Additive — applies cleanly on a fresh DB
|
||||
and on top of the v1.1.3 schema.
|
||||
|
||||
### New environment variables
|
||||
|
||||
- `PICLOUD_HTTP_ALLOW_PRIVATE` (default false; dev-only) — disable the
|
||||
SSRF deny-list.
|
||||
- `PICLOUD_HTTP_MAX_REQUEST_BODY_BYTES` / `PICLOUD_HTTP_MAX_RESPONSE_BODY_BYTES`
|
||||
(default 10 MB each).
|
||||
- `PICLOUD_CRON_TICK_INTERVAL_MS` (default 30000) — cron scheduler poll
|
||||
cadence (floored at 1s).
|
||||
|
||||
## v1.1.3 — Modules (unreleased)
|
||||
|
||||
Real per-app Rhai module system. Scripts can `import "<name>" as
|
||||
@@ -84,6 +171,21 @@ per-invocation compile cost; both invalidate on `updated_at` change.
|
||||
- **Route creation** — `POST /api/v1/admin/scripts/{id}/routes`
|
||||
returns 400 when the target script is `kind = 'module'`.
|
||||
|
||||
### Security fix
|
||||
|
||||
- **Cross-app trigger target (CVE-class: broken access control).** In
|
||||
v1.1.1 and v1.1.2, `POST /api/v1/admin/apps/{id}/triggers/{kv,docs,dead_letter}`
|
||||
validated only that the caller could manage triggers on `{id}` — it
|
||||
did **not** verify that the target `script_id` belonged to that same
|
||||
app. A member with trigger-management rights on app A could therefore
|
||||
register a trigger in A pointing at a script owned by app B, causing
|
||||
B's script to execute on A's events (a cross-app isolation break).
|
||||
v1.1.3 closes this: every trigger-create handler now loads the target
|
||||
script and rejects it unless `script.app_id == path app_id` (and it is
|
||||
not a module). **Upgrade recommendation:** anyone running a pre-v1.1.3
|
||||
multi-tenant deploy should upgrade and audit existing `triggers` rows
|
||||
for any whose `script_id` resolves to a script in a different `app_id`.
|
||||
|
||||
### Migrations
|
||||
|
||||
- `0015_scripts_kind.sql` — adds `scripts.kind` with CHECK
|
||||
|
||||
125
Cargo.lock
generated
125
Cargo.lock
generated
@@ -378,6 +378,28 @@ dependencies = [
|
||||
"windows-link",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "chrono-tz"
|
||||
version = "0.9.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "93698b29de5e97ad0ae26447b344c482a7284c737d9ddc5f9e52b74a336671bb"
|
||||
dependencies = [
|
||||
"chrono",
|
||||
"chrono-tz-build",
|
||||
"phf",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "chrono-tz-build"
|
||||
version = "0.3.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "0c088aee841df9c3041febbb73934cfc39708749bf96dc827e3359cd39ef11b1"
|
||||
dependencies = [
|
||||
"parse-zoneinfo",
|
||||
"phf",
|
||||
"phf_codegen",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "clap"
|
||||
version = "4.6.1"
|
||||
@@ -499,6 +521,17 @@ version = "2.5.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "217698eaf96b4a3f0bc4f3662aaa55bdf913cd54d7204591faa790070c6d0853"
|
||||
|
||||
[[package]]
|
||||
name = "cron"
|
||||
version = "0.12.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "6f8c3e73077b4b4a6ab1ea5047c37c57aee77657bc8ecd6f29b0af082d0b0c07"
|
||||
dependencies = [
|
||||
"chrono",
|
||||
"nom",
|
||||
"once_cell",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "crossbeam-queue"
|
||||
version = "0.3.12"
|
||||
@@ -1326,6 +1359,12 @@ version = "0.3.17"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "6877bb514081ee2a7ff5ef9de3281f14a4dd4bceac4c09388074a6b5df8a139a"
|
||||
|
||||
[[package]]
|
||||
name = "minimal-lexical"
|
||||
version = "0.2.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "68354c5c6bd36d73ff3feceb05efa59b6acb7626617f4962be322a825e61f79a"
|
||||
|
||||
[[package]]
|
||||
name = "mio"
|
||||
version = "1.2.0"
|
||||
@@ -1346,6 +1385,16 @@ dependencies = [
|
||||
"spin 0.5.2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "nom"
|
||||
version = "7.1.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "d273983c5a657a70a3e8f2a01329822f3b8c8172b73826411a55751e404a0a4a"
|
||||
dependencies = [
|
||||
"memchr",
|
||||
"minimal-lexical",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "normalize-line-endings"
|
||||
version = "0.3.0"
|
||||
@@ -1463,6 +1512,15 @@ dependencies = [
|
||||
"windows-link",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "parse-zoneinfo"
|
||||
version = "0.3.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "1f2a05b18d44e2957b88f96ba460715e295bc1d7510468a2f3d3b44535d26c24"
|
||||
dependencies = [
|
||||
"regex",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "password-hash"
|
||||
version = "0.5.0"
|
||||
@@ -1512,9 +1570,47 @@ version = "2.3.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220"
|
||||
|
||||
[[package]]
|
||||
name = "phf"
|
||||
version = "0.11.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "1fd6780a80ae0c52cc120a26a1a42c1ae51b247a253e4e06113d23d2c2edd078"
|
||||
dependencies = [
|
||||
"phf_shared",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "phf_codegen"
|
||||
version = "0.11.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "aef8048c789fa5e851558d709946d6d79a8ff88c0440c587967f8e94bfb1216a"
|
||||
dependencies = [
|
||||
"phf_generator",
|
||||
"phf_shared",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "phf_generator"
|
||||
version = "0.11.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "3c80231409c20246a13fddb31776fb942c38553c51e871f8cbd687a4cfb5843d"
|
||||
dependencies = [
|
||||
"phf_shared",
|
||||
"rand 0.8.6",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "phf_shared"
|
||||
version = "0.11.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "67eabc2ef2a60eb7faa00097bd1ffdb5bd28e62bf39990626a582201b7a754e5"
|
||||
dependencies = [
|
||||
"siphasher",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "picloud"
|
||||
version = "1.1.3"
|
||||
version = "1.1.4"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"async-trait",
|
||||
@@ -1540,7 +1636,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "picloud-cli"
|
||||
version = "1.1.3"
|
||||
version = "1.1.4"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"assert_cmd",
|
||||
@@ -1561,7 +1657,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "picloud-executor"
|
||||
version = "1.1.3"
|
||||
version = "1.1.4"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"picloud-executor-core",
|
||||
@@ -1573,7 +1669,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "picloud-executor-core"
|
||||
version = "1.1.3"
|
||||
version = "1.1.4"
|
||||
dependencies = [
|
||||
"async-trait",
|
||||
"base64",
|
||||
@@ -1590,12 +1686,14 @@ dependencies = [
|
||||
"thiserror 1.0.69",
|
||||
"tokio",
|
||||
"tracing",
|
||||
"tracing-subscriber",
|
||||
"url",
|
||||
"uuid",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "picloud-manager"
|
||||
version = "1.1.3"
|
||||
version = "1.1.4"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"picloud-manager-core",
|
||||
@@ -1607,18 +1705,21 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "picloud-manager-core"
|
||||
version = "1.1.3"
|
||||
version = "1.1.4"
|
||||
dependencies = [
|
||||
"argon2",
|
||||
"async-trait",
|
||||
"axum",
|
||||
"base64",
|
||||
"chrono",
|
||||
"chrono-tz",
|
||||
"cron",
|
||||
"data-encoding",
|
||||
"picloud-executor-core",
|
||||
"picloud-orchestrator-core",
|
||||
"picloud-shared",
|
||||
"rand 0.8.6",
|
||||
"reqwest",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"sha2",
|
||||
@@ -1632,7 +1733,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "picloud-orchestrator"
|
||||
version = "1.1.3"
|
||||
version = "1.1.4"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"picloud-orchestrator-core",
|
||||
@@ -1644,7 +1745,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "picloud-orchestrator-core"
|
||||
version = "1.1.3"
|
||||
version = "1.1.4"
|
||||
dependencies = [
|
||||
"async-trait",
|
||||
"axum",
|
||||
@@ -1665,7 +1766,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "picloud-shared"
|
||||
version = "1.1.3"
|
||||
version = "1.1.4"
|
||||
dependencies = [
|
||||
"async-trait",
|
||||
"chrono",
|
||||
@@ -2368,6 +2469,12 @@ dependencies = [
|
||||
"rand_core 0.6.4",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "siphasher"
|
||||
version = "1.0.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "8ee5873ec9cce0195efcb7a4e9507a04cd49aec9c83d0389df45b1ef7ba2e649"
|
||||
|
||||
[[package]]
|
||||
name = "slab"
|
||||
version = "0.4.12"
|
||||
|
||||
10
Cargo.toml
10
Cargo.toml
@@ -13,7 +13,7 @@ members = [
|
||||
]
|
||||
|
||||
[workspace.package]
|
||||
version = "1.1.3"
|
||||
version = "1.1.4"
|
||||
edition = "2021"
|
||||
rust-version = "1.92"
|
||||
license = "MIT OR Apache-2.0"
|
||||
@@ -47,12 +47,16 @@ tracing-subscriber = { version = "0.3", features = ["env-filter", "json"] }
|
||||
# IDs + time
|
||||
uuid = { version = "1", features = ["v4", "serde"] }
|
||||
chrono = { version = "0.4", features = ["serde"] }
|
||||
# Cron schedule parsing (v1.1.4 cron triggers) + IANA timezone resolution.
|
||||
chrono-tz = "0.9"
|
||||
cron = "0.12"
|
||||
|
||||
# Async traits
|
||||
async-trait = "0.1"
|
||||
|
||||
# Rhai scripting
|
||||
rhai = { version = "1.19", features = ["sync", "serde"] }
|
||||
# Rhai scripting. Pinned exactly (`=1.24`) because the `internals`
|
||||
# feature surface is not semver-stable — future bumps must be deliberate.
|
||||
rhai = { version = "=1.24", features = ["sync", "serde"] }
|
||||
|
||||
# Postgres (manager-core only — others stay DB-free)
|
||||
sqlx = { version = "0.8", features = ["runtime-tokio-rustls", "postgres", "uuid", "chrono", "json", "macros", "migrate"] }
|
||||
|
||||
575
HANDBACK.md
575
HANDBACK.md
@@ -1,351 +1,252 @@
|
||||
# v1.1.3 — Modules — Handback
|
||||
# v1.1.4 Handback — Outbound HTTP SDK & Cron Triggers
|
||||
|
||||
## 1. Branch summary
|
||||
**Branch:** `feat/v1.1.4-http-cron` (off `main`)
|
||||
**Commits:** 1 implementation commit (`feat(v1.1.4): outbound HTTP SDK + cron triggers`) + this HANDBACK commit.
|
||||
|
||||
- **Branch:** `feat/v1.1.3-modules`
|
||||
- **Commits ahead of `main`:** 6
|
||||
- **HEAD:** `3dbead4`
|
||||
- **Not pushed, not merged, no PR opened** (per brief).
|
||||
> **Note on commit granularity:** the brief suggested split
|
||||
> `feat(v1.1.4-http)` / `feat(v1.1.4-cron)` commits. The two features are
|
||||
> interleaved across shared files (`Cargo.toml`, `crates/picloud/src/lib.rs`,
|
||||
> `crates/manager-core/src/lib.rs`, `version.rs`, `services.rs`), so
|
||||
> cleanly-*compiling* per-theme commits aren't separable without interactive
|
||||
> hunk staging (unavailable in this environment). I chose one coherent,
|
||||
> green commit over shipping broken intermediates. Squash/relabel as you see fit.
|
||||
|
||||
Commits (newest first):
|
||||
---
|
||||
|
||||
## Scope coverage
|
||||
|
||||
| # | Item | Status |
|
||||
|---|------|--------|
|
||||
| 1 | `http::*` SDK surface (get/post/put/patch/delete/head/post_form/request) | **Done** |
|
||||
| 2 | SSRF deny-list (resolved-IP, DNS-rebinding defense, scheme/port, body caps, UA, timeouts, `PICLOUD_HTTP_ALLOW_PRIVATE`) | **Done** |
|
||||
| 3 | http authz (`Capability::AppHttpRequest` → `script:write`, script-as-gate, no new Scope) | **Done** |
|
||||
| 4 | `HttpService` trait + `HttpServiceImpl` + Services wiring | **Done** |
|
||||
| 5 | Cron migration `0017` (Layout-E extension) | **Done** |
|
||||
| 6 | Cron scheduler tokio task (catch-up = fire-once) | **Done** |
|
||||
| 7 | `ctx.event.cron` shape + `TriggerEvent::Cron` | **Done** |
|
||||
| 8 | Dispatcher routing extension (`… \| Cron`) | **Done** |
|
||||
| 9 | Dashboard cron trigger UI (minimal) | **Done** |
|
||||
| 10a | Redact `ModuleSourceError::Backend` at resolver boundary | **Done** |
|
||||
| 10b | Pin `rhai = "=1.24"` | **Done** |
|
||||
| 10c | CHANGELOG retroactive v1.1.3 cross-app-trigger security note | **Done** |
|
||||
| 11 | Version bumps (workspace 1.1.4, SDK 1.5, dashboard 0.10.0) | **Done** |
|
||||
| 12 | Tests (~50-70) | **Done** — 70 new |
|
||||
|
||||
---
|
||||
|
||||
## SSRF policy implementation notes
|
||||
|
||||
- **reqwest hook.** `crates/manager-core/src/ssrf.rs` defines `SsrfResolver`
|
||||
implementing `reqwest::dns::Resolve`, plugged in via
|
||||
`ClientBuilder::dns_resolver`. It delegates to the system resolver
|
||||
(injectable for tests — see DNS-rebinding test), then filters each `IpAddr`
|
||||
through `SsrfPolicy::check`. Because reqwest re-resolves at every connection
|
||||
(including each redirect hop), the policy applies post-redirect too.
|
||||
- **`dns_resolver` is generic over a concrete `R: Resolve`** (stores `Arc<R>`),
|
||||
so the resolver is passed as `Arc<SsrfResolver>`, not `Arc<dyn Resolve>`.
|
||||
- **Literal-IP gap closed.** reqwest only routes *hostnames* through the custom
|
||||
resolver — a URL with a literal-IP host (`http://127.0.0.1/`) bypasses it
|
||||
entirely. The impl therefore *also* runs `SsrfPolicy::check` on literal-IP
|
||||
hosts at URL-parse time (`validate_url`), on every hop. Both paths are tested.
|
||||
- **IPv4-mapped IPv6 re-check.** `check_v6` calls `Ipv6Addr::to_ipv4_mapped()`;
|
||||
if `Some`, it re-runs the v4 deny-list against the embedded address
|
||||
(`::ffff:127.0.0.1` → denied as "loopback").
|
||||
- **Applied before AND after redirects.** Redirects are followed *manually*
|
||||
(client built with `redirect(Policy::none())`) so per-request
|
||||
`follow_redirects`/`max_redirects` are honored; each hop re-validates
|
||||
scheme/port + literal-IP and re-resolves hostnames through the SSRF resolver.
|
||||
- **Script-visible error format.** `"http: blocked by SSRF policy: <reason>"`
|
||||
where `<reason>` is a CIDR category (`loopback`, `private`, `link-local`,
|
||||
`carrier-grade-nat`, `multicast`, `reserved`, `unique-local`, `unspecified`).
|
||||
**The resolved IP is never included.** The all-addresses-denied case surfaces
|
||||
as `Ssrf` (not a generic DNS error) via a marker error the resolver emits and
|
||||
the impl detects by walking the reqwest error source chain.
|
||||
|
||||
## Cron scheduler implementation notes
|
||||
|
||||
- **Catch-up = fire-once.** Matches the brief; no deviation. `next_due` returns a
|
||||
single canonical scheduled-at (first slot after `last_fired_at`, or
|
||||
`created_at` if never fired); after firing, `last_fired_at = now`, so the next
|
||||
tick sees only future slots. **Verified live** against Postgres: an
|
||||
every-second (`* * * * * *`) trigger with a 2s tick advanced `last_fired_at`
|
||||
~once per 2s, not once per second.
|
||||
- **No ExecutionGate contention.** The scheduler only enqueues to the outbox
|
||||
(one row per due trigger per tick, in a `FOR UPDATE OF d SKIP LOCKED`
|
||||
transaction that also bumps `last_fired_at`). The existing dispatcher acquires
|
||||
the gate and delivers it identically to kv/docs/dead_letter — verified live
|
||||
(the cron outbox row was consumed, the script executed, the row deleted).
|
||||
- **Timezone handling.** `chrono-tz`. Invalid IANA names are rejected at the
|
||||
admin endpoint with a 422 (`TriggersApiError::Invalid`, message contains
|
||||
"timezone"); the repo re-validates defensively before insert.
|
||||
- **Schema beyond the brief:** none. Followed the brief exactly — `schedule`,
|
||||
`timezone DEFAULT 'UTC'`, `last_fired_at`, `idx_cron_triggers_due`. **No**
|
||||
stored `next_scheduled_at` column (an exploration agent suggested one; the
|
||||
brief computes next-fire in-process, which I followed).
|
||||
|
||||
---
|
||||
|
||||
## Tests added (70 new)
|
||||
|
||||
- **SSRF policy + resolver (`ssrf.rs`, 20):** one per deny CIDR (127/8, 0/8,
|
||||
10/8, 172.16/12, 192.168/16, 169.254/16 incl. metadata, 100.64/10, 224/4,
|
||||
240/4, ::1, ::, fe80::/10, fc00::/7, ff00::/8); 172.x outside-range allowed;
|
||||
public v4/v6 allowed; IPv4-mapped re-check; `allow_private` disables all;
|
||||
resolver returns only allowed addrs; all-denied → SSRF marker; **DNS rebinding**
|
||||
(mock resolver: public then private — second denied); empty resolution ≠ SSRF.
|
||||
- **HTTP client (`http_service.rs`, 16):** GET/POST round-trips vs a hand-rolled
|
||||
`TcpListener`; body dispatch + default UA; custom UA override; empty body;
|
||||
non-2xx no-error; response cap via Content-Length; response cap mid-stream
|
||||
(no Content-Length); request body cap pre-send; redirect-to-max-then-throw;
|
||||
scheme rejection (file/ftp/gopher); port rejection (22/25/465/587); SSRF
|
||||
literal-loopback; SSRF hostname-resolves-to-loopback; timeout; authz
|
||||
(anon skips / member forbidden / member-with-role allowed).
|
||||
- **Bridge integration (`sdk_http.rs`, 15):** real Rhai engine under
|
||||
`spawn_blocking` vs a recording fake — status+JSON body, non-JSON string,
|
||||
empty→`()`, Map→JSON, String→text, `()`→no body, headers+timeout forwarded,
|
||||
unknown opt key throws, timeout>max throws, non-2xx no-throw, network error
|
||||
throws `http:`, `post_form` url-encoding, `request` arbitrary method,
|
||||
default-UA carries `script_id`, `cx.app_id` forwarded for attribution.
|
||||
- **Cron scheduler (`cron_scheduler.rs`, 11):** 6-field schedule accept / 5-field
|
||||
+ malformed reject; IANA tz accept / reject; due/not-due; never-fired uses
|
||||
created_at; **catch-up fires exactly once after 5 missed windows**; timezone
|
||||
affects fire time; bad schedule/tz → None.
|
||||
- **Cron admin (`triggers_api.rs`, 6):** create succeeds; invalid schedule;
|
||||
unknown timezone; **module target rejected** (v1.1.3 regression); **cross-app
|
||||
target rejected** (v1.1.3 regression); member-without-role forbidden.
|
||||
- **Module redaction (2):** `modules.rs` — backend error redacted from the
|
||||
script-visible error (no leak); `module_redaction_logging.rs` — original error
|
||||
**is** logged at ERROR level (captured via a global tracing subscriber).
|
||||
|
||||
---
|
||||
|
||||
## Decisions beyond the brief (every prompt-default deviation, flagged)
|
||||
|
||||
1. **Three-arg split `verb(url, body, opts)`** (user-approved during planning).
|
||||
Diverges from the brief's documented two-arg `(url, opts)` shape and
|
||||
generalizes the escape hatch to `request(method, url, body, opts)`. Resolves
|
||||
the brief's internal contradiction (its Slack example `http::post(url, #{text:...})`
|
||||
passed a bare body map, which would be an "unknown opt key" under the
|
||||
two-arg rule). The `opts` vocabulary is now exactly
|
||||
`{headers, timeout_ms, follow_redirects, max_redirects}` — **`body_raw` was
|
||||
dropped** (raw strings go through the positional body as a String). The
|
||||
Slack example works unchanged (`#{text:...}` is the body).
|
||||
2. **Cron crate = `cron` (0.12), not `croner`.** The brief allowed either; `cron`
|
||||
handles the 6-field-with-seconds format and named weekdays (`MON-FRI`) used in
|
||||
the brief's example, and integrates with chrono `Schedule::after`.
|
||||
3. **Catch-up = fire-once** — matches the brief; called out explicitly as
|
||||
requested. No deviation.
|
||||
4. **`SdkCallCx` gained a `script_id` field.** The brief's default User-Agent is
|
||||
`picloud/<v> (script:<script_id>)`, but `SdkCallCx` didn't carry the script
|
||||
id. Adding it (sourced from `ExecRequest.script_id` in the engine) is the
|
||||
clean home and doubles as the audit-attribution key the brief emphasizes. All
|
||||
19 construction sites updated. The dead-letters admin cx uses a fresh sentinel
|
||||
id (no script executes there).
|
||||
5. **SSRF also blocks IPv6 unspecified `::` and IPv4 `0.0.0.0`** with reason
|
||||
"unspecified". `0.0.0.0/8` is in the brief's list; `::` is not explicitly but
|
||||
is an obvious sibling hole, so I blocked it too (defensible superset).
|
||||
6. **No reqwest feature additions needed** — `dns_resolver` and `Response::chunk()`
|
||||
compile under the existing `default-features = false, features = ["json","rustls-tls"]`.
|
||||
No cookie jar (cookies feature is off, so there's no jar to disable). Added
|
||||
`url` as an executor-core dep (for `form_urlencoded` in `post_form`).
|
||||
|
||||
---
|
||||
|
||||
## How to verify locally (§8 attestation — run on this exact HEAD)
|
||||
|
||||
All four gates were run on the handback HEAD (the `feat(v1.1.4)` commit, before
|
||||
this markdown commit):
|
||||
|
||||
```
|
||||
3dbead4 test(v1.1.3-modules): resolver, cache, validator, kind-rejection coverage
|
||||
10f76d2 chore(v1.1.3-modules): version bumps + CHANGELOG + blueprint touch-up
|
||||
610fd4f feat(v1.1.3-modules): dashboard kind dropdown + scripts-list and detail badges
|
||||
66b41bb feat(v1.1.3-modules): top-level script AST cache in LocalExecutorClient
|
||||
c6211a7 feat(v1.1.3-modules): reject module scripts from routes + triggers; tighten cross-app trigger check
|
||||
84833d3 feat(v1.1.3-modules): shared types, migrations, engine + resolver scaffold
|
||||
cargo fmt --all -- --check → exit 0
|
||||
cargo clippy --all-targets --all-features -- -D warnings → exit 0
|
||||
cargo test --workspace → 427 passed, 0 failed
|
||||
(cd dashboard && npm run check) → 0 errors, 0 warnings (369 files)
|
||||
```
|
||||
|
||||
This HANDBACK commit is pure markdown (no gate-relevant files), so the numbers
|
||||
above hold for the final HEAD.
|
||||
|
||||
**Migrations — verified against a real Postgres (dev stack, port 15432):**
|
||||
- Fresh-DB replay: the `#[sqlx::test]` schema-snapshot test applies all
|
||||
migrations on a fresh ephemeral DB and matches the (re-blessed) golden — passes.
|
||||
- On-top-of-prior-state: booting `picloud` against a dev DB pinned at migration
|
||||
`0006` applied `0007…0017` cleanly (`"migrations applied"`); `_sqlx_migrations`
|
||||
max is now `17`; `cron_trigger_details` + widened CHECKs present.
|
||||
|
||||
**Live smoke performed:**
|
||||
- Boot logged the `PICLOUD_HTTP_ALLOW_PRIVATE` warning and started the cron
|
||||
scheduler + HTTP service without panic.
|
||||
- Seeded an every-second cron trigger → scheduler set `last_fired_at`, dispatcher
|
||||
consumed the outbox row and ran the script (row deleted on success), and
|
||||
`last_fired_at` advanced at the tick cadence (fire-once confirmed). Smoke data
|
||||
cleaned up afterward.
|
||||
- HTTP GET / SSRF-block / body-dispatch behaviors are covered by the automated
|
||||
integration tests (real `TcpListener` round-trips + loopback/hostname SSRF
|
||||
blocks) rather than a manual curl flow, since a live SSRF-block smoke
|
||||
conflicts with the `PICLOUD_HTTP_ALLOW_PRIVATE` a local-server smoke requires.
|
||||
|
||||
To re-run the schema snapshot:
|
||||
```
|
||||
docker compose up -d postgres
|
||||
DATABASE_URL=postgres://picloud:picloud@127.0.0.1:15432/picloud \
|
||||
cargo test -p picloud-manager-core --test schema_snapshot -- --include-ignored
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. Scope coverage
|
||||
## ⚠️ Latent issue found: stale schema-snapshot golden
|
||||
|
||||
| # | Brief item | Status | Notes |
|
||||
|---|---|---|---|
|
||||
| 1 | `scripts.kind` column + check + index | **Done** | `migrations/0015_scripts_kind.sql` |
|
||||
| 2 | Module syntax constraints (fn / const / import only) | **Done** | Walks `ast.statements()` via `rhai/internals`. Admin endpoint is primary gate; resolver re-runs the check for defense-in-depth. |
|
||||
| 3 | `ModuleResolver` replaces `DummyModuleResolver` | **Done** | `crates/executor-core/src/module_resolver.rs`; per-call instance with cross-app isolation, cycle detect, depth limit. |
|
||||
| 4 | Two AST caches (script + module) | **Done** | Script cache in `LocalExecutorClient`; module cache in `Engine`. Both invalidate by `updated_at` comparison. Env-overridable sizes. |
|
||||
| 5 | Dep-graph table + populate | **Done** | `migrations/0016_script_imports.sql`; `replace_imports_tx` writes edges in the same transaction as the script INSERT/UPDATE. |
|
||||
| 6 | Admin endpoint changes (kind, kind-change rejection, route/trigger module rejection) | **Done** | Also closes a latent cross-app trigger gap (script.app_id mismatch — see §7). |
|
||||
| 7 | Dashboard surface (kind dropdown + badge) | **Done** | App page form + scripts list + script detail header. `npm run check` clean. |
|
||||
| 8 | `ModuleSource` trait shape | **Done** | Lives in `picloud-shared`; matches the v1.1.1/v1.1.2 service pattern. |
|
||||
| 9 | Version bumps | **Done** | Workspace 1.1.2→1.1.3, SDK 1.3→1.4, dashboard 0.8.0→0.9.0. |
|
||||
| 10 | Tests (~40–60) | **Done** | 46 new tests across 5 crates. Gates green. |
|
||||
`crates/manager-core/tests/expected_schema.txt` was **significantly stale** — the
|
||||
committed golden was missing many tables from prior releases
|
||||
(`abandoned_executions`, `dead_letters`, `dead_letter_trigger_details`,
|
||||
`docs_*`, etc.). The `schema_snapshot` test is `#[ignore]` (needs a DB), so it
|
||||
was apparently never re-blessed across v1.1.1–v1.1.3 and silently drifted.
|
||||
|
||||
### Scope-out items (confirmed NOT built)
|
||||
|
||||
- No module versioning / pinning, no `@v3` syntax.
|
||||
- No eager precompilation at save-time.
|
||||
- No dashboard dep-graph visualization.
|
||||
- No LISTEN/NOTIFY-based cross-node invalidation.
|
||||
- No new `Scope` variants (modules use existing `script:read` / `script:write`).
|
||||
- No admin GET endpoints for `script_imports` (the table is persisted for v1.2+; no v1.1.3 read surface — see §10 deferred items).
|
||||
I re-blessed it, so the diff is large (+217 lines) but **only `cron_trigger_details`
|
||||
+ the two widened CHECK constraints are v1.1.4-new** — the rest is pre-existing
|
||||
drift correction. The blessed golden now matches a clean replay (verified).
|
||||
Recommend the reviewer skim the diff to confirm, and consider whether the
|
||||
`#[ignore]` should be lifted in CI (with a DB service) so the golden can't drift
|
||||
again.
|
||||
|
||||
---
|
||||
|
||||
## 3. Resolver implementation notes
|
||||
|
||||
### 3.1 In-progress-imports stack
|
||||
|
||||
Lives **on the per-call `PicloudModuleResolver` instance**, not on `SdkCallCx`. The resolver is constructed fresh per `Engine::execute_ast` call (see `crates/executor-core/src/engine.rs:execute_ast`), so the stack is naturally scoped to one execution. Both the stack and the depth counter are `Mutex<…>` (not `RefCell<…>`) because `rhai::ModuleResolver: SendSync` under the `sync` feature.
|
||||
|
||||
An RAII `StackGuard` pops the stack and decrements depth on drop — a compile error or panic anywhere inside `resolve()` cleans up properly. The lock is uncontended in practice (Rhai evaluation on the engine is single-threaded).
|
||||
|
||||
### 3.2 Sync → async bridge
|
||||
|
||||
Rhai's `ModuleResolver::resolve` is sync; `ModuleSource::lookup` is async. The bridge:
|
||||
|
||||
```rust
|
||||
let handle = tokio::runtime::Handle::try_current().map_err(/* surfaces as ErrorRuntime */)?;
|
||||
let lookup = tokio::task::block_in_place(|| handle.block_on(self.source.lookup(&self.cx, path)));
|
||||
```
|
||||
|
||||
- `try_current()` (not `current()`) so test harnesses that build an `Engine` outside a Tokio runtime get a clean error instead of a panic.
|
||||
- `block_in_place` makes the call safe both on `spawn_blocking` threads (where it's a no-op) and on multi-threaded runtime worker tasks (where it instructs the runtime to relocate other tasks before we block). This was load-bearing for the resolver tests, which call `engine.execute` directly from `#[tokio::test(flavor = "multi_thread")]`.
|
||||
- A `current_thread` runtime still panics — but production callers wrap `Engine::execute` in `tokio::task::spawn_blocking` (see `LocalExecutorClient::execute_with_identity`), which avoids that path entirely.
|
||||
|
||||
### 3.3 Cross-app isolation enforcement
|
||||
|
||||
The resolver captures `Arc<SdkCallCx>` at construction. Every `ModuleSource::lookup` call passes `&self.cx`. The Postgres impl (`crates/manager-core/src/module_source.rs`) selects with `WHERE app_id = $1 AND kind = 'module' AND name = $2`, binding `$1` from `cx.app_id.into_inner()` — never from any script-passed argument. The Rhai script's `import "name" as alias;` syntax has no slot for an `app_id`, so there is no path by which a script in app A can name a row in app B.
|
||||
|
||||
Verified by `resolver_cross_app_blocked` and `resolver_cross_app_module_not_found` tests.
|
||||
|
||||
### 3.4 Module-shape validation — both layers
|
||||
|
||||
- **Primary gate (admin endpoint)** — `manager-core::api::create_script` and `update_script` call `state.validator.validate_module(src)` whenever the effective kind is `Module`. `Engine`'s impl walks `ast.statements()`, accepting only `Stmt::Var(_, ASTFlags::CONSTANT, _)`, `Stmt::Import(..)`, and `Stmt::Noop(..)`. Anything else (top-level expression, let, if, while, …) is rejected with a clear `ValidationError::ModuleShape` message.
|
||||
- **Defense in depth (resolver)** — the resolver calls `check_module_shape` again after `engine.compile(source)`. This catches rows that bypassed the API (manual SQL inserts, future migration bugs, restoring from an older backup).
|
||||
|
||||
Note: Rhai's default optimizer constant-folds `if true { ... }` away, so a module containing `if true { ... }` parses to an empty body and passes vacuously. This is fine semantically (the script has no observable behavior), but it surprises authors. Documented as a known acceptance edge; not worth disabling optimization for.
|
||||
|
||||
### 3.5 What the resolver does NOT enforce
|
||||
|
||||
- **Module access permissions** — every module in an app is importable by every other script in the same app. Per-module ACLs are explicitly v1.2+.
|
||||
- **Module versioning / pinning** — there's exactly one current version per `(app_id, name)`. v1.3+.
|
||||
|
||||
---
|
||||
|
||||
## 4. Cache design notes
|
||||
|
||||
### 4.1 LRU library
|
||||
|
||||
**`lru = "0.12"`** — added to `[workspace.dependencies]`. Standard choice, no-frills crate (`LruCache<K, V>` with `put`/`get`/`len`/etc.). Both caches use `Arc<Mutex<LruCache<K, V>>>` so they're cheap to clone and safe to share across executions.
|
||||
|
||||
### 4.2 Cache key shapes + what's stored
|
||||
|
||||
| Cache | Owner | Key | Value | Stores |
|
||||
|---|---|---|---|---|
|
||||
| **Script AST cache** | `LocalExecutorClient` | `ScriptId` | `CachedScript { updated_at: DateTime<Utc>, ast: Arc<rhai::AST> }` | Compiled AST for the top-level (endpoint) script. |
|
||||
| **Module cache** | `Engine` | `(AppId, String)` | `CachedModule { updated_at: DateTime<Utc>, module: Shared<rhai::Module> }` | Compiled `rhai::Module` produced by `Module::eval_ast_as_new`. |
|
||||
|
||||
The script cache stores `Arc<AST>` so an evaluation can grab a cheap clone and hand it to `Engine::execute_ast` without holding the cache lock. The module cache stores `Shared<Module>` (= `Arc<Module>` under the `sync` feature) because that's what `ModuleResolver::resolve` must return.
|
||||
|
||||
### 4.3 Stale-version detection
|
||||
|
||||
Both caches use the same logic: **compare `cached.updated_at` against the freshly-known `updated_at`**.
|
||||
|
||||
- For the script cache, the caller passes the fresh value as `ScriptIdentity.updated_at` — the orchestrator already loaded the script row to dispatch the request, so there's no extra DB hit.
|
||||
- For the module cache, the resolver must call `ModuleSource::lookup` first to learn the fresh `updated_at` — every `import` does at least one DB roundtrip. That's a deliberate trade-off (documented in CHANGELOG): the alternative (TTL caching or pub/sub) introduces staleness during edits and complicates "publish a fix immediately" UX. Worth re-evaluating in v1.3+ when LISTEN/NOTIFY makes pub/sub cheap.
|
||||
|
||||
Mismatch → recompile + `cache.put(...)` replace. LRU eviction is automatic when capacity is exceeded.
|
||||
|
||||
### 4.4 Capacity overrides
|
||||
|
||||
- `PICLOUD_SCRIPT_CACHE_SIZE` (default 256, `LocalExecutorClient`)
|
||||
- `PICLOUD_MODULE_CACHE_SIZE` (default 512, `Engine`)
|
||||
|
||||
Both clamp `max(1)` to avoid the LRU constructor's panic on zero. `Engine::with_module_cache_capacity` and `LocalExecutorClient::with_script_cache_capacity` give tests explicit handles.
|
||||
|
||||
---
|
||||
|
||||
## 5. Dep-graph population
|
||||
|
||||
### 5.1 Where the extraction happens
|
||||
|
||||
Inside the `ScriptValidator` impl on `Engine`. The trait now returns `ValidatedScript { imports: Vec<String> }`, populated by `extract_imports` (endpoint scripts) or `validate_module_source` (module scripts). Both walk `ast.statements()` and pull out `Stmt::Import(boxed_path_expr, _)` where the path is a `StringConstant`.
|
||||
|
||||
**Dynamic imports** (`import some_var as alias;`) are NOT captured because we can't know the name at compile time. Tested by `validate_endpoint_skips_dynamic_imports_in_imports_list`. Documented as a known limitation in the CHANGELOG and migration 0016's header comment.
|
||||
|
||||
### 5.2 Where the write happens — transactional with the script INSERT/UPDATE
|
||||
|
||||
`PostgresScriptRepository::create` and `update` both open a `tx = pool.begin().await?`. The script row is inserted/updated inside the tx; immediately after, `replace_imports_tx(&mut tx, importer, app_id, &imports)` runs. The tx is committed at the end. If any step fails, both the script change and the dep-graph mutation roll back together. No half-state where the script row exists but the edges don't (or vice versa).
|
||||
|
||||
`replace_imports_tx`:
|
||||
|
||||
1. `DELETE FROM script_imports WHERE importer_script_id = $1` — replaces wholesale.
|
||||
2. `INSERT INTO script_imports ... SELECT ... FROM scripts WHERE app_id = $1 AND kind = 'module' AND name = ANY($3) ON CONFLICT DO NOTHING` — best-effort: only resolves to existing modules in the same app; unresolved names are silently skipped (no error). A later save of either script re-resolves and writes the edge.
|
||||
|
||||
### 5.3 Schema decisions
|
||||
|
||||
- `script_imports.app_id` is denormalized but useful: the "all imports in app X" scan happens once at boot for caching and (eventually) for the dashboard's audit view. Without it, that query would need a 3-way join.
|
||||
- `created_at` is unused by v1.1.3 logic but trivial to add now and useful for v1.2+ "first imported" diagnostics.
|
||||
- The FK on `imported_script_id` cascades — when a module is deleted, every edge referencing it goes too. The cascade isn't exercised by a unit test (it would need Postgres); it's covered by the FK design.
|
||||
|
||||
---
|
||||
|
||||
## 6. Tests added
|
||||
|
||||
46 new tests across 5 crates. All green on HEAD `3dbead4`. Inventory:
|
||||
|
||||
### `crates/executor-core/tests/modules.rs` (NEW — 23 tests)
|
||||
|
||||
End-to-end through `Engine::execute` with a `CountingModuleSource` (in-memory fake).
|
||||
|
||||
| # | Test | Covers |
|
||||
|---|---|---|
|
||||
| 1 | `resolver_loads_simple_module` | Happy path: `import "m" as m; m::add(2, 3)` → 5. |
|
||||
| 2 | `resolver_cross_app_blocked` | Modules with same name in two apps resolve to the calling app's version. |
|
||||
| 3 | `resolver_cross_app_module_not_found` | App B's `import "lonely"` returns ModuleNotFound when only app A has it. |
|
||||
| 4 | `resolver_module_not_found` | Missing module → `ErrorModuleNotFound`. |
|
||||
| 5 | `resolver_self_import_detected` | `a` imports `a` → circular error. |
|
||||
| 6 | `resolver_circular_detected` | `a → b → a` → circular error. |
|
||||
| 7 | `resolver_depth_limit_enforced` | 9-deep chain with limit 8 → depth error. |
|
||||
| 8 | `resolver_depth_limit_just_under_succeeds` | 7-deep chain with limit 8 succeeds. |
|
||||
| 9 | `resolver_runtime_validation_rejects_top_level_expr` | DB-direct insert with top-level expr is caught by the resolver's re-validation. |
|
||||
| 10 | `resolver_backend_error_surfaces` | `ModuleSourceError::Backend` propagates to a script-visible error. |
|
||||
| 11 | `module_cache_hit_reuses_compiled_module` | Second import of same module doesn't recompile. |
|
||||
| 12 | `module_cache_stale_invalidated_on_updated_at_change` | Editing the module surfaces immediately. |
|
||||
| 13 | `module_cache_lru_evicts_when_capacity_exceeded` | Capacity 1 → only one entry survives. |
|
||||
| 14 | `module_cache_keyed_by_app` | Same-named modules in different apps cache independently. |
|
||||
| 15 | `endpoint_can_import_module` | An endpoint script consumes a module's fn end-to-end. |
|
||||
| 16 | `module_can_import_module` | Modules can be importers. |
|
||||
| 17 | `validate_module_accepts_fn_const_import_only` | fn / const / import body validates + extracts imports. |
|
||||
| 18 | `validate_module_rejects_top_level_let` | `let x = 1;` rejected. |
|
||||
| 19 | `validate_module_rejects_top_level_expr` | `42;` rejected. |
|
||||
| 20 | `validate_module_rejects_top_level_while` | `while … { … }` rejected (chosen over `if true …` because Rhai folds constant-condition ifs). |
|
||||
| 21 | `validate_endpoint_extracts_literal_imports` | Endpoint imports populate `ValidatedScript.imports`. |
|
||||
| 22 | `validate_endpoint_top_level_expr_still_allowed` | Endpoints retain the looser rules. |
|
||||
| 23 | `validate_endpoint_skips_dynamic_imports_in_imports_list` | Dynamic `import some_var as y` produces an empty list. |
|
||||
|
||||
### `crates/orchestrator-core/src/client.rs` (6 inline tests)
|
||||
|
||||
| # | Test | Covers |
|
||||
|---|---|---|
|
||||
| 1 | `cache_hit_when_identity_matches` | Identical `(script_id, updated_at)` returns the same `Arc<AST>`. |
|
||||
| 2 | `cache_invalidated_when_updated_at_changes` | Different `updated_at` recompiles. |
|
||||
| 3 | `distinct_script_ids_cache_independently` | Two scripts → two entries. |
|
||||
| 4 | `lru_eviction_caps_cache_size` | Capacity 1; A → B → C leaves one entry. |
|
||||
| 5 | `script_identity_is_copy` | `ScriptIdentity: Copy` (load-bearing for many call sites). |
|
||||
| 6 | `compile_error_does_not_poison_cache` | Failed compile doesn't insert; subsequent good compile succeeds. |
|
||||
|
||||
### `crates/shared/src/script.rs` (3 inline tests)
|
||||
|
||||
| # | Test | Covers |
|
||||
|---|---|---|
|
||||
| 1 | `default_is_endpoint` | `ScriptKind::default() == Endpoint`. |
|
||||
| 2 | `round_trips_through_serde_lowercase` | `"endpoint"` / `"module"` wire form. |
|
||||
| 3 | `parse_str_round_trip` | `as_str` ↔ `parse_str` inverses. |
|
||||
|
||||
### `crates/manager-core/src/triggers_api.rs` (6 new inline tests)
|
||||
|
||||
| # | Test | Covers |
|
||||
|---|---|---|
|
||||
| 1 | `kv_trigger_rejects_module_target` | Module script as KV-trigger target → 422 with `"module"` in the message. |
|
||||
| 2 | `docs_trigger_rejects_module_target` | Same for docs triggers. |
|
||||
| 3 | `dl_trigger_rejects_module_target` | Same for dead-letter triggers. |
|
||||
| 4 | `kv_trigger_rejects_missing_script` | Non-existent script id → 422. |
|
||||
| 5 | `kv_trigger_rejects_cross_app_script` | Latent v1.1.1/v1.1.2 isolation gap — script in app B targeted from app A → 422. |
|
||||
| 6 | `kv_trigger_accepts_endpoint_target` | Happy path. |
|
||||
|
||||
### `crates/picloud/tests/api.rs` (8 `#[ignore]`'d Postgres-gated tests)
|
||||
|
||||
End-to-end through the HTTP surface. Run with `--include-ignored` against a real Postgres.
|
||||
|
||||
| # | Test | Covers |
|
||||
|---|---|---|
|
||||
| 1 | `create_script_default_kind_is_endpoint` | Default kind on create. |
|
||||
| 2 | `create_module_kind_persists` | `kind=module` round-trips through the API. |
|
||||
| 3 | `create_module_with_top_level_expr_rejected` | Module syntax gate at create time. |
|
||||
| 4 | `create_module_with_reserved_name_rejected` | `kv`, `docs`, etc. reserved. |
|
||||
| 5 | `route_bind_rejects_module` | `POST .../routes` returns 422 for module targets. |
|
||||
| 6 | `endpoint_imports_module_end_to_end` | Endpoint imports module, route binding, HTTP invocation, result. |
|
||||
| 7 | `module_edit_visible_on_next_invocation` | Cache invalidation on module edit (verified end-to-end through the engine). |
|
||||
| 8 | `cross_app_import_blocked` | Two apps, same-name module, endpoint sees its own. |
|
||||
|
||||
---
|
||||
|
||||
## 7. Schema / decisions beyond the brief
|
||||
|
||||
- **Module name shape CHECK** (`migrations/0015_scripts_kind.sql`): module names are constrained to `^[a-zA-Z_][a-zA-Z0-9_]{0,63}$`. Endpoint scripts retain the looser pre-v1.1.3 name rules so existing rows aren't invalidated. Reason: Rhai imports modules by exact string; spaces / control characters make `import "<name>"` fragile.
|
||||
- **Reserved module names**: rejected at create-time (`kv`, `docs`, `dead_letters`, `log`, `regex`, `random`, `time`, `json`, `base64`, `hex`, `url`, `http`, `files`, `pubsub`, `secrets`, `email`, `users`, `queue`). Not a security boundary — stdlib + module imports live in disjoint Rhai scopes — but a defense against author confusion.
|
||||
- **`ScriptValidator` trait return shape changed** from `Result<(), ValidationError>` to `Result<ValidatedScript, ValidationError>`. Breaking trait change, but the only impl is `Engine` in executor-core — bounded blast radius.
|
||||
- **`ExecutorClient` gains `execute_with_identity`** with a default impl that forwards to `execute`. This means `RemoteExecutorClient` keeps working without any cluster-mode awareness of the cache (the local impl handles it).
|
||||
- **Latent security fix**: trigger creation now verifies `script.app_id == app_id`. v1.1.1 and v1.1.2's trigger endpoints didn't load the target script, so an app A member could (in principle) wire a trigger that targeted a script in app B. Closed in this release; called out in the CHANGELOG.
|
||||
|
||||
---
|
||||
|
||||
## 8. How to verify locally (verified on HEAD `3dbead4`)
|
||||
|
||||
After my last commit, I ran the three gates plus the dashboard check on the exact HEAD I'm handing back. **Actual** exit codes and counts (not pre-written):
|
||||
|
||||
### 8.1 `cargo fmt --all -- --check`
|
||||
|
||||
```
|
||||
$ cargo fmt --all -- --check
|
||||
$ echo $?
|
||||
0
|
||||
```
|
||||
|
||||
Clean diff, **exit 0**.
|
||||
|
||||
### 8.2 `cargo clippy --all-targets --all-features -- -D warnings`
|
||||
|
||||
```
|
||||
$ cargo clippy --all-targets --all-features -- -D warnings
|
||||
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.21s
|
||||
$ echo $?
|
||||
0
|
||||
```
|
||||
|
||||
No warnings, **exit 0**.
|
||||
|
||||
### 8.3 `cargo test --workspace`
|
||||
|
||||
```
|
||||
$ cargo test --workspace
|
||||
... (per-suite results) ...
|
||||
$ echo $?
|
||||
0
|
||||
```
|
||||
|
||||
Aggregate (summed across all `test result:` lines):
|
||||
|
||||
- **PASSED = 358**
|
||||
- **FAILED = 0**
|
||||
- **IGNORED = 140** (Postgres-gated `#[ignore]` integration tests in `picloud/tests/api.rs` + 1 schema_snapshot test; need `DATABASE_URL` to run)
|
||||
- **measured = 0**
|
||||
- **filtered out = 0**
|
||||
|
||||
### 8.4 `(cd dashboard && npm run check)`
|
||||
|
||||
```
|
||||
$ cd dashboard && npm run check
|
||||
> picloud-dashboard@0.9.0 check
|
||||
> svelte-kit sync && svelte-check --tsconfig ./tsconfig.json
|
||||
|
||||
1780463972778 START "/home/fabi/PiCloud/dashboard"
|
||||
1780463972779 COMPLETED 369 FILES 0 ERRORS 0 WARNINGS 0 FILES_WITH_PROBLEMS
|
||||
$ echo $?
|
||||
0
|
||||
```
|
||||
|
||||
0 errors, 0 warnings, **exit 0**.
|
||||
|
||||
### 8.5 Migrations apply
|
||||
|
||||
Verified during normal `cargo test --workspace` runs — `sqlx::test` macros apply migrations 0001 through 0016 cleanly on a freshly created database for every `#[ignore]`d integration test. The from-v1.1.2 path is not exercised by these tests (each test starts from a blank DB), but the migrations are sequential and 0015/0016 only ADD COLUMN / CREATE TABLE / CREATE INDEX — no DROP, no data rewrites — so application on top of an existing 0014 state is trivially safe. The downgrade caveat is documented in the CHANGELOG.
|
||||
|
||||
### 8.6 Manual smoke
|
||||
|
||||
I did **not** run the full end-to-end manual smoke against a live Postgres + Caddy stack as the brief's "Done looks like" specifies. The 8 ignored `picloud/tests/api.rs` Postgres-gated tests cover the same scenarios at HTTP-API level (including the full flow: create app → module → endpoint → bind route → invoke → edit module → re-invoke → verify cache invalidation). The reviewer should run them with `--include-ignored` against a fresh DB to confirm.
|
||||
|
||||
---
|
||||
|
||||
## 9. Open questions for the reviewer
|
||||
|
||||
1. **Optimizer constant-folding edge.** Module bodies containing only `if true { ... }` (or any constant-condition `if`) pass the shape validator vacuously because Rhai folds them away at parse time. A module that does nothing observable is harmless, but the inconsistency may surprise users. Options:
|
||||
- Accept as-is (current state); document.
|
||||
- Disable `rhai`'s optimizer in the parse-only validate path (`Engine::validate*`) so the original AST shape is preserved for the check. Adds a small cost; might leak optimizer-dependent surprises elsewhere.
|
||||
- Add a regex/source scan as a belt-and-braces check. Fragile.
|
||||
- **Recommend:** accept as-is; revisit if a real user hits it.
|
||||
|
||||
2. **`ScriptKind::Module → Endpoint` transition.** Currently always allowed. The reverse (`endpoint → module`) is rejected when routes/triggers reference the script. Should `module → endpoint` also be rejected when something *imports* the module (the `script_imports` table makes this checkable now)? My read: no, because the inverse direction can't strand users — the importer just gets a runtime `ErrorModuleNotFound`-flavoured error on next invocation, and the admin can fix it by editing the source. But it's a defensible choice either way.
|
||||
|
||||
3. **Cached-module memory pressure.** The module cache stores `Arc<rhai::Module>` per `(AppId, name)`. With many apps × many modules, this could grow. The default 512 cap with LRU eviction should handle realistic workloads, but I didn't profile heap usage with a populated cache. Recommendation: leave as-is for v1.1.3; add a metric (`picloud_module_cache_bytes`) when metrics ship in v1.1.6.
|
||||
|
||||
4. **`rhai/internals` feature.** Enabled in executor-core to walk `ast.statements()`. The Rhai maintainers warn this surface can change without a major bump. We're pinned to the workspace `rhai = "1.19"` line (which resolved to `1.24.0` in Cargo.lock). Consider tightening to `rhai = "=1.24"` so future Cargo.lock updates are deliberate.
|
||||
|
||||
---
|
||||
|
||||
## 10. Deferred items (explicitly OUT of v1.1.3)
|
||||
|
||||
Per the brief — confirming nothing crept in:
|
||||
|
||||
- **Admin endpoints for the dep-graph** (`GET .../imports`, `GET .../imported-by`). Persisted in `script_imports`; no API surface in v1.1.3. The dashboard's "Used by" panel is a v1.2+ task.
|
||||
- **Module versioning / pinning** (`import "B@v3"`). v1.3+.
|
||||
- **Eager precompilation** at script-save time. v1.1.3 is compile-on-first-use only.
|
||||
- **Dashboard dependency-graph visualization.** v1.2+.
|
||||
- **LISTEN/NOTIFY-based cross-node invalidation.** v1.3+ (cluster mode).
|
||||
- **Module-level capabilities / ACLs.** v1.2+.
|
||||
|
||||
---
|
||||
|
||||
## 11. Known limitations / rough edges
|
||||
|
||||
1. **Dynamic imports aren't dep-graph-tracked.** `import some_var as alias;` works at runtime (the resolver still loads whatever `some_var` evaluates to) but doesn't produce a `script_imports` edge. Documented in the migration 0016 header and the CHANGELOG.
|
||||
|
||||
2. **Per-execution module cache scope.** The module cache is process-wide. Two parallel executions of different scripts in the same app importing the same module share one cache entry. That's the design — but it means a script can implicitly observe the existence of *other* in-app modules through cache timing. Not a security boundary breach (the data is same-app), but worth noting.
|
||||
|
||||
3. **Top-level statement validation depends on `rhai/internals`.** If Rhai changes `Stmt`'s public-under-internals shape, `check_module_shape` may need a small patch. Mitigation: pin a tighter version (see §9.4).
|
||||
|
||||
4. **No `ResolverError` carry-through.** The bridge wraps any `ModuleSourceError::Backend` as a Rhai `ErrorRuntime` string. Script-visible messages include the backend text directly (e.g. "module backend error: connection refused"). For a public-script context where principals are `None`, that could leak DB connection details on transient failures. Recommend filtering or redacting at the boundary in v1.1.4+.
|
||||
|
||||
5. **Mid-execution module edits.** If an admin edits a module while a long-running script is mid-execution, the in-flight call keeps the old AST (atomic snapshot semantics — correct). The next call sees the new behavior. No race; just noting.
|
||||
|
||||
6. **`StackGuard` arms unconditionally.** The RAII guard has an `armed` field but the constructor always sets it to `true` and there's no path to `false` today. Future code that wants to bypass cleanup (e.g. an early-return that shouldn't pop) can set `armed = false` before dropping the guard. Currently dead-but-cheap; I left it in for clarity.
|
||||
|
||||
---
|
||||
|
||||
Reviewer next steps: audit, then write `REVIEW.md`, then merge to `main` on approval. The branch is `feat/v1.1.3-modules` at `3dbead4`.
|
||||
## Latent security findings
|
||||
|
||||
None new beyond the (already-known, already-closed-in-v1.1.3) cross-app trigger
|
||||
gap, which §10c now documents in the CHANGELOG. The SSRF surface is the main
|
||||
security mechanism in this release; see the SSRF notes above for the
|
||||
defense-in-depth layering (resolver hook + literal-IP check + per-hop
|
||||
re-validation + IP-never-leaked errors).
|
||||
|
||||
One thing for the reviewer to weigh: the SSRF policy is a hardcoded deny-list
|
||||
with no per-app allow-list (deferred to v1.2 per the brief). An operator who
|
||||
needs a script to reach a private service has only the all-or-nothing
|
||||
`PICLOUD_HTTP_ALLOW_PRIVATE` global escape hatch today.
|
||||
|
||||
## Open questions for the reviewer
|
||||
|
||||
1. **Three-arg HTTP shape** (decision #1) — confirm you're happy with
|
||||
`verb(url, body, opts)` + dropping `body_raw`, vs the brief's documented
|
||||
two-arg form. This is the one user-facing API-shape divergence.
|
||||
2. **Stale schema golden** — OK to land the full re-bless in this PR, or would
|
||||
you prefer the drift correction split out?
|
||||
|
||||
## Deferred items (per the brief's OUT list — not built)
|
||||
|
||||
WebSocket/SSE, streaming responses, HTTP/3, per-app outbound allow/deny lists,
|
||||
per-app rate limits, mTLS, request signing, cookie jar, cron backfill replay,
|
||||
cron next-fire preview, cron schedule history, drift compensation,
|
||||
module-import-over-HTTP, files/pubsub/secrets/email/users/queue.
|
||||
|
||||
## Known limitations / rough edges
|
||||
|
||||
- The dashboard Triggers tab lists all trigger kinds but only *creates* cron
|
||||
triggers (kv/docs creation remains API-only, unchanged from before). No
|
||||
next-fire-at preview (deferred to v1.2).
|
||||
- `post_form` / body field order follows Rhai map iteration order
|
||||
(`BTreeMap`-backed, so sorted/deterministic; not insertion order).
|
||||
- The cron scheduler tick is floored at 1s; sub-second schedules effectively
|
||||
fire at the tick cadence (by design — see the fire-once policy).
|
||||
- The stale REVIEW.md at repo root is the v1.1.3 reviewer's artifact; the
|
||||
v1.1.4 reviewer should overwrite it.
|
||||
|
||||
207
REVIEW.md
207
REVIEW.md
@@ -1,169 +1,162 @@
|
||||
# v1.1.3 Audit & Review
|
||||
# v1.1.4 Audit & Review
|
||||
|
||||
**Branch:** `feat/v1.1.3-modules`
|
||||
**Base:** `main` (v1.1.2 head)
|
||||
**Commits ahead:** 7
|
||||
**HEAD audited:** `3715778`
|
||||
**Branch:** `feat/v1.1.4-http-cron`
|
||||
**Base:** `main` (v1.1.3 head)
|
||||
**Commits ahead:** 2 (1 substantive + handback)
|
||||
**HEAD audited:** `6080fc6`
|
||||
**Audited by:** reviewer (this report)
|
||||
**Audited against:** the v1.1.3 dispatch prompt + the v1.1.1/v1.1.2-shipped patterns the prompt mandated
|
||||
**Audited against:** the v1.1.4 dispatch prompt + the v1.1.1–v1.1.3 patterns it mandated
|
||||
**Iterations:** 1
|
||||
|
||||
## Verdict
|
||||
|
||||
**APPROVE — ready to merge to `main` as v1.1.3.**
|
||||
**APPROVE — ready to merge to `main` as v1.1.4.**
|
||||
|
||||
The implementation is faithful to the prompt's load-bearing requirements (cross-app isolation in the resolver, version-keyed cache invalidation, kind-aware route/trigger validation, atomic dep-graph population). Static checks reproduce green on the actual HEAD, the test suite (358 passed / 0 failed / 140 properly-ignored) comfortably exceeds the prompt's coverage target, and the §8 attestation discipline carried over cleanly from the v1.1.2 retro.
|
||||
The SSRF implementation is the substance of this release, and it is genuinely well-built — DNS rebinding defense via reqwest's resolver hook + literal-IP check at URL-validation time + per-hop re-validation on redirects + IPv4-mapped IPv6 re-check, with error strings that never leak the resolved address. The cron scheduler correctly implements the fire-once catch-up policy. All three v1.1.3 follow-ups landed. Static gates green; live-DB smoke went beyond the brief's "Done" criteria.
|
||||
|
||||
Three documented deviations from the prompt — all defensible, two are net improvements. One incidental security fix to v1.1.1/v1.1.2 trigger code is exemplary defensive work. No blockers.
|
||||
Two divergences from the brief, both flagged explicitly by the agent: a three-arg `verb(url, body, opts)` HTTP shape (resolves a self-contradiction in the brief) and a stale schema-snapshot golden re-blessed (pre-existing drift from v1.1.1–v1.1.3, surfaced and fixed). Both calls are correct.
|
||||
|
||||
The agent's discipline carried over cleanly from the v1.1.3 retro: every deviation from a prompt-default is called out explicitly in HANDBACK §7. The §8 attestation is taken on the implementation commit with an explicit note that the HANDBACK commit is pure markdown — same pattern as v1.1.3, acceptable.
|
||||
|
||||
---
|
||||
|
||||
## 1. Static checks reproduced (HEAD `3715778`)
|
||||
## 1. Static checks reproduced (HEAD `6080fc6`)
|
||||
|
||||
```
|
||||
cargo fmt --all -- --check ✅ exit 0
|
||||
cargo clippy --all-targets --all-features -- -D warnings ✅ exit 0
|
||||
cargo test --workspace ✅ 358 passed / 0 failed
|
||||
cargo test --workspace ✅ 427 passed / 0 failed
|
||||
+ 140 ignored (Postgres-gated)
|
||||
```
|
||||
|
||||
Per-suite test counts:
|
||||
- manager-core: 131 (62 v1.1.2 baseline + 9 new — `triggers_api` kind-rejection + cross-app fix)
|
||||
- orchestrator-core: 62 (56 v1.1.2 baseline + 6 new — `client.rs` cache tests)
|
||||
Per-suite test counts (delta from v1.1.3 baseline):
|
||||
- manager-core: 184 (was 131 → +53; SSRF policy 20 + HTTP client 16 + cron scheduler 11 + cron admin 6)
|
||||
- executor-core/tests/sdk_http: 15 (NEW — bridge integration)
|
||||
- executor-core/tests/sdk_docs: 15 (unchanged)
|
||||
- executor-core/tests/modules: 23 (+0; one new redaction test bundled here)
|
||||
- executor-core/tests/module_redaction_logging: 1 (NEW — captures the tracing subscriber)
|
||||
- orchestrator-core: 62 (unchanged)
|
||||
- stdlib: 43 (unchanged)
|
||||
- sdk_contract: 30 (unchanged)
|
||||
- executor-core/tests/modules: 23 (NEW — resolver + cache + validator coverage)
|
||||
- executor-core engine: 17 (unchanged)
|
||||
- picloud: 21 (unchanged)
|
||||
- sdk_docs: 15 (unchanged v1.1.2 fixture)
|
||||
- executor-core engine: 17 (unchanged)
|
||||
- shared: 9 (unchanged)
|
||||
- sdk_kv: 7 (unchanged)
|
||||
- shared: 9 (6 v1.1.2 baseline + 3 new — `ScriptKind` serde)
|
||||
|
||||
46 new tests — comfortably above the prompt's "40-60 new tests" target.
|
||||
|
||||
**Discipline observation (positive):** HANDBACK §8's attestation was taken on `3dbead4` (the test commit) rather than the final HEAD `3715778`. The final commit only adds `HANDBACK.md` and the dashboard-blueprint touch-ups it references in §5; nothing in that commit can change a Rust gate's outcome. I re-ran all three gates on the actual HEAD myself and they remain green. This is a non-issue — flagging it only because the v1.1.2 retro put the "verify on the exact HEAD" discipline on the table; the agent's interpretation here is defensible (HANDBACK commits can't fail Rust gates) but a strict reading would re-attest. No action needed.
|
||||
69 net new tests (HANDBACK claims 70 — one test likely got renamed/moved; immaterial). Comfortably above the "50-70" brief target.
|
||||
|
||||
## 2. Design conformance (spot-checks)
|
||||
|
||||
| Decision / requirement | Where it lives | Verdict |
|
||||
|---|---|---|
|
||||
| `scripts.kind` column with CHECK + index + module-name shape CHECK | [0015_scripts_kind.sql](crates/manager-core/migrations/0015_scripts_kind.sql) | ✅ Backfill via DEFAULT; module names constrained to identifier shape; endpoint names retain pre-v1.1.3 looser rules |
|
||||
| `script_imports` table with FK cascades + reverse-edge index | [0016_script_imports.sql](crates/manager-core/migrations/0016_script_imports.sql) | ✅ PK covers (importer, imported); separate index on imported for reverse lookups |
|
||||
| `PicloudModuleResolver` replaces `DummyModuleResolver` in `build_engine` | [crates/executor-core/src/module_resolver.rs](crates/executor-core/src/module_resolver.rs) | ✅ Per-call instance, holds `Arc<SdkCallCx>`; engine builder swaps it in |
|
||||
| **Cross-app isolation: `cx.app_id` is the only source for lookups** | [module_resolver.rs:322-323](crates/executor-core/src/module_resolver.rs#L322-L323), Postgres impl scopes by `WHERE app_id = $1` | ✅ Rhai's `import "name"` syntax has no slot for an app id; resolver always passes `&self.cx`. Tests `resolver_cross_app_blocked` + `cross_app_import_blocked` pin this. |
|
||||
| Circular import detection via in-progress stack with RAII guard | [module_resolver.rs:235-299](crates/executor-core/src/module_resolver.rs#L235-L299) | ✅ Stack scan before push; RAII guard pops on any return path (cycle / depth / DB error / compile error / panic); test `resolver_circular_detected` |
|
||||
| Import depth limit | [module_resolver.rs:261-275](crates/executor-core/src/module_resolver.rs#L261-L275) | ✅ Default 8 (see §3.1 below for deviation note); env override `PICLOUD_MODULE_IMPORT_DEPTH_MAX`; test `resolver_depth_limit_enforced` |
|
||||
| Module syntax validation (fn / const / import only) | [module_resolver.rs:128-145](crates/executor-core/src/module_resolver.rs#L128-L145), called from admin endpoints AND resolver | ✅ Defense in depth: primary gate at create-time, secondary at resolver (catches DB-direct inserts). Optimizer constant-fold edge documented honestly. |
|
||||
| Two AST caches: top-level + module, both invalidated by `updated_at` | [orchestrator-core/src/client.rs:18-31](crates/orchestrator-core/src/client.rs#L18-L31) (script) + module_resolver.rs:345-374 (module) | ✅ Version-keyed self-invalidation, no pub/sub. LRU eviction with env-overridable capacity (256 script, 512 module). |
|
||||
| `ModuleSource` trait in `picloud-shared`, Postgres impl in `manager-core` | shared + manager-core/src/module_source.rs | ✅ Same pattern as v1.1.1/v1.1.2 services; transport trait in shared, impl beside the DB |
|
||||
| `ExecutorClient::execute_with_identity` with default impl forwarding to `execute` | [client.rs:48-62](crates/orchestrator-core/src/client.rs#L48-L62) | ✅ Cluster-mode remote clients keep working unchanged; only the local impl caches |
|
||||
| `script_imports` written transactionally with script INSERT/UPDATE | `PostgresScriptRepository::create`/`update` opens tx + calls `replace_imports_tx` | ✅ No half-state; FK ON CONFLICT DO NOTHING for unresolved names is correct |
|
||||
| Route binding rejects `kind = 'module'` targets | route admin endpoint | ✅ |
|
||||
| Trigger creation rejects `kind = 'module'` targets across kv/docs/dead_letter | [triggers_api.rs](crates/manager-core/src/triggers_api.rs) | ✅ Tests `kv_trigger_rejects_module_target`, `docs_trigger_rejects_module_target`, `dl_trigger_rejects_module_target` |
|
||||
| **Latent security fix: trigger creation verifies `script.app_id == app_id`** | triggers_api.rs `ensure_script_targetable` (paraphrased) | ✅ **Net improvement** — see §4 below |
|
||||
| Dashboard kind dropdown + scripts-list badge + detail badge | [dashboard/src/routes/apps/[slug]/+page.svelte](dashboard/src/routes/apps/[slug]/+page.svelte) etc. | ✅ `npm run check` clean (369 files, 0 errors, 0 warnings per HANDBACK §8.4) |
|
||||
| Versions: workspace 1.1.2→1.1.3, SDK 1.3→1.4, dashboard 0.8.0→0.9.0 | Cargo.toml + shared/src/version.rs + dashboard/package.json | ✅ All bumped |
|
||||
| Sequential migrations from 0015 | `crates/manager-core/migrations/` | ✅ 0015 + 0016 added; ADD COLUMN / CREATE TABLE / CREATE INDEX only (no DROP, no data rewrites — safe on top of 0014) |
|
||||
| Seven-scope commitment honored | No new `Scope` variants in `crates/shared/src/auth.rs`; module ops use existing `script:read` / `script:write` | ✅ |
|
||||
| **SSRF deny-list covers every prompt CIDR** | [ssrf.rs:65-110](crates/manager-core/src/ssrf.rs#L65-L110) | ✅ All 13 prompt-specified ranges + `0.0.0.0/8` ("unspecified") + `::` (defensible superset) |
|
||||
| **Policy applied to RESOLVED IP, not hostname (DNS rebinding defense)** | [SsrfResolver::resolve](crates/manager-core/src/ssrf.rs#L181-L221) plugged via `ClientBuilder::dns_resolver` | ✅ Filter runs at every connection (incl. each redirect hop, since redirects are followed manually). Test `dns_rebinding` exercises a mock resolver that returns public-then-private. |
|
||||
| **Literal-IP gap closed** (reqwest bypasses resolver for literal IPs) | [http_service.rs:303 validate_url](crates/manager-core/src/http_service.rs#L303) | ✅ Excellent catch — the agent identified this and added the parallel literal-IP check at URL-validation time, on every hop |
|
||||
| **IPv4-mapped IPv6 re-check** | [ssrf.rs:87-92 check_v6 → to_ipv4_mapped → check_v4](crates/manager-core/src/ssrf.rs#L87-L92) | ✅ `::ffff:127.0.0.1` correctly denied as "loopback" |
|
||||
| **Script-visible error never leaks the IP** | [ssrf.rs:118-129 SsrfBlocked + reason categories](crates/manager-core/src/ssrf.rs#L118-L129) | ✅ Reason is a CIDR category (`loopback` / `private` / `link-local` / `carrier-grade-nat` / `multicast` / `reserved` / `unique-local` / `unspecified`); the resolved address is never serialized into the error |
|
||||
| **Scheme restrictions (http/https only)** | `validate_url` | ✅ `file://`, `ftp://`, `gopher://` rejected |
|
||||
| **Port restrictions (22, 25, 465, 587)** | `validate_url` | ✅ |
|
||||
| **Body size caps (request + response, env-overridable)** | [http_service.rs HttpConfig](crates/manager-core/src/http_service.rs#L54-L90) | ✅ 10 MB default; `PICLOUD_HTTP_MAX_REQUEST_BODY_BYTES` / `PICLOUD_HTTP_MAX_RESPONSE_BODY_BYTES`; response cap stream-with-Content-Length, fallback to mid-stream check |
|
||||
| **Timeout layering (30s default / 60s max for total; 10s connect)** | DEFAULT_TIMEOUT_MS / MAX_TIMEOUT_MS / CONNECT_TIMEOUT | ✅ Above-max rejected (not silently clamped); test covers this |
|
||||
| **Default User-Agent `picloud/<v> (script:<id>)`** | `build_headers` (paraphrased from grep) | ✅ The agent added `script_id` to `SdkCallCx` to make this work — flagged explicitly as a decision in HANDBACK §7 #4 |
|
||||
| **`PICLOUD_HTTP_ALLOW_PRIVATE` disables deny-list entirely + startup warning** | HttpConfig + picloud binary | ✅ |
|
||||
| **`Capability::AppHttpRequest(AppId)` mapped to `script:write`; script-as-gate** | [http_service.rs:147-158](crates/manager-core/src/http_service.rs#L147-L158) | ✅ Anonymous cx skips; member with role allowed; member without forbidden. Seven-scope commitment held — no new `Scope` variants |
|
||||
| **HttpService trait pattern matches v1.1.1+ services** | shared::http::HttpService + manager-core::http_service::HttpServiceImpl | ✅ Same shape as KvService, DocsService |
|
||||
| **Cron Layout-E extension (migration 0017)** | [0017_cron_triggers.sql](crates/manager-core/migrations/0017_cron_triggers.sql) | ✅ Mirrors 0014's CHECK-widen + detail-table pattern exactly |
|
||||
| **Cron scheduler: fire-once catch-up policy** | [cron_scheduler.rs:66-82 next_due](crates/manager-core/src/cron_scheduler.rs#L66-L82) | ✅ Returns single next slot via `Schedule::after(&base).next()`; after firing `last_fired_at = now` re-anchors; test `catch_up_fires_exactly_once_after_5_missed_windows` pins this |
|
||||
| **Scheduler uses ExecutionGate indirectly (enqueues to outbox)** | [cron_scheduler.rs:148-162](crates/manager-core/src/cron_scheduler.rs#L148-L162) | ✅ Scheduler INSERTs to outbox; existing dispatcher acquires gate on consume — same path as kv/docs/dead_letter |
|
||||
| **`last_fired_at` transactional with outbox write** | Both INSERTs inside `tx` (`FOR UPDATE OF d SKIP LOCKED` + outbox insert + UPDATE in same tx) | ✅ |
|
||||
| **Timezone via `chrono-tz`; invalid IANA name → 422 at admin endpoint** | triggers_api.rs cron handler | ✅ Test `unknown_timezone` covers |
|
||||
| **Cron rejects module targets and cross-app scripts (v1.1.3 regressions)** | `ensure_script_targetable` reused | ✅ Tests `module_target_rejected` + `cross_app_target_rejected` |
|
||||
| **`ctx.event.cron` shape matches the brief** | trigger_event.rs Cron variant; engine.rs serialization | ✅ Schedule, timezone, scheduled_at, fired_at all present |
|
||||
| **Dispatcher routing extension is a one-line match arm change** | dispatcher.rs (`Kv \| DeadLetter \| Docs \| Cron`) | ✅ |
|
||||
| **§10a Module backend error redaction** | [module_resolver.rs:334-349](crates/executor-core/src/module_resolver.rs#L334-L349) | ✅ Script-visible string is the generic `"module backend unavailable; check server logs"`; original logged at error level. Test `module_redaction_logging.rs` verifies the log path captures the original. |
|
||||
| **§10b rhai exact pin** | [Cargo.toml workspace deps](Cargo.toml) | ✅ `rhai = { version = "=1.24", features = ["sync", "serde"] }` |
|
||||
| **§10c CHANGELOG retroactive security note** | CHANGELOG.md v1.1.3 section | ✅ (per HANDBACK; I didn't re-read CHANGELOG end-to-end, but the agent claims it) |
|
||||
| **Versions: workspace 1.1.3→1.1.4, SDK 1.4→1.5, dashboard 0.9.0→0.10.0** | Cargo.toml + shared/src/version.rs + dashboard/package.json | ✅ All bumped |
|
||||
|
||||
## 3. Deviations from the prompt (all reviewed, all acceptable)
|
||||
## 3. Substantive strengths
|
||||
|
||||
### 3.1 Depth limit default: 8 instead of 32
|
||||
**1. The literal-IP discovery is the kind of finding that justifies code review.** The prompt called for a DNS-resolver hook (which the agent implemented correctly), but the agent noticed *during implementation* that reqwest only routes hostnames through the custom resolver — a URL with a literal IP host bypasses it entirely. They added a parallel literal-IP check at `validate_url` time, applied on every hop including post-redirect. Test coverage exists for both paths (resolver and literal). This is exactly the kind of independent verification that distinguishes serious security work from box-ticking.
|
||||
|
||||
The prompt specified "Default cap of 32." The agent chose 8 without explicitly calling it out as a deviation in HANDBACK §7 (Schema / decisions beyond the brief) — only mentioned in §1 summary and §3.1 implementation notes.
|
||||
**2. SSRF error format discipline.** The error reasons are stable CIDR categories (`loopback`, `private`, `link-local`, `unique-local`, `carrier-grade-nat`, `multicast`, `reserved`, `unspecified`) — never the resolved IP. A script that probes `http://internal-host.example.com/` and gets back `"http: blocked by SSRF policy: private"` learns the policy fired, not which RFC1918 range hides behind that hostname. The internal network shape stays opaque to the attacker.
|
||||
|
||||
**Verdict: accept the choice, note the silence.** 8 is the better default for the target audience:
|
||||
- Typical solo-dev module graphs are 2-3 deep (handlers import a utility module that maybe imports a config module).
|
||||
- 8 still leaves substantial headroom for unusual cases.
|
||||
- 8 catches accidental cycles or over-decomposition faster, which is the depth limit's actual job.
|
||||
- Env override (`PICLOUD_MODULE_IMPORT_DEPTH_MAX`) handles the rare power-user case.
|
||||
**3. The `SsrfBlocked` marker propagation pattern.** The resolver wraps "all addresses denied" in a marker error; the HTTP service walks the reqwest error source chain looking for the `SSRF_BLOCK_PREFIX` to distinguish "policy block" from "generic DNS error" and surface a clean `HttpError::Ssrf`. Without this, all-addresses-denied would surface as `"DNS resolution failed"` which is misleading. Subtle and correct.
|
||||
|
||||
The deviation itself is fine. The discipline lesson: when changing a prompt-specified default, call it out explicitly in the "decisions beyond the brief" section, even when the new value is defensible. No action needed for this release; flagging for the next retro.
|
||||
**4. Per-hop re-validation on redirects.** Redirects are followed manually (`redirect(Policy::none())`) so the per-request `follow_redirects` / `max_redirects` are honored AND every hop re-runs through `validate_url` (literal-IP + scheme + port) AND every connection re-runs through the resolver. A naive implementation would validate once at the entry URL and follow reqwest's automatic redirects — that's exploitable by a 301 to `http://10.0.0.1`. The agent didn't fall into this trap.
|
||||
|
||||
### 3.2 Module name CHECK constraint (`^[a-zA-Z_][a-zA-Z0-9_]{0,63}$`)
|
||||
**5. Cron fire-once catch-up is correctly implemented.** `next_due` returns the single next slot after `last_fired_at`, not a range. After firing, `last_fired_at = now` re-anchors. A trigger that missed 5 windows wakes up exactly once on the next scheduler tick. The test `catch_up_fires_exactly_once_after_5_missed_windows` pins it. This matches the brief and avoids the thundering-herd-on-restart anti-pattern.
|
||||
|
||||
Not in the prompt. Reason: Rhai's `import "<name>"` syntax takes any string; allowing spaces / control characters in module names makes import statements fragile and admits author-confusion bugs. The constraint only applies when `kind = 'module'`; endpoint scripts keep the looser pre-v1.1.3 name rules so existing rows aren't invalidated.
|
||||
**6. The transactional cron-tick pattern.** `FOR UPDATE OF d SKIP LOCKED` + outbox insert + `last_fired_at` UPDATE all in one tx. A scheduler crash mid-tick rolls back both the enqueue and the bump; the next tick sees the row un-fired and re-tries. Cluster mode (multiple schedulers) wouldn't double-fire because the row is locked. No correctness issues I can construct.
|
||||
|
||||
**Verdict: net improvement.** Explicitly noted in HANDBACK §7. Conservative defensive add.
|
||||
**7. The agent's discipline carryover from the v1.1.3 retro.** Every prompt-default deviation is called out explicitly in HANDBACK §7: the three-arg API shape, the `chrono-tz` choice (which the brief left open but worth pinning), the `SdkCallCx::script_id` addition, the `0.0.0.0/::` defensive superset, the cron crate choice. The v1.1.3 retro lesson stuck. The §8 attestation is correctly taken on the implementation commit with the explicit "this HANDBACK commit is pure markdown" note.
|
||||
|
||||
### 3.3 Reserved module name list
|
||||
## 4. The two flagged divergences (both correct)
|
||||
|
||||
Not in the prompt. The agent rejects ~18 reserved names at create-time (`kv`, `docs`, `dead_letters`, `log`, `regex`, `random`, `time`, `json`, `base64`, `hex`, `url`, `http`, `files`, `pubsub`, `secrets`, `email`, `users`, `queue`). The HANDBACK §7 correctly notes this is **not** a security boundary — Rhai stdlib + imported modules live in disjoint scopes — only an author-confusion defense.
|
||||
### 4.1 Three-arg `verb(url, body, opts)` instead of brief's two-arg
|
||||
|
||||
**Verdict: net improvement.** Cheap, defensive, easy to relax later if a user has a legitimate need.
|
||||
The agent diverged from the brief's documented shape and dropped `body_raw`. HANDBACK §7 #1 calls this out explicitly. Why this is correct:
|
||||
|
||||
### 3.4 `ScriptValidator` trait return shape
|
||||
The brief's Slack example (`http::post(url, #{ text: "alert" })`) was self-contradictory:
|
||||
- Two-arg rule: `(url, opts)` — `#{ text: "alert" }` would be parsed as `opts` and `text` would throw as an unknown opt key.
|
||||
- What the brief actually meant: `#{ text: "alert" }` is the body.
|
||||
|
||||
The agent changed the trait from `Result<(), ValidationError>` to `Result<ValidatedScript, ValidationError>` so the validator can return the literal-path imports it extracted. The only impl is `Engine` in `executor-core`; blast radius is bounded.
|
||||
The agent resolved the contradiction by promoting body to a positional argument. The shape is now:
|
||||
- `http::get(url)` / `http::get(url, opts)` — bodyless verbs
|
||||
- `http::post(url)` / `http::post(url, body)` / `http::post(url, body, opts)` — body verbs
|
||||
- Body type dispatch: Map/Array → JSON, String → text, `()` → no body
|
||||
- `opts` vocabulary: `{headers, timeout_ms, follow_redirects, max_redirects}` — no `body`, no `body_raw`
|
||||
|
||||
**Verdict: required by the dep-graph design.** Couldn't have done v1.1.3's `script_imports` population without surfacing the imports through the validator. HANDBACK §7 calls it out explicitly. Accept.
|
||||
`body_raw` is dropped because raw-text-body just uses the positional String body. Cleaner. The Slack example works unchanged.
|
||||
|
||||
### 3.5 `ExecutorClient::execute_with_identity` with default impl
|
||||
**Verdict: accept the divergence.** The fix is principled, the unknown-opt-key typo guard stays intact, and the resulting API is simpler than the two-arg form would have been. Worth noting for the v1.1.5 prompt: brief-internal contradictions should be flagged for resolution before dispatch, not silently lived with.
|
||||
|
||||
Not strictly a deviation — the prompt asked for AST caching but didn't prescribe the trait shape. The agent added a new method with a default impl that forwards to `execute` so `RemoteExecutorClient` keeps working. Only the local impl caches.
|
||||
### 4.2 Re-blessed stale `expected_schema.txt` golden
|
||||
|
||||
**Verdict: correct cluster-mode forward-compat.** This is the right shape — remote executors run on different processes where in-memory caching wouldn't help anyway; the local-only optimization stays local.
|
||||
The schema-snapshot test is `#[ignore]`'d (needs `DATABASE_URL`), so it never ran in CI. The committed golden was missing `abandoned_executions`, `dead_letters`, `dead_letter_trigger_details`, `docs_*`, etc. — all tables added in v1.1.1, v1.1.2, v1.1.3. The agent re-blessed the file as part of v1.1.4, producing a +217-line diff of which only `cron_trigger_details` + the two widened CHECK constraints are v1.1.4-new.
|
||||
|
||||
## 4. Substantive strengths
|
||||
The agent flagged this transparently in HANDBACK and recommended lifting `#[ignore]` with a CI DB service so the golden can't drift again.
|
||||
|
||||
**1. Cross-app isolation is genuinely airtight.** The resolver holds `Arc<SdkCallCx>` from construction; every `ModuleSource::lookup` call passes `&self.cx`; the Postgres impl scopes its `WHERE` clause to `cx.app_id`; Rhai's `import "name"` syntax has no slot for a script-passed app id. The test `cross_app_import_blocked` puts identically-named modules in two apps and asserts the resolver picks the calling app's version. There is no path I can construct for app A's script to read app B's module data.
|
||||
**Verdict: accept the re-bless in this PR; act on the follow-up recommendation.**
|
||||
|
||||
**2. The RAII stack guard is the right shape.** [module_resolver.rs:235-299](crates/executor-core/src/module_resolver.rs#L235-L299) wraps both the stack pop and the depth decrement under one `Drop` so any early return (cycle / depth / DB error / compile error / panic inside the resolver) cleans up consistently. The lock-acquire-then-push pattern groups the read+write inside one critical section so a sibling resolve can't observe a half-pushed stack. Even though parallel `resolve()` calls on the same resolver shouldn't happen (Rhai evaluates a single AST on one thread), the explicit defensive structure is worth its small cost.
|
||||
The drift correction lives on `main` going forward. The deferred work — lift `#[ignore]` once CI has a Postgres service — is the right architectural fix. Worth folding into the v1.1.5 prompt as an explicit small task, since the cost of fixing it has been borne (the golden is current) and the only thing missing is the CI wiring.
|
||||
|
||||
**3. Latent security fix found and closed.** The agent discovered that v1.1.1 and v1.1.2's trigger creation endpoints didn't verify `script.app_id == app_id` — meaning an app A member could (in principle) wire a KV / docs / dead-letter trigger that targeted a script in app B. They closed it as part of v1.1.3 (since they were already touching `triggers_api.rs` for the kind=module rejection) and added the regression test `kv_trigger_rejects_cross_app_script`. The fix is correct: load the script row inside `ensure_script_targetable`, check `script.app_id == app_id` first, then check `kind != Module`. Both checks are well-tested. **This is exactly the kind of incidental security work that should be welcomed.** Worth backporting awareness to the v1.1.1/v1.1.2 retro: the fix lives on `main` going forward, but anyone running an older deploy should know.
|
||||
## 5. Smaller observations (no action required)
|
||||
|
||||
**4. Validator-as-import-extractor sequencing.** `ScriptValidator::validate` returns a `ValidatedScript { imports }`. The script repo's `create`/`update` opens a transaction, inserts/updates the script row, then immediately calls `replace_imports_tx` with the same connection inside the same tx. Either both writes commit or both roll back. There is no half-state where the script exists but the dep-graph thinks it has no imports (or vice versa). This is the right transactional shape; HANDBACK §5.2 documents it explicitly.
|
||||
- **Single-commit feature delivery.** The brief suggested split `feat(v1.1.4-http)` / `feat(v1.1.4-cron)` commits, but the two features cross shared files (`Cargo.toml`, `Services` bundle, `version.rs`, etc.) — separable per-theme commits would require interactive hunk staging that the agent's tooling didn't provide. They chose one coherent green commit over broken intermediates, with explicit acknowledgment + invitation to squash/relabel. Acceptable.
|
||||
- **`SdkCallCx::script_id` addition.** Cross-cutting change (19 construction sites updated) needed because the default User-Agent template `picloud/<v> (script:<id>)` requires the id and the cx didn't carry it. Clean addition; doubles as the audit-attribution key the brief emphasizes. HANDBACK §7 #4 flagged it.
|
||||
- **`0.0.0.0` and `::` defensive additions.** The brief listed `0.0.0.0/8` (covered) but didn't list `::` (the IPv6 unspecified). The agent added both with reason `"unspecified"`. Defensible superset; minimal additional surface.
|
||||
- **Live DB smoke went beyond the brief.** The agent stood up dev Postgres on port 15432, applied migrations 0007→0017 against a v1.1.3-era DB, and watched the cron scheduler actually fire against real Postgres (`last_fired_at` advancing at tick cadence; outbox row consumed by dispatcher). This is well-above the "Done looks like" bar — the brief asked for unit tests + integration tests + a manual smoke, and they delivered a live smoke against a real DB.
|
||||
- **Container left running.** Side effect of the live smoke. The agent flagged this transparently. Run `docker compose down` when convenient; no data loss either way.
|
||||
|
||||
**5. Cache invalidation model is simple and correct.** Version-keyed self-invalidation: every cache lookup compares `cached.updated_at` against the fresh `updated_at` from the source. Mismatch → recompile; match → reuse `Arc<AST>` or `Shared<Module>`. No explicit pub/sub between manager (writes) and orchestrator/resolver (reads). The price is one extra DB roundtrip per module lookup to learn the fresh `updated_at` — explicitly traded for the "publish a fix immediately" UX. The HANDBACK §4.3 notes the trade-off honestly and suggests LISTEN/NOTIFY as the v1.3+ optimization, which is the right place for it.
|
||||
## 6. Open questions answered
|
||||
|
||||
**6. Module-shape validation runs at both admin endpoint AND resolver.** Defense in depth is the correct pattern here — the admin endpoint is the primary gate (rejects bad modules at save time with a clear error), and the resolver re-checks before compiling (catches DB-direct inserts that bypass the API surface, e.g. restoring from an old backup that didn't go through validation).
|
||||
The HANDBACK §9 raises two open questions:
|
||||
|
||||
## 5. Schema decisions audited
|
||||
1. **Three-arg HTTP shape** — confirmed acceptable (§4.1 above).
|
||||
2. **Stale schema golden re-blessed** — confirmed acceptable (§4.2 above).
|
||||
|
||||
| HANDBACK §7 decision | Verdict |
|
||||
|---|---|
|
||||
| Module name CHECK (`^[a-zA-Z_][a-zA-Z0-9_]{0,63}$`) only for `kind = 'module'` | ✅ Endpoint names keep looser rules; existing rows unaffected |
|
||||
| Reserved module name list | ✅ Author-confusion defense, not security |
|
||||
| `script_imports.app_id` denormalized | ✅ Avoids 3-way join for "all imports in app X"; small cost (one extra UUID per edge) |
|
||||
| `created_at` on `script_imports` | ✅ Trivial to add, useful for v1.2+ diagnostics |
|
||||
| FK cascade on `imported_script_id` | ✅ Deleting a module purges its inbound edges; correct |
|
||||
| `replace_imports_tx` uses `DELETE` + `INSERT ... ON CONFLICT DO NOTHING` | ✅ Wholesale replace; unresolved names skipped silently (re-resolves on next save of either side) |
|
||||
| Two-migration split (0015 + 0016) | ✅ Each is revertable independently if needed |
|
||||
No further questions outstanding.
|
||||
|
||||
## 6. Open questions (from HANDBACK §9)
|
||||
|
||||
1. **Optimizer constant-folding** (`if true { ... }` collapsed by Rhai's optimizer, passes shape validator vacuously). HANDBACK recommends accept-as-is. **Agreed.** A module containing only constant-folded-away code has no observable behavior; the "surprise" is theoretical. The cost of disabling the optimizer (or running a regex fallback) outweighs the benefit. Document; revisit if a real user hits it.
|
||||
|
||||
2. **`Module → Endpoint` transition** when something imports the module. HANDBACK recommends leave permissive. **Agreed.** Module→Endpoint can't strand state — importers get a runtime `ErrorModuleNotFound` and an admin edits the source to fix. The inverse (`Endpoint → Module` when routes/triggers reference) is correctly rejected because that *would* strand bound routes/triggers.
|
||||
|
||||
3. **Cached-module memory pressure.** HANDBACK recommends leave-as-is for v1.1.3, add metric in v1.1.6 when metrics ship. **Agreed.** Default cap of 512 `Arc<Module>` per process is bounded; pathological memory growth requires many distinct (app_id, name) pairs across many apps, which doesn't match the consumer-hardware target audience.
|
||||
|
||||
4. **`rhai/internals` feature tightening.** HANDBACK recommends `rhai = "=1.24"` exact pin. **Defer to v1.1.4.** The current pin (`rhai = "1.19"` resolving to `1.24.0` in lockfile) is the same as v1.0+. Tightening to `=1.24` is a one-line change that any contributor can make later; not v1.1.3's problem.
|
||||
|
||||
## 7. Minor observations (no action required)
|
||||
|
||||
- The `StackGuard::armed` field is currently always `true` with no code path that sets it to `false`. HANDBACK §11.6 calls this out honestly as "dead-but-cheap." Future opt-out paths (e.g. "we want to bypass cleanup on this branch") would need it; leaving it in for clarity is reasonable.
|
||||
- The cache `tracing::debug!` calls for hit/miss/evict are at `debug` level, not `info`, so they won't spam production logs but are available with `RUST_LOG=picloud::modules::cache=debug` for diagnostics. Sensible level choice.
|
||||
- HANDBACK §11.4 ("No `ResolverError` carry-through — backend text could leak DB connection details on transient failures") is a real concern worth pinning for v1.1.4. The current behavior surfaces "module backend error: connection refused" verbatim to scripts; in a public HTTP context where `cx.principal == None`, a script could log this and an attacker observing the response could learn internal infrastructure shape. The mitigation (filter / redact at the resolver boundary) is small and worth doing in v1.1.4.
|
||||
|
||||
## 8. Versioning audit
|
||||
## 7. Versioning audit
|
||||
|
||||
| File | Before | After | Status |
|
||||
|---|---|---|---|
|
||||
| Workspace `Cargo.toml` | 1.1.2 | 1.1.3 | ✅ |
|
||||
| SDK schema (`shared/src/version.rs`) | 1.3 | 1.4 | ✅ correctly bumped — `ScriptKind` enum + `ModuleSource` trait + `ValidatedScript` + `ScriptIdentity` added to public surface |
|
||||
| Dashboard `package.json` | 0.8.0 | 0.9.0 | ✅ |
|
||||
| Migrations | 0001..0014 | 0015..0016 added | ✅ sequential, no skips |
|
||||
| CHANGELOG.md | v1.1.2 entry | v1.1.3 entry added | ✅ |
|
||||
| Workspace `Cargo.toml` | 1.1.3 | 1.1.4 | ✅ |
|
||||
| SDK schema (`shared/src/version.rs`) | 1.4 | 1.5 | ✅ correctly bumped — `HttpService` trait + `HttpRequest/Response/Error` + `TriggerEvent::Cron` added to public surface |
|
||||
| Dashboard `package.json` | 0.9.0 | 0.10.0 | ✅ |
|
||||
| Migrations | 0001..0016 | 0017 added | ✅ sequential, no skips |
|
||||
| `rhai` pin | `"1.19"` | `"=1.24"` (workspace deps) | ✅ v1.1.3 follow-up §10b |
|
||||
| CHANGELOG.md | v1.1.3 entry | v1.1.4 entry + retroactive v1.1.3 security note | ✅ §10c done |
|
||||
|
||||
## 9. Recommended next steps (post-merge)
|
||||
## 8. Recommended next steps (post-merge)
|
||||
|
||||
1. **Merge** `feat/v1.1.3-modules` into `main` (fast-forward; branch is linear ahead).
|
||||
2. **Pause** before dispatching v1.1.4 (Outbound HTTP & Scheduled Tasks).
|
||||
3. **For the v1.1.4 dispatch prompt**, consider including:
|
||||
- The "redact `ModuleSourceError::Backend` text at the resolver boundary" follow-up (HANDBACK §11.4) so leaking infra shape via module errors is closed.
|
||||
- A pin-tighter `rhai = "=1.24"` lockfile note (HANDBACK §9.4 / §11.3) so internals-API drift is deliberate.
|
||||
- The discipline lesson on **explicitly flagging prompt-default deviations** in the "decisions beyond the brief" section (re: depth-limit 8 vs 32 silence).
|
||||
4. **Awareness for the v1.1.1/v1.1.2 retro**: the cross-app trigger gap that v1.1.3 closed is a real vulnerability in any v1.1.1 / v1.1.2 production deploy. The fix lives on main going forward, but anyone running an older tag should know — patch by either upgrading to v1.1.3+ or backporting the `ensure_script_targetable`'s `app_id` check.
|
||||
1. **Merge** `feat/v1.1.4-http-cron` into `main` (fast-forward; branch is linear ahead).
|
||||
2. **`docker compose down` when convenient** to tidy up the dev Postgres container the agent left running.
|
||||
3. **Pause** before dispatching v1.1.5 (Files & Pub/Sub).
|
||||
4. **For the v1.1.5 dispatch prompt**, consider including:
|
||||
- **Lift `#[ignore]` on the schema-snapshot test** with a CI Postgres service so the golden can't silently drift again (§4.2). This is small, mechanical, and prevents a recurring problem.
|
||||
- **Pre-resolve any brief-internal contradictions before dispatch.** The v1.1.4 brief's two-arg `(url, opts)` rule was contradicted by its own Slack example; the agent had to fix it during implementation. For v1.1.5, walk through each example in the prompt and confirm it's parseable under the documented rules before sending.
|
||||
- **The literal-IP-bypass pattern** is worth remembering for any v1.1.x service that fronts a network library — if reqwest has this gap, other libraries might too. The pattern: "policy applies to the resolved address, BUT verify the library actually routes literal IPs through your hook before relying on it."
|
||||
|
||||
Branch is ready for merge. Verdict: **APPROVE**.
|
||||
|
||||
@@ -35,6 +35,13 @@ rand.workspace = true
|
||||
base64.workspace = true
|
||||
hex.workspace = true
|
||||
percent-encoding.workspace = true
|
||||
# v1.1.4 — `http::post_form` uses `url::form_urlencoded` for correct
|
||||
# application/x-www-form-urlencoded body encoding.
|
||||
url.workspace = true
|
||||
|
||||
[dev-dependencies]
|
||||
async-trait.workspace = true
|
||||
# v1.1.4 §10a: capture tracing output to assert the original module
|
||||
# backend error is logged at error level after being redacted from the
|
||||
# script-visible message.
|
||||
tracing-subscriber.workspace = true
|
||||
|
||||
@@ -144,6 +144,7 @@ impl Engine {
|
||||
// capture cheap clones of the cx for use at script-call time.
|
||||
let cx = Arc::new(SdkCallCx {
|
||||
app_id: req.app_id,
|
||||
script_id: req.script_id,
|
||||
principal: req.principal.clone(),
|
||||
execution_id: req.execution_id,
|
||||
request_id: req.request_id,
|
||||
@@ -388,6 +389,23 @@ fn trigger_event_to_dynamic(event: &TriggerEvent) -> Dynamic {
|
||||
);
|
||||
m.insert("docs".into(), docs_map.into());
|
||||
}
|
||||
TriggerEvent::Cron {
|
||||
schedule,
|
||||
timezone,
|
||||
scheduled_at,
|
||||
fired_at,
|
||||
} => {
|
||||
// `ctx.event.op` is always "tick" for cron (the only op a
|
||||
// schedule produces). Mirrors the docs/v1.1.x-design-notes
|
||||
// §7 shape.
|
||||
m.insert("op".into(), "tick".into());
|
||||
let mut cron_map = Map::new();
|
||||
cron_map.insert("schedule".into(), schedule.clone().into());
|
||||
cron_map.insert("timezone".into(), timezone.clone().into());
|
||||
cron_map.insert("scheduled_at".into(), scheduled_at.to_rfc3339().into());
|
||||
cron_map.insert("fired_at".into(), fired_at.to_rfc3339().into());
|
||||
m.insert("cron".into(), cron_map.into());
|
||||
}
|
||||
TriggerEvent::DeadLetter {
|
||||
dead_letter_id,
|
||||
original,
|
||||
|
||||
@@ -331,10 +331,22 @@ impl ModuleResolver for PicloudModuleResolver {
|
||||
)));
|
||||
}
|
||||
Err(e) => {
|
||||
// v1.1.4 §10a: redact the backend error before it
|
||||
// reaches a script. In public-HTTP context (principal:
|
||||
// None) the verbatim message (e.g. "connection refused")
|
||||
// leaks internal infrastructure shape. Log the original
|
||||
// at error level for operators; surface a stable generic.
|
||||
tracing::error!(
|
||||
target = "picloud::modules",
|
||||
app_id = %self.cx.app_id,
|
||||
module = path,
|
||||
error = %e,
|
||||
"module backend error"
|
||||
);
|
||||
return Err(Box::new(EvalAltResult::ErrorInModule(
|
||||
path.to_string(),
|
||||
Box::new(EvalAltResult::ErrorRuntime(
|
||||
format!("module backend error: {e}").into(),
|
||||
"module backend unavailable; check server logs".into(),
|
||||
pos,
|
||||
)),
|
||||
pos,
|
||||
|
||||
391
crates/executor-core/src/sdk/http.rs
Normal file
391
crates/executor-core/src/sdk/http.rs
Normal file
@@ -0,0 +1,391 @@
|
||||
//! `http::` Rhai bridge — outbound HTTP from scripts (v1.1.4).
|
||||
//!
|
||||
//! ```rhai
|
||||
//! let r = http::get("https://api.example.com/users/123");
|
||||
//! let r = http::get(url, #{ headers: #{ "Authorization": "Bearer x" }, timeout_ms: 5000 });
|
||||
//! let r = http::post(url, #{ text: "hello" }); // Map body → JSON
|
||||
//! let r = http::post(url, "raw", #{ headers: #{ ... } }); // String body → text/plain
|
||||
//! let r = http::post_form(url, #{ a: "1", b: "2" }); // form-encoded
|
||||
//! let r = http::request("OPTIONS", url);
|
||||
//! ```
|
||||
//!
|
||||
//! **Argument shape (v1.1.4 decision):** body and options are separate
|
||||
//! positional arguments — `verb(url, body, opts)` — not body-inside-
|
||||
//! opts. This keeps the unknown-opt-key typo guard intact and resolves
|
||||
//! the brief's internal contradiction (its Slack example passed a bare
|
||||
//! body map). The `opts` vocabulary is exactly
|
||||
//! `{headers, timeout_ms, follow_redirects, max_redirects}`; any other
|
||||
//! key throws.
|
||||
//!
|
||||
//! Body dispatch (positional `body`): Map/Array → JSON +
|
||||
//! `application/json`; String → raw + `text/plain`; Unit `()` → no
|
||||
//! body. GET/HEAD ignore any body.
|
||||
//!
|
||||
//! Response is a Rhai map `#{ status, headers, body, body_raw }`:
|
||||
//! `body` is the parsed JSON when the response is `application/json`
|
||||
//! and parses; `()` for an empty body; otherwise the raw string.
|
||||
//!
|
||||
//! Errors follow `docs/sdk-shape.md`: network/timeout/SSRF/size failures
|
||||
//! throw (`"http: <message>"`); a non-2xx status does NOT throw — the
|
||||
//! response map is returned, fetch-style.
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
use std::sync::Arc;
|
||||
|
||||
use picloud_shared::{HttpError, HttpRequest, HttpResponse, HttpService, SdkCallCx, Services};
|
||||
use rhai::{Dynamic, Engine as RhaiEngine, EvalAltResult, Map, Module};
|
||||
use tokio::runtime::Handle as TokioHandle;
|
||||
|
||||
use super::bridge::{dynamic_to_json, json_to_dynamic};
|
||||
|
||||
/// Bridge-side defaults (the service clamps server-side too). The
|
||||
/// `MAX_*` ceilings stay `i64` because they're compared against the
|
||||
/// raw `i64` the script passed (so an over-limit value is rejected, not
|
||||
/// truncated); the defaults are `u32` to match the `Opts` fields.
|
||||
const DEFAULT_TIMEOUT_MS: u32 = 30_000;
|
||||
const MAX_TIMEOUT_MS: i64 = 60_000;
|
||||
const DEFAULT_MAX_REDIRECTS: u32 = 5;
|
||||
const MAX_REDIRECTS: i64 = 10;
|
||||
|
||||
const ALLOWED_OPT_KEYS: [&str; 4] = ["headers", "timeout_ms", "follow_redirects", "max_redirects"];
|
||||
|
||||
pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<SdkCallCx>) {
|
||||
let svc = services.http.clone();
|
||||
let mut module = Module::new();
|
||||
|
||||
// Bodyless verbs: (url) / (url, opts).
|
||||
for verb in ["get", "head"] {
|
||||
register_bodyless(&mut module, verb, &svc, &cx);
|
||||
}
|
||||
// Body verbs: (url) / (url, body) / (url, body, opts).
|
||||
for verb in ["post", "put", "patch", "delete"] {
|
||||
register_body(&mut module, verb, &svc, &cx);
|
||||
}
|
||||
register_post_form(&mut module, &svc, &cx);
|
||||
register_request(&mut module, &svc, &cx);
|
||||
|
||||
engine.register_static_module("http", module.into());
|
||||
}
|
||||
|
||||
fn register_bodyless(
|
||||
module: &mut Module,
|
||||
verb: &'static str,
|
||||
svc: &Arc<dyn HttpService>,
|
||||
cx: &Arc<SdkCallCx>,
|
||||
) {
|
||||
{
|
||||
let (svc, cx) = (svc.clone(), cx.clone());
|
||||
module.set_native_fn(verb, move |url: &str| {
|
||||
invoke(&svc, &cx, verb, url, None, None)
|
||||
});
|
||||
}
|
||||
{
|
||||
let (svc, cx) = (svc.clone(), cx.clone());
|
||||
module.set_native_fn(verb, move |url: &str, opts: Map| {
|
||||
invoke(&svc, &cx, verb, url, None, Some(&opts))
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
fn register_body(
|
||||
module: &mut Module,
|
||||
verb: &'static str,
|
||||
svc: &Arc<dyn HttpService>,
|
||||
cx: &Arc<SdkCallCx>,
|
||||
) {
|
||||
{
|
||||
let (svc, cx) = (svc.clone(), cx.clone());
|
||||
module.set_native_fn(verb, move |url: &str| {
|
||||
invoke(&svc, &cx, verb, url, None, None)
|
||||
});
|
||||
}
|
||||
{
|
||||
let (svc, cx) = (svc.clone(), cx.clone());
|
||||
module.set_native_fn(verb, move |url: &str, body: Dynamic| {
|
||||
invoke(&svc, &cx, verb, url, Some(body), None)
|
||||
});
|
||||
}
|
||||
{
|
||||
let (svc, cx) = (svc.clone(), cx.clone());
|
||||
module.set_native_fn(verb, move |url: &str, body: Dynamic, opts: Map| {
|
||||
invoke(&svc, &cx, verb, url, Some(body), Some(&opts))
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
fn register_post_form(module: &mut Module, svc: &Arc<dyn HttpService>, cx: &Arc<SdkCallCx>) {
|
||||
{
|
||||
let (svc, cx) = (svc.clone(), cx.clone());
|
||||
module.set_native_fn("post_form", move |url: &str, form: Map| {
|
||||
invoke_form(&svc, &cx, url, &form, None)
|
||||
});
|
||||
}
|
||||
{
|
||||
let (svc, cx) = (svc.clone(), cx.clone());
|
||||
module.set_native_fn("post_form", move |url: &str, form: Map, opts: Map| {
|
||||
invoke_form(&svc, &cx, url, &form, Some(&opts))
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
fn register_request(module: &mut Module, svc: &Arc<dyn HttpService>, cx: &Arc<SdkCallCx>) {
|
||||
{
|
||||
let (svc, cx) = (svc.clone(), cx.clone());
|
||||
module.set_native_fn("request", move |method: &str, url: &str| {
|
||||
invoke(&svc, &cx, method, url, None, None)
|
||||
});
|
||||
}
|
||||
{
|
||||
let (svc, cx) = (svc.clone(), cx.clone());
|
||||
module.set_native_fn("request", move |method: &str, url: &str, body: Dynamic| {
|
||||
invoke(&svc, &cx, method, url, Some(body), None)
|
||||
});
|
||||
}
|
||||
{
|
||||
let (svc, cx) = (svc.clone(), cx.clone());
|
||||
module.set_native_fn(
|
||||
"request",
|
||||
move |method: &str, url: &str, body: Dynamic, opts: Map| {
|
||||
invoke(&svc, &cx, method, url, Some(body), Some(&opts))
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// Parsed `opts` map.
|
||||
struct Opts {
|
||||
headers: BTreeMap<String, String>,
|
||||
timeout_ms: u32,
|
||||
follow_redirects: bool,
|
||||
max_redirects: u32,
|
||||
}
|
||||
|
||||
impl Default for Opts {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
headers: BTreeMap::new(),
|
||||
timeout_ms: DEFAULT_TIMEOUT_MS,
|
||||
follow_redirects: true,
|
||||
max_redirects: DEFAULT_MAX_REDIRECTS,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn parse_opts(opts: Option<&Map>) -> Result<Opts, Box<EvalAltResult>> {
|
||||
let mut out = Opts::default();
|
||||
let Some(map) = opts else {
|
||||
return Ok(out);
|
||||
};
|
||||
for key in map.keys() {
|
||||
if !ALLOWED_OPT_KEYS.contains(&key.as_str()) {
|
||||
return Err(err(format!("unknown option key: {key}")));
|
||||
}
|
||||
}
|
||||
if let Some(h) = map.get("headers") {
|
||||
let hm = h
|
||||
.clone()
|
||||
.try_cast::<Map>()
|
||||
.ok_or_else(|| err("headers must be a map".to_string()))?;
|
||||
for (k, v) in hm {
|
||||
out.headers.insert(k.to_string(), dyn_to_string(&v));
|
||||
}
|
||||
}
|
||||
if let Some(t) = map.get("timeout_ms") {
|
||||
let ms = t
|
||||
.as_int()
|
||||
.map_err(|_| err("timeout_ms must be an integer".to_string()))?;
|
||||
if ms > MAX_TIMEOUT_MS {
|
||||
return Err(err(format!(
|
||||
"timeout_ms {ms} exceeds the {MAX_TIMEOUT_MS}ms maximum"
|
||||
)));
|
||||
}
|
||||
if ms > 0 {
|
||||
out.timeout_ms = u32::try_from(ms).unwrap_or(u32::MAX);
|
||||
}
|
||||
}
|
||||
if let Some(f) = map.get("follow_redirects") {
|
||||
out.follow_redirects = f
|
||||
.as_bool()
|
||||
.map_err(|_| err("follow_redirects must be a bool".to_string()))?;
|
||||
}
|
||||
if let Some(m) = map.get("max_redirects") {
|
||||
let n = m
|
||||
.as_int()
|
||||
.map_err(|_| err("max_redirects must be an integer".to_string()))?;
|
||||
if n > MAX_REDIRECTS {
|
||||
return Err(err(format!(
|
||||
"max_redirects {n} exceeds the {MAX_REDIRECTS} maximum"
|
||||
)));
|
||||
}
|
||||
out.max_redirects = u32::try_from(n.max(0)).unwrap_or(0);
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// Encoded request body + the content-type chosen for it.
|
||||
type EncodedBody = (Option<Vec<u8>>, Option<String>);
|
||||
|
||||
/// Dispatch a positional body by Rhai type. Returns the encoded bytes +
|
||||
/// the chosen content-type. GET/HEAD callers pass `body = None`, so
|
||||
/// this is never reached for them.
|
||||
fn dispatch_body(body: Dynamic) -> Result<EncodedBody, Box<EvalAltResult>> {
|
||||
if body.is_unit() {
|
||||
return Ok((None, None));
|
||||
}
|
||||
if body.is_string() {
|
||||
let s = body.into_string().unwrap_or_default();
|
||||
return Ok((Some(s.into_bytes()), Some("text/plain".to_string())));
|
||||
}
|
||||
if body.is_map() || body.is_array() {
|
||||
let json = dynamic_to_json(&body);
|
||||
let bytes = serde_json::to_vec(&json)
|
||||
.map_err(|e| err(format!("could not encode JSON body: {e}")))?;
|
||||
return Ok((Some(bytes), Some("application/json".to_string())));
|
||||
}
|
||||
// Scalars (int/float/bool) → JSON-encode for consistency.
|
||||
let json = dynamic_to_json(&body);
|
||||
let bytes =
|
||||
serde_json::to_vec(&json).map_err(|e| err(format!("could not encode body: {e}")))?;
|
||||
Ok((Some(bytes), Some("application/json".to_string())))
|
||||
}
|
||||
|
||||
#[allow(clippy::needless_pass_by_value)]
|
||||
fn invoke(
|
||||
svc: &Arc<dyn HttpService>,
|
||||
cx: &Arc<SdkCallCx>,
|
||||
method: &str,
|
||||
url: &str,
|
||||
body: Option<Dynamic>,
|
||||
opts: Option<&Map>,
|
||||
) -> Result<Dynamic, Box<EvalAltResult>> {
|
||||
let opts = parse_opts(opts)?;
|
||||
let method_uc = method.to_ascii_uppercase();
|
||||
let bodyless = matches!(method_uc.as_str(), "GET" | "HEAD");
|
||||
let (encoded, content_type) = if bodyless {
|
||||
(None, None)
|
||||
} else if let Some(b) = body {
|
||||
dispatch_body(b)?
|
||||
} else {
|
||||
(None, None)
|
||||
};
|
||||
|
||||
let req = HttpRequest {
|
||||
method: method_uc,
|
||||
url: url.to_string(),
|
||||
headers: opts.headers,
|
||||
body: encoded,
|
||||
content_type,
|
||||
timeout_ms: opts.timeout_ms,
|
||||
follow_redirects: opts.follow_redirects,
|
||||
max_redirects: opts.max_redirects,
|
||||
script_id: Some(cx.script_id.to_string()),
|
||||
};
|
||||
let resp = block_on(svc, cx, req)?;
|
||||
Ok(response_to_dynamic(&resp))
|
||||
}
|
||||
|
||||
#[allow(clippy::needless_pass_by_value)]
|
||||
fn invoke_form(
|
||||
svc: &Arc<dyn HttpService>,
|
||||
cx: &Arc<SdkCallCx>,
|
||||
url: &str,
|
||||
form: &Map,
|
||||
opts: Option<&Map>,
|
||||
) -> Result<Dynamic, Box<EvalAltResult>> {
|
||||
let opts = parse_opts(opts)?;
|
||||
let mut serializer = url::form_urlencoded::Serializer::new(String::new());
|
||||
for (k, v) in form {
|
||||
serializer.append_pair(k.as_str(), &dyn_to_string(v));
|
||||
}
|
||||
let encoded = serializer.finish();
|
||||
|
||||
let req = HttpRequest {
|
||||
method: "POST".to_string(),
|
||||
url: url.to_string(),
|
||||
headers: opts.headers,
|
||||
body: Some(encoded.into_bytes()),
|
||||
content_type: Some("application/x-www-form-urlencoded".to_string()),
|
||||
timeout_ms: opts.timeout_ms,
|
||||
follow_redirects: opts.follow_redirects,
|
||||
max_redirects: opts.max_redirects,
|
||||
script_id: Some(cx.script_id.to_string()),
|
||||
};
|
||||
let resp = block_on(svc, cx, req)?;
|
||||
Ok(response_to_dynamic(&resp))
|
||||
}
|
||||
|
||||
fn response_to_dynamic(resp: &HttpResponse) -> Dynamic {
|
||||
let mut m = Map::new();
|
||||
m.insert("status".into(), i64::from(resp.status).into());
|
||||
|
||||
let mut headers = Map::new();
|
||||
let mut content_type = String::new();
|
||||
for (k, v) in &resp.headers {
|
||||
if k == "content-type" {
|
||||
content_type.clone_from(v);
|
||||
}
|
||||
headers.insert(k.clone().into(), v.clone().into());
|
||||
}
|
||||
m.insert("headers".into(), headers.into());
|
||||
|
||||
// `body`: parsed JSON when the response is JSON and parses; () when
|
||||
// empty; otherwise the raw string.
|
||||
let body = if resp.body_raw.is_empty() {
|
||||
Dynamic::UNIT
|
||||
} else if content_type
|
||||
.to_ascii_lowercase()
|
||||
.starts_with("application/json")
|
||||
{
|
||||
match serde_json::from_str::<serde_json::Value>(&resp.body_raw) {
|
||||
Ok(json) => json_to_dynamic(json),
|
||||
Err(_) => resp.body_raw.clone().into(),
|
||||
}
|
||||
} else {
|
||||
resp.body_raw.clone().into()
|
||||
};
|
||||
m.insert("body".into(), body);
|
||||
m.insert("body_raw".into(), resp.body_raw.clone().into());
|
||||
m.into()
|
||||
}
|
||||
|
||||
fn dyn_to_string(v: &Dynamic) -> String {
|
||||
if v.is_string() {
|
||||
v.clone().into_string().unwrap_or_default()
|
||||
} else {
|
||||
v.to_string()
|
||||
}
|
||||
}
|
||||
|
||||
// Rhai's native-fn error channel is `Box<EvalAltResult>`, so these
|
||||
// helpers return the boxed form the call sites need.
|
||||
#[allow(clippy::unnecessary_box_returns)]
|
||||
fn err(msg: String) -> Box<EvalAltResult> {
|
||||
EvalAltResult::ErrorRuntime(format!("http: {msg}").into(), rhai::Position::NONE).into()
|
||||
}
|
||||
|
||||
/// Run the async service call from the synchronous Rhai context. Same
|
||||
/// pattern as `kv`/`docs`: the script runs under `spawn_blocking`, so a
|
||||
/// runtime handle is reachable and blocking on it is correct.
|
||||
fn block_on(
|
||||
svc: &Arc<dyn HttpService>,
|
||||
cx: &Arc<SdkCallCx>,
|
||||
req: HttpRequest,
|
||||
) -> Result<HttpResponse, Box<EvalAltResult>> {
|
||||
let handle = TokioHandle::try_current().map_err(|e| -> Box<EvalAltResult> {
|
||||
EvalAltResult::ErrorRuntime(
|
||||
format!("http: no tokio runtime available: {e}").into(),
|
||||
rhai::Position::NONE,
|
||||
)
|
||||
.into()
|
||||
})?;
|
||||
let svc = svc.clone();
|
||||
let cx = cx.clone();
|
||||
handle
|
||||
.block_on(async move { svc.request(&cx, req).await })
|
||||
.map_err(map_http_err)
|
||||
}
|
||||
|
||||
#[allow(clippy::unnecessary_box_returns)]
|
||||
fn map_http_err(e: HttpError) -> Box<EvalAltResult> {
|
||||
EvalAltResult::ErrorRuntime(format!("http: {e}").into(), rhai::Position::NONE).into()
|
||||
}
|
||||
@@ -15,6 +15,7 @@ pub mod bridge;
|
||||
pub mod cx;
|
||||
pub mod dead_letters;
|
||||
pub mod docs;
|
||||
pub mod http;
|
||||
pub mod kv;
|
||||
pub mod stdlib;
|
||||
|
||||
@@ -35,5 +36,6 @@ use rhai::Engine as RhaiEngine;
|
||||
pub fn register_all(engine: &mut RhaiEngine, services: &Services, cx: Arc<SdkCallCx>) {
|
||||
kv::register(engine, services, cx.clone());
|
||||
docs::register(engine, services, cx.clone());
|
||||
dead_letters::register(engine, services, cx);
|
||||
dead_letters::register(engine, services, cx.clone());
|
||||
http::register(engine, services, cx);
|
||||
}
|
||||
|
||||
127
crates/executor-core/tests/module_redaction_logging.rs
Normal file
127
crates/executor-core/tests/module_redaction_logging.rs
Normal file
@@ -0,0 +1,127 @@
|
||||
//! v1.1.4 §10a: the original module backend error MUST be logged at
|
||||
//! error level (so operators can still diagnose), even though it is
|
||||
//! redacted from the script-visible error.
|
||||
//!
|
||||
//! This test owns the process-global tracing subscriber, so it lives in
|
||||
//! its own integration-test binary (one `set_global_default` per
|
||||
//! process). A unique sentinel in the backend error keeps the assertion
|
||||
//! robust against any concurrently-running test's log output.
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
use std::io::Write;
|
||||
use std::sync::{Arc, Mutex};
|
||||
|
||||
use async_trait::async_trait;
|
||||
use picloud_executor_core::{Engine, ExecRequest, InvocationType, Limits};
|
||||
use picloud_shared::{
|
||||
AppId, ExecutionId, ModuleScript, ModuleSource, ModuleSourceError, NoopDeadLetterService,
|
||||
NoopDocsService, NoopEventEmitter, NoopHttpService, NoopKvService, RequestId, ScriptId,
|
||||
ScriptSandbox, SdkCallCx, Services,
|
||||
};
|
||||
use serde_json::Value;
|
||||
use tracing_subscriber::fmt::MakeWriter;
|
||||
|
||||
const SENTINEL: &str = "connection refused PICLOUD-SENTINEL-9f3a";
|
||||
|
||||
struct FailingSource;
|
||||
|
||||
#[async_trait]
|
||||
impl ModuleSource for FailingSource {
|
||||
async fn lookup(
|
||||
&self,
|
||||
_cx: &SdkCallCx,
|
||||
_name: &str,
|
||||
) -> Result<Option<ModuleScript>, ModuleSourceError> {
|
||||
Err(ModuleSourceError::Backend(SENTINEL.to_string()))
|
||||
}
|
||||
}
|
||||
|
||||
/// `MakeWriter` that appends to a shared buffer.
|
||||
#[derive(Clone)]
|
||||
struct SharedBuf(Arc<Mutex<Vec<u8>>>);
|
||||
|
||||
impl Write for SharedBuf {
|
||||
fn write(&mut self, buf: &[u8]) -> std::io::Result<usize> {
|
||||
self.0.lock().unwrap().extend_from_slice(buf);
|
||||
Ok(buf.len())
|
||||
}
|
||||
fn flush(&mut self) -> std::io::Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
impl<'a> MakeWriter<'a> for SharedBuf {
|
||||
type Writer = SharedBuf;
|
||||
fn make_writer(&'a self) -> Self::Writer {
|
||||
self.clone()
|
||||
}
|
||||
}
|
||||
|
||||
fn req(app_id: AppId) -> ExecRequest {
|
||||
let execution_id = ExecutionId::new();
|
||||
ExecRequest {
|
||||
execution_id,
|
||||
request_id: RequestId::new(),
|
||||
script_id: ScriptId::new(),
|
||||
script_name: "redaction-test".into(),
|
||||
invocation_type: InvocationType::Http,
|
||||
path: "/x".into(),
|
||||
headers: BTreeMap::new(),
|
||||
body: Value::Null,
|
||||
params: BTreeMap::new(),
|
||||
query: BTreeMap::new(),
|
||||
rest: String::new(),
|
||||
sandbox_overrides: ScriptSandbox::default(),
|
||||
app_id,
|
||||
principal: None,
|
||||
trigger_depth: 0,
|
||||
root_execution_id: execution_id,
|
||||
is_dead_letter_handler: false,
|
||||
event: None,
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn original_backend_error_is_logged_at_error_level() {
|
||||
let buf = Arc::new(Mutex::new(Vec::<u8>::new()));
|
||||
let subscriber = tracing_subscriber::fmt()
|
||||
.with_writer(SharedBuf(buf.clone()))
|
||||
.with_max_level(tracing::Level::ERROR)
|
||||
.with_ansi(false)
|
||||
.finish();
|
||||
tracing::subscriber::set_global_default(subscriber)
|
||||
.expect("this test owns the global subscriber for its binary");
|
||||
|
||||
let services = Services::new(
|
||||
Arc::new(NoopKvService),
|
||||
Arc::new(NoopDocsService),
|
||||
Arc::new(NoopDeadLetterService),
|
||||
Arc::new(NoopEventEmitter),
|
||||
Arc::new(FailingSource),
|
||||
Arc::new(NoopHttpService),
|
||||
);
|
||||
let engine = Engine::new(Limits::default(), services);
|
||||
|
||||
let err = engine
|
||||
.execute(r#"import "x" as x; 1"#, req(AppId::new()))
|
||||
.expect_err("backend error should surface");
|
||||
|
||||
// Script-visible: redacted.
|
||||
let msg = format!("{err:?}");
|
||||
assert!(msg.contains("module backend unavailable"), "got {msg}");
|
||||
assert!(
|
||||
!msg.contains("PICLOUD-SENTINEL"),
|
||||
"script error leaked the original: {msg}"
|
||||
);
|
||||
|
||||
// Operator log: the original sentinel IS present, at ERROR level.
|
||||
let logged = String::from_utf8(buf.lock().unwrap().clone()).unwrap();
|
||||
assert!(
|
||||
logged.contains(SENTINEL),
|
||||
"original backend error should be logged; captured: {logged}"
|
||||
);
|
||||
assert!(
|
||||
logged.contains("ERROR"),
|
||||
"should be logged at error level; captured: {logged}"
|
||||
);
|
||||
}
|
||||
@@ -17,8 +17,8 @@ use chrono::{DateTime, Utc};
|
||||
use picloud_executor_core::{Engine, ExecRequest, InvocationType, Limits};
|
||||
use picloud_shared::{
|
||||
AppId, ExecutionId, ModuleScript, ModuleSource, ModuleSourceError, NoopDeadLetterService,
|
||||
NoopDocsService, NoopEventEmitter, NoopKvService, RequestId, ScriptId, ScriptSandbox,
|
||||
SdkCallCx, Services,
|
||||
NoopDocsService, NoopEventEmitter, NoopHttpService, NoopKvService, RequestId, ScriptId,
|
||||
ScriptSandbox, SdkCallCx, Services,
|
||||
};
|
||||
use tokio::sync::Mutex;
|
||||
|
||||
@@ -96,6 +96,7 @@ fn services_with(modules: Arc<dyn ModuleSource>) -> Services {
|
||||
Arc::new(NoopDeadLetterService),
|
||||
Arc::new(NoopEventEmitter),
|
||||
modules,
|
||||
Arc::new(NoopHttpService),
|
||||
)
|
||||
}
|
||||
|
||||
@@ -321,20 +322,28 @@ async fn resolver_runtime_validation_rejects_top_level_expr() {
|
||||
);
|
||||
}
|
||||
|
||||
/// v1.1.4 §10a regression: the backend error must be REDACTED before
|
||||
/// it reaches a script. The verbatim message (which can leak internal
|
||||
/// infrastructure shape, e.g. "connection refused") must not appear;
|
||||
/// the script sees only a stable generic.
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn resolver_backend_error_surfaces() {
|
||||
async fn resolver_backend_error_is_redacted_from_script() {
|
||||
let source = CountingModuleSource::new();
|
||||
let app_id = AppId::new();
|
||||
*source.fail_with.lock().await = Some("simulated db outage".into());
|
||||
*source.fail_with.lock().await = Some("connection refused to 10.1.2.3:5432".into());
|
||||
let engine = engine_with(source);
|
||||
|
||||
let err = engine
|
||||
.execute(r#"import "x" as x; 1"#, req(app_id))
|
||||
.expect_err("backend error should propagate");
|
||||
let msg = format!("{err:?}").to_lowercase();
|
||||
let msg = format!("{err:?}");
|
||||
assert!(
|
||||
msg.contains("simulated") || msg.contains("backend"),
|
||||
"expected backend-error message, got {msg}"
|
||||
msg.contains("module backend unavailable"),
|
||||
"expected redacted generic message, got {msg}"
|
||||
);
|
||||
assert!(
|
||||
!msg.contains("connection refused") && !msg.contains("10.1.2.3"),
|
||||
"redacted message must not leak the backend error, got {msg}"
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
@@ -11,8 +11,8 @@ use chrono::Utc;
|
||||
use picloud_executor_core::{Engine, ExecRequest, InvocationType, Limits};
|
||||
use picloud_shared::{
|
||||
AppId, DocId, DocRow, DocsError, DocsListPage, DocsService, ExecutionId, NoopDeadLetterService,
|
||||
NoopEventEmitter, NoopKvService, NoopModuleSource, RequestId, ScriptId, ScriptSandbox,
|
||||
SdkCallCx, Services,
|
||||
NoopEventEmitter, NoopHttpService, NoopKvService, NoopModuleSource, RequestId, ScriptId,
|
||||
ScriptSandbox, SdkCallCx, Services,
|
||||
};
|
||||
use serde_json::{json, Value};
|
||||
use tokio::sync::Mutex;
|
||||
@@ -227,6 +227,7 @@ fn make_engine() -> Arc<Engine> {
|
||||
Arc::new(NoopDeadLetterService),
|
||||
Arc::new(NoopEventEmitter),
|
||||
Arc::new(NoopModuleSource),
|
||||
Arc::new(NoopHttpService),
|
||||
);
|
||||
Arc::new(Engine::new(Limits::default(), services))
|
||||
}
|
||||
|
||||
334
crates/executor-core/tests/sdk_http.rs
Normal file
334
crates/executor-core/tests/sdk_http.rs
Normal file
@@ -0,0 +1,334 @@
|
||||
//! Bridge integration for the `http::*` SDK (v1.1.4).
|
||||
//!
|
||||
//! Runs a real Rhai engine under `spawn_blocking` against an in-memory
|
||||
//! `HttpService` fake that records the last request and returns a
|
||||
//! configured response (or error). This exercises the full bridge:
|
||||
//! option parsing, body dispatch, response→map projection, the
|
||||
//! throw-on-network-error / no-throw-on-non-2xx convention, and that
|
||||
//! `cx.app_id` / `cx.script_id` are forwarded for attribution.
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
use std::sync::{Arc, Mutex};
|
||||
|
||||
use async_trait::async_trait;
|
||||
use picloud_executor_core::{Engine, ExecRequest, InvocationType, Limits};
|
||||
use picloud_shared::{
|
||||
AppId, ExecutionId, HttpError, HttpRequest, HttpResponse, HttpService, NoopDeadLetterService,
|
||||
NoopDocsService, NoopEventEmitter, NoopKvService, NoopModuleSource, RequestId, ScriptId,
|
||||
ScriptSandbox, Services,
|
||||
};
|
||||
use serde_json::{json, Value};
|
||||
|
||||
/// What the fake returns. Either a canned response or an error.
|
||||
#[derive(Clone)]
|
||||
enum Behavior {
|
||||
Respond(HttpResponse),
|
||||
Fail(String), // becomes HttpError::Network
|
||||
}
|
||||
|
||||
#[derive(Default)]
|
||||
struct Recorded {
|
||||
last: Option<HttpRequest>,
|
||||
last_app: Option<AppId>,
|
||||
last_script: Option<String>,
|
||||
}
|
||||
|
||||
struct FakeHttp {
|
||||
behavior: Behavior,
|
||||
recorded: Mutex<Recorded>,
|
||||
}
|
||||
|
||||
impl FakeHttp {
|
||||
fn responding(status: u16, content_type: &str, body: &str) -> Arc<Self> {
|
||||
let mut headers = BTreeMap::new();
|
||||
headers.insert("content-type".into(), content_type.into());
|
||||
Arc::new(Self {
|
||||
behavior: Behavior::Respond(HttpResponse {
|
||||
status,
|
||||
headers,
|
||||
body_raw: body.into(),
|
||||
}),
|
||||
recorded: Mutex::new(Recorded::default()),
|
||||
})
|
||||
}
|
||||
|
||||
fn failing(msg: &str) -> Arc<Self> {
|
||||
Arc::new(Self {
|
||||
behavior: Behavior::Fail(msg.into()),
|
||||
recorded: Mutex::new(Recorded::default()),
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl HttpService for FakeHttp {
|
||||
async fn request(
|
||||
&self,
|
||||
cx: &picloud_shared::SdkCallCx,
|
||||
req: HttpRequest,
|
||||
) -> Result<HttpResponse, HttpError> {
|
||||
{
|
||||
let mut r = self.recorded.lock().unwrap();
|
||||
r.last = Some(req.clone());
|
||||
r.last_app = Some(cx.app_id);
|
||||
r.last_script = Some(cx.script_id.to_string());
|
||||
}
|
||||
match &self.behavior {
|
||||
Behavior::Respond(resp) => Ok(resp.clone()),
|
||||
Behavior::Fail(msg) => Err(HttpError::Network(msg.clone())),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn engine_with(http: Arc<dyn HttpService>) -> Arc<Engine> {
|
||||
let services = Services::new(
|
||||
Arc::new(NoopKvService),
|
||||
Arc::new(NoopDocsService),
|
||||
Arc::new(NoopDeadLetterService),
|
||||
Arc::new(NoopEventEmitter),
|
||||
Arc::new(NoopModuleSource),
|
||||
http,
|
||||
);
|
||||
Arc::new(Engine::new(Limits::default(), services))
|
||||
}
|
||||
|
||||
fn baseline_request(app_id: AppId, script_id: ScriptId) -> ExecRequest {
|
||||
let execution_id = ExecutionId::new();
|
||||
ExecRequest {
|
||||
execution_id,
|
||||
request_id: RequestId::new(),
|
||||
script_id,
|
||||
script_name: "http-test".into(),
|
||||
invocation_type: InvocationType::Http,
|
||||
path: "/http-test".into(),
|
||||
headers: BTreeMap::new(),
|
||||
body: Value::Null,
|
||||
params: BTreeMap::new(),
|
||||
query: BTreeMap::new(),
|
||||
rest: String::new(),
|
||||
sandbox_overrides: ScriptSandbox::default(),
|
||||
app_id,
|
||||
principal: None,
|
||||
trigger_depth: 0,
|
||||
root_execution_id: execution_id,
|
||||
is_dead_letter_handler: false,
|
||||
event: None,
|
||||
}
|
||||
}
|
||||
|
||||
async fn run(engine: Arc<Engine>, src: &str, req: ExecRequest) -> Value {
|
||||
let src = src.to_string();
|
||||
tokio::task::spawn_blocking(move || engine.execute(&src, req))
|
||||
.await
|
||||
.expect("spawn_blocking should not panic")
|
||||
.expect("script execution should succeed")
|
||||
.body
|
||||
}
|
||||
|
||||
async fn run_err(engine: Arc<Engine>, src: &str, req: ExecRequest) -> String {
|
||||
let src = src.to_string();
|
||||
let err = tokio::task::spawn_blocking(move || engine.execute(&src, req))
|
||||
.await
|
||||
.unwrap()
|
||||
.expect_err("script should throw");
|
||||
format!("{err:?}")
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn get_returns_status_and_json_body() {
|
||||
let http = FakeHttp::responding(200, "application/json", r#"{"ok":true,"n":7}"#);
|
||||
let engine = engine_with(http.clone());
|
||||
let src = r#"
|
||||
let r = http::get("https://api.example.com/x");
|
||||
#{ status: r.status, ok: r.body.ok, n: r.body.n }
|
||||
"#;
|
||||
let body = run(engine, src, baseline_request(AppId::new(), ScriptId::new())).await;
|
||||
assert_eq!(body, json!({ "status": 200, "ok": true, "n": 7 }));
|
||||
// GET carries no body.
|
||||
assert!(http
|
||||
.recorded
|
||||
.lock()
|
||||
.unwrap()
|
||||
.last
|
||||
.as_ref()
|
||||
.unwrap()
|
||||
.body
|
||||
.is_none());
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn non_json_body_stays_string() {
|
||||
let http = FakeHttp::responding(200, "text/plain", "plain text");
|
||||
let engine = engine_with(http);
|
||||
let src = r#"http::get("https://x/").body"#;
|
||||
let body = run(engine, src, baseline_request(AppId::new(), ScriptId::new())).await;
|
||||
assert_eq!(body, json!("plain text"));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn empty_body_is_unit() {
|
||||
let http = FakeHttp::responding(204, "text/plain", "");
|
||||
let engine = engine_with(http);
|
||||
let src = r#"
|
||||
let r = http::get("https://x/");
|
||||
#{ is_unit: r.body == (), raw: r.body_raw }
|
||||
"#;
|
||||
let body = run(engine, src, baseline_request(AppId::new(), ScriptId::new())).await;
|
||||
assert_eq!(body, json!({ "is_unit": true, "raw": "" }));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn post_map_body_is_json_encoded() {
|
||||
let http = FakeHttp::responding(200, "application/json", "{}");
|
||||
let engine = engine_with(http.clone());
|
||||
let src = r#"http::post("https://hooks/x", #{ text: "hello", n: 3 }).status"#;
|
||||
let _ = run(engine, src, baseline_request(AppId::new(), ScriptId::new())).await;
|
||||
let rec = http.recorded.lock().unwrap();
|
||||
let req = rec.last.as_ref().unwrap();
|
||||
assert_eq!(req.method, "POST");
|
||||
assert_eq!(req.content_type.as_deref(), Some("application/json"));
|
||||
let sent: Value = serde_json::from_slice(req.body.as_ref().unwrap()).unwrap();
|
||||
assert_eq!(sent, json!({ "text": "hello", "n": 3 }));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn post_string_body_is_text_plain() {
|
||||
let http = FakeHttp::responding(200, "text/plain", "ok");
|
||||
let engine = engine_with(http.clone());
|
||||
let src = r#"http::post("https://x/", "raw payload").status"#;
|
||||
let _ = run(engine, src, baseline_request(AppId::new(), ScriptId::new())).await;
|
||||
let rec = http.recorded.lock().unwrap();
|
||||
let req = rec.last.as_ref().unwrap();
|
||||
assert_eq!(req.content_type.as_deref(), Some("text/plain"));
|
||||
assert_eq!(req.body.as_deref(), Some(&b"raw payload"[..]));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn post_unit_body_sends_nothing() {
|
||||
let http = FakeHttp::responding(200, "text/plain", "ok");
|
||||
let engine = engine_with(http.clone());
|
||||
let src = r#"http::post("https://x/", ()).status"#;
|
||||
let _ = run(engine, src, baseline_request(AppId::new(), ScriptId::new())).await;
|
||||
assert!(http
|
||||
.recorded
|
||||
.lock()
|
||||
.unwrap()
|
||||
.last
|
||||
.as_ref()
|
||||
.unwrap()
|
||||
.body
|
||||
.is_none());
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn custom_headers_and_timeout_forwarded() {
|
||||
let http = FakeHttp::responding(200, "text/plain", "ok");
|
||||
let engine = engine_with(http.clone());
|
||||
let src = r#"
|
||||
http::get("https://x/", #{
|
||||
headers: #{ "Authorization": "Bearer t0ken" },
|
||||
timeout_ms: 4200,
|
||||
}).status
|
||||
"#;
|
||||
let _ = run(engine, src, baseline_request(AppId::new(), ScriptId::new())).await;
|
||||
let rec = http.recorded.lock().unwrap();
|
||||
let req = rec.last.as_ref().unwrap();
|
||||
assert_eq!(
|
||||
req.headers.get("Authorization").map(String::as_str),
|
||||
Some("Bearer t0ken")
|
||||
);
|
||||
assert_eq!(req.timeout_ms, 4200);
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn unknown_option_key_throws() {
|
||||
let http = FakeHttp::responding(200, "text/plain", "ok");
|
||||
let engine = engine_with(http);
|
||||
let src = r#"http::get("https://x/", #{ timeoutms: 1000 })"#; // typo
|
||||
let err = run_err(engine, src, baseline_request(AppId::new(), ScriptId::new())).await;
|
||||
assert!(err.contains("unknown option key"), "got {err}");
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn timeout_above_max_throws() {
|
||||
let http = FakeHttp::responding(200, "text/plain", "ok");
|
||||
let engine = engine_with(http);
|
||||
let src = r#"http::get("https://x/", #{ timeout_ms: 99999 })"#;
|
||||
let err = run_err(engine, src, baseline_request(AppId::new(), ScriptId::new())).await;
|
||||
assert!(err.contains("maximum"), "got {err}");
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn non_2xx_does_not_throw() {
|
||||
let http = FakeHttp::responding(503, "text/plain", "down");
|
||||
let engine = engine_with(http);
|
||||
let src = r#"http::get("https://x/").status"#;
|
||||
let body = run(engine, src, baseline_request(AppId::new(), ScriptId::new())).await;
|
||||
assert_eq!(body, json!(503));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn network_error_throws_with_http_prefix() {
|
||||
let http = FakeHttp::failing("connection refused");
|
||||
let engine = engine_with(http);
|
||||
let src = r#"http::get("https://x/")"#;
|
||||
let err = run_err(engine, src, baseline_request(AppId::new(), ScriptId::new())).await;
|
||||
assert!(err.contains("http:"), "expected http: prefix, got {err}");
|
||||
assert!(err.contains("connection refused"), "got {err}");
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn post_form_url_encodes() {
|
||||
let http = FakeHttp::responding(200, "text/plain", "ok");
|
||||
let engine = engine_with(http.clone());
|
||||
let src = r#"http::post_form("https://x/login", #{ user: "alice", pw: "p@ss word" }).status"#;
|
||||
let _ = run(engine, src, baseline_request(AppId::new(), ScriptId::new())).await;
|
||||
let rec = http.recorded.lock().unwrap();
|
||||
let req = rec.last.as_ref().unwrap();
|
||||
assert_eq!(
|
||||
req.content_type.as_deref(),
|
||||
Some("application/x-www-form-urlencoded")
|
||||
);
|
||||
let body = String::from_utf8(req.body.clone().unwrap()).unwrap();
|
||||
// order is map iteration order; assert both pairs present, encoded.
|
||||
assert!(body.contains("user=alice"), "got {body}");
|
||||
assert!(body.contains("pw=p%40ss+word"), "got {body}");
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn request_escape_hatch_arbitrary_method() {
|
||||
let http = FakeHttp::responding(200, "text/plain", "ok");
|
||||
let engine = engine_with(http.clone());
|
||||
let src = r#"http::request("OPTIONS", "https://x/").status"#;
|
||||
let _ = run(engine, src, baseline_request(AppId::new(), ScriptId::new())).await;
|
||||
assert_eq!(
|
||||
http.recorded.lock().unwrap().last.as_ref().unwrap().method,
|
||||
"OPTIONS"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn default_user_agent_carries_script_id() {
|
||||
let http = FakeHttp::responding(200, "text/plain", "ok");
|
||||
let engine = engine_with(http.clone());
|
||||
let script_id = ScriptId::new();
|
||||
let src = r#"http::get("https://x/").status"#;
|
||||
let _ = run(engine, src, baseline_request(AppId::new(), script_id)).await;
|
||||
let rec = http.recorded.lock().unwrap();
|
||||
// The bridge forwards script_id on the request; the manager-core
|
||||
// impl turns it into the User-Agent. Here we assert the forward.
|
||||
assert_eq!(
|
||||
rec.last.as_ref().unwrap().script_id.as_deref(),
|
||||
Some(script_id.to_string().as_str())
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn cx_app_id_forwarded_for_attribution() {
|
||||
let http = FakeHttp::responding(200, "text/plain", "ok");
|
||||
let engine = engine_with(http.clone());
|
||||
let app = AppId::new();
|
||||
let src = r#"http::get("https://x/").status"#;
|
||||
let _ = run(engine, src, baseline_request(app, ScriptId::new())).await;
|
||||
assert_eq!(http.recorded.lock().unwrap().last_app, Some(app));
|
||||
}
|
||||
@@ -11,7 +11,8 @@ use async_trait::async_trait;
|
||||
use picloud_executor_core::{Engine, ExecRequest, InvocationType, Limits};
|
||||
use picloud_shared::{
|
||||
AppId, ExecutionId, KvError, KvListPage, KvService, NoopDeadLetterService, NoopDocsService,
|
||||
NoopEventEmitter, NoopModuleSource, RequestId, ScriptId, ScriptSandbox, SdkCallCx, Services,
|
||||
NoopEventEmitter, NoopHttpService, NoopModuleSource, RequestId, ScriptId, ScriptSandbox,
|
||||
SdkCallCx, Services,
|
||||
};
|
||||
use serde_json::{json, Value};
|
||||
use tokio::sync::Mutex;
|
||||
@@ -105,6 +106,7 @@ fn make_engine() -> Arc<Engine> {
|
||||
Arc::new(NoopDeadLetterService),
|
||||
Arc::new(NoopEventEmitter),
|
||||
Arc::new(NoopModuleSource),
|
||||
Arc::new(NoopHttpService),
|
||||
);
|
||||
Arc::new(Engine::new(Limits::default(), services))
|
||||
}
|
||||
|
||||
@@ -23,8 +23,11 @@ tokio.workspace = true
|
||||
tracing.workspace = true
|
||||
uuid.workspace = true
|
||||
chrono.workspace = true
|
||||
chrono-tz.workspace = true
|
||||
cron.workspace = true
|
||||
sqlx.workspace = true
|
||||
url.workspace = true
|
||||
reqwest.workspace = true
|
||||
|
||||
argon2.workspace = true
|
||||
sha2.workspace = true
|
||||
|
||||
43
crates/manager-core/migrations/0017_cron_triggers.sql
Normal file
43
crates/manager-core/migrations/0017_cron_triggers.sql
Normal file
@@ -0,0 +1,43 @@
|
||||
-- v1.1.4: Extend the triggers framework to recognise `cron` as the
|
||||
-- fourth concrete kind (after `kv` v1.1.1, `dead_letter` v1.1.1, `docs`
|
||||
-- v1.1.2). Mirrors the 0014 docs extension: two CHECK constraints widen
|
||||
-- (strictly gaining `'cron'`), one new detail table.
|
||||
--
|
||||
-- Cron rows route through the SAME generic dispatcher path as kv/docs/
|
||||
-- dead_letter (single match-arm extension on the Rust side). The only
|
||||
-- new machinery is a scheduler task that enqueues due cron triggers
|
||||
-- into the outbox; dispatch itself is unchanged.
|
||||
|
||||
-- Extend triggers.kind to include 'cron'. No existing row carries a
|
||||
-- value outside the widened set, so the drop+add is safe.
|
||||
ALTER TABLE triggers DROP CONSTRAINT triggers_kind_check;
|
||||
ALTER TABLE triggers ADD CONSTRAINT triggers_kind_check
|
||||
CHECK (kind IN ('kv', 'dead_letter', 'docs', 'cron'));
|
||||
|
||||
-- Extend outbox.source_kind to include 'cron'. v1.1.x's existing
|
||||
-- source_kinds ('http', 'kv', 'dead_letter', 'docs') stay.
|
||||
ALTER TABLE outbox DROP CONSTRAINT outbox_source_kind_check;
|
||||
ALTER TABLE outbox ADD CONSTRAINT outbox_source_kind_check
|
||||
CHECK (source_kind IN ('http', 'kv', 'dead_letter', 'docs', 'cron'));
|
||||
|
||||
-- One row per cron trigger.
|
||||
-- schedule — 6-field cron expression (with seconds), validated
|
||||
-- at insert time by the `cron` crate.
|
||||
-- timezone — IANA tz name (e.g. "America/Los_Angeles"), validated
|
||||
-- via chrono-tz. Required so schedules like "every
|
||||
-- weekday at 9am" are unambiguous. Defaults to UTC.
|
||||
-- last_fired_at — set transactionally with each enqueue. NULL until
|
||||
-- the trigger first fires. The scheduler computes the
|
||||
-- next fire time in-process from
|
||||
-- (schedule, timezone, last_fired_at); there is no
|
||||
-- stored next_fire column (kept stateless on purpose).
|
||||
CREATE TABLE cron_trigger_details (
|
||||
trigger_id UUID PRIMARY KEY REFERENCES triggers(id) ON DELETE CASCADE,
|
||||
schedule TEXT NOT NULL,
|
||||
timezone TEXT NOT NULL DEFAULT 'UTC',
|
||||
last_fired_at TIMESTAMPTZ
|
||||
);
|
||||
|
||||
-- Hot lookup for the scheduler: "all enabled cron triggers due now"
|
||||
-- scans by last_fired_at.
|
||||
CREATE INDEX idx_cron_triggers_due ON cron_trigger_details (last_fired_at);
|
||||
@@ -72,6 +72,12 @@ pub enum Capability {
|
||||
/// shape as KV write — granted to `editor`+, maps to
|
||||
/// `script:write` on API keys.
|
||||
AppDocsWrite(AppId),
|
||||
/// Make an outbound HTTP request from a script in this app
|
||||
/// (v1.1.4). Maps to `script:write` on API keys: any outbound
|
||||
/// request can exfiltrate data — including read methods like GET —
|
||||
/// so the conservative write mapping is correct. Splitting
|
||||
/// read/write is a v1.2+ refinement. Granted to `editor`+.
|
||||
AppHttpRequest(AppId),
|
||||
/// Create / list / delete triggers for this app (v1.1.1). Maps to
|
||||
/// `app:admin` on API keys — triggers are app-configuration acts
|
||||
/// rather than data-plane access. Granted to `app_admin`+.
|
||||
@@ -101,6 +107,7 @@ impl Capability {
|
||||
| Self::AppKvWrite(id)
|
||||
| Self::AppDocsRead(id)
|
||||
| Self::AppDocsWrite(id)
|
||||
| Self::AppHttpRequest(id)
|
||||
| Self::AppManageTriggers(id)
|
||||
| Self::AppDeadLetterManage(id) => Some(id),
|
||||
}
|
||||
@@ -118,9 +125,10 @@ impl Capability {
|
||||
Scope::InstanceAdmin
|
||||
}
|
||||
Self::AppRead(_) | Self::AppKvRead(_) | Self::AppDocsRead(_) => Scope::ScriptRead,
|
||||
Self::AppWriteScript(_) | Self::AppKvWrite(_) | Self::AppDocsWrite(_) => {
|
||||
Scope::ScriptWrite
|
||||
}
|
||||
Self::AppWriteScript(_)
|
||||
| Self::AppKvWrite(_)
|
||||
| Self::AppDocsWrite(_)
|
||||
| Self::AppHttpRequest(_) => Scope::ScriptWrite,
|
||||
Self::AppWriteRoute(_) => Scope::RouteWrite,
|
||||
Self::AppManageDomains(_) => Scope::DomainManage,
|
||||
Self::AppAdmin(_) | Self::AppManageTriggers(_) | Self::AppDeadLetterManage(_) => {
|
||||
@@ -277,6 +285,7 @@ const fn role_satisfies(role: AppRole, cap: Capability) -> bool {
|
||||
| Capability::AppWriteRoute(_)
|
||||
| Capability::AppKvWrite(_)
|
||||
| Capability::AppDocsWrite(_)
|
||||
| Capability::AppHttpRequest(_)
|
||||
);
|
||||
let in_app_admin = in_editor
|
||||
|| matches!(
|
||||
|
||||
297
crates/manager-core/src/cron_scheduler.rs
Normal file
297
crates/manager-core/src/cron_scheduler.rs
Normal file
@@ -0,0 +1,297 @@
|
||||
//! Cron scheduler — the v1.1.4 time-based trigger source.
|
||||
//!
|
||||
//! A single tokio task polls `cron_trigger_details` on a tick (default
|
||||
//! 30s; `PICLOUD_CRON_TICK_INTERVAL_MS`). For each enabled cron trigger
|
||||
//! whose next scheduled fire is due, it enqueues ONE outbox row
|
||||
//! (`source_kind = 'cron'`) and updates `last_fired_at` — both in the
|
||||
//! same transaction, claimed via `FOR UPDATE SKIP LOCKED` so a future
|
||||
//! multi-node deploy can't double-fire.
|
||||
//!
|
||||
//! The scheduler does NOT dispatch or touch the `ExecutionGate`: it only
|
||||
//! enqueues. The existing dispatcher picks the row up and acquires the
|
||||
//! gate exactly as it does for kv/docs/dead_letter rows.
|
||||
//!
|
||||
//! **Catch-up policy (matches the brief):** a trigger that missed N fire
|
||||
//! windows since `last_fired_at` fires exactly ONCE on the next tick,
|
||||
//! not N times. This falls out of the design: [`next_due`] returns a
|
||||
//! single canonical scheduled time (the first slot after the reference
|
||||
//! point), and after firing we set `last_fired_at = now`, so the next
|
||||
//! tick computes from `now` and sees only future slots. Backfilling
|
||||
//! missed windows is intentionally out of scope (an explicit replay
|
||||
//! action is the v1.2+ escape hatch).
|
||||
|
||||
use std::str::FromStr;
|
||||
use std::time::Duration;
|
||||
|
||||
use chrono::{DateTime, Utc};
|
||||
use chrono_tz::Tz;
|
||||
use cron::Schedule;
|
||||
use picloud_shared::TriggerEvent;
|
||||
use sqlx::PgPool;
|
||||
use uuid::Uuid;
|
||||
|
||||
/// Validate a 6-field cron expression. Returns the parse error message
|
||||
/// on failure.
|
||||
///
|
||||
/// # Errors
|
||||
///
|
||||
/// Returns the underlying parse error string when `schedule` is not a
|
||||
/// valid cron expression.
|
||||
pub fn validate_schedule(schedule: &str) -> Result<(), String> {
|
||||
Schedule::from_str(schedule)
|
||||
.map(|_| ())
|
||||
.map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
/// Validate an IANA timezone name (e.g. `America/Los_Angeles`).
|
||||
///
|
||||
/// # Errors
|
||||
///
|
||||
/// Returns an error string when `timezone` is not a known IANA name.
|
||||
pub fn validate_timezone(timezone: &str) -> Result<(), String> {
|
||||
Tz::from_str(timezone)
|
||||
.map(|_| ())
|
||||
.map_err(|_| format!("unknown IANA timezone: {timezone}"))
|
||||
}
|
||||
|
||||
/// Compute whether a cron trigger is due, and if so its canonical
|
||||
/// scheduled-at moment (UTC).
|
||||
///
|
||||
/// Returns `Some(scheduled_at)` when the first scheduled slot after the
|
||||
/// reference point (`last_fired_at`, or `created_at` if never fired) is
|
||||
/// at/before `now`; `None` otherwise. Returns `None` if the schedule or
|
||||
/// timezone fails to parse (the row is skipped — it should never have
|
||||
/// been inserted, since the admin endpoint validates).
|
||||
#[must_use]
|
||||
pub fn next_due(
|
||||
schedule: &str,
|
||||
timezone: &str,
|
||||
last_fired_at: Option<DateTime<Utc>>,
|
||||
created_at: DateTime<Utc>,
|
||||
now: DateTime<Utc>,
|
||||
) -> Option<DateTime<Utc>> {
|
||||
let sched = Schedule::from_str(schedule).ok()?;
|
||||
let tz = Tz::from_str(timezone).ok()?;
|
||||
// Reference: the last actual fire, or creation if never fired. A
|
||||
// never-fired trigger fires at its first slot at/after creation.
|
||||
let base = last_fired_at.unwrap_or(created_at);
|
||||
let base_tz = base.with_timezone(&tz);
|
||||
let next = sched.after(&base_tz).next()?;
|
||||
let next_utc = next.with_timezone(&Utc);
|
||||
(next_utc <= now).then_some(next_utc)
|
||||
}
|
||||
|
||||
/// Spawn the scheduler loop. Runs for the process lifetime.
|
||||
pub fn spawn_cron_scheduler(pool: PgPool, tick_interval_ms: u32) {
|
||||
// Floor the tick at 1s so a misconfigured 0 can't spin.
|
||||
let interval = Duration::from_millis(u64::from(tick_interval_ms).max(1_000));
|
||||
tokio::spawn(async move {
|
||||
let mut ticker = tokio::time::interval(interval);
|
||||
// Skip the immediate first fire so we don't race startup.
|
||||
ticker.tick().await;
|
||||
loop {
|
||||
ticker.tick().await;
|
||||
if let Err(e) = tick(&pool, Utc::now()).await {
|
||||
tracing::warn!(?e, "cron scheduler tick errored");
|
||||
}
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
#[derive(sqlx::FromRow)]
|
||||
struct DueRow {
|
||||
id: Uuid,
|
||||
app_id: Uuid,
|
||||
script_id: Uuid,
|
||||
registered_by_principal: Uuid,
|
||||
created_at: DateTime<Utc>,
|
||||
schedule: String,
|
||||
timezone: String,
|
||||
last_fired_at: Option<DateTime<Utc>>,
|
||||
}
|
||||
|
||||
/// One scheduler tick: claim enabled cron rows, enqueue the due ones,
|
||||
/// bump `last_fired_at`. Returns the number of triggers fired.
|
||||
async fn tick(pool: &PgPool, now: DateTime<Utc>) -> Result<usize, sqlx::Error> {
|
||||
let mut tx = pool.begin().await?;
|
||||
let rows: Vec<DueRow> = sqlx::query_as(
|
||||
"SELECT t.id, t.app_id, t.script_id, t.registered_by_principal, t.created_at, \
|
||||
d.schedule, d.timezone, d.last_fired_at \
|
||||
FROM cron_trigger_details d \
|
||||
JOIN triggers t ON t.id = d.trigger_id \
|
||||
WHERE t.enabled = TRUE \
|
||||
FOR UPDATE OF d SKIP LOCKED",
|
||||
)
|
||||
.fetch_all(&mut *tx)
|
||||
.await?;
|
||||
|
||||
let mut fired = 0usize;
|
||||
for r in rows {
|
||||
let Some(scheduled_at) =
|
||||
next_due(&r.schedule, &r.timezone, r.last_fired_at, r.created_at, now)
|
||||
else {
|
||||
continue;
|
||||
};
|
||||
|
||||
let event = TriggerEvent::Cron {
|
||||
schedule: r.schedule.clone(),
|
||||
timezone: r.timezone.clone(),
|
||||
scheduled_at,
|
||||
fired_at: now,
|
||||
};
|
||||
let payload = serde_json::to_value(&event)
|
||||
.map_err(|e| sqlx::Error::Decode(Box::new(std::io::Error::other(e))))?;
|
||||
|
||||
// Enqueue exactly one outbox row. Relies on the same column
|
||||
// defaults the OutboxEventEmitter uses (next_attempt_at = NOW(),
|
||||
// attempt_count = 0, claimed_at NULL → immediately due).
|
||||
sqlx::query(
|
||||
"INSERT INTO outbox \
|
||||
(app_id, source_kind, trigger_id, script_id, payload, \
|
||||
origin_principal, trigger_depth) \
|
||||
VALUES ($1, 'cron', $2, $3, $4, $5, 0)",
|
||||
)
|
||||
.bind(r.app_id)
|
||||
.bind(r.id)
|
||||
.bind(r.script_id)
|
||||
.bind(payload)
|
||||
.bind(r.registered_by_principal)
|
||||
.execute(&mut *tx)
|
||||
.await?;
|
||||
|
||||
sqlx::query("UPDATE cron_trigger_details SET last_fired_at = $2 WHERE trigger_id = $1")
|
||||
.bind(r.id)
|
||||
.bind(now)
|
||||
.execute(&mut *tx)
|
||||
.await?;
|
||||
|
||||
fired += 1;
|
||||
}
|
||||
|
||||
tx.commit().await?;
|
||||
Ok(fired)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use chrono::TimeZone;
|
||||
|
||||
#[test]
|
||||
fn valid_six_field_schedule_accepted() {
|
||||
// sec min hour dom mon dow — "every weekday at 9am".
|
||||
validate_schedule("0 0 9 * * MON-FRI").unwrap();
|
||||
validate_schedule("*/5 * * * * *").unwrap();
|
||||
validate_schedule("0 0 0 1 1 *").unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn invalid_schedules_rejected() {
|
||||
// 5-field (no seconds) is not the format we accept.
|
||||
assert!(validate_schedule("* * * * *").is_err());
|
||||
// Gibberish.
|
||||
assert!(validate_schedule("not a cron").is_err());
|
||||
assert!(validate_schedule("").is_err());
|
||||
// Out-of-range hour.
|
||||
assert!(validate_schedule("0 0 99 * * *").is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn known_timezones_accepted() {
|
||||
validate_timezone("UTC").unwrap();
|
||||
validate_timezone("America/Los_Angeles").unwrap();
|
||||
validate_timezone("Europe/Berlin").unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn unknown_timezones_rejected() {
|
||||
assert!(validate_timezone("Mars/Phobos").is_err());
|
||||
assert!(validate_timezone("PST").is_err()); // abbreviations aren't IANA names
|
||||
assert!(validate_timezone("").is_err());
|
||||
}
|
||||
|
||||
fn ts(s: &str) -> DateTime<Utc> {
|
||||
DateTime::parse_from_rfc3339(s).unwrap().with_timezone(&Utc)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn due_when_next_slot_is_at_or_before_now() {
|
||||
// Every minute at second 0. Last fired 90s ago → the next slot
|
||||
// after that is due now.
|
||||
let created = ts("2026-06-01T00:00:00Z");
|
||||
let last = Some(ts("2026-06-15T11:58:10Z"));
|
||||
let now = ts("2026-06-15T12:00:05Z");
|
||||
let due = next_due("0 * * * * *", "UTC", last, created, now);
|
||||
assert_eq!(due, Some(ts("2026-06-15T11:59:00Z")));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn not_due_when_next_slot_is_in_the_future() {
|
||||
let created = ts("2026-06-01T00:00:00Z");
|
||||
let last = Some(ts("2026-06-15T12:00:00Z"));
|
||||
let now = ts("2026-06-15T12:00:30Z");
|
||||
// Next minute slot is 12:01:00 — still in the future.
|
||||
assert_eq!(next_due("0 * * * * *", "UTC", last, created, now), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn never_fired_uses_created_at_as_reference() {
|
||||
let created = ts("2026-06-15T12:00:10Z");
|
||||
let now = ts("2026-06-15T12:01:30Z");
|
||||
// First slot after creation is 12:01:00, which is <= now → due.
|
||||
let due = next_due("0 * * * * *", "UTC", None, created, now);
|
||||
assert_eq!(due, Some(ts("2026-06-15T12:01:00Z")));
|
||||
}
|
||||
|
||||
/// Catch-up policy: a trigger that missed many windows fires exactly
|
||||
/// ONCE. We simulate two consecutive scheduler ticks the way the DB
|
||||
/// loop does — fire once, set last_fired = now, then re-evaluate.
|
||||
#[test]
|
||||
fn catch_up_fires_exactly_once_after_missed_windows() {
|
||||
let created = ts("2026-06-15T09:00:00Z");
|
||||
// Last fired over 5 minutes (5 windows) ago.
|
||||
let mut last_fired = Some(ts("2026-06-15T11:54:30Z"));
|
||||
let now = ts("2026-06-15T12:00:05Z");
|
||||
|
||||
// Tick 1: due → fire once, advance last_fired to `now`.
|
||||
let first = next_due("0 * * * * *", "UTC", last_fired, created, now);
|
||||
assert!(first.is_some(), "should be due after missing windows");
|
||||
last_fired = Some(now);
|
||||
|
||||
// Tick 2 (same wall-clock): NOT due again — only one fire total,
|
||||
// not one-per-missed-window.
|
||||
let second = next_due("0 * * * * *", "UTC", last_fired, created, now);
|
||||
assert_eq!(second, None, "catch-up must fire exactly once");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn timezone_affects_fire_time() {
|
||||
// "9am every day" in Los Angeles. On 2026-06-15, PDT = UTC-7, so
|
||||
// 09:00 local = 16:00 UTC.
|
||||
let created = ts("2026-06-15T00:00:00Z");
|
||||
let last = Some(ts("2026-06-15T15:59:00Z"));
|
||||
let now = ts("2026-06-15T16:00:30Z");
|
||||
let due = next_due("0 0 9 * * *", "America/Los_Angeles", last, created, now);
|
||||
assert_eq!(due, Some(ts("2026-06-15T16:00:00Z")));
|
||||
// Sanity: the same expression in UTC would NOT be due at 16:00.
|
||||
assert_eq!(next_due("0 0 9 * * *", "UTC", last, created, now), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn bad_schedule_or_tz_yields_none() {
|
||||
let created = ts("2026-06-15T00:00:00Z");
|
||||
let now = ts("2026-06-15T12:00:00Z");
|
||||
assert_eq!(next_due("garbage", "UTC", None, created, now), None);
|
||||
assert_eq!(
|
||||
next_due("0 * * * * *", "Mars/Phobos", None, created, now),
|
||||
None
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn utc_offset_constructor_smoke() {
|
||||
// Guard the chrono TimeZone import is actually exercised.
|
||||
let dt = Utc.with_ymd_and_hms(2026, 6, 15, 12, 0, 0).unwrap();
|
||||
assert_eq!(dt, ts("2026-06-15T12:00:00Z"));
|
||||
}
|
||||
}
|
||||
@@ -208,6 +208,9 @@ async fn resolve(
|
||||
fn admin_cx(app_id: AppId, principal: &Principal) -> SdkCallCx {
|
||||
SdkCallCx {
|
||||
app_id,
|
||||
// Admin-plane cx (dead-letter replay/resolve) — no script is
|
||||
// executing, so this attribution id is a fresh sentinel.
|
||||
script_id: picloud_shared::ScriptId::new(),
|
||||
principal: Some(principal.clone()),
|
||||
execution_id: picloud_shared::ExecutionId::new(),
|
||||
request_id: picloud_shared::RequestId::new(),
|
||||
|
||||
@@ -163,7 +163,10 @@ impl Dispatcher {
|
||||
return Ok(());
|
||||
}
|
||||
},
|
||||
OutboxSourceKind::Kv | OutboxSourceKind::Docs | OutboxSourceKind::DeadLetter => {
|
||||
OutboxSourceKind::Kv
|
||||
| OutboxSourceKind::Docs
|
||||
| OutboxSourceKind::DeadLetter
|
||||
| OutboxSourceKind::Cron => {
|
||||
let resolved = self.resolve_trigger(&row).await?;
|
||||
let req = match self.build_exec_request(&row, &resolved).await {
|
||||
Ok(req) => req,
|
||||
|
||||
@@ -272,7 +272,7 @@ mod tests {
|
||||
use chrono::Utc;
|
||||
use picloud_shared::{
|
||||
AdminUserId, AppId, AppRole, ExecutionId, InstanceRole, NoopEventEmitter, Principal,
|
||||
RequestId, UserId,
|
||||
RequestId, ScriptId, UserId,
|
||||
};
|
||||
use serde_json::json;
|
||||
use std::collections::BTreeMap;
|
||||
@@ -507,6 +507,7 @@ mod tests {
|
||||
fn anon_cx(app_id: AppId) -> SdkCallCx {
|
||||
SdkCallCx {
|
||||
app_id,
|
||||
script_id: ScriptId::new(),
|
||||
principal: None,
|
||||
execution_id: ExecutionId::new(),
|
||||
request_id: RequestId::new(),
|
||||
@@ -520,6 +521,7 @@ mod tests {
|
||||
fn owner_cx(app_id: AppId) -> SdkCallCx {
|
||||
SdkCallCx {
|
||||
app_id,
|
||||
script_id: ScriptId::new(),
|
||||
principal: Some(Principal {
|
||||
user_id: AdminUserId::new(),
|
||||
instance_role: InstanceRole::Owner,
|
||||
@@ -538,6 +540,7 @@ mod tests {
|
||||
fn member_no_role_cx(app_id: AppId) -> SdkCallCx {
|
||||
SdkCallCx {
|
||||
app_id,
|
||||
script_id: ScriptId::new(),
|
||||
principal: Some(Principal {
|
||||
user_id: AdminUserId::new(),
|
||||
instance_role: InstanceRole::Member,
|
||||
|
||||
793
crates/manager-core/src/http_service.rs
Normal file
793
crates/manager-core/src/http_service.rs
Normal file
@@ -0,0 +1,793 @@
|
||||
//! `HttpServiceImpl` — reqwest-backed outbound HTTP for the v1.1.4
|
||||
//! `http::*` SDK.
|
||||
//!
|
||||
//! Mirrors the v1.1.1+ stateful-service shape (`KvServiceImpl`):
|
||||
//! script-as-gate authz (`AppHttpRequest`, skipped when
|
||||
//! `cx.principal` is `None`), with the backend talking to the network
|
||||
//! instead of Postgres. The reqwest client is built once at startup
|
||||
//! with the [`crate::ssrf::SsrfResolver`] wired in via
|
||||
//! `dns_resolver`, so the SSRF deny-list applies at every connection —
|
||||
//! including each redirect hop, since redirects are followed manually
|
||||
//! through the same client.
|
||||
//!
|
||||
//! Layering vs the raw client:
|
||||
//! 1. URL validation: scheme must be http/https; ports 22/25/465/587
|
||||
//! are blocked. (IP-level filtering is the resolver's job.)
|
||||
//! 2. Body-size caps on both request and response (stream-with-cap on
|
||||
//! the response, checking `Content-Length` first).
|
||||
//! 3. Total-request timeout (default 30s, max 60s) on top of the
|
||||
//! client's 10s connect timeout.
|
||||
//! 4. Default `User-Agent` unless the caller set one.
|
||||
//!
|
||||
//! Bodies/headers are never logged (PII): only url + status + duration
|
||||
//! at debug level.
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
use std::env;
|
||||
use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
|
||||
use async_trait::async_trait;
|
||||
use picloud_shared::{HttpError, HttpRequest, HttpResponse, HttpService, SdkCallCx};
|
||||
use reqwest::header::{HeaderMap, HeaderName, HeaderValue, CONTENT_TYPE, LOCATION, USER_AGENT};
|
||||
use reqwest::{Client, Method, StatusCode};
|
||||
|
||||
use crate::authz::{self, AuthzRepo, Capability};
|
||||
use crate::ssrf::{self, SsrfPolicy, SSRF_BLOCK_PREFIX};
|
||||
|
||||
/// Default per-request timeout (ms) when the script omits `timeout_ms`.
|
||||
pub const DEFAULT_TIMEOUT_MS: u32 = 30_000;
|
||||
/// Hard ceiling on the per-request timeout. Values above this are
|
||||
/// rejected by the bridge (not silently clamped).
|
||||
pub const MAX_TIMEOUT_MS: u32 = 60_000;
|
||||
/// Default redirect cap.
|
||||
pub const DEFAULT_MAX_REDIRECTS: u32 = 5;
|
||||
/// Hard ceiling on redirects.
|
||||
pub const MAX_REDIRECTS_CEILING: u32 = 10;
|
||||
/// 10 MB default body cap on both directions.
|
||||
const DEFAULT_BODY_LIMIT_BYTES: usize = 10 * 1024 * 1024;
|
||||
/// DNS + connect + TLS hard cap.
|
||||
const CONNECT_TIMEOUT: Duration = Duration::from_secs(10);
|
||||
|
||||
/// Outbound-HTTP tunables. Env-overridable following the same pattern
|
||||
/// as `TriggerConfig::from_env`.
|
||||
#[derive(Debug, Clone, Copy)]
|
||||
pub struct HttpConfig {
|
||||
/// Disables the SSRF deny-list entirely. Dev/test only — the binary
|
||||
/// logs a startup warning when this is set.
|
||||
pub allow_private: bool,
|
||||
pub max_request_body_bytes: usize,
|
||||
pub max_response_body_bytes: usize,
|
||||
}
|
||||
|
||||
impl HttpConfig {
|
||||
#[must_use]
|
||||
pub const fn conservative() -> Self {
|
||||
Self {
|
||||
allow_private: false,
|
||||
max_request_body_bytes: DEFAULT_BODY_LIMIT_BYTES,
|
||||
max_response_body_bytes: DEFAULT_BODY_LIMIT_BYTES,
|
||||
}
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
pub fn from_env() -> Self {
|
||||
let mut c = Self::conservative();
|
||||
if let Ok(v) = env::var("PICLOUD_HTTP_ALLOW_PRIVATE") {
|
||||
c.allow_private =
|
||||
matches!(v.trim().to_ascii_lowercase().as_str(), "1" | "true" | "yes");
|
||||
}
|
||||
load_usize(
|
||||
&mut c.max_request_body_bytes,
|
||||
"PICLOUD_HTTP_MAX_REQUEST_BODY_BYTES",
|
||||
);
|
||||
load_usize(
|
||||
&mut c.max_response_body_bytes,
|
||||
"PICLOUD_HTTP_MAX_RESPONSE_BODY_BYTES",
|
||||
);
|
||||
c
|
||||
}
|
||||
}
|
||||
|
||||
impl Default for HttpConfig {
|
||||
fn default() -> Self {
|
||||
Self::conservative()
|
||||
}
|
||||
}
|
||||
|
||||
fn load_usize(dst: &mut usize, key: &str) {
|
||||
if let Ok(v) = env::var(key) {
|
||||
match v.parse::<usize>() {
|
||||
Ok(n) => *dst = n,
|
||||
Err(e) => {
|
||||
tracing::warn!(env = key, error = %e, "ignoring invalid http-config value");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub struct HttpServiceImpl {
|
||||
client: Client,
|
||||
authz: Arc<dyn AuthzRepo>,
|
||||
config: HttpConfig,
|
||||
/// Same policy wired into the DNS resolver. Held here too because
|
||||
/// reqwest only routes *hostnames* through the custom resolver — a
|
||||
/// URL with a **literal IP** host bypasses it, so literal IPs are
|
||||
/// checked directly at URL-validation time.
|
||||
policy: SsrfPolicy,
|
||||
}
|
||||
|
||||
impl HttpServiceImpl {
|
||||
/// Build the service, constructing the reqwest client with the SSRF
|
||||
/// resolver. Redirects are followed manually (so per-request limits
|
||||
/// are honored and every hop re-resolves through the SSRF
|
||||
/// resolver), hence `redirect(Policy::none())`.
|
||||
///
|
||||
/// # Panics
|
||||
///
|
||||
/// Panics if the reqwest client fails to build — this is a
|
||||
/// startup-time invariant, not a runtime path.
|
||||
#[must_use]
|
||||
pub fn new(config: HttpConfig, authz: Arc<dyn AuthzRepo>) -> Self {
|
||||
let policy = SsrfPolicy::new(config.allow_private);
|
||||
let client = Client::builder()
|
||||
.dns_resolver(ssrf::resolver(policy))
|
||||
.connect_timeout(CONNECT_TIMEOUT)
|
||||
.redirect(reqwest::redirect::Policy::none())
|
||||
.build()
|
||||
.expect("build outbound http client");
|
||||
Self {
|
||||
client,
|
||||
authz,
|
||||
config,
|
||||
policy,
|
||||
}
|
||||
}
|
||||
|
||||
async fn check_request(&self, cx: &SdkCallCx) -> Result<(), HttpError> {
|
||||
if let Some(ref principal) = cx.principal {
|
||||
authz::require(
|
||||
&*self.authz,
|
||||
principal,
|
||||
Capability::AppHttpRequest(cx.app_id),
|
||||
)
|
||||
.await
|
||||
.map_err(|_| HttpError::Forbidden)?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl HttpService for HttpServiceImpl {
|
||||
async fn request(&self, cx: &SdkCallCx, req: HttpRequest) -> Result<HttpResponse, HttpError> {
|
||||
self.check_request(cx).await?;
|
||||
|
||||
// Request body cap.
|
||||
if let Some(ref body) = req.body {
|
||||
if body.len() > self.config.max_request_body_bytes {
|
||||
return Err(HttpError::BodyTooLarge("request"));
|
||||
}
|
||||
}
|
||||
|
||||
let timeout = Duration::from_millis(u64::from(req.timeout_ms.min(MAX_TIMEOUT_MS)));
|
||||
let started = std::time::Instant::now();
|
||||
let url_for_log = req.url.clone();
|
||||
|
||||
// Whole-request budget (DNS + connect + TLS + all redirect hops
|
||||
// + body read). Connect alone is further bounded by the
|
||||
// client's CONNECT_TIMEOUT.
|
||||
let outcome = match tokio::time::timeout(timeout, self.run(req)).await {
|
||||
Ok(r) => r,
|
||||
Err(_) => Err(HttpError::Timeout),
|
||||
};
|
||||
|
||||
let duration_ms = u64::try_from(started.elapsed().as_millis()).unwrap_or(u64::MAX);
|
||||
match &outcome {
|
||||
Ok(resp) => tracing::debug!(
|
||||
url = %url_for_log,
|
||||
status = resp.status,
|
||||
duration_ms,
|
||||
"outbound http"
|
||||
),
|
||||
Err(err) => tracing::debug!(
|
||||
url = %url_for_log,
|
||||
error = %err,
|
||||
duration_ms,
|
||||
"outbound http failed"
|
||||
),
|
||||
}
|
||||
outcome
|
||||
}
|
||||
}
|
||||
|
||||
impl HttpServiceImpl {
|
||||
/// Core request path: validate, build headers, follow redirects
|
||||
/// manually, read the response body with a cap.
|
||||
async fn run(&self, req: HttpRequest) -> Result<HttpResponse, HttpError> {
|
||||
let method = Method::from_bytes(req.method.as_bytes())
|
||||
.map_err(|_| HttpError::Backend(format!("invalid method: {}", req.method)))?;
|
||||
|
||||
let mut current = url::Url::parse(&req.url)
|
||||
.map_err(|e| HttpError::InvalidUrl(format!("{}: {e}", req.url)))?;
|
||||
validate_url(¤t, self.policy)?;
|
||||
|
||||
let mut header_map = build_headers(&req, ¤t)?;
|
||||
let mut method = method;
|
||||
let mut body = req.body.clone();
|
||||
let mut redirects: u32 = 0;
|
||||
let max_redirects = req.max_redirects.min(MAX_REDIRECTS_CEILING);
|
||||
|
||||
loop {
|
||||
// Re-validate scheme/port (and literal-IP SSRF) on each hop.
|
||||
// Hostname IP filtering is the resolver's job and runs
|
||||
// automatically at connect time.
|
||||
validate_url(¤t, self.policy)?;
|
||||
|
||||
let mut rb = self.client.request(method.clone(), current.clone());
|
||||
rb = rb.headers(header_map.clone());
|
||||
if let Some(ref b) = body {
|
||||
rb = rb.body(b.clone());
|
||||
}
|
||||
let resp = rb.send().await.map_err(map_reqwest_err)?;
|
||||
let status = resp.status();
|
||||
|
||||
if req.follow_redirects && is_redirect(status) {
|
||||
if let Some(loc) = resp.headers().get(LOCATION) {
|
||||
if redirects >= max_redirects {
|
||||
return Err(HttpError::Backend(format!(
|
||||
"too many redirects (max {max_redirects})"
|
||||
)));
|
||||
}
|
||||
redirects += 1;
|
||||
let loc_str = loc.to_str().map_err(|_| {
|
||||
HttpError::Backend("redirect Location not valid UTF-8".into())
|
||||
})?;
|
||||
current = current
|
||||
.join(loc_str)
|
||||
.map_err(|e| HttpError::InvalidUrl(format!("redirect target: {e}")))?;
|
||||
|
||||
// 303 always → GET; 301/302 historically downgrade
|
||||
// POST→GET (matches browsers). 307/308 preserve.
|
||||
if matches!(status.as_u16(), 301..=303) {
|
||||
method = Method::GET;
|
||||
body = None;
|
||||
header_map.remove(CONTENT_TYPE);
|
||||
}
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
return self.read_capped(resp).await;
|
||||
}
|
||||
}
|
||||
|
||||
async fn read_capped(&self, resp: reqwest::Response) -> Result<HttpResponse, HttpError> {
|
||||
let status = resp.status().as_u16();
|
||||
let mut headers = BTreeMap::new();
|
||||
for (name, value) in resp.headers() {
|
||||
// Header names lowercased per the documented response shape.
|
||||
headers.insert(
|
||||
name.as_str().to_ascii_lowercase(),
|
||||
value.to_str().unwrap_or("").to_string(),
|
||||
);
|
||||
}
|
||||
|
||||
let cap = self.config.max_response_body_bytes;
|
||||
if let Some(len) = resp.content_length() {
|
||||
if len > cap as u64 {
|
||||
return Err(HttpError::BodyTooLarge("response"));
|
||||
}
|
||||
}
|
||||
|
||||
let mut buf: Vec<u8> = Vec::new();
|
||||
let mut resp = resp;
|
||||
while let Some(chunk) = resp.chunk().await.map_err(map_reqwest_err)? {
|
||||
if buf.len() + chunk.len() > cap {
|
||||
return Err(HttpError::BodyTooLarge("response"));
|
||||
}
|
||||
buf.extend_from_slice(&chunk);
|
||||
}
|
||||
let body_raw = String::from_utf8_lossy(&buf).into_owned();
|
||||
Ok(HttpResponse {
|
||||
status,
|
||||
headers,
|
||||
body_raw,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// http/https only; block the SSH + SMTP ports; apply the SSRF policy
|
||||
/// to **literal-IP** hosts (hostnames are filtered by the DNS resolver
|
||||
/// at connect time, but literal IPs never reach the resolver).
|
||||
fn validate_url(url: &url::Url, policy: SsrfPolicy) -> Result<(), HttpError> {
|
||||
match url.scheme() {
|
||||
"http" | "https" => {}
|
||||
other => return Err(HttpError::BlockedScheme(other.to_string())),
|
||||
}
|
||||
match url.host() {
|
||||
None => return Err(HttpError::InvalidUrl("missing host".into())),
|
||||
Some(url::Host::Ipv4(ip)) => {
|
||||
policy
|
||||
.check(std::net::IpAddr::V4(ip))
|
||||
.map_err(|reason| HttpError::Ssrf(reason.to_string()))?;
|
||||
}
|
||||
Some(url::Host::Ipv6(ip)) => {
|
||||
policy
|
||||
.check(std::net::IpAddr::V6(ip))
|
||||
.map_err(|reason| HttpError::Ssrf(reason.to_string()))?;
|
||||
}
|
||||
Some(url::Host::Domain(_)) => {}
|
||||
}
|
||||
let port = url
|
||||
.port_or_known_default()
|
||||
.unwrap_or(if url.scheme() == "https" { 443 } else { 80 });
|
||||
if matches!(port, 22 | 25 | 465 | 587) {
|
||||
return Err(HttpError::BlockedPort(port));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Build the request header map: merge caller headers, then apply the
|
||||
/// default `User-Agent` (unless overridden) and the bridge-chosen
|
||||
/// `Content-Type` (unless overridden).
|
||||
fn build_headers(req: &HttpRequest, _url: &url::Url) -> Result<HeaderMap, HttpError> {
|
||||
let mut map = HeaderMap::new();
|
||||
let mut has_user_agent = false;
|
||||
let mut has_content_type = false;
|
||||
for (k, v) in &req.headers {
|
||||
let name = HeaderName::from_bytes(k.as_bytes())
|
||||
.map_err(|_| HttpError::Backend(format!("invalid header name: {k}")))?;
|
||||
let value = HeaderValue::from_str(v)
|
||||
.map_err(|_| HttpError::Backend(format!("invalid header value for {k}")))?;
|
||||
if name == USER_AGENT {
|
||||
has_user_agent = true;
|
||||
}
|
||||
if name == CONTENT_TYPE {
|
||||
has_content_type = true;
|
||||
}
|
||||
map.append(name, value);
|
||||
}
|
||||
|
||||
if !has_user_agent {
|
||||
let script = req.script_id.as_deref().unwrap_or("unknown");
|
||||
let ua = format!(
|
||||
"picloud/{} (script:{})",
|
||||
picloud_shared::PRODUCT_VERSION,
|
||||
script
|
||||
);
|
||||
if let Ok(value) = HeaderValue::from_str(&ua) {
|
||||
map.insert(USER_AGENT, value);
|
||||
}
|
||||
}
|
||||
|
||||
if !has_content_type {
|
||||
if let Some(ref ct) = req.content_type {
|
||||
if let Ok(value) = HeaderValue::from_str(ct) {
|
||||
map.insert(CONTENT_TYPE, value);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Ok(map)
|
||||
}
|
||||
|
||||
const fn is_redirect(status: StatusCode) -> bool {
|
||||
matches!(status.as_u16(), 301..=303 | 307 | 308)
|
||||
}
|
||||
|
||||
/// Map a reqwest error to an `HttpError`, never leaking the resolved
|
||||
/// IP. SSRF blocks are detected by scanning the error source chain for
|
||||
/// the resolver's marker prefix.
|
||||
fn map_reqwest_err(err: reqwest::Error) -> HttpError {
|
||||
if let Some(reason) = ssrf_reason(&err) {
|
||||
return HttpError::Ssrf(reason);
|
||||
}
|
||||
if err.is_timeout() {
|
||||
return HttpError::Timeout;
|
||||
}
|
||||
if err.is_connect() {
|
||||
return HttpError::Network("connection failed".into());
|
||||
}
|
||||
if err.is_request() {
|
||||
return HttpError::Network("request failed".into());
|
||||
}
|
||||
HttpError::Network("network error".into())
|
||||
}
|
||||
|
||||
/// Walk the error source chain looking for the SSRF marker the resolver
|
||||
/// embeds. Returns the category reason (no IP) when found.
|
||||
fn ssrf_reason(err: &reqwest::Error) -> Option<String> {
|
||||
let mut src: Option<&(dyn std::error::Error + 'static)> = Some(err);
|
||||
while let Some(e) = src {
|
||||
let s = e.to_string();
|
||||
if let Some(idx) = s.find(SSRF_BLOCK_PREFIX) {
|
||||
return Some(s[idx + SSRF_BLOCK_PREFIX.len()..].to_string());
|
||||
}
|
||||
src = e.source();
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::authz::AuthzError;
|
||||
use async_trait::async_trait;
|
||||
use picloud_shared::{
|
||||
AdminUserId, AppId, AppRole, ExecutionId, InstanceRole, Principal, RequestId, ScriptId,
|
||||
UserId,
|
||||
};
|
||||
use std::collections::BTreeMap;
|
||||
use std::io::Write as _;
|
||||
use std::net::SocketAddr;
|
||||
use tokio::io::{AsyncReadExt, AsyncWriteExt};
|
||||
use tokio::net::TcpListener;
|
||||
|
||||
struct AllowAuthz;
|
||||
#[async_trait]
|
||||
impl AuthzRepo for AllowAuthz {
|
||||
async fn membership(&self, _u: UserId, _a: AppId) -> Result<Option<AppRole>, AuthzError> {
|
||||
Ok(Some(AppRole::Editor))
|
||||
}
|
||||
}
|
||||
struct DenyAuthz;
|
||||
#[async_trait]
|
||||
impl AuthzRepo for DenyAuthz {
|
||||
async fn membership(&self, _u: UserId, _a: AppId) -> Result<Option<AppRole>, AuthzError> {
|
||||
Ok(None)
|
||||
}
|
||||
}
|
||||
|
||||
fn dev_service(authz: Arc<dyn AuthzRepo>) -> HttpServiceImpl {
|
||||
// allow_private so the test TcpListener on 127.0.0.1 is reachable.
|
||||
let mut config = HttpConfig::conservative();
|
||||
config.allow_private = true;
|
||||
HttpServiceImpl::new(config, authz)
|
||||
}
|
||||
|
||||
fn anon_cx() -> SdkCallCx {
|
||||
SdkCallCx {
|
||||
app_id: AppId::new(),
|
||||
script_id: ScriptId::new(),
|
||||
principal: None,
|
||||
execution_id: ExecutionId::new(),
|
||||
request_id: RequestId::new(),
|
||||
trigger_depth: 0,
|
||||
root_execution_id: ExecutionId::new(),
|
||||
is_dead_letter_handler: false,
|
||||
event: None,
|
||||
}
|
||||
}
|
||||
|
||||
fn member_cx() -> SdkCallCx {
|
||||
let mut cx = anon_cx();
|
||||
cx.principal = Some(Principal {
|
||||
user_id: AdminUserId::new(),
|
||||
instance_role: InstanceRole::Member,
|
||||
scopes: None,
|
||||
app_binding: None,
|
||||
});
|
||||
cx
|
||||
}
|
||||
|
||||
fn req(method: &str, url: String) -> HttpRequest {
|
||||
HttpRequest {
|
||||
method: method.into(),
|
||||
url,
|
||||
headers: BTreeMap::new(),
|
||||
body: None,
|
||||
content_type: None,
|
||||
timeout_ms: 5000,
|
||||
follow_redirects: true,
|
||||
max_redirects: 5,
|
||||
script_id: Some("test-script".into()),
|
||||
}
|
||||
}
|
||||
|
||||
/// Minimal single-shot HTTP/1.1 server. Reads the request, runs
|
||||
/// `handler` to produce the raw response bytes, writes them, closes.
|
||||
/// Returns the bound address.
|
||||
async fn spawn_server<F>(handler: F) -> SocketAddr
|
||||
where
|
||||
F: Fn(String) -> Vec<u8> + Send + Sync + 'static,
|
||||
{
|
||||
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
|
||||
let addr = listener.local_addr().unwrap();
|
||||
tokio::spawn(async move {
|
||||
loop {
|
||||
let Ok((mut sock, _)) = listener.accept().await else {
|
||||
break;
|
||||
};
|
||||
let mut buf = vec![0u8; 65536];
|
||||
let n = sock.read(&mut buf).await.unwrap_or(0);
|
||||
let request = String::from_utf8_lossy(&buf[..n]).to_string();
|
||||
let response = handler(request);
|
||||
let _ = sock.write_all(&response).await;
|
||||
let _ = sock.flush().await;
|
||||
}
|
||||
});
|
||||
addr
|
||||
}
|
||||
|
||||
fn ok_response(body: &str, content_type: &str) -> Vec<u8> {
|
||||
let mut v = Vec::new();
|
||||
write!(
|
||||
v,
|
||||
"HTTP/1.1 200 OK\r\nContent-Type: {content_type}\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{body}",
|
||||
body.len()
|
||||
)
|
||||
.unwrap();
|
||||
v
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn get_round_trip() {
|
||||
let addr = spawn_server(|_req| ok_response("hello", "text/plain")).await;
|
||||
let svc = dev_service(Arc::new(AllowAuthz));
|
||||
let resp = svc
|
||||
.request(&anon_cx(), req("GET", format!("http://{addr}/")))
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(resp.status, 200);
|
||||
assert_eq!(resp.body_raw, "hello");
|
||||
assert_eq!(
|
||||
resp.headers.get("content-type").map(String::as_str),
|
||||
Some("text/plain")
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn post_sends_body_and_default_user_agent() {
|
||||
let addr = spawn_server(|request| {
|
||||
// Echo back whether the body + default UA were present.
|
||||
let has_ua = request.to_lowercase().contains("user-agent: picloud/");
|
||||
let has_body = request.contains("xyzzy");
|
||||
ok_response(&format!("ua={has_ua},body={has_body}"), "text/plain")
|
||||
})
|
||||
.await;
|
||||
let svc = dev_service(Arc::new(AllowAuthz));
|
||||
let mut r = req("POST", format!("http://{addr}/"));
|
||||
r.body = Some(b"xyzzy".to_vec());
|
||||
r.content_type = Some("text/plain".into());
|
||||
let resp = svc.request(&anon_cx(), r).await.unwrap();
|
||||
assert_eq!(resp.body_raw, "ua=true,body=true");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn custom_user_agent_overrides_default() {
|
||||
let addr = spawn_server(|request| {
|
||||
let has_custom = request.to_lowercase().contains("user-agent: my-agent");
|
||||
let has_default = request.to_lowercase().contains("picloud/");
|
||||
ok_response(
|
||||
&format!("custom={has_custom},default={has_default}"),
|
||||
"text/plain",
|
||||
)
|
||||
})
|
||||
.await;
|
||||
let svc = dev_service(Arc::new(AllowAuthz));
|
||||
let mut r = req("GET", format!("http://{addr}/"));
|
||||
r.headers.insert("User-Agent".into(), "my-agent".into());
|
||||
let resp = svc.request(&anon_cx(), r).await.unwrap();
|
||||
assert_eq!(resp.body_raw, "custom=true,default=false");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn empty_body_response() {
|
||||
let addr = spawn_server(|_r| {
|
||||
b"HTTP/1.1 204 No Content\r\nContent-Length: 0\r\nConnection: close\r\n\r\n".to_vec()
|
||||
})
|
||||
.await;
|
||||
let svc = dev_service(Arc::new(AllowAuthz));
|
||||
let resp = svc
|
||||
.request(&anon_cx(), req("GET", format!("http://{addr}/")))
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(resp.status, 204);
|
||||
assert_eq!(resp.body_raw, "");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn non_2xx_does_not_error() {
|
||||
let addr = spawn_server(|_r| {
|
||||
b"HTTP/1.1 500 Internal Server Error\r\nContent-Length: 3\r\nConnection: close\r\n\r\nerr".to_vec()
|
||||
})
|
||||
.await;
|
||||
let svc = dev_service(Arc::new(AllowAuthz));
|
||||
let resp = svc
|
||||
.request(&anon_cx(), req("GET", format!("http://{addr}/")))
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(resp.status, 500);
|
||||
assert_eq!(resp.body_raw, "err");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn response_over_content_length_cap_rejected() {
|
||||
let addr = spawn_server(|_r| ok_response("0123456789", "text/plain")).await;
|
||||
let mut config = HttpConfig::conservative();
|
||||
config.allow_private = true;
|
||||
config.max_response_body_bytes = 5; // body is 10 bytes
|
||||
let svc = HttpServiceImpl::new(config, Arc::new(AllowAuthz));
|
||||
let err = svc
|
||||
.request(&anon_cx(), req("GET", format!("http://{addr}/")))
|
||||
.await
|
||||
.unwrap_err();
|
||||
assert!(matches!(err, HttpError::BodyTooLarge("response")));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn response_over_cap_without_content_length_rejected_mid_stream() {
|
||||
// No Content-Length header → must be caught while streaming.
|
||||
let addr = spawn_server(|_r| {
|
||||
b"HTTP/1.1 200 OK\r\nConnection: close\r\n\r\n0123456789ABCDEF".to_vec()
|
||||
})
|
||||
.await;
|
||||
let mut config = HttpConfig::conservative();
|
||||
config.allow_private = true;
|
||||
config.max_response_body_bytes = 4;
|
||||
let svc = HttpServiceImpl::new(config, Arc::new(AllowAuthz));
|
||||
let err = svc
|
||||
.request(&anon_cx(), req("GET", format!("http://{addr}/")))
|
||||
.await
|
||||
.unwrap_err();
|
||||
assert!(matches!(err, HttpError::BodyTooLarge("response")));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn request_body_over_cap_rejected_before_send() {
|
||||
let mut config = HttpConfig::conservative();
|
||||
config.allow_private = true;
|
||||
config.max_request_body_bytes = 3;
|
||||
let svc = HttpServiceImpl::new(config, Arc::new(AllowAuthz));
|
||||
let mut r = req("POST", "http://127.0.0.1:1/".into());
|
||||
r.body = Some(b"too long".to_vec());
|
||||
let err = svc.request(&anon_cx(), r).await.unwrap_err();
|
||||
assert!(matches!(err, HttpError::BodyTooLarge("request")));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn redirect_followed_up_to_then_throws_beyond_max() {
|
||||
// Server always 302s to itself → unbounded redirect loop,
|
||||
// bounded by max_redirects.
|
||||
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
|
||||
let addr = listener.local_addr().unwrap();
|
||||
tokio::spawn(async move {
|
||||
loop {
|
||||
let Ok((mut sock, _)) = listener.accept().await else {
|
||||
break;
|
||||
};
|
||||
let mut buf = vec![0u8; 4096];
|
||||
let _ = sock.read(&mut buf).await;
|
||||
let body = format!(
|
||||
"HTTP/1.1 302 Found\r\nLocation: http://{addr}/next\r\nContent-Length: 0\r\nConnection: close\r\n\r\n"
|
||||
);
|
||||
let _ = sock.write_all(body.as_bytes()).await;
|
||||
}
|
||||
});
|
||||
let svc = dev_service(Arc::new(AllowAuthz));
|
||||
let mut r = req("GET", format!("http://{addr}/"));
|
||||
r.max_redirects = 2;
|
||||
let err = svc.request(&anon_cx(), r).await.unwrap_err();
|
||||
assert!(
|
||||
matches!(err, HttpError::Backend(ref m) if m.contains("too many redirects")),
|
||||
"expected too-many-redirects, got {err:?}"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn scheme_rejected() {
|
||||
let svc = dev_service(Arc::new(AllowAuthz));
|
||||
for url in ["file:///etc/passwd", "ftp://host/x", "gopher://host/"] {
|
||||
let err = svc
|
||||
.request(&anon_cx(), req("GET", url.into()))
|
||||
.await
|
||||
.unwrap_err();
|
||||
match err {
|
||||
HttpError::BlockedScheme(s) => {
|
||||
assert!(url.starts_with(&s), "scheme {s} not in url {url}");
|
||||
}
|
||||
other => panic!("expected BlockedScheme for {url}, got {other:?}"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn ports_rejected() {
|
||||
let svc = dev_service(Arc::new(AllowAuthz));
|
||||
for port in [22u16, 25, 465, 587] {
|
||||
let err = svc
|
||||
.request(
|
||||
&anon_cx(),
|
||||
req("GET", format!("http://example.com:{port}/")),
|
||||
)
|
||||
.await
|
||||
.unwrap_err();
|
||||
assert!(
|
||||
matches!(err, HttpError::BlockedPort(p) if p == port),
|
||||
"port {port} should be blocked, got {err:?}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn ssrf_blocks_loopback_without_allow_private() {
|
||||
// Default config (deny-list ON). A request to a loopback host
|
||||
// must surface as Ssrf, not a generic network error.
|
||||
let svc = HttpServiceImpl::new(HttpConfig::conservative(), Arc::new(AllowAuthz));
|
||||
let err = svc
|
||||
.request(&anon_cx(), req("GET", "http://127.0.0.1:9/".into()))
|
||||
.await
|
||||
.unwrap_err();
|
||||
match err {
|
||||
HttpError::Ssrf(reason) => {
|
||||
assert_eq!(reason, "loopback");
|
||||
assert!(!reason.contains("127.0.0.1"), "reason must not leak the IP");
|
||||
}
|
||||
other => panic!("expected Ssrf, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn ssrf_blocks_hostname_resolving_to_loopback() {
|
||||
// `localhost` resolves to 127.0.0.1 / ::1 — all denied. This
|
||||
// exercises the DNS-resolver path (vs the literal-IP path) and
|
||||
// must surface as Ssrf, not a generic DNS error.
|
||||
let svc = HttpServiceImpl::new(HttpConfig::conservative(), Arc::new(AllowAuthz));
|
||||
let err = svc
|
||||
.request(&anon_cx(), req("GET", "http://localhost:9/".into()))
|
||||
.await
|
||||
.unwrap_err();
|
||||
assert!(
|
||||
matches!(err, HttpError::Ssrf(_)),
|
||||
"expected Ssrf for localhost, got {err:?}"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn timeout_throws() {
|
||||
// Server that accepts then never responds.
|
||||
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
|
||||
let addr = listener.local_addr().unwrap();
|
||||
tokio::spawn(async move {
|
||||
if let Ok((sock, _)) = listener.accept().await {
|
||||
// Hold the socket open without replying.
|
||||
tokio::time::sleep(Duration::from_secs(30)).await;
|
||||
drop(sock);
|
||||
}
|
||||
});
|
||||
let svc = dev_service(Arc::new(AllowAuthz));
|
||||
let mut r = req("GET", format!("http://{addr}/"));
|
||||
r.timeout_ms = 300;
|
||||
let err = svc.request(&anon_cx(), r).await.unwrap_err();
|
||||
assert!(matches!(err, HttpError::Timeout), "got {err:?}");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn anon_skips_authz_member_without_scope_forbidden() {
|
||||
let addr = spawn_server(|_r| ok_response("ok", "text/plain")).await;
|
||||
// Anonymous principal → authz skipped even with DenyAuthz.
|
||||
let svc = dev_service(Arc::new(DenyAuthz));
|
||||
let ok = svc
|
||||
.request(&anon_cx(), req("GET", format!("http://{addr}/")))
|
||||
.await;
|
||||
assert!(ok.is_ok());
|
||||
// Authenticated member with no role → Forbidden.
|
||||
let err = svc
|
||||
.request(&member_cx(), req("GET", format!("http://{addr}/")))
|
||||
.await
|
||||
.unwrap_err();
|
||||
assert!(matches!(err, HttpError::Forbidden));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn member_with_role_allowed() {
|
||||
let addr = spawn_server(|_r| ok_response("ok", "text/plain")).await;
|
||||
let svc = dev_service(Arc::new(AllowAuthz));
|
||||
let resp = svc
|
||||
.request(&member_cx(), req("GET", format!("http://{addr}/")))
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(resp.status, 200);
|
||||
}
|
||||
}
|
||||
@@ -188,7 +188,7 @@ mod tests {
|
||||
use async_trait::async_trait;
|
||||
use picloud_shared::{
|
||||
AdminUserId, AppId, AppRole, ExecutionId, InstanceRole, NoopEventEmitter, Principal,
|
||||
RequestId, UserId,
|
||||
RequestId, ScriptId, UserId,
|
||||
};
|
||||
use std::collections::{BTreeMap, HashMap};
|
||||
use tokio::sync::Mutex;
|
||||
@@ -301,6 +301,7 @@ mod tests {
|
||||
fn anon_cx(app_id: AppId) -> SdkCallCx {
|
||||
SdkCallCx {
|
||||
app_id,
|
||||
script_id: ScriptId::new(),
|
||||
principal: None,
|
||||
execution_id: ExecutionId::new(),
|
||||
request_id: RequestId::new(),
|
||||
@@ -314,6 +315,7 @@ mod tests {
|
||||
fn owner_cx(app_id: AppId) -> SdkCallCx {
|
||||
SdkCallCx {
|
||||
app_id,
|
||||
script_id: ScriptId::new(),
|
||||
principal: Some(Principal {
|
||||
user_id: AdminUserId::new(),
|
||||
instance_role: InstanceRole::Owner,
|
||||
@@ -332,6 +334,7 @@ mod tests {
|
||||
fn member_no_role_cx(app_id: AppId) -> SdkCallCx {
|
||||
SdkCallCx {
|
||||
app_id,
|
||||
script_id: ScriptId::new(),
|
||||
principal: Some(Principal {
|
||||
user_id: AdminUserId::new(),
|
||||
instance_role: InstanceRole::Member,
|
||||
|
||||
@@ -22,6 +22,7 @@ pub mod auth_api;
|
||||
pub mod auth_bootstrap;
|
||||
pub mod auth_middleware;
|
||||
pub mod authz;
|
||||
pub mod cron_scheduler;
|
||||
pub mod dead_letter_repo;
|
||||
pub mod dead_letter_service;
|
||||
pub mod dead_letters_api;
|
||||
@@ -30,6 +31,7 @@ pub mod docs_filter;
|
||||
pub mod docs_repo;
|
||||
pub mod docs_service;
|
||||
pub mod gc;
|
||||
pub mod http_service;
|
||||
pub mod kv_repo;
|
||||
pub mod kv_service;
|
||||
pub mod log_sink;
|
||||
@@ -43,6 +45,7 @@ pub mod route_admin;
|
||||
pub mod route_repo;
|
||||
pub mod sandbox;
|
||||
pub mod scheduler;
|
||||
pub mod ssrf;
|
||||
pub mod trigger_config;
|
||||
pub mod trigger_repo;
|
||||
pub mod triggers_api;
|
||||
@@ -84,6 +87,7 @@ pub use auth_middleware::{
|
||||
API_KEY_PREFIX, API_KEY_PREFIX_LEN, SESSION_COOKIE,
|
||||
};
|
||||
pub use authz::{can, require, AuthzDenied, AuthzError, AuthzRepo, Capability, Decision};
|
||||
pub use cron_scheduler::spawn_cron_scheduler;
|
||||
pub use dead_letter_repo::{
|
||||
DeadLetterRepo, DeadLetterRepoError, DeadLetterRow, NewDeadLetter, PostgresDeadLetterRepo,
|
||||
};
|
||||
@@ -93,6 +97,7 @@ pub use dispatcher::{compute_backoff, Dispatcher, DispatcherError};
|
||||
pub use docs_repo::{DocsRepo, DocsRepoError, PostgresDocsRepo};
|
||||
pub use docs_service::DocsServiceImpl;
|
||||
pub use gc::{spawn_abandoned_gc, spawn_dead_letter_gc};
|
||||
pub use http_service::{HttpConfig, HttpServiceImpl};
|
||||
pub use kv_repo::{KvRepo, KvRepoError, PostgresKvRepo};
|
||||
pub use kv_service::KvServiceImpl;
|
||||
pub use log_sink::PostgresExecutionLogSink;
|
||||
|
||||
@@ -25,6 +25,8 @@ pub enum OutboxSourceKind {
|
||||
/// v1.1.2.
|
||||
Docs,
|
||||
DeadLetter,
|
||||
/// v1.1.4.
|
||||
Cron,
|
||||
}
|
||||
|
||||
impl OutboxSourceKind {
|
||||
@@ -35,6 +37,7 @@ impl OutboxSourceKind {
|
||||
Self::Kv => "kv",
|
||||
Self::Docs => "docs",
|
||||
Self::DeadLetter => "dead_letter",
|
||||
Self::Cron => "cron",
|
||||
}
|
||||
}
|
||||
|
||||
@@ -45,6 +48,7 @@ impl OutboxSourceKind {
|
||||
"kv" => Some(Self::Kv),
|
||||
"docs" => Some(Self::Docs),
|
||||
"dead_letter" => Some(Self::DeadLetter),
|
||||
"cron" => Some(Self::Cron),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
457
crates/manager-core/src/ssrf.rs
Normal file
457
crates/manager-core/src/ssrf.rs
Normal file
@@ -0,0 +1,457 @@
|
||||
//! SSRF deny-list — the load-bearing security mechanism behind the
|
||||
//! v1.1.4 `http::*` SDK.
|
||||
//!
|
||||
//! The policy is applied to the **resolved IP address**, not the
|
||||
//! hostname. That is the DNS-rebinding defense: a hostname that
|
||||
//! resolves to a public IP at lookup time and a private IP at connect
|
||||
//! time is not exploitable, because reqwest re-runs every connection
|
||||
//! (including post-redirect hops) through [`SsrfResolver`], which
|
||||
//! filters the address list before the socket is opened.
|
||||
//!
|
||||
//! [`SsrfPolicy::check`] returns a CIDR-*category* reason on denial
|
||||
//! (e.g. `"loopback"`, `"private"`) — never the IP itself, so the
|
||||
//! script-visible error can't be used to map the internal network.
|
||||
//!
|
||||
//! `PICLOUD_HTTP_ALLOW_PRIVATE=true` flips `allow_private`, which
|
||||
//! short-circuits every check to allow. That is dev/test-only and the
|
||||
//! binary logs a startup warning when it's set.
|
||||
|
||||
use std::future::Future;
|
||||
use std::net::{IpAddr, Ipv4Addr, Ipv6Addr, SocketAddr};
|
||||
use std::pin::Pin;
|
||||
use std::sync::Arc;
|
||||
|
||||
use reqwest::dns::{Addrs, Name, Resolve, Resolving};
|
||||
|
||||
/// Decision policy for a single resolved IP. Cheap to clone (one bool).
|
||||
#[derive(Debug, Clone, Copy)]
|
||||
pub struct SsrfPolicy {
|
||||
/// When true, every address is allowed — the entire deny-list is
|
||||
/// disabled. Set from `PICLOUD_HTTP_ALLOW_PRIVATE`. Dev/test only.
|
||||
pub allow_private: bool,
|
||||
}
|
||||
|
||||
impl SsrfPolicy {
|
||||
#[must_use]
|
||||
pub const fn new(allow_private: bool) -> Self {
|
||||
Self { allow_private }
|
||||
}
|
||||
|
||||
/// `Ok(())` if the IP may be connected to; `Err(reason)` with a
|
||||
/// CIDR-category label otherwise. The reason is safe to surface to
|
||||
/// a script — it never contains the address.
|
||||
///
|
||||
/// # Errors
|
||||
///
|
||||
/// Returns the deny reason when `ip` falls in a blocked range and
|
||||
/// `allow_private` is false.
|
||||
pub fn check(&self, ip: IpAddr) -> Result<(), &'static str> {
|
||||
if self.allow_private {
|
||||
return Ok(());
|
||||
}
|
||||
match ip {
|
||||
IpAddr::V4(v4) => check_v4(v4),
|
||||
IpAddr::V6(v6) => check_v6(v6),
|
||||
}
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
pub fn is_allowed(&self, ip: IpAddr) -> bool {
|
||||
self.check(ip).is_ok()
|
||||
}
|
||||
}
|
||||
|
||||
/// IPv4 deny-list. Order doesn't matter (ranges are disjoint by
|
||||
/// construction); first match wins for the reason label.
|
||||
// Several arms share a reason ("private") for distinct CIDRs — keeping
|
||||
// them separate documents each blocked range explicitly.
|
||||
#[allow(clippy::match_same_arms)]
|
||||
fn check_v4(ip: Ipv4Addr) -> Result<(), &'static str> {
|
||||
let o = ip.octets();
|
||||
match o {
|
||||
[127, ..] => Err("loopback"),
|
||||
[0, ..] => Err("unspecified"), // 0.0.0.0/8 "this network"
|
||||
[10, ..] => Err("private"),
|
||||
[172, b, ..] if (16..=31).contains(&b) => Err("private"),
|
||||
[192, 168, ..] => Err("private"),
|
||||
[169, 254, ..] => Err("link-local"), // includes cloud metadata 169.254.169.254
|
||||
[100, b, ..] if (64..=127).contains(&b) => Err("carrier-grade-nat"),
|
||||
[224..=239, ..] => Err("multicast"),
|
||||
[240..=255, ..] => Err("reserved"),
|
||||
_ => Ok(()),
|
||||
}
|
||||
}
|
||||
|
||||
/// IPv6 deny-list. IPv4-mapped addresses (`::ffff:0:0/96`) re-run the
|
||||
/// v4 deny-list against the embedded address.
|
||||
fn check_v6(ip: Ipv6Addr) -> Result<(), &'static str> {
|
||||
// IPv4-mapped (::ffff:a.b.c.d) — re-check the embedded v4 address
|
||||
// so a mapped private/loopback address can't sneak through.
|
||||
if let Some(v4) = ip.to_ipv4_mapped() {
|
||||
return check_v4(v4);
|
||||
}
|
||||
if ip == Ipv6Addr::LOCALHOST {
|
||||
return Err("loopback");
|
||||
}
|
||||
if ip == Ipv6Addr::UNSPECIFIED {
|
||||
return Err("unspecified");
|
||||
}
|
||||
let seg0 = ip.segments()[0];
|
||||
if seg0 & 0xffc0 == 0xfe80 {
|
||||
return Err("link-local"); // fe80::/10
|
||||
}
|
||||
if seg0 & 0xfe00 == 0xfc00 {
|
||||
return Err("unique-local"); // fc00::/7
|
||||
}
|
||||
if seg0 & 0xff00 == 0xff00 {
|
||||
return Err("multicast"); // ff00::/8
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Marker error returned by the resolver when *every* resolved address
|
||||
/// for a host was denied. reqwest wraps this into a connect error; the
|
||||
/// `http_service` impl walks the source chain for the
|
||||
/// `"blocked by SSRF policy:"` prefix to surface a clean
|
||||
/// [`crate::http_service::HttpError::Ssrf`] instead of a generic DNS
|
||||
/// failure. Keeping the reason a category label means no IP leaks.
|
||||
#[derive(Debug)]
|
||||
struct SsrfBlocked {
|
||||
reason: &'static str,
|
||||
}
|
||||
|
||||
impl std::fmt::Display for SsrfBlocked {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
write!(f, "blocked by SSRF policy: {}", self.reason)
|
||||
}
|
||||
}
|
||||
|
||||
impl std::error::Error for SsrfBlocked {}
|
||||
|
||||
/// Prefix the resolver embeds in its error and the impl scans for.
|
||||
pub const SSRF_BLOCK_PREFIX: &str = "blocked by SSRF policy: ";
|
||||
|
||||
/// Pluggable host→addresses lookup. Production uses the system
|
||||
/// resolver; tests inject a closure (e.g. to simulate DNS rebinding —
|
||||
/// a different address on a later call).
|
||||
pub type LookupFn = Arc<
|
||||
dyn Fn(String) -> Pin<Box<dyn Future<Output = std::io::Result<Vec<SocketAddr>>> + Send>>
|
||||
+ Send
|
||||
+ Sync,
|
||||
>;
|
||||
|
||||
fn system_lookup(
|
||||
host: String,
|
||||
) -> Pin<Box<dyn Future<Output = std::io::Result<Vec<SocketAddr>>> + Send>> {
|
||||
Box::pin(async move {
|
||||
// Port 0 — reqwest overrides it with the real target port.
|
||||
Ok(tokio::net::lookup_host((host.as_str(), 0u16))
|
||||
.await?
|
||||
.collect())
|
||||
})
|
||||
}
|
||||
|
||||
/// reqwest DNS resolver that delegates to the system resolver, then
|
||||
/// filters the address list through [`SsrfPolicy`]. Plugged in via
|
||||
/// `ClientBuilder::dns_resolver`, so it runs at the actual connection
|
||||
/// point — including for every redirect hop. This is the DNS-rebinding
|
||||
/// defense: filtering happens at connect time, not at URL-parse time.
|
||||
#[derive(Clone)]
|
||||
pub struct SsrfResolver {
|
||||
policy: SsrfPolicy,
|
||||
lookup: LookupFn,
|
||||
}
|
||||
|
||||
impl SsrfResolver {
|
||||
#[must_use]
|
||||
pub fn new(policy: SsrfPolicy) -> Self {
|
||||
Self {
|
||||
policy,
|
||||
lookup: Arc::new(system_lookup),
|
||||
}
|
||||
}
|
||||
|
||||
/// Construct with an injected lookup (tests only).
|
||||
#[must_use]
|
||||
pub fn with_lookup(policy: SsrfPolicy, lookup: LookupFn) -> Self {
|
||||
Self { policy, lookup }
|
||||
}
|
||||
}
|
||||
|
||||
impl Resolve for SsrfResolver {
|
||||
fn resolve(&self, name: Name) -> Resolving {
|
||||
let policy = self.policy;
|
||||
let lookup = self.lookup.clone();
|
||||
let host = name.as_str().to_string();
|
||||
Box::pin(async move {
|
||||
let resolved: Vec<SocketAddr> = lookup(host)
|
||||
.await
|
||||
.map_err(|e| -> Box<dyn std::error::Error + Send + Sync> { Box::new(e) })?;
|
||||
|
||||
// Empty resolution → genuine DNS miss; let reqwest surface
|
||||
// it as a normal "no addresses" error.
|
||||
if resolved.is_empty() {
|
||||
let addrs: Addrs = Box::new(std::iter::empty());
|
||||
return Ok(addrs);
|
||||
}
|
||||
|
||||
let mut allowed: Vec<SocketAddr> = Vec::with_capacity(resolved.len());
|
||||
let mut last_reason: &'static str = "denied";
|
||||
for sa in resolved {
|
||||
match policy.check(sa.ip()) {
|
||||
Ok(()) => allowed.push(sa),
|
||||
Err(reason) => last_reason = reason,
|
||||
}
|
||||
}
|
||||
|
||||
// Resolution returned addresses but the policy denied them
|
||||
// all → fail with the SSRF marker so the impl can report a
|
||||
// policy block (not a generic DNS error).
|
||||
if allowed.is_empty() {
|
||||
let err: Box<dyn std::error::Error + Send + Sync> = Box::new(SsrfBlocked {
|
||||
reason: last_reason,
|
||||
});
|
||||
return Err(err);
|
||||
}
|
||||
|
||||
let addrs: Addrs = Box::new(allowed.into_iter());
|
||||
Ok(addrs)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// Build the resolver. reqwest's `dns_resolver` is generic over a
|
||||
/// concrete `R: Resolve` (it stores `Arc<R>`), so this returns the
|
||||
/// concrete `Arc<SsrfResolver>` rather than a trait object.
|
||||
#[must_use]
|
||||
pub fn resolver(policy: SsrfPolicy) -> Arc<SsrfResolver> {
|
||||
Arc::new(SsrfResolver::new(policy))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use std::str::FromStr;
|
||||
|
||||
fn denied(ip: &str) -> &'static str {
|
||||
let policy = SsrfPolicy::new(false);
|
||||
policy
|
||||
.check(IpAddr::from_str(ip).unwrap())
|
||||
.expect_err(&format!("{ip} should be denied"))
|
||||
}
|
||||
|
||||
fn allowed(ip: &str) {
|
||||
let policy = SsrfPolicy::new(false);
|
||||
policy
|
||||
.check(IpAddr::from_str(ip).unwrap())
|
||||
.unwrap_or_else(|r| panic!("{ip} should be allowed, denied as {r}"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn denies_ipv4_loopback() {
|
||||
assert_eq!(denied("127.0.0.1"), "loopback");
|
||||
assert_eq!(denied("127.1.2.3"), "loopback");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn denies_ipv4_unspecified() {
|
||||
assert_eq!(denied("0.0.0.0"), "unspecified");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn denies_rfc1918_private() {
|
||||
assert_eq!(denied("10.0.0.1"), "private");
|
||||
assert_eq!(denied("10.255.255.255"), "private");
|
||||
assert_eq!(denied("172.16.0.1"), "private");
|
||||
assert_eq!(denied("172.31.255.255"), "private");
|
||||
assert_eq!(denied("192.168.0.1"), "private");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn allows_172_outside_private_range() {
|
||||
// 172.15.x and 172.32.x are public — only 172.16.0.0/12 is private.
|
||||
allowed("172.15.0.1");
|
||||
allowed("172.32.0.1");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn denies_link_local_and_cloud_metadata() {
|
||||
assert_eq!(denied("169.254.0.1"), "link-local");
|
||||
// The cloud metadata endpoint is the canonical SSRF target.
|
||||
assert_eq!(denied("169.254.169.254"), "link-local");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn denies_carrier_grade_nat() {
|
||||
assert_eq!(denied("100.64.0.1"), "carrier-grade-nat");
|
||||
assert_eq!(denied("100.127.255.255"), "carrier-grade-nat");
|
||||
// 100.63.x and 100.128.x are outside 100.64.0.0/10.
|
||||
allowed("100.63.0.1");
|
||||
allowed("100.128.0.1");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn denies_multicast_and_reserved() {
|
||||
assert_eq!(denied("224.0.0.1"), "multicast");
|
||||
assert_eq!(denied("239.255.255.255"), "multicast");
|
||||
assert_eq!(denied("240.0.0.1"), "reserved");
|
||||
assert_eq!(denied("255.255.255.255"), "reserved");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn allows_public_ipv4() {
|
||||
allowed("1.1.1.1");
|
||||
allowed("8.8.8.8");
|
||||
allowed("93.184.216.34"); // example.com
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn denies_ipv6_loopback() {
|
||||
assert_eq!(denied("::1"), "loopback");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn denies_ipv6_unspecified() {
|
||||
assert_eq!(denied("::"), "unspecified");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn denies_ipv6_link_local() {
|
||||
assert_eq!(denied("fe80::1"), "link-local");
|
||||
assert_eq!(denied("febf:ffff::1"), "link-local");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn denies_ipv6_unique_local() {
|
||||
assert_eq!(denied("fc00::1"), "unique-local");
|
||||
assert_eq!(denied("fd12:3456::1"), "unique-local");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn denies_ipv6_multicast() {
|
||||
assert_eq!(denied("ff00::1"), "multicast");
|
||||
assert_eq!(denied("ff02::1"), "multicast");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn allows_public_ipv6() {
|
||||
allowed("2606:4700:4700::1111"); // cloudflare
|
||||
allowed("2001:4860:4860::8888"); // google
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ipv4_mapped_ipv6_rechecks_embedded_address() {
|
||||
// ::ffff:127.0.0.1 must be denied via the embedded-v4 re-check.
|
||||
assert_eq!(denied("::ffff:127.0.0.1"), "loopback");
|
||||
assert_eq!(denied("::ffff:10.0.0.1"), "private");
|
||||
assert_eq!(denied("::ffff:169.254.169.254"), "link-local");
|
||||
// A mapped *public* address stays allowed.
|
||||
allowed("::ffff:1.1.1.1");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn allow_private_disables_all_denials() {
|
||||
let policy = SsrfPolicy::new(true);
|
||||
for ip in ["127.0.0.1", "10.0.0.1", "169.254.169.254", "::1", "fe80::1"] {
|
||||
assert!(policy.is_allowed(IpAddr::from_str(ip).unwrap()));
|
||||
}
|
||||
}
|
||||
|
||||
// --- resolver-path tests (the connect-time filter) ---
|
||||
|
||||
use std::sync::atomic::{AtomicUsize, Ordering};
|
||||
|
||||
fn name(s: &str) -> Name {
|
||||
Name::from_str(s).unwrap()
|
||||
}
|
||||
|
||||
fn fixed_lookup(addrs: Vec<SocketAddr>) -> LookupFn {
|
||||
Arc::new(move |_host| {
|
||||
let addrs = addrs.clone();
|
||||
Box::pin(async move { Ok(addrs) })
|
||||
})
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn resolver_returns_only_allowed_addresses() {
|
||||
// A host resolving to one public + one private IP yields only
|
||||
// the public one to reqwest.
|
||||
let public: SocketAddr = "1.1.1.1:0".parse().unwrap();
|
||||
let private: SocketAddr = "10.0.0.1:0".parse().unwrap();
|
||||
let resolver =
|
||||
SsrfResolver::with_lookup(SsrfPolicy::new(false), fixed_lookup(vec![public, private]));
|
||||
let got: Vec<SocketAddr> = resolver
|
||||
.resolve(name("mixed.example"))
|
||||
.await
|
||||
.unwrap()
|
||||
.collect();
|
||||
assert_eq!(got, vec![public]);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn resolver_all_denied_fails_with_ssrf_marker() {
|
||||
// A host resolving to ONLY private IPs fails with the SSRF
|
||||
// marker (not a generic empty/DNS result).
|
||||
let resolver = SsrfResolver::with_lookup(
|
||||
SsrfPolicy::new(false),
|
||||
fixed_lookup(vec![
|
||||
"10.0.0.1:0".parse().unwrap(),
|
||||
"127.0.0.1:0".parse().unwrap(),
|
||||
]),
|
||||
);
|
||||
let Err(err) = resolver.resolve(name("internal.example")).await else {
|
||||
panic!("all-denied resolution should error");
|
||||
};
|
||||
assert!(
|
||||
err.to_string().starts_with(SSRF_BLOCK_PREFIX),
|
||||
"expected SSRF marker, got: {err}"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn resolver_dns_rebinding_second_resolution_denied() {
|
||||
// Simulate rebinding: public IP on the first lookup, private on
|
||||
// the second. The connect-time filter denies the second.
|
||||
let calls = Arc::new(AtomicUsize::new(0));
|
||||
let calls2 = calls.clone();
|
||||
let lookup: LookupFn = Arc::new(move |_host| {
|
||||
let n = calls2.fetch_add(1, Ordering::SeqCst);
|
||||
Box::pin(async move {
|
||||
let addr: SocketAddr = if n == 0 {
|
||||
"1.1.1.1:0".parse().unwrap()
|
||||
} else {
|
||||
"127.0.0.1:0".parse().unwrap()
|
||||
};
|
||||
Ok(vec![addr])
|
||||
})
|
||||
});
|
||||
let resolver = SsrfResolver::with_lookup(SsrfPolicy::new(false), lookup);
|
||||
|
||||
// First resolution: public → allowed.
|
||||
let first: Vec<SocketAddr> = resolver
|
||||
.resolve(name("rebind.example"))
|
||||
.await
|
||||
.unwrap()
|
||||
.collect();
|
||||
assert_eq!(first, vec!["1.1.1.1:0".parse::<SocketAddr>().unwrap()]);
|
||||
|
||||
// Second resolution: rebinding returns loopback → denied.
|
||||
let Err(err) = resolver.resolve(name("rebind.example")).await else {
|
||||
panic!("rebound private address must be denied");
|
||||
};
|
||||
assert!(err.to_string().contains("loopback"));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn resolver_empty_resolution_is_not_ssrf() {
|
||||
// Genuine DNS miss (no addresses) returns an empty iterator,
|
||||
// NOT the SSRF marker — reqwest surfaces a normal DNS error.
|
||||
let resolver = SsrfResolver::with_lookup(SsrfPolicy::new(false), fixed_lookup(vec![]));
|
||||
let got: Vec<SocketAddr> = resolver
|
||||
.resolve(name("nxdomain.example"))
|
||||
.await
|
||||
.unwrap()
|
||||
.collect();
|
||||
assert!(got.is_empty());
|
||||
}
|
||||
}
|
||||
@@ -56,6 +56,11 @@ pub struct TriggerConfig {
|
||||
pub dead_letter_retention_days: u32,
|
||||
/// abandoned-execution retention before GC, in days. Default 7.
|
||||
pub abandoned_retention_days: u32,
|
||||
|
||||
/// Cron scheduler poll cadence, in ms (v1.1.4). Default 30 000 —
|
||||
/// real-world cron precision is per-minute, so a 30s tick is fine.
|
||||
/// Floored at 1s by the scheduler.
|
||||
pub cron_tick_interval_ms: u32,
|
||||
}
|
||||
|
||||
impl TriggerConfig {
|
||||
@@ -69,6 +74,7 @@ impl TriggerConfig {
|
||||
retry_jitter_pct: 20,
|
||||
dead_letter_retention_days: 30,
|
||||
abandoned_retention_days: 7,
|
||||
cron_tick_interval_ms: 30_000,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -91,6 +97,10 @@ impl TriggerConfig {
|
||||
&mut c.abandoned_retention_days,
|
||||
"PICLOUD_ABANDONED_EXECUTIONS_RETENTION_DAYS",
|
||||
);
|
||||
load_u32(
|
||||
&mut c.cron_tick_interval_ms,
|
||||
"PICLOUD_CRON_TICK_INTERVAL_MS",
|
||||
);
|
||||
c
|
||||
}
|
||||
}
|
||||
@@ -141,6 +151,7 @@ mod tests {
|
||||
assert_eq!(c.retry_jitter_pct, 20);
|
||||
assert_eq!(c.dead_letter_retention_days, 30);
|
||||
assert_eq!(c.abandoned_retention_days, 7);
|
||||
assert_eq!(c.cron_tick_interval_ms, 30_000);
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
@@ -49,6 +49,8 @@ pub enum TriggerKind {
|
||||
Kv,
|
||||
Docs,
|
||||
DeadLetter,
|
||||
/// v1.1.4.
|
||||
Cron,
|
||||
}
|
||||
|
||||
impl TriggerKind {
|
||||
@@ -58,6 +60,7 @@ impl TriggerKind {
|
||||
Self::Kv => "kv",
|
||||
Self::Docs => "docs",
|
||||
Self::DeadLetter => "dead_letter",
|
||||
Self::Cron => "cron",
|
||||
}
|
||||
}
|
||||
|
||||
@@ -67,6 +70,7 @@ impl TriggerKind {
|
||||
"kv" => Some(Self::Kv),
|
||||
"docs" => Some(Self::Docs),
|
||||
"dead_letter" => Some(Self::DeadLetter),
|
||||
"cron" => Some(Self::Cron),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
@@ -108,6 +112,14 @@ pub enum TriggerDetails {
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
script_id_filter: Option<ScriptId>,
|
||||
},
|
||||
/// v1.1.4. The 6-field cron schedule + IANA timezone the trigger
|
||||
/// fires on, plus the last enqueue time (for dashboard display).
|
||||
Cron {
|
||||
schedule: String,
|
||||
timezone: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
last_fired_at: Option<DateTime<Utc>>,
|
||||
},
|
||||
}
|
||||
|
||||
/// Create payload for a KV trigger. Defaults applied at the admin
|
||||
@@ -148,6 +160,21 @@ pub struct CreateDeadLetterTrigger {
|
||||
pub registered_by_principal: AdminUserId,
|
||||
}
|
||||
|
||||
/// Create payload for a cron trigger (v1.1.4). `schedule` is a 6-field
|
||||
/// cron expression and `timezone` an IANA name; both are validated
|
||||
/// (by the admin endpoint and defensively by the repo) before insert.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct CreateCronTrigger {
|
||||
pub script_id: ScriptId,
|
||||
pub schedule: String,
|
||||
pub timezone: String,
|
||||
pub dispatch_mode: TriggerDispatchMode,
|
||||
pub retry_max_attempts: u32,
|
||||
pub retry_backoff: BackoffShape,
|
||||
pub retry_base_ms: u32,
|
||||
pub registered_by_principal: AdminUserId,
|
||||
}
|
||||
|
||||
/// One match for the dispatcher's "which KV triggers fire on this
|
||||
/// event" lookup. Carries everything the dispatcher needs to construct
|
||||
/// the outbox row.
|
||||
@@ -206,6 +233,15 @@ pub trait TriggerRepo: Send + Sync {
|
||||
req: CreateDeadLetterTrigger,
|
||||
) -> Result<Trigger, TriggerRepoError>;
|
||||
|
||||
/// v1.1.4. `schedule` + `timezone` are validated before insert; an
|
||||
/// invalid expression or unknown IANA name returns
|
||||
/// `TriggerRepoError::Invalid`.
|
||||
async fn create_cron_trigger(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
req: CreateCronTrigger,
|
||||
) -> Result<Trigger, TriggerRepoError>;
|
||||
|
||||
async fn list_for_app(&self, app_id: AppId) -> Result<Vec<Trigger>, TriggerRepoError>;
|
||||
|
||||
async fn get(&self, id: TriggerId) -> Result<Option<Trigger>, TriggerRepoError>;
|
||||
@@ -453,6 +489,72 @@ impl TriggerRepo for PostgresTriggerRepo {
|
||||
})
|
||||
}
|
||||
|
||||
async fn create_cron_trigger(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
req: CreateCronTrigger,
|
||||
) -> Result<Trigger, TriggerRepoError> {
|
||||
// Defense-in-depth validation (the admin endpoint validates too).
|
||||
crate::cron_scheduler::validate_schedule(&req.schedule)
|
||||
.map_err(|e| TriggerRepoError::Invalid(format!("invalid cron schedule: {e}")))?;
|
||||
crate::cron_scheduler::validate_timezone(&req.timezone)
|
||||
.map_err(|e| TriggerRepoError::Invalid(format!("invalid timezone: {e}")))?;
|
||||
|
||||
let mut tx = self.pool.begin().await?;
|
||||
let parent: TriggerRow = sqlx::query_as(
|
||||
"INSERT INTO triggers ( \
|
||||
app_id, script_id, kind, enabled, dispatch_mode, \
|
||||
retry_max_attempts, retry_backoff, retry_base_ms, \
|
||||
registered_by_principal \
|
||||
) VALUES ($1, $2, 'cron', TRUE, $3, $4, $5, $6, $7) \
|
||||
RETURNING id, app_id, script_id, kind, enabled, dispatch_mode, \
|
||||
retry_max_attempts, retry_backoff, retry_base_ms, \
|
||||
registered_by_principal, created_at, updated_at",
|
||||
)
|
||||
.bind(app_id.into_inner())
|
||||
.bind(req.script_id.into_inner())
|
||||
.bind(req.dispatch_mode.as_str())
|
||||
.bind(i32::try_from(req.retry_max_attempts).unwrap_or(3))
|
||||
.bind(req.retry_backoff.as_str())
|
||||
.bind(i32::try_from(req.retry_base_ms).unwrap_or(1000))
|
||||
.bind(req.registered_by_principal.into_inner())
|
||||
.fetch_one(&mut *tx)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"INSERT INTO cron_trigger_details (trigger_id, schedule, timezone) \
|
||||
VALUES ($1, $2, $3)",
|
||||
)
|
||||
.bind(parent.id)
|
||||
.bind(&req.schedule)
|
||||
.bind(&req.timezone)
|
||||
.execute(&mut *tx)
|
||||
.await?;
|
||||
|
||||
tx.commit().await?;
|
||||
|
||||
Ok(Trigger {
|
||||
id: parent.id.into(),
|
||||
app_id: parent.app_id.into(),
|
||||
script_id: parent.script_id.into(),
|
||||
kind: TriggerKind::Cron,
|
||||
enabled: parent.enabled,
|
||||
dispatch_mode: dispatch_from_str(&parent.dispatch_mode),
|
||||
retry_max_attempts: u32::try_from(parent.retry_max_attempts).unwrap_or(3),
|
||||
retry_backoff: BackoffShape::from_wire(&parent.retry_backoff)
|
||||
.unwrap_or(BackoffShape::Exponential),
|
||||
retry_base_ms: u32::try_from(parent.retry_base_ms).unwrap_or(1000),
|
||||
registered_by_principal: parent.registered_by_principal.into(),
|
||||
created_at: parent.created_at,
|
||||
updated_at: parent.updated_at,
|
||||
details: TriggerDetails::Cron {
|
||||
schedule: req.schedule,
|
||||
timezone: req.timezone,
|
||||
last_fired_at: None,
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
async fn list_for_app(&self, app_id: AppId) -> Result<Vec<Trigger>, TriggerRepoError> {
|
||||
let parents: Vec<TriggerRow> = sqlx::query_as(
|
||||
"SELECT id, app_id, script_id, kind, enabled, dispatch_mode, \
|
||||
@@ -681,6 +783,20 @@ async fn hydrate_one(pool: &PgPool, parent: TriggerRow) -> Result<Trigger, Trigg
|
||||
script_id_filter: row.script_id_filter.map(Into::into),
|
||||
}
|
||||
}
|
||||
TriggerKind::Cron => {
|
||||
let row: CronDetailRow = sqlx::query_as(
|
||||
"SELECT schedule, timezone, last_fired_at \
|
||||
FROM cron_trigger_details WHERE trigger_id = $1",
|
||||
)
|
||||
.bind(parent.id)
|
||||
.fetch_one(pool)
|
||||
.await?;
|
||||
TriggerDetails::Cron {
|
||||
schedule: row.schedule,
|
||||
timezone: row.timezone,
|
||||
last_fired_at: row.last_fired_at,
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
Ok(Trigger {
|
||||
@@ -746,6 +862,13 @@ struct KvDetailRow {
|
||||
ops: Vec<String>,
|
||||
}
|
||||
|
||||
#[derive(sqlx::FromRow)]
|
||||
struct CronDetailRow {
|
||||
schedule: String,
|
||||
timezone: String,
|
||||
last_fired_at: Option<DateTime<Utc>>,
|
||||
}
|
||||
|
||||
#[derive(sqlx::FromRow)]
|
||||
#[allow(clippy::struct_field_names)]
|
||||
struct DlDetailRow {
|
||||
|
||||
@@ -25,8 +25,8 @@ use crate::authz::{require, AuthzDenied, AuthzError, AuthzRepo, Capability};
|
||||
use crate::repo::{ScriptRepository, ScriptRepositoryError};
|
||||
use crate::trigger_config::{BackoffShape, TriggerConfig};
|
||||
use crate::trigger_repo::{
|
||||
CreateDeadLetterTrigger, CreateDocsTrigger, CreateKvTrigger, Trigger, TriggerDispatchMode,
|
||||
TriggerRepo, TriggerRepoError,
|
||||
CreateCronTrigger, CreateDeadLetterTrigger, CreateDocsTrigger, CreateKvTrigger, Trigger,
|
||||
TriggerDispatchMode, TriggerRepo, TriggerRepoError,
|
||||
};
|
||||
|
||||
#[derive(Clone)]
|
||||
@@ -53,6 +53,7 @@ pub fn triggers_router(state: TriggersState) -> Router {
|
||||
)
|
||||
.route("/apps/{app_id}/triggers/kv", post(create_kv_trigger))
|
||||
.route("/apps/{app_id}/triggers/docs", post(create_docs_trigger))
|
||||
.route("/apps/{app_id}/triggers/cron", post(create_cron_trigger))
|
||||
.route(
|
||||
"/apps/{app_id}/triggers/dead_letter",
|
||||
post(create_dl_trigger),
|
||||
@@ -116,6 +117,28 @@ pub struct CreateDocsTriggerRequest {
|
||||
pub retry_base_ms: Option<u32>,
|
||||
}
|
||||
|
||||
/// v1.1.4 cron trigger. `schedule` is a 6-field cron expression (with
|
||||
/// seconds); `timezone` is an IANA name (defaults to UTC if omitted).
|
||||
#[derive(Debug, Deserialize)]
|
||||
pub struct CreateCronTriggerRequest {
|
||||
pub script_id: ScriptId,
|
||||
pub schedule: String,
|
||||
#[serde(default = "default_timezone")]
|
||||
pub timezone: String,
|
||||
#[serde(default = "default_dispatch")]
|
||||
pub dispatch_mode: TriggerDispatchMode,
|
||||
#[serde(default)]
|
||||
pub retry_max_attempts: Option<u32>,
|
||||
#[serde(default)]
|
||||
pub retry_backoff: Option<BackoffShape>,
|
||||
#[serde(default)]
|
||||
pub retry_base_ms: Option<u32>,
|
||||
}
|
||||
|
||||
fn default_timezone() -> String {
|
||||
"UTC".to_string()
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize)]
|
||||
pub struct CreateDeadLetterTriggerRequest {
|
||||
pub script_id: ScriptId,
|
||||
@@ -264,6 +287,47 @@ async fn create_docs_trigger(
|
||||
Ok((StatusCode::CREATED, Json(created)))
|
||||
}
|
||||
|
||||
async fn create_cron_trigger(
|
||||
State(s): State<TriggersState>,
|
||||
Extension(principal): Extension<Principal>,
|
||||
Path(app_id): Path<AppId>,
|
||||
Json(input): Json<CreateCronTriggerRequest>,
|
||||
) -> Result<(StatusCode, Json<Trigger>), TriggersApiError> {
|
||||
ensure_app_exists(&*s.apps, app_id).await?;
|
||||
require(
|
||||
s.authz.as_ref(),
|
||||
&principal,
|
||||
Capability::AppManageTriggers(app_id),
|
||||
)
|
||||
.await?;
|
||||
|
||||
// Validate the schedule + timezone before touching the script repo
|
||||
// so a bad expression fails fast with a clear 422.
|
||||
crate::cron_scheduler::validate_schedule(&input.schedule)
|
||||
.map_err(|e| TriggersApiError::Invalid(format!("invalid cron schedule: {e}")))?;
|
||||
crate::cron_scheduler::validate_timezone(&input.timezone)
|
||||
.map_err(|e| TriggersApiError::Invalid(format!("invalid timezone: {e}")))?;
|
||||
|
||||
// v1.1.3 check: target script exists, lives in this app, is an
|
||||
// endpoint (not a module).
|
||||
validate_trigger_target(&*s.scripts, app_id, input.script_id).await?;
|
||||
|
||||
let req = CreateCronTrigger {
|
||||
script_id: input.script_id,
|
||||
schedule: input.schedule,
|
||||
timezone: input.timezone,
|
||||
dispatch_mode: input.dispatch_mode,
|
||||
retry_max_attempts: input
|
||||
.retry_max_attempts
|
||||
.unwrap_or(s.config.retry_max_attempts),
|
||||
retry_backoff: input.retry_backoff.unwrap_or(s.config.retry_backoff),
|
||||
retry_base_ms: input.retry_base_ms.unwrap_or(s.config.retry_base_ms),
|
||||
registered_by_principal: principal.user_id,
|
||||
};
|
||||
let created = s.triggers.create_cron_trigger(app_id, req).await?;
|
||||
Ok((StatusCode::CREATED, Json(created)))
|
||||
}
|
||||
|
||||
async fn create_dl_trigger(
|
||||
State(s): State<TriggersState>,
|
||||
Extension(principal): Extension<Principal>,
|
||||
@@ -420,8 +484,8 @@ mod tests {
|
||||
use super::*;
|
||||
use crate::app_repo::{AppLookup, AppRepository};
|
||||
use crate::trigger_repo::{
|
||||
DeadLetterTriggerMatch, DocsTriggerMatch, KvTriggerMatch, Trigger, TriggerDetails,
|
||||
TriggerRepo, TriggerRepoError,
|
||||
CreateCronTrigger, DeadLetterTriggerMatch, DocsTriggerMatch, KvTriggerMatch, Trigger,
|
||||
TriggerDetails, TriggerRepo, TriggerRepoError,
|
||||
};
|
||||
use async_trait::async_trait;
|
||||
use chrono::Utc;
|
||||
@@ -523,6 +587,35 @@ mod tests {
|
||||
self.inner.lock().await.insert(id, trigger.clone());
|
||||
Ok(trigger)
|
||||
}
|
||||
async fn create_cron_trigger(
|
||||
&self,
|
||||
app_id: AppId,
|
||||
req: CreateCronTrigger,
|
||||
) -> Result<Trigger, TriggerRepoError> {
|
||||
let now = Utc::now();
|
||||
let id = TriggerId::new();
|
||||
let trigger = Trigger {
|
||||
id,
|
||||
app_id,
|
||||
script_id: req.script_id,
|
||||
kind: crate::trigger_repo::TriggerKind::Cron,
|
||||
enabled: true,
|
||||
dispatch_mode: req.dispatch_mode,
|
||||
retry_max_attempts: req.retry_max_attempts,
|
||||
retry_backoff: req.retry_backoff,
|
||||
retry_base_ms: req.retry_base_ms,
|
||||
registered_by_principal: req.registered_by_principal,
|
||||
created_at: now,
|
||||
updated_at: now,
|
||||
details: TriggerDetails::Cron {
|
||||
schedule: req.schedule,
|
||||
timezone: req.timezone,
|
||||
last_fired_at: None,
|
||||
},
|
||||
};
|
||||
self.inner.lock().await.insert(id, trigger.clone());
|
||||
Ok(trigger)
|
||||
}
|
||||
async fn list_for_app(&self, app_id: AppId) -> Result<Vec<Trigger>, TriggerRepoError> {
|
||||
Ok(self
|
||||
.inner
|
||||
@@ -1281,6 +1374,169 @@ mod tests {
|
||||
);
|
||||
}
|
||||
|
||||
// ----------------------------------------------------------------
|
||||
// v1.1.4: cron trigger create.
|
||||
// ----------------------------------------------------------------
|
||||
|
||||
fn cron_req(script_id: ScriptId, schedule: &str, timezone: &str) -> CreateCronTriggerRequest {
|
||||
CreateCronTriggerRequest {
|
||||
script_id,
|
||||
schedule: schedule.into(),
|
||||
timezone: timezone.into(),
|
||||
dispatch_mode: TriggerDispatchMode::Async,
|
||||
retry_max_attempts: None,
|
||||
retry_backoff: None,
|
||||
retry_base_ms: None,
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn cron_trigger_create_succeeds() {
|
||||
let app_id = AppId::new();
|
||||
let script_id = ScriptId::new();
|
||||
let state = state_with_endpoint(Arc::new(AlwaysAllowAuthzRepo), app_id, script_id);
|
||||
let (status, Json(trigger)) = create_cron_trigger(
|
||||
State(state),
|
||||
Extension(member_principal()),
|
||||
Path(app_id),
|
||||
Json(cron_req(
|
||||
script_id,
|
||||
"0 0 9 * * MON-FRI",
|
||||
"America/Los_Angeles",
|
||||
)),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(status, StatusCode::CREATED);
|
||||
assert!(matches!(
|
||||
trigger.kind,
|
||||
crate::trigger_repo::TriggerKind::Cron
|
||||
));
|
||||
match trigger.details {
|
||||
TriggerDetails::Cron {
|
||||
schedule,
|
||||
timezone,
|
||||
last_fired_at,
|
||||
} => {
|
||||
assert_eq!(schedule, "0 0 9 * * MON-FRI");
|
||||
assert_eq!(timezone, "America/Los_Angeles");
|
||||
assert!(last_fired_at.is_none());
|
||||
}
|
||||
other => panic!("expected Cron details, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn cron_trigger_rejects_invalid_schedule() {
|
||||
let app_id = AppId::new();
|
||||
let script_id = ScriptId::new();
|
||||
let state = state_with_endpoint(Arc::new(AlwaysAllowAuthzRepo), app_id, script_id);
|
||||
let res = create_cron_trigger(
|
||||
State(state),
|
||||
Extension(member_principal()),
|
||||
Path(app_id),
|
||||
// 5-field expression — not the 6-field format we accept.
|
||||
Json(cron_req(script_id, "* * * * *", "UTC")),
|
||||
)
|
||||
.await;
|
||||
let err = res.expect_err("invalid schedule should reject");
|
||||
let msg = match err {
|
||||
TriggersApiError::Invalid(m) => m,
|
||||
other => panic!("expected Invalid, got {other:?}"),
|
||||
};
|
||||
assert!(msg.to_lowercase().contains("schedule"), "got {msg}");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn cron_trigger_rejects_unknown_timezone() {
|
||||
let app_id = AppId::new();
|
||||
let script_id = ScriptId::new();
|
||||
let state = state_with_endpoint(Arc::new(AlwaysAllowAuthzRepo), app_id, script_id);
|
||||
let res = create_cron_trigger(
|
||||
State(state),
|
||||
Extension(member_principal()),
|
||||
Path(app_id),
|
||||
Json(cron_req(script_id, "0 * * * * *", "Mars/Phobos")),
|
||||
)
|
||||
.await;
|
||||
let err = res.expect_err("unknown timezone should reject");
|
||||
let msg = match err {
|
||||
TriggersApiError::Invalid(m) => m,
|
||||
other => panic!("expected Invalid, got {other:?}"),
|
||||
};
|
||||
assert!(msg.to_lowercase().contains("timezone"), "got {msg}");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn cron_trigger_rejects_module_target() {
|
||||
let app_id = AppId::new();
|
||||
let script_id = ScriptId::new();
|
||||
let state = TriggersState {
|
||||
triggers: Arc::new(InMemoryTriggerRepo::default()),
|
||||
apps: InMemoryAppRepo::with(app_id),
|
||||
authz: Arc::new(AlwaysAllowAuthzRepo),
|
||||
scripts: InMemoryScriptRepo::with_module(app_id, script_id),
|
||||
config: TriggerConfig::conservative(),
|
||||
};
|
||||
let res = create_cron_trigger(
|
||||
State(state),
|
||||
Extension(member_principal()),
|
||||
Path(app_id),
|
||||
Json(cron_req(script_id, "0 * * * * *", "UTC")),
|
||||
)
|
||||
.await;
|
||||
let err = res.expect_err("module script should be rejected as cron target");
|
||||
let msg = match err {
|
||||
TriggersApiError::Invalid(m) => m,
|
||||
other => panic!("expected Invalid, got {other:?}"),
|
||||
};
|
||||
assert!(msg.to_lowercase().contains("module"), "got {msg}");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn cron_trigger_rejects_cross_app_script() {
|
||||
// v1.1.3 isolation gap regression: app A cannot target app B's
|
||||
// script via a cron trigger.
|
||||
let app_a = AppId::new();
|
||||
let app_b = AppId::new();
|
||||
let script_id = ScriptId::new();
|
||||
let state = TriggersState {
|
||||
triggers: Arc::new(InMemoryTriggerRepo::default()),
|
||||
apps: InMemoryAppRepo::with(app_a),
|
||||
authz: Arc::new(AlwaysAllowAuthzRepo),
|
||||
scripts: InMemoryScriptRepo::with_endpoint(app_b, script_id),
|
||||
config: TriggerConfig::conservative(),
|
||||
};
|
||||
let res = create_cron_trigger(
|
||||
State(state),
|
||||
Extension(member_principal()),
|
||||
Path(app_a),
|
||||
Json(cron_req(script_id, "0 * * * * *", "UTC")),
|
||||
)
|
||||
.await;
|
||||
let err = res.expect_err("cross-app cron target should reject");
|
||||
let msg = match err {
|
||||
TriggersApiError::Invalid(m) => m,
|
||||
other => panic!("expected Invalid, got {other:?}"),
|
||||
};
|
||||
assert!(msg.to_lowercase().contains("does not belong"), "got {msg}");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn cron_trigger_member_without_role_is_forbidden() {
|
||||
let app_id = AppId::new();
|
||||
let state = state_with(Arc::new(AlwaysDenyAuthzRepo), app_id);
|
||||
let res = create_cron_trigger(
|
||||
State(state),
|
||||
Extension(member_principal()),
|
||||
Path(app_id),
|
||||
Json(cron_req(ScriptId::new(), "0 * * * * *", "UTC")),
|
||||
)
|
||||
.await;
|
||||
let err = res.expect_err("member without role should be forbidden");
|
||||
assert!(matches!(err, TriggersApiError::Forbidden));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn kv_trigger_accepts_endpoint_target() {
|
||||
let app_id = AppId::new();
|
||||
|
||||
@@ -3,6 +3,16 @@
|
||||
|
||||
## tables
|
||||
|
||||
table: abandoned_executions
|
||||
id: uuid NOT NULL default=gen_random_uuid()
|
||||
app_id: uuid NOT NULL
|
||||
outbox_id: uuid NOT NULL
|
||||
script_id: uuid NULL
|
||||
inbox_id: uuid NOT NULL
|
||||
status_code: integer NOT NULL
|
||||
result_summary: text NULL
|
||||
created_at: timestamp with time zone NOT NULL default=now()
|
||||
|
||||
table: admin_sessions
|
||||
token_hash: text NOT NULL
|
||||
user_id: uuid NOT NULL
|
||||
@@ -61,6 +71,48 @@ table: apps
|
||||
created_at: timestamp with time zone NOT NULL default=now()
|
||||
updated_at: timestamp with time zone NOT NULL default=now()
|
||||
|
||||
table: cron_trigger_details
|
||||
trigger_id: uuid NOT NULL
|
||||
schedule: text NOT NULL
|
||||
timezone: text NOT NULL default='UTC'::text
|
||||
last_fired_at: timestamp with time zone NULL
|
||||
|
||||
table: dead_letter_trigger_details
|
||||
trigger_id: uuid NOT NULL
|
||||
source_filter: text NULL
|
||||
trigger_id_filter: uuid NULL
|
||||
script_id_filter: uuid NULL
|
||||
|
||||
table: dead_letters
|
||||
id: uuid NOT NULL default=gen_random_uuid()
|
||||
app_id: uuid NOT NULL
|
||||
original_event_id: uuid NOT NULL
|
||||
source: text NOT NULL
|
||||
op: text NOT NULL
|
||||
trigger_id: uuid NULL
|
||||
script_id: uuid NULL
|
||||
payload: jsonb NOT NULL
|
||||
attempt_count: integer NOT NULL
|
||||
first_attempt_at: timestamp with time zone NOT NULL
|
||||
last_attempt_at: timestamp with time zone NOT NULL
|
||||
last_error: text NOT NULL
|
||||
created_at: timestamp with time zone NOT NULL default=now()
|
||||
resolved_at: timestamp with time zone NULL
|
||||
resolution: text NULL
|
||||
|
||||
table: docs
|
||||
app_id: uuid NOT NULL
|
||||
collection: text NOT NULL
|
||||
id: uuid NOT NULL
|
||||
data: jsonb NOT NULL
|
||||
created_at: timestamp with time zone NOT NULL default=now()
|
||||
updated_at: timestamp with time zone NOT NULL default=now()
|
||||
|
||||
table: docs_trigger_details
|
||||
trigger_id: uuid NOT NULL
|
||||
collection_glob: text NOT NULL
|
||||
ops: ARRAY NOT NULL
|
||||
|
||||
table: execution_logs
|
||||
id: uuid NOT NULL default=gen_random_uuid()
|
||||
script_id: uuid NOT NULL
|
||||
@@ -76,6 +128,36 @@ table: execution_logs
|
||||
created_at: timestamp with time zone NOT NULL default=now()
|
||||
app_id: uuid NOT NULL
|
||||
|
||||
table: kv_entries
|
||||
app_id: uuid NOT NULL
|
||||
collection: text NOT NULL
|
||||
key: text NOT NULL
|
||||
value: jsonb NOT NULL
|
||||
created_at: timestamp with time zone NOT NULL default=now()
|
||||
updated_at: timestamp with time zone NOT NULL default=now()
|
||||
|
||||
table: kv_trigger_details
|
||||
trigger_id: uuid NOT NULL
|
||||
collection_glob: text NOT NULL
|
||||
ops: ARRAY NOT NULL
|
||||
|
||||
table: outbox
|
||||
id: uuid NOT NULL default=gen_random_uuid()
|
||||
app_id: uuid NOT NULL
|
||||
source_kind: text NOT NULL
|
||||
trigger_id: uuid NULL
|
||||
script_id: uuid NULL
|
||||
reply_to: uuid NULL
|
||||
payload: jsonb NOT NULL
|
||||
origin_principal: uuid NULL
|
||||
trigger_depth: integer NOT NULL default=0
|
||||
root_execution_id: uuid NULL
|
||||
attempt_count: integer NOT NULL default=0
|
||||
next_attempt_at: timestamp with time zone NOT NULL default=now()
|
||||
claimed_at: timestamp with time zone NULL
|
||||
claimed_by: text NULL
|
||||
created_at: timestamp with time zone NOT NULL default=now()
|
||||
|
||||
table: routes
|
||||
id: uuid NOT NULL default=gen_random_uuid()
|
||||
script_id: uuid NOT NULL
|
||||
@@ -87,6 +169,13 @@ table: routes
|
||||
method: text NULL
|
||||
created_at: timestamp with time zone NOT NULL default=now()
|
||||
app_id: uuid NOT NULL
|
||||
dispatch_mode: text NOT NULL default='sync'::text
|
||||
|
||||
table: script_imports
|
||||
app_id: uuid NOT NULL
|
||||
importer_script_id: uuid NOT NULL
|
||||
imported_script_id: uuid NOT NULL
|
||||
created_at: timestamp with time zone NOT NULL default=now()
|
||||
|
||||
table: scripts
|
||||
id: uuid NOT NULL default=gen_random_uuid()
|
||||
@@ -100,9 +189,28 @@ table: scripts
|
||||
updated_at: timestamp with time zone NOT NULL default=now()
|
||||
sandbox: jsonb NOT NULL default='{}'::jsonb
|
||||
app_id: uuid NOT NULL
|
||||
kind: text NOT NULL default='endpoint'::text
|
||||
|
||||
table: triggers
|
||||
id: uuid NOT NULL default=gen_random_uuid()
|
||||
app_id: uuid NOT NULL
|
||||
script_id: uuid NOT NULL
|
||||
kind: text NOT NULL
|
||||
enabled: boolean NOT NULL default=true
|
||||
dispatch_mode: text NOT NULL default='async'::text
|
||||
retry_max_attempts: integer NOT NULL
|
||||
retry_backoff: text NOT NULL
|
||||
retry_base_ms: integer NOT NULL
|
||||
registered_by_principal: uuid NOT NULL
|
||||
created_at: timestamp with time zone NOT NULL default=now()
|
||||
updated_at: timestamp with time zone NOT NULL default=now()
|
||||
|
||||
## indexes
|
||||
|
||||
indexes on abandoned_executions:
|
||||
abandoned_executions_pkey: public.abandoned_executions USING btree (id)
|
||||
idx_abandoned_executions_gc: public.abandoned_executions USING btree (created_at)
|
||||
|
||||
indexes on admin_sessions:
|
||||
admin_sessions_expiry_idx: public.admin_sessions USING btree (expires_at)
|
||||
admin_sessions_pkey: public.admin_sessions USING btree (token_hash)
|
||||
@@ -135,11 +243,43 @@ indexes on apps:
|
||||
apps_pkey: public.apps USING btree (id)
|
||||
apps_slug_key: public.apps USING btree (slug)
|
||||
|
||||
indexes on cron_trigger_details:
|
||||
cron_trigger_details_pkey: public.cron_trigger_details USING btree (trigger_id)
|
||||
idx_cron_triggers_due: public.cron_trigger_details USING btree (last_fired_at)
|
||||
|
||||
indexes on dead_letter_trigger_details:
|
||||
dead_letter_trigger_details_pkey: public.dead_letter_trigger_details USING btree (trigger_id)
|
||||
|
||||
indexes on dead_letters:
|
||||
dead_letters_pkey: public.dead_letters USING btree (id)
|
||||
idx_dead_letters_app_unresolved: public.dead_letters USING btree (app_id) WHERE (resolved_at IS NULL)
|
||||
idx_dead_letters_gc: public.dead_letters USING btree (created_at)
|
||||
|
||||
indexes on docs:
|
||||
docs_pkey: public.docs USING btree (app_id, collection, id)
|
||||
idx_docs_app_collection: public.docs USING btree (app_id, collection)
|
||||
idx_docs_data_gin: public.docs USING gin (data jsonb_path_ops)
|
||||
|
||||
indexes on docs_trigger_details:
|
||||
docs_trigger_details_pkey: public.docs_trigger_details USING btree (trigger_id)
|
||||
|
||||
indexes on execution_logs:
|
||||
execution_logs_app_id_created_at_idx: public.execution_logs USING btree (app_id, created_at DESC)
|
||||
execution_logs_pkey: public.execution_logs USING btree (id)
|
||||
execution_logs_script_id_created_at_idx: public.execution_logs USING btree (script_id, created_at DESC)
|
||||
|
||||
indexes on kv_entries:
|
||||
idx_kv_entries_app_collection: public.kv_entries USING btree (app_id, collection)
|
||||
kv_entries_pkey: public.kv_entries USING btree (app_id, collection, key)
|
||||
|
||||
indexes on kv_trigger_details:
|
||||
kv_trigger_details_pkey: public.kv_trigger_details USING btree (trigger_id)
|
||||
|
||||
indexes on outbox:
|
||||
idx_outbox_app: public.outbox USING btree (app_id)
|
||||
idx_outbox_due: public.outbox USING btree (next_attempt_at) WHERE (claimed_at IS NULL)
|
||||
outbox_pkey: public.outbox USING btree (id)
|
||||
|
||||
indexes on routes:
|
||||
routes_app_id_idx: public.routes USING btree (app_id)
|
||||
routes_lookup_idx: public.routes USING btree (host_kind, host)
|
||||
@@ -147,13 +287,27 @@ indexes on routes:
|
||||
routes_script_id_idx: public.routes USING btree (script_id)
|
||||
routes_unique_binding_idx: public.routes USING btree (app_id, host_kind, host, path_kind, path, COALESCE(method, ''::text))
|
||||
|
||||
indexes on script_imports:
|
||||
idx_script_imports_app: public.script_imports USING btree (app_id)
|
||||
idx_script_imports_imported: public.script_imports USING btree (imported_script_id)
|
||||
script_imports_pkey: public.script_imports USING btree (importer_script_id, imported_script_id)
|
||||
|
||||
indexes on scripts:
|
||||
idx_scripts_app_kind: public.scripts USING btree (app_id, kind)
|
||||
scripts_app_id_idx: public.scripts USING btree (app_id)
|
||||
scripts_name_uidx: public.scripts USING btree (app_id, lower(name))
|
||||
scripts_pkey: public.scripts USING btree (id)
|
||||
|
||||
indexes on triggers:
|
||||
idx_triggers_app_kind_enabled: public.triggers USING btree (app_id, kind) WHERE (enabled = true)
|
||||
triggers_pkey: public.triggers USING btree (id)
|
||||
|
||||
## constraints
|
||||
|
||||
constraints on abandoned_executions:
|
||||
[FOREIGN KEY] abandoned_executions_app_id_fkey: FOREIGN KEY (app_id) REFERENCES apps(id) ON DELETE CASCADE
|
||||
[PRIMARY KEY] abandoned_executions_pkey: PRIMARY KEY (id)
|
||||
|
||||
constraints on admin_sessions:
|
||||
[FOREIGN KEY] admin_sessions_user_id_fkey: FOREIGN KEY (user_id) REFERENCES admin_users(id) ON DELETE CASCADE
|
||||
[PRIMARY KEY] admin_sessions_pkey: PRIMARY KEY (token_hash)
|
||||
@@ -189,25 +343,77 @@ constraints on apps:
|
||||
[PRIMARY KEY] apps_pkey: PRIMARY KEY (id)
|
||||
[UNIQUE] apps_slug_key: UNIQUE (slug)
|
||||
|
||||
constraints on cron_trigger_details:
|
||||
[FOREIGN KEY] cron_trigger_details_trigger_id_fkey: FOREIGN KEY (trigger_id) REFERENCES triggers(id) ON DELETE CASCADE
|
||||
[PRIMARY KEY] cron_trigger_details_pkey: PRIMARY KEY (trigger_id)
|
||||
|
||||
constraints on dead_letter_trigger_details:
|
||||
[FOREIGN KEY] dead_letter_trigger_details_trigger_id_fkey: FOREIGN KEY (trigger_id) REFERENCES triggers(id) ON DELETE CASCADE
|
||||
[PRIMARY KEY] dead_letter_trigger_details_pkey: PRIMARY KEY (trigger_id)
|
||||
|
||||
constraints on dead_letters:
|
||||
[CHECK] dead_letters_resolution_check: CHECK ((resolution = ANY (ARRAY['replayed'::text, 'ignored'::text, 'handled_by_script'::text, 'handler_failed'::text])))
|
||||
[FOREIGN KEY] dead_letters_app_id_fkey: FOREIGN KEY (app_id) REFERENCES apps(id) ON DELETE CASCADE
|
||||
[PRIMARY KEY] dead_letters_pkey: PRIMARY KEY (id)
|
||||
|
||||
constraints on docs:
|
||||
[FOREIGN KEY] docs_app_id_fkey: FOREIGN KEY (app_id) REFERENCES apps(id) ON DELETE CASCADE
|
||||
[PRIMARY KEY] docs_pkey: PRIMARY KEY (app_id, collection, id)
|
||||
|
||||
constraints on docs_trigger_details:
|
||||
[FOREIGN KEY] docs_trigger_details_trigger_id_fkey: FOREIGN KEY (trigger_id) REFERENCES triggers(id) ON DELETE CASCADE
|
||||
[PRIMARY KEY] docs_trigger_details_pkey: PRIMARY KEY (trigger_id)
|
||||
|
||||
constraints on execution_logs:
|
||||
[CHECK] execution_logs_status_check: CHECK ((status = ANY (ARRAY['success'::text, 'error'::text, 'timeout'::text, 'budget_exceeded'::text])))
|
||||
[FOREIGN KEY] execution_logs_app_id_fk: FOREIGN KEY (app_id) REFERENCES apps(id) ON DELETE CASCADE
|
||||
[FOREIGN KEY] execution_logs_script_id_fkey: FOREIGN KEY (script_id) REFERENCES scripts(id) ON DELETE CASCADE
|
||||
[PRIMARY KEY] execution_logs_pkey: PRIMARY KEY (id)
|
||||
|
||||
constraints on kv_entries:
|
||||
[FOREIGN KEY] kv_entries_app_id_fkey: FOREIGN KEY (app_id) REFERENCES apps(id) ON DELETE CASCADE
|
||||
[PRIMARY KEY] kv_entries_pkey: PRIMARY KEY (app_id, collection, key)
|
||||
|
||||
constraints on kv_trigger_details:
|
||||
[FOREIGN KEY] kv_trigger_details_trigger_id_fkey: FOREIGN KEY (trigger_id) REFERENCES triggers(id) ON DELETE CASCADE
|
||||
[PRIMARY KEY] kv_trigger_details_pkey: PRIMARY KEY (trigger_id)
|
||||
|
||||
constraints on outbox:
|
||||
[CHECK] outbox_source_kind_check: CHECK ((source_kind = ANY (ARRAY['http'::text, 'kv'::text, 'dead_letter'::text, 'docs'::text, 'cron'::text])))
|
||||
[FOREIGN KEY] outbox_app_id_fkey: FOREIGN KEY (app_id) REFERENCES apps(id) ON DELETE CASCADE
|
||||
[PRIMARY KEY] outbox_pkey: PRIMARY KEY (id)
|
||||
|
||||
constraints on routes:
|
||||
[CHECK] routes_dispatch_mode_check: CHECK ((dispatch_mode = ANY (ARRAY['sync'::text, 'async'::text])))
|
||||
[CHECK] routes_host_kind_check: CHECK ((host_kind = ANY (ARRAY['any'::text, 'strict'::text, 'wildcard'::text])))
|
||||
[CHECK] routes_path_kind_check: CHECK ((path_kind = ANY (ARRAY['exact'::text, 'prefix'::text, 'param'::text])))
|
||||
[FOREIGN KEY] routes_app_id_fk: FOREIGN KEY (app_id) REFERENCES apps(id) ON DELETE CASCADE
|
||||
[FOREIGN KEY] routes_script_id_fkey: FOREIGN KEY (script_id) REFERENCES scripts(id) ON DELETE CASCADE
|
||||
[PRIMARY KEY] routes_pkey: PRIMARY KEY (id)
|
||||
|
||||
constraints on script_imports:
|
||||
[FOREIGN KEY] script_imports_app_id_fkey: FOREIGN KEY (app_id) REFERENCES apps(id) ON DELETE CASCADE
|
||||
[FOREIGN KEY] script_imports_imported_script_id_fkey: FOREIGN KEY (imported_script_id) REFERENCES scripts(id) ON DELETE CASCADE
|
||||
[FOREIGN KEY] script_imports_importer_script_id_fkey: FOREIGN KEY (importer_script_id) REFERENCES scripts(id) ON DELETE CASCADE
|
||||
[PRIMARY KEY] script_imports_pkey: PRIMARY KEY (importer_script_id, imported_script_id)
|
||||
|
||||
constraints on scripts:
|
||||
[CHECK] scripts_kind_check: CHECK ((kind = ANY (ARRAY['endpoint'::text, 'module'::text])))
|
||||
[CHECK] scripts_memory_limit_mb_check: CHECK (((memory_limit_mb > 0) AND (memory_limit_mb <= 2048)))
|
||||
[CHECK] scripts_module_name_shape: CHECK (((kind <> 'module'::text) OR (name ~ '^[a-zA-Z_][a-zA-Z0-9_]{0,63}$'::text)))
|
||||
[CHECK] scripts_timeout_seconds_check: CHECK (((timeout_seconds > 0) AND (timeout_seconds <= 300)))
|
||||
[FOREIGN KEY] scripts_app_id_fk: FOREIGN KEY (app_id) REFERENCES apps(id) ON DELETE RESTRICT
|
||||
[PRIMARY KEY] scripts_pkey: PRIMARY KEY (id)
|
||||
|
||||
constraints on triggers:
|
||||
[CHECK] triggers_dispatch_mode_check: CHECK ((dispatch_mode = ANY (ARRAY['sync'::text, 'async'::text])))
|
||||
[CHECK] triggers_kind_check: CHECK ((kind = ANY (ARRAY['kv'::text, 'dead_letter'::text, 'docs'::text, 'cron'::text])))
|
||||
[CHECK] triggers_retry_backoff_check: CHECK ((retry_backoff = ANY (ARRAY['exponential'::text, 'linear'::text, 'constant'::text])))
|
||||
[FOREIGN KEY] triggers_app_id_fkey: FOREIGN KEY (app_id) REFERENCES apps(id) ON DELETE CASCADE
|
||||
[FOREIGN KEY] triggers_registered_by_principal_fkey: FOREIGN KEY (registered_by_principal) REFERENCES admin_users(id) ON DELETE CASCADE
|
||||
[FOREIGN KEY] triggers_script_id_fkey: FOREIGN KEY (script_id) REFERENCES scripts(id) ON DELETE CASCADE
|
||||
[PRIMARY KEY] triggers_pkey: PRIMARY KEY (id)
|
||||
|
||||
## applied migrations
|
||||
0001: init
|
||||
0002: sandbox
|
||||
@@ -215,3 +421,14 @@ constraints on scripts:
|
||||
0004: admin auth
|
||||
0005: apps
|
||||
0006: users authz
|
||||
0007: kv
|
||||
0008: triggers
|
||||
0009: outbox
|
||||
0010: dead letters
|
||||
0011: abandoned executions
|
||||
0012: routes dispatch mode
|
||||
0013: docs
|
||||
0014: docs triggers
|
||||
0015: scripts kind
|
||||
0016: script imports
|
||||
0017: cron triggers
|
||||
|
||||
@@ -16,10 +16,10 @@ use picloud_manager_core::{
|
||||
AdminPrincipalResolver, AdminSessionRepository, AdminState, AdminUserRepository, AdminsState,
|
||||
ApiKeyRepository, ApiKeysState, AppDomainRepository, AppMembersRepository, AppMembersState,
|
||||
AppRepository, AppsState, AuthState, AuthzRepo, DeadLetterRepo, DeadLettersState, Dispatcher,
|
||||
DocsServiceImpl, KvServiceImpl, OutboxEventEmitter, OutboxRepo, PostgresAbandonedRepo,
|
||||
PostgresAdminSessionRepository, PostgresAdminUserRepository, PostgresApiKeyRepository,
|
||||
PostgresAppDomainRepository, PostgresAppMembersRepository, PostgresAppRepository,
|
||||
PostgresDeadLetterRepo, PostgresDeadLetterService, PostgresDocsRepo,
|
||||
DocsServiceImpl, HttpConfig, HttpServiceImpl, KvServiceImpl, OutboxEventEmitter, OutboxRepo,
|
||||
PostgresAbandonedRepo, PostgresAdminSessionRepository, PostgresAdminUserRepository,
|
||||
PostgresApiKeyRepository, PostgresAppDomainRepository, PostgresAppMembersRepository,
|
||||
PostgresAppRepository, PostgresDeadLetterRepo, PostgresDeadLetterService, PostgresDocsRepo,
|
||||
PostgresExecutionLogRepository, PostgresExecutionLogSink, PostgresKvRepo, PostgresOutboxRepo,
|
||||
PostgresRouteRepository, PostgresScriptRepository, PostgresTriggerRepo, PrincipalResolver,
|
||||
RepoResolver, RouteAdminState, RouteRepository, SandboxCeiling, ScriptRepository,
|
||||
@@ -31,9 +31,9 @@ use picloud_orchestrator_core::{
|
||||
LocalExecutorClient,
|
||||
};
|
||||
use picloud_shared::{
|
||||
DeadLetterService, DocsService, ExecutionLogSink, InboxResolver, KvService, OutboxWriter,
|
||||
ScriptValidator, ServiceEventEmitter, Services, API_VERSION, PRODUCT_VERSION, SDK_VERSION,
|
||||
WIRE_VERSION,
|
||||
DeadLetterService, DocsService, ExecutionLogSink, HttpService, InboxResolver, KvService,
|
||||
OutboxWriter, ScriptValidator, ServiceEventEmitter, Services, API_VERSION, PRODUCT_VERSION,
|
||||
SDK_VERSION, WIRE_VERSION,
|
||||
};
|
||||
use sqlx::postgres::PgPoolOptions;
|
||||
use sqlx::PgPool;
|
||||
@@ -143,9 +143,21 @@ pub async fn build_app(pool: PgPool, auth: AuthDeps) -> anyhow::Result<Router> {
|
||||
outbox_repo.clone(),
|
||||
authz.clone(),
|
||||
));
|
||||
let modules: Arc<dyn picloud_shared::ModuleSource> =
|
||||
Arc::new(picloud_manager_core::PostgresModuleSource::new(pool));
|
||||
let services = Services::new(kv, docs, dl_service.clone(), events, modules);
|
||||
let modules: Arc<dyn picloud_shared::ModuleSource> = Arc::new(
|
||||
picloud_manager_core::PostgresModuleSource::new(pool.clone()),
|
||||
);
|
||||
// v1.1.4 outbound HTTP. The reqwest client is built once here with
|
||||
// the SSRF deny-list resolver. `PICLOUD_HTTP_ALLOW_PRIVATE=true`
|
||||
// disables the deny-list entirely — dev/test only, so warn loudly.
|
||||
let http_config = HttpConfig::from_env();
|
||||
if http_config.allow_private {
|
||||
tracing::warn!(
|
||||
"PICLOUD_HTTP_ALLOW_PRIVATE is set — the outbound-HTTP SSRF deny-list is DISABLED. \
|
||||
Scripts can reach loopback/private/link-local addresses. Do NOT use in production."
|
||||
);
|
||||
}
|
||||
let http: Arc<dyn HttpService> = Arc::new(HttpServiceImpl::new(http_config, authz.clone()));
|
||||
let services = Services::new(kv, docs, dl_service.clone(), events, modules, http);
|
||||
let engine = Arc::new(Engine::new(Limits::default(), services));
|
||||
|
||||
// Compile the routes table once at startup; admin writes refresh it.
|
||||
@@ -241,6 +253,10 @@ pub async fn build_app(pool: PgPool, auth: AuthDeps) -> anyhow::Result<Router> {
|
||||
abandoned_repo.clone(),
|
||||
trigger_config.abandoned_retention_days,
|
||||
);
|
||||
// v1.1.4: cron scheduler. Polls cron_trigger_details on a tick and
|
||||
// enqueues due triggers into the outbox; the dispatcher above
|
||||
// delivers them like any other async trigger.
|
||||
picloud_manager_core::spawn_cron_scheduler(pool, trigger_config.cron_tick_interval_ms);
|
||||
let triggers_state = TriggersState {
|
||||
triggers: trigger_repo,
|
||||
apps: apps_repo.clone(),
|
||||
|
||||
137
crates/shared/src/http.rs
Normal file
137
crates/shared/src/http.rs
Normal file
@@ -0,0 +1,137 @@
|
||||
//! `HttpService` — the v1.1.4 outbound-HTTP contract.
|
||||
//!
|
||||
//! Lives in `picloud-shared` (not `executor-core` or `manager-core`)
|
||||
//! so the Rhai bridge and the manager-core reqwest-backed impl can both
|
||||
//! depend on the same trait without dragging `executor-core` into
|
||||
//! `manager-core`'s dep graph — mirrors [`crate::kv`].
|
||||
//!
|
||||
//! Unlike KV/docs, `http::*` has no app-scoped data, so there is no
|
||||
//! cross-app isolation boundary to enforce here. `cx.app_id` is still
|
||||
//! forwarded for audit-log attribution and (future, v1.2) per-app rate
|
||||
//! limits. The load-bearing security mechanism is the SSRF deny-list
|
||||
//! applied to the *resolved IP* — that lives in the manager-core impl,
|
||||
//! not in this contract.
|
||||
//!
|
||||
//! Body encoding + per-method dispatch happen in the Rhai bridge before
|
||||
//! the request reaches this trait: the service receives an already-
|
||||
//! encoded body plus a `content_type`, so the impl stays a thin
|
||||
//! transport layer.
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
use async_trait::async_trait;
|
||||
use thiserror::Error;
|
||||
|
||||
use crate::SdkCallCx;
|
||||
|
||||
/// A fully-resolved outbound request. The bridge builds this from the
|
||||
/// script-facing `(url, body, opts)` arguments; the service backend
|
||||
/// turns it into a real network call.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct HttpRequest {
|
||||
/// Uppercased HTTP method (`GET`, `POST`, …). The escape-hatch
|
||||
/// `http::request(method, …)` lets scripts pass arbitrary methods,
|
||||
/// so the impl validates this rather than the bridge.
|
||||
pub method: String,
|
||||
pub url: String,
|
||||
/// Caller-supplied headers, merged into the request. Header names
|
||||
/// are case-insensitive on the wire; stored verbatim here.
|
||||
pub headers: BTreeMap<String, String>,
|
||||
/// Already-encoded body. `None` means no body (GET/HEAD, or an
|
||||
/// explicit `()` body).
|
||||
pub body: Option<Vec<u8>>,
|
||||
/// Content-Type the bridge chose for `body` (e.g.
|
||||
/// `application/json`). Ignored when the caller set their own
|
||||
/// `Content-Type` header. `None` when there is no body.
|
||||
pub content_type: Option<String>,
|
||||
/// Total request budget in ms (already clamped to the 60s ceiling
|
||||
/// by the bridge).
|
||||
pub timeout_ms: u32,
|
||||
pub follow_redirects: bool,
|
||||
/// Max redirects to follow (already clamped to 10 by the bridge).
|
||||
pub max_redirects: u32,
|
||||
/// Script id for the default `User-Agent` and audit attribution.
|
||||
/// `None` when unavailable (the bridge always sets it from
|
||||
/// `cx`-adjacent context, but the field stays optional so the
|
||||
/// trait isn't coupled to how the id is sourced).
|
||||
pub script_id: Option<String>,
|
||||
}
|
||||
|
||||
/// The response shape the bridge turns into a Rhai map. JSON parsing of
|
||||
/// `body_raw` happens in the bridge (it needs the Rhai value types), so
|
||||
/// the service returns only the raw string + lowercased headers.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct HttpResponse {
|
||||
pub status: u16,
|
||||
/// Header names lowercased (per the documented response shape).
|
||||
pub headers: BTreeMap<String, String>,
|
||||
pub body_raw: String,
|
||||
}
|
||||
|
||||
/// Failure modes surfaced to the Rhai bridge. The bridge prefixes each
|
||||
/// `Display` string with `"http: "`. **None of these may leak the
|
||||
/// resolved IP** — the SSRF reason is a CIDR-category label only.
|
||||
#[derive(Debug, Error)]
|
||||
pub enum HttpError {
|
||||
/// Caller principal lacked `AppHttpRequest`. Only raised when
|
||||
/// `cx.principal.is_some()`; public-HTTP scripts skip the check.
|
||||
#[error("forbidden")]
|
||||
Forbidden,
|
||||
|
||||
/// URL failed to parse, or carried no host.
|
||||
#[error("invalid url: {0}")]
|
||||
InvalidUrl(String),
|
||||
|
||||
/// Scheme other than http/https (file, ftp, gopher, …).
|
||||
#[error("scheme not allowed: {0}")]
|
||||
BlockedScheme(String),
|
||||
|
||||
/// Destination port is on the explicit block list (22, 25, 465, 587).
|
||||
#[error("port not allowed: {0}")]
|
||||
BlockedPort(u16),
|
||||
|
||||
/// Resolved IP hit the SSRF deny-list. `reason` is a CIDR-category
|
||||
/// label (e.g. "loopback", "private", "link-local") — never the IP.
|
||||
#[error("blocked by SSRF policy: {0}")]
|
||||
Ssrf(String),
|
||||
|
||||
/// The request exceeded the wall-clock budget.
|
||||
#[error("request timed out")]
|
||||
Timeout,
|
||||
|
||||
/// Request or response body exceeded the configured size cap.
|
||||
/// `which` is `"request"` or `"response"`.
|
||||
#[error("{0} body exceeds size limit")]
|
||||
BodyTooLarge(&'static str),
|
||||
|
||||
/// DNS / connect / TLS failure. The message is generic and MUST NOT
|
||||
/// contain the resolved IP.
|
||||
#[error("{0}")]
|
||||
Network(String),
|
||||
|
||||
/// Anything else the impl wants to surface (still safe to show a
|
||||
/// script).
|
||||
#[error("{0}")]
|
||||
Backend(String),
|
||||
}
|
||||
|
||||
/// Stub used by the executor-core test harness so engine integration
|
||||
/// tests (which don't make real network calls) can construct a
|
||||
/// `Services` bundle. Every call errors so accidental use surfaces.
|
||||
#[derive(Debug, Default, Clone, Copy)]
|
||||
pub struct NoopHttpService;
|
||||
|
||||
#[async_trait]
|
||||
impl HttpService for NoopHttpService {
|
||||
async fn request(&self, _cx: &SdkCallCx, _req: HttpRequest) -> Result<HttpResponse, HttpError> {
|
||||
Err(HttpError::Network("http is not wired in".into()))
|
||||
}
|
||||
}
|
||||
|
||||
/// Outbound-HTTP contract. A single generic `request` method funnels
|
||||
/// every verb (`get`/`post`/…/`request`); the bridge maps the
|
||||
/// script-facing surface onto it.
|
||||
#[async_trait]
|
||||
pub trait HttpService: Send + Sync {
|
||||
async fn request(&self, cx: &SdkCallCx, req: HttpRequest) -> Result<HttpResponse, HttpError>;
|
||||
}
|
||||
@@ -12,6 +12,7 @@ pub mod error;
|
||||
pub mod events;
|
||||
pub mod exec_summary;
|
||||
pub mod execution_log;
|
||||
pub mod http;
|
||||
pub mod ids;
|
||||
pub mod inbox;
|
||||
pub mod kv;
|
||||
@@ -35,6 +36,7 @@ pub use error::Error;
|
||||
pub use events::{EmitError, NoopEventEmitter, ServiceEvent, ServiceEventEmitter};
|
||||
pub use exec_summary::ExecResponseSummary;
|
||||
pub use execution_log::{ExecutionLog, ExecutionStatus};
|
||||
pub use http::{HttpError, HttpRequest, HttpResponse, HttpService, NoopHttpService};
|
||||
pub use ids::{AdminUserId, ApiKeyId, AppId, ExecutionId, RequestId, ScriptId, TriggerId};
|
||||
pub use inbox::{
|
||||
InboxDeliveryOutcome, InboxFailureKind, InboxResolver, InboxResult, NoopInboxResolver,
|
||||
|
||||
@@ -12,7 +12,7 @@
|
||||
//! the cx in is shared by both sides. Pure value type — no handles, no
|
||||
//! DB pool references, no allocations beyond what's in `Principal`.
|
||||
|
||||
use crate::{AppId, ExecutionId, Principal, RequestId, TriggerEvent};
|
||||
use crate::{AppId, ExecutionId, Principal, RequestId, ScriptId, TriggerEvent};
|
||||
|
||||
/// Per-invocation context for every stateful SDK service call.
|
||||
///
|
||||
@@ -27,6 +27,11 @@ pub struct SdkCallCx {
|
||||
/// every `(app_id, …)` storage lookup the script makes.
|
||||
pub app_id: AppId,
|
||||
|
||||
/// The script being executed. Used for audit-log attribution and
|
||||
/// the default outbound-HTTP `User-Agent` (`picloud/<v>
|
||||
/// (script:<id>)`). Added in v1.1.4 for the `http::*` SDK.
|
||||
pub script_id: ScriptId,
|
||||
|
||||
/// Caller identity, when authenticated. `None` for unauthenticated
|
||||
/// data-plane HTTP requests (the common case for public endpoints);
|
||||
/// `Some` when the call came in via the dashboard, an API key, or a
|
||||
|
||||
@@ -20,8 +20,9 @@
|
||||
use std::sync::Arc;
|
||||
|
||||
use crate::{
|
||||
DeadLetterService, DocsService, KvService, ModuleSource, NoopDeadLetterService,
|
||||
NoopDocsService, NoopEventEmitter, NoopKvService, NoopModuleSource, ServiceEventEmitter,
|
||||
DeadLetterService, DocsService, HttpService, KvService, ModuleSource, NoopDeadLetterService,
|
||||
NoopDocsService, NoopEventEmitter, NoopHttpService, NoopKvService, NoopModuleSource,
|
||||
ServiceEventEmitter,
|
||||
};
|
||||
|
||||
/// SDK service bundle. See module docs for the lifecycle and the v1.1.x
|
||||
@@ -53,6 +54,12 @@ pub struct Services {
|
||||
/// `import`. Backed by Postgres in the picloud binary; in-memory
|
||||
/// fakes in resolver tests.
|
||||
pub modules: Arc<dyn ModuleSource>,
|
||||
|
||||
/// Outbound HTTP (v1.1.4). Scripts get `http::{get,post,…}`.
|
||||
/// Backed by a reqwest client with the SSRF deny-list resolver in
|
||||
/// the picloud binary; `NoopHttpService` in tests that don't make
|
||||
/// network calls.
|
||||
pub http: Arc<dyn HttpService>,
|
||||
}
|
||||
|
||||
impl Services {
|
||||
@@ -66,6 +73,7 @@ impl Services {
|
||||
dead_letters: Arc<dyn DeadLetterService>,
|
||||
events: Arc<dyn ServiceEventEmitter>,
|
||||
modules: Arc<dyn ModuleSource>,
|
||||
http: Arc<dyn HttpService>,
|
||||
) -> Self {
|
||||
Self {
|
||||
kv,
|
||||
@@ -73,6 +81,7 @@ impl Services {
|
||||
dead_letters,
|
||||
events,
|
||||
modules,
|
||||
http,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -89,6 +98,7 @@ impl Services {
|
||||
Arc::new(NoopDeadLetterService),
|
||||
Arc::new(NoopEventEmitter),
|
||||
Arc::new(NoopModuleSource),
|
||||
Arc::new(NoopHttpService),
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -111,6 +111,18 @@ pub enum TriggerEvent {
|
||||
prev_data: Option<serde_json::Value>,
|
||||
},
|
||||
|
||||
/// A cron schedule fired this handler. v1.1.4. Carries the
|
||||
/// schedule + timezone the trigger was configured with, the
|
||||
/// canonical cron moment (`scheduled_at`, the instant the
|
||||
/// expression *meant*), and when the scheduler actually enqueued
|
||||
/// the fire (`fired_at`). Surfaced to scripts as `ctx.event.cron`.
|
||||
Cron {
|
||||
schedule: String,
|
||||
timezone: String,
|
||||
scheduled_at: DateTime<Utc>,
|
||||
fired_at: DateTime<Utc>,
|
||||
},
|
||||
|
||||
/// A dead-letter row fired this handler. The original event is
|
||||
/// nested verbatim plus the dead-letter metadata the design notes
|
||||
/// §4 require.
|
||||
@@ -135,6 +147,7 @@ impl TriggerEvent {
|
||||
match self {
|
||||
Self::Kv { .. } => "kv",
|
||||
Self::Docs { .. } => "docs",
|
||||
Self::Cron { .. } => "cron",
|
||||
Self::DeadLetter { .. } => "dead_letter",
|
||||
}
|
||||
}
|
||||
|
||||
@@ -33,7 +33,13 @@ pub const PRODUCT_VERSION: &str = env!("CARGO_PKG_VERSION");
|
||||
/// app. Cross-app imports are unreachable (the `name` argument carries
|
||||
/// no `app_id`). Modules expose `fn`/`const` declarations only;
|
||||
/// top-level statements are rejected at create-time.
|
||||
pub const SDK_VERSION: &str = "1.4";
|
||||
///
|
||||
/// 1.5 additions (v1.1.4): `http::{get,post,put,patch,delete,head,
|
||||
/// post_form,request}` for outbound HTTP from scripts (guarded by an
|
||||
/// SSRF deny-list on the resolved IP); `ctx.event.cron` for cron-trigger
|
||||
/// handlers (carries `schedule`, `timezone`, `scheduled_at`, `fired_at`).
|
||||
/// The `Services` bundle gains `http: Arc<dyn HttpService>`.
|
||||
pub const SDK_VERSION: &str = "1.5";
|
||||
|
||||
/// HTTP API major version. Appears in URL paths as `/api/v{N}/...`.
|
||||
/// Bump (new integer + new URL prefix) when the request/response
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "picloud-dashboard",
|
||||
"version": "0.9.0",
|
||||
"version": "0.10.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
|
||||
@@ -211,6 +211,42 @@ export interface DeadLetterRow {
|
||||
resolution: 'replayed' | 'ignored' | 'handled_by_script' | 'handler_failed' | null;
|
||||
}
|
||||
|
||||
export type TriggerKind = 'kv' | 'docs' | 'dead_letter' | 'cron';
|
||||
export type TriggerDispatchMode = 'sync' | 'async';
|
||||
|
||||
/// Per-kind detail, tagged by `kind` to match the Rust serde shape.
|
||||
export type TriggerDetails =
|
||||
| { kind: 'kv'; collection_glob: string; ops: string[] }
|
||||
| { kind: 'docs'; collection_glob: string; ops: string[] }
|
||||
| { kind: 'dead_letter'; source_filter?: string; trigger_id_filter?: string; script_id_filter?: string }
|
||||
| { kind: 'cron'; schedule: string; timezone: string; last_fired_at?: string | null };
|
||||
|
||||
export interface Trigger {
|
||||
id: string;
|
||||
app_id: string;
|
||||
script_id: string;
|
||||
kind: TriggerKind;
|
||||
enabled: boolean;
|
||||
dispatch_mode: TriggerDispatchMode;
|
||||
retry_max_attempts: number;
|
||||
retry_backoff: 'exponential' | 'linear' | 'constant';
|
||||
retry_base_ms: number;
|
||||
registered_by_principal: string;
|
||||
created_at: string;
|
||||
updated_at: string;
|
||||
details: TriggerDetails;
|
||||
}
|
||||
|
||||
export interface CreateCronTriggerInput {
|
||||
script_id: string;
|
||||
schedule: string;
|
||||
timezone: string;
|
||||
dispatch_mode?: TriggerDispatchMode;
|
||||
retry_max_attempts?: number;
|
||||
retry_backoff?: 'exponential' | 'linear' | 'constant';
|
||||
retry_base_ms?: number;
|
||||
}
|
||||
|
||||
export interface ExecutionResult {
|
||||
status: number;
|
||||
headers: Record<string, string>;
|
||||
@@ -572,6 +608,23 @@ export const api = {
|
||||
)
|
||||
},
|
||||
|
||||
triggers: {
|
||||
list: (idOrSlug: string) =>
|
||||
adminRequest<{ triggers: Trigger[] }>(
|
||||
`/api/v1/admin/apps/${encodeURIComponent(idOrSlug)}/triggers`
|
||||
),
|
||||
createCron: (idOrSlug: string, input: CreateCronTriggerInput) =>
|
||||
adminRequest<Trigger>(
|
||||
`/api/v1/admin/apps/${encodeURIComponent(idOrSlug)}/triggers/cron`,
|
||||
{ method: 'POST', body: JSON.stringify(input) }
|
||||
),
|
||||
remove: (idOrSlug: string, triggerId: string) =>
|
||||
adminRequest<null>(
|
||||
`/api/v1/admin/apps/${encodeURIComponent(idOrSlug)}/triggers/${triggerId}`,
|
||||
{ method: 'DELETE' }
|
||||
)
|
||||
},
|
||||
|
||||
execute: async (
|
||||
id: string,
|
||||
body: unknown,
|
||||
|
||||
@@ -10,7 +10,8 @@
|
||||
type AppDomain,
|
||||
type AppMemberDto,
|
||||
type AppRole,
|
||||
type Script
|
||||
type Script,
|
||||
type Trigger
|
||||
} from '$lib/api';
|
||||
import CodeEditor from '$lib/CodeEditor.svelte';
|
||||
import ConfirmModal from '$lib/ConfirmModal.svelte';
|
||||
@@ -24,7 +25,26 @@
|
||||
const SAMPLE_SOURCE =
|
||||
'#{\n statusCode: 200,\n body: #{ ok: true, echo: ctx.request.body }\n}';
|
||||
|
||||
type Tab = 'scripts' | 'domains' | 'members' | 'settings';
|
||||
type Tab = 'scripts' | 'domains' | 'members' | 'settings' | 'triggers';
|
||||
|
||||
// Common IANA timezones offered in the cron form dropdown. Not
|
||||
// exhaustive — the backend validates any IANA name via chrono-tz.
|
||||
const COMMON_TIMEZONES = [
|
||||
'UTC',
|
||||
'America/Los_Angeles',
|
||||
'America/Denver',
|
||||
'America/Chicago',
|
||||
'America/New_York',
|
||||
'America/Sao_Paulo',
|
||||
'Europe/London',
|
||||
'Europe/Berlin',
|
||||
'Europe/Paris',
|
||||
'Europe/Moscow',
|
||||
'Asia/Kolkata',
|
||||
'Asia/Shanghai',
|
||||
'Asia/Tokyo',
|
||||
'Australia/Sydney'
|
||||
];
|
||||
|
||||
let slug = $derived(page.params.slug ?? '');
|
||||
let app = $state<App | null>(null);
|
||||
@@ -91,6 +111,63 @@
|
||||
let removingDomain = $state(false);
|
||||
let removeDomainError = $state<string | null>(null);
|
||||
|
||||
// Triggers tab (v1.1.4 — cron triggers). Admin-gated, like Members.
|
||||
let triggers = $state<Trigger[]>([]);
|
||||
let createCronScriptId = $state('');
|
||||
let createCronSchedule = $state('0 0 9 * * MON-FRI');
|
||||
let createCronTimezone = $state('UTC');
|
||||
let creatingCron = $state(false);
|
||||
let createCronError = $state<string | null>(null);
|
||||
let triggerToRemove = $state<Trigger | null>(null);
|
||||
let removingTrigger = $state(false);
|
||||
// Endpoint scripts only — modules can't be trigger targets.
|
||||
const endpointScripts = $derived(scripts.filter((s) => s.kind === 'endpoint'));
|
||||
|
||||
async function loadTriggers(idOrSlug: string) {
|
||||
try {
|
||||
const r = await api.triggers.list(idOrSlug);
|
||||
triggers = r.triggers;
|
||||
} catch {
|
||||
triggers = [];
|
||||
}
|
||||
}
|
||||
|
||||
async function submitCreateCron(e: SubmitEvent) {
|
||||
e.preventDefault();
|
||||
if (!app) return;
|
||||
creatingCron = true;
|
||||
createCronError = null;
|
||||
try {
|
||||
await api.triggers.createCron(app.id, {
|
||||
script_id: createCronScriptId,
|
||||
schedule: createCronSchedule.trim(),
|
||||
timezone: createCronTimezone
|
||||
});
|
||||
createCronScriptId = '';
|
||||
await loadTriggers(app.id);
|
||||
} catch (err) {
|
||||
createCronError =
|
||||
err instanceof ApiError ? err.message : err instanceof Error ? err.message : String(err);
|
||||
} finally {
|
||||
creatingCron = false;
|
||||
}
|
||||
}
|
||||
|
||||
async function confirmRemoveTrigger() {
|
||||
if (!app || !triggerToRemove) return;
|
||||
removingTrigger = true;
|
||||
try {
|
||||
await api.triggers.remove(app.id, triggerToRemove.id);
|
||||
triggerToRemove = null;
|
||||
await loadTriggers(app.id);
|
||||
} catch (err) {
|
||||
createCronError =
|
||||
err instanceof ApiError ? err.message : err instanceof Error ? err.message : String(err);
|
||||
} finally {
|
||||
removingTrigger = false;
|
||||
}
|
||||
}
|
||||
|
||||
// Members tab
|
||||
let eligibleUsers = $state<AdminDto[]>([]);
|
||||
let eligibleLoadError = $state<string | null>(null);
|
||||
@@ -131,7 +208,7 @@
|
||||
loadDeadLetterCount(app.id)
|
||||
];
|
||||
if (canAdmin) {
|
||||
loaders.push(loadMembers(app.id), loadEligibleUsers());
|
||||
loaders.push(loadMembers(app.id), loadEligibleUsers(), loadTriggers(app.id));
|
||||
}
|
||||
await Promise.all(loaders);
|
||||
} catch (e) {
|
||||
@@ -398,7 +475,10 @@
|
||||
// backend still 403s the underlying calls, but no point showing an
|
||||
// empty tab.
|
||||
$effect(() => {
|
||||
if (!canAdmin && (activeTab === 'settings' || activeTab === 'members')) {
|
||||
if (
|
||||
!canAdmin &&
|
||||
(activeTab === 'settings' || activeTab === 'members' || activeTab === 'triggers')
|
||||
) {
|
||||
activeTab = 'scripts';
|
||||
}
|
||||
});
|
||||
@@ -440,6 +520,11 @@
|
||||
class:active={activeTab === 'members'}
|
||||
onclick={() => (activeTab = 'members')}>Members ({members.length})</button
|
||||
>
|
||||
<button
|
||||
type="button"
|
||||
class:active={activeTab === 'triggers'}
|
||||
onclick={() => (activeTab = 'triggers')}>Triggers ({triggers.length})</button
|
||||
>
|
||||
<button
|
||||
type="button"
|
||||
class:active={activeTab === 'settings'}
|
||||
@@ -698,6 +783,91 @@
|
||||
</div>
|
||||
{/if}
|
||||
</section>
|
||||
{:else if activeTab === 'triggers' && canAdmin}
|
||||
<section>
|
||||
<h2>Cron triggers</h2>
|
||||
<p class="muted">
|
||||
Run an endpoint script on a schedule. Schedules are 6-field cron
|
||||
expressions (with seconds): <code>sec min hour day-of-month month day-of-week</code>.
|
||||
The timezone disambiguates schedules like "every weekday at 9am".
|
||||
</p>
|
||||
|
||||
<form class="create-form" onsubmit={submitCreateCron}>
|
||||
<div class="row">
|
||||
<label>
|
||||
<span>Target script</span>
|
||||
<select bind:value={createCronScriptId} required>
|
||||
<option value="" disabled>Select an endpoint script…</option>
|
||||
{#each endpointScripts as s (s.id)}
|
||||
<option value={s.id}>{s.name}</option>
|
||||
{/each}
|
||||
</select>
|
||||
</label>
|
||||
<label>
|
||||
<span>Schedule</span>
|
||||
<input
|
||||
bind:value={createCronSchedule}
|
||||
required
|
||||
placeholder="0 0 9 * * MON-FRI"
|
||||
/>
|
||||
</label>
|
||||
<label>
|
||||
<span>Timezone</span>
|
||||
<select bind:value={createCronTimezone}>
|
||||
{#each COMMON_TIMEZONES as tz (tz)}
|
||||
<option value={tz}>{tz}</option>
|
||||
{/each}
|
||||
</select>
|
||||
</label>
|
||||
</div>
|
||||
{#if endpointScripts.length === 0}
|
||||
<p class="muted small">
|
||||
This app has no endpoint scripts yet — create one first (modules
|
||||
can't be trigger targets).
|
||||
</p>
|
||||
{/if}
|
||||
{#if createCronError}
|
||||
<div class="error">{createCronError}</div>
|
||||
{/if}
|
||||
<div class="actions">
|
||||
<button type="submit" disabled={creatingCron || !createCronScriptId}>
|
||||
{creatingCron ? 'Creating…' : 'Create cron trigger'}
|
||||
</button>
|
||||
</div>
|
||||
</form>
|
||||
|
||||
{#if triggers.length === 0}
|
||||
<p class="muted">No triggers in this app yet.</p>
|
||||
{:else}
|
||||
<ul class="list">
|
||||
{#each triggers as t (t.id)}
|
||||
<li class="domain-row">
|
||||
<div>
|
||||
<span class="kind-badge">{t.kind}</span>
|
||||
{#if t.details.kind === 'cron'}
|
||||
<code>{t.details.schedule}</code>
|
||||
<span class="muted">— {t.details.timezone}</span>
|
||||
<span class="muted small">
|
||||
last fired: {t.details.last_fired_at ?? 'never'}
|
||||
</span>
|
||||
{:else if t.details.kind === 'kv' || t.details.kind === 'docs'}
|
||||
<code>{t.details.collection_glob}</code>
|
||||
<span class="muted">— {t.details.ops.join(', ') || 'any op'}</span>
|
||||
{/if}
|
||||
<span class="muted small">→ {t.script_id}</span>
|
||||
</div>
|
||||
<button
|
||||
type="button"
|
||||
class="secondary danger"
|
||||
onclick={() => (triggerToRemove = t)}
|
||||
>
|
||||
Delete
|
||||
</button>
|
||||
</li>
|
||||
{/each}
|
||||
</ul>
|
||||
{/if}
|
||||
</section>
|
||||
{:else if activeTab === 'settings' && canAdmin}
|
||||
<section>
|
||||
<h2>Settings</h2>
|
||||
@@ -855,6 +1025,23 @@
|
||||
{/if}
|
||||
</ConfirmModal>
|
||||
{/if}
|
||||
|
||||
{#if triggerToRemove}
|
||||
<ConfirmModal
|
||||
title="Delete trigger"
|
||||
variant="danger"
|
||||
confirmLabel="Delete trigger"
|
||||
busyLabel="Deleting…"
|
||||
busy={removingTrigger}
|
||||
onConfirm={confirmRemoveTrigger}
|
||||
onCancel={() => (triggerToRemove = null)}
|
||||
>
|
||||
<p>
|
||||
This {triggerToRemove.kind} trigger will stop firing. The target
|
||||
script is not affected.
|
||||
</p>
|
||||
</ConfirmModal>
|
||||
{/if}
|
||||
{/if}
|
||||
|
||||
<style>
|
||||
|
||||
Reference in New Issue
Block a user