Compare commits

...

3 Commits

Author SHA1 Message Date
MechaCat02
3b650a2b14 feat: declarative project-tool foundation (pull/plan/apply/prune)
Add a server-side, atomic, declarative reconcile loop for a single app —
the foundation of the project-tool design. Developers describe an app's
scripts, routes, triggers, and secret-names in `picloud.toml`, then
`pic pull / plan / apply [--prune]` to converge live state to the manifest.

Server (manager-core):
- apply_service: a pure diff engine (compute_diff) shared by plan and
  apply, plus an ApplyService that composes the existing per-repo writes
  into ONE Postgres transaction. Identity keys mirror the DB UNIQUE
  constraints (script=lower(name); route=(method,host_kind,host,
  path_kind,path); trigger=per-kind semantic tuple; secret=name).
  Apply takes a per-app advisory lock, recomputes the diff in-tx, applies
  scripts -> routes -> triggers, prunes dependents-first, commits, then
  refreshes the route table once post-commit.
- apply_api: POST /apps/{id}/plan (AppRead) and /apps/{id}/apply.
  Apply requires the per-kind write caps the bundle exercises (all three
  when --prune), plus AppSecretsRead when it binds an email trigger.
- tx-accepting repo siblings (insert/update/delete *_tx) so the existing
  create/update/delete delegate to one SQL definition each.
- email triggers reference an inbound secret by NAME; the value is
  resolved, decrypted (AAD-bound), and re-sealed server-side at apply —
  it never travels in the manifest.

CLI (picloud-cli):
- manifest.rs (picloud.toml model), client plan/apply, and the pull/plan/
  apply commands. pull rejects filesystem-unsafe script names up front.

Safety properties enforced and tested:
- idempotent: a freshly-pulled manifest re-applies as all-NoOp.
- atomic: a mid-bundle failure rolls back with nothing written.
- routes delete-before-insert so a freed binding is reusable in one apply.
- queue one-consumer invariant held inside the shared tx.
- email triggers are never pruned, and a script that still owns an
  email/dead-letter trigger can't be pruned (the FK cascade would destroy
  the sealed secret) — refused with a pointer to `pic triggers rm`.
- plan and apply agree on unset email-secret references.

No migration: the existing schema's UNIQUE constraints serve as identity
keys. Groups, env-scoping, and the `enabled` toggle are later milestones.

Tested: manager-core lib (360) + CLI bins (27) + 8 project-tool journeys
(pull/plan/apply/prune/email+queue), all green; clippy -D warnings clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 21:52:21 +02:00
MechaCat02
c600177fd6 docs(design): groups & declarative project-tool spec
The architecture spec for the full vision — a declarative project tool
(picloud.toml + pull/plan/apply/prune) and a server-side nested-groups
hierarchy with live-resolve inheritance. Reviewed across several passes
for consistency, gaps, and contradictions; resolutions are folded in
beside the topics they settle.

This commit lands the spec only. The first slice of implementation (the
single-app reconcile foundation) follows; groups, env-scoping, and the
`enabled` toggle are later milestones called out in the doc.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 21:52:01 +02:00
MechaCat02
51f14fa2b1 feat: E2E #2 (Stash) gap remediation + S6 hardening
Some checks failed
CI / Rust — fmt, clippy, test (push) Failing after 6m19s
CI / Dashboard — check (push) Successful in 9m48s
Closes the gaps and the one security finding from the second end-to-end
CLI test (E2E_STASH_REPORT.md), plus the H1 boot-regression found while
re-reviewing those fixes.

Security
- S6: reserved-path validation (`check_reserved`) now case-folds before
  comparing, so `/API/v2/x`, `/HEALTHZ`, `/Admin/x` are rejected like
  their lowercase forms. Request-time matching stays case-sensitive.
- S10: "public route != public data" callout in sdk-shape.md (script_gate
  skips authz when the principal is anonymous).

Observability / features
- G1: trigger executions now write `execution_logs`. Migration 0043 adds
  a `source` column (CHECK mirrors ExecutionSource/OutboxSourceKind,
  DEFAULT 'http' backfills history); a shared `build_execution_log` helper
  in executor-core; dispatcher logging for outbox triggers + queue
  consumers (skips sync-HTTP rows the orchestrator already logs). `pic
  logs` gains a source column + `--source` filter.
- G5: dev-only in-memory email capture under PICLOUD_DEV_MODE with no SMTP
  (email::send succeeds locally), readable at GET /api/v1/admin/dev/emails
  (Owner/Admin only; route mounted only in capture mode).
- G6: generalized the Rhai in-place-mutation footgun note (trim/replace/
  make_upper/make_lower/crop/truncate/pad return ()).
- G2/G3/G4 (CLI): `pic members`, `pic files`, `pic queues`, read-only
  `pic kv` (+ new kv_api.rs); `pic deploy --timeout/--memory/--kind/
  --sandbox`; first-class `pic triggers create-{docs,files,pubsub,queue,
  email}` wrappers. All new client path segments percent-encoded via seg().

H1 regression fix (found in re-review)
- The S6 change also runs in `compile_routes`, which compiles every stored
  route at boot and on each route CRUD. A single stored route the new
  validation rejects (creatable while the S6 gap existed) made the whole
  compile Err and aborted startup. `compile_routes` is now lenient: it
  skips an un-compilable row with a warning instead of bricking boot
  (route creation still validates separately). Migration 0044 sweeps
  pre-existing reserved-path routes on upgrade (WHERE mirrors
  check_reserved exactly). Added regression tests for both.

Verified: cargo fmt, clippy --all-targets --all-features -D warnings, the
schema_snapshot test, and the new S6/lenient-compile unit tests all pass;
boot-resilience and G1/G5 confirmed live.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-13 15:01:04 +02:00
48 changed files with 7126 additions and 323 deletions

View File

@@ -117,7 +117,7 @@ Environment variables consumed by the `picloud` binary:
| `PICLOUD_MAX_CONCURRENT_EXECUTIONS` | `32` | Global concurrency cap on data-plane script executions. Overflow returns HTTP 503 with `Retry-After: 1` immediately (no queue). | | `PICLOUD_MAX_CONCURRENT_EXECUTIONS` | `32` | Global concurrency cap on data-plane script executions. Overflow returns HTTP 503 with `Retry-After: 1` immediately (no queue). |
| `DATABASE_URL` | — | Required. Postgres connection string. | | `DATABASE_URL` | — | Required. Postgres connection string. |
| `PICLOUD_SECRET_KEY` | — | Master encryption key (base64). Required at startup unless dev mode is acknowledged (below). | | `PICLOUD_SECRET_KEY` | — | Master encryption key (base64). Required at startup unless dev mode is acknowledged (below). |
| `PICLOUD_DEV_MODE` | `false` | `true` enables local-dev conveniences. Without `PICLOUD_SECRET_KEY` it ALSO requires the acknowledgement var below — `PICLOUD_DEV_MODE=true` alone aborts at startup. | | `PICLOUD_DEV_MODE` | `false` | `true` enables local-dev conveniences. Without `PICLOUD_SECRET_KEY` it ALSO requires the acknowledgement var below — `PICLOUD_DEV_MODE=true` alone aborts at startup. Also: when no SMTP relay is configured, `email::send` switches from disabled (`NotConfigured`) to an **in-memory dev sink** — sends succeed and the last 100 messages are readable at `GET /api/v1/admin/dev/emails` (instance Owner/Admin only; route exists only in this mode). Never in production. |
| `PICLOUD_DEV_INSECURE_KEY` | — | Set to the literal `i-understand-this-is-insecure` to let dev mode boot without `PICLOUD_SECRET_KEY`, using a deterministic, world-known dev master key. Never set in production — it would encrypt everything with a public value. | | `PICLOUD_DEV_INSECURE_KEY` | — | Set to the literal `i-understand-this-is-insecure` to let dev mode boot without `PICLOUD_SECRET_KEY`, using a deterministic, world-known dev master key. Never set in production — it would encrypt everything with a public value. |
| `PICLOUD_DB_MAX_CONNECTIONS` | `32` | Postgres pool size. Matched to `PICLOUD_MAX_CONCURRENT_EXECUTIONS` so the data plane can't starve background workers. | | `PICLOUD_DB_MAX_CONNECTIONS` | `32` | Postgres pool size. Matched to `PICLOUD_MAX_CONCURRENT_EXECUTIONS` so the data plane can't starve background workers. |
| `PICLOUD_SESSION_TTL_HOURS` | `24` | Sliding-window session lifetime. | | `PICLOUD_SESSION_TTL_HOURS` | `24` | Sliding-window session lifetime. |

138
E2E_STASH_REPORT.md Normal file
View File

@@ -0,0 +1,138 @@
# E2E Test #2 — "Stash" (paste + file-drop) + security matrix
**Date:** 2026-06-13 · **Build:** v1.1.9 · **Instance:** host `picloud` on `127.0.0.1:8099`
(dev mode, `PICLOUD_FILES_MAX_FILE_SIZE_BYTES=1048576`, SSRF guard on, SMTP unset), Postgres in
Docker. Driven entirely through the `pic` CLI; data-plane traffic via `curl`.
## Goal
Second end-to-end pass. The first test (To-Do, `E2E_TODO_REPORT.md`) covered users/docs/pubsub/
cron/routes/domains/topics. This one builds a **new** app to exercise the *untested* half of the
platform — **kv, files, queue, invoke, secrets, api-keys, dead-letters, async + two-param routes**
— and then runs a **10-probe security matrix**. Findings: feature/CLI gaps + security verdicts.
## Verdict
**Every feature worked** end-to-end, and **every security control held except one Low-severity
gap** (reserved-path validation is case-sensitive). Highlights of what's *good*: cross-app
isolation is airtight, the SSRF guard blocks cloud-metadata, size/op caps fire before authz, and
internal script errors are scrubbed from responses with a correlation id. Two notable
**observability/ergonomics gaps** (trigger executions invisible to `pic logs`; the `.trim()`/`.replace()`
return-`()` footgun) and the expected CLI-coverage gaps round it out.
---
## The app — "Stash" (built CLI-only, no raw admin curl)
A pastebin with file attachments and a background word-count worker. 9 scripts, 6 routes, a queue
consumer trigger, a dead-letter trigger, 2 API keys, 1 secret — all via `pic`.
| Feature exercised | How | Result |
|---|---|---|
| **kv** | paste store, hit counter, `/stats` aggregates, event log | ✅ get/set/list round-trip |
| **files** | `files::create` from base64 blob + `get`/`head`, two-param download route | ✅ bytes round-trip (`"hello attachment bytes"`, size 22) |
| **queue** | `queue::enqueue("ingest")` + `create-from-json --kind queue` consumer | ✅ worker fired |
| **invoke** | worker → `invoke("analyzer", #{text})` function-to-function | ✅ returns word count |
| **secrets** | `pic secrets set` (stdin) + `secrets::get` to gate `/stats` | ✅ 401 w/o token, 200 with |
| **async route** | `routes create --dispatch async` for `POST /events` | ✅ **202** + execution_id |
| **dead-letters** | poison job → 3 attempts → DL row; `pic dead-letters replay` | ✅ captured + replayed (+9 words) |
| **api-keys** | `pic api-keys mint --scope … [--app …]` | ✅ minted, scopes enforced (below) |
| **two-param route** | `/pastes/:code/files/:fid` | ✅ both params captured |
End-to-end stats after the run: `{total_pastes: 2, total_words: 13}` — proving queue → worker →
invoke → kv all fired on real Postgres rows + on-disk file bytes.
---
## Security matrix (10 probes, all reproduced live)
| # | Probe | Verdict | Evidence |
|---|---|---|---|
| **S1** | Cross-app isolation | ✅ HOLDS | A script in app `evil` doing `kv…list()` / `secrets::get("admin-token")` / `files…list()` returned `{paste_keys:[], attachment_count:0, stolen_secret:null, my_secret_names:[]}`. `app_id` derives from `cx.app_id`; no SDK call takes a script-passed app_id. |
| **S2** | Secret confidentiality | ✅ HOLDS | `pic secrets ls` shows names only; the plaintext `sup3r-s3cret-admin` does **not** appear anywhere in the server log even though `/stats` reads it each request. |
| **S3** | API-key scope + app-binding | ✅ HOLDS | `script:read` key → **403** on `POST /apps` and `GET /admins`, **200** on in-scope `GET /scripts`. `app:admin` key bound to `stash`**403** on app `evil`. `instance:admin` + `--app`**422**. Unknown scope `bogus:scope` → rejected by CLI. |
| **S4** | Sandbox op-budget | ✅ HOLDS | `loop {}`**HTTP 507 "execution exceeded operation budget"** in **0.30 s**; `/healthz` still `ok` (no hang). |
| **S5** | Value/size caps | ✅ HOLDS | Oversized kv write rejected (HTTP 502). Notably the detail (`"Length of string too large"`) was **scrubbed from the response** and logged with `correlation_id` — good error hygiene. Caps are checked before authz, so anonymous callers can't DoS. |
| **S6** | Reserved-path prefixes | ⚠️ **GAP (Low)** | `/api/x`, `/admin/x`, `/healthz`, `/version` → correctly **422**. But case variants `/API/v2/x`, `/Admin/x`, `/HEALTHZ` were **accepted**. See finding below. |
| **S7** | Files path traversal | ✅ HOLDS | `files::collection("../../../etc")``"invalid collection name: must not contain '/', '\\', '..', or NUL"`; a non-UUID file id → not found. No FS escape. |
| **S8** | docs filter injection | ✅ HOLDS | `docs::find(#{ "x'); DROP TABLE docs;--": 1 })` → ran safely, `count=0`; the `docs` table still exists (`to_regclass` = true). Values/paths are bound as `$N` params. |
| **S9** | SSRF via `http::` | ✅ HOLDS | `http::get("http://127.0.0.1:8099/healthz")``"blocked by SSRF policy: loopback"`; `http://169.254.169.254/…` (cloud metadata) → `"blocked by SSRF policy: link-local"`. |
| **S10** | Anonymous data-plane access | By design (caveat) | Every Stash route is public (no auth) yet freely uses kv/secrets/files/queue. `script_gate` returns `Ok` when `principal.is_none()` — the *script* is the gate. Correct per design, but a sharp threat-model edge (see note). |
### S6 — reserved-path validation is case-sensitive (Low)
**What:** route-creation rejects the exact reserved prefixes but accepts case variants:
```
pic routes create --path /api/x -> 422 "path '/api/x' is reserved"
pic routes create --path /API/v2/x -> Created route … (GET * /API/v2/x) # accepted
pic routes create --path /HEALTHZ -> Created route … # accepted
pic routes create --path /Admin/x -> Created route … # accepted
```
**Current impact — Low, *not* a full bypass:** path matching is case-*sensitive*, so a request to
the real lowercase `/api/v2/x` still 404s (the system namespace is safe); only the exact
mixed-case path serves the script (`GET /API/v2/x` → 200, `/HEALTHZ` → 200). Real `/healthz`
still returns `ok` from the top-level handler.
**Why it still matters:** (1) inconsistent enforcement of a stated security boundary; (2) lets a
tenant publish convincing look-alike paths (`/Admin/login`, `/API/v1/…`) for phishing or log
confusion; (3) **fragile** — method matching is already case-insensitive (`matcher.rs:275`); if
path matching is ever made case-insensitive too, this instantly becomes a real shadowing bypass
of `/api/`, `/admin/`, `/healthz`. The validation should be the durable guard.
**Fix:** normalize case before the reserved check in `orchestrator-core/src/routing/pattern.rs`
(`check_reserved` ~line 110) — compare `raw.to_ascii_lowercase()` against the reserved list (or
reject any case-insensitive match).
### S10 — anonymous public scripts have full app data-plane access (design caveat)
Not a bug, but worth surfacing for users: a *public* (unauthenticated) route's script can read/write
**all** of the app's kv, docs, files, secrets, and queue — the platform only gates *authenticated*
principals; for anonymous ingress the script itself must enforce access. A developer who assumes
"this route is public" ≠ "this code can read every secret in my app" could over-expose data. Worth
a prominent doc callout near the SDK auth model.
---
## Feature / CLI gaps
- **G1 — trigger executions are invisible to `pic logs`.** After successful queue-worker and
analyzer runs, `pic logs <worker>` and `pic logs <analyzer>` were **empty**. Trigger-dispatched
executions (queue/cron/dead-letter/invoke) don't surface in the per-script log tail — the only
built-in visibility into worker activity is **dead-letters (failures only)** or the script's own
side effects. This is a real observability gap for background workloads. *Suggest: include
trigger executions in the logs surface, or add a `pic logs --trigger`/events view.*
- **G2 — no `pic kv` / `files` / `queues` / `members` commands.** Data-plane stores are
script/HTTP-only (no admin CLI to inspect kv/files/queue contents; queues are read-only HTTP).
App **membership** (`/apps/{id}/members`) also has no CLI — multi-user app roles can't be managed
with `pic`. *(Confirmed absent via `pic <cmd> --help`.)*
- **G3 — no per-script sandbox/timeout flags in `pic deploy`/`scripts`.** `timeout_seconds`,
`memory_limit_mb`, and sandbox overrides are only settable via the raw scripts API or instance
env (`PICLOUD_SANDBOX_MAX_*`). A developer can't cap a single script's runtime from the CLI.
- **G4 — uneven trigger wrappers.** Only `kv`, `cron`, `dead-letter` have first-class
`pic triggers create-*` wrappers; `docs`/`files`/`pubsub`/`queue`/`email` require hand-built
`create-from-json --kind … --body '{…}'`. Works, but the developer must know each body schema.
- **G5 — `email::send` unusable in dev.** Returns `"email is not configured: set
PICLOUD_SMTP_HOST/USER/PASSWORD"`. Expected (no silent drop), but the email feature can't be
exercised at all without an SMTP relay — worth a documented local-dev fake/sink.
- **G6 — Rhai `.trim()` returns `()` (in-place mutation footgun).** Same family as `.replace()`
(already documented). `let t = text.trim()` sets `t` to unit; my analyzer dead-lettered with
`Function not found: split((), …)` until rewritten to `text.trim();` as a statement. *Suggest:
extend the existing stdlib footgun note to list `trim`/`to_upper`/`to_lower` etc., not just
`replace`.*
## Things done especially well (no action)
- Cross-app isolation has no script-controlled `app_id` anywhere — the boundary is structural.
- SSRF guard covers loopback **and** link-local (cloud-metadata `169.254.169.254`), and blocks at
every redirect hop.
- Size/operation caps are enforced **before** authz, so anonymous public scripts can't DoS the DB.
- Script runtime errors are **scrubbed** from the HTTP response and logged with a `correlation_id`
— no internal detail leaks to the caller.
- `deploy`/`apps create` honor `--output json` (the earlier F1 fix), so the whole build scripts
cleanly with captured ids.
## Reproduction / teardown
Scripts under `/tmp/stash/*.rhai`; apps `stash` + `evil` and their data persist in the dev
Postgres. Server: `target/debug/picloud` on `:8099` (see Phase 0 env). Teardown: kill the host
`picloud`; `docker compose down [-v]`. (Note: the S6 probe left harmless `/API/v2/x`, `/HEALTHZ`,
`/Admin/x`, `/Api/y` routes on `stash` — they vanish with the app.)
## Priority
**S6** is the only code-level finding (Low — confirm-and-harden the reserved-path check). **G1**
(trigger-execution observability) is the most impactful *developer-experience* gap for anyone
running background workers. Everything else is documented-behavior or known CLI-coverage gaps.

View File

@@ -19,5 +19,6 @@ pub use module_resolver::{
}; };
pub use sandbox::Limits; pub use sandbox::Limits;
pub use types::{ pub use types::{
ExecError, ExecRequest, ExecResponse, ExecStats, InvocationType, LogEntry, LogLevel, build_execution_log, ExecError, ExecRequest, ExecResponse, ExecStats, InvocationType, LogEntry,
LogLevel,
}; };

View File

@@ -2,10 +2,12 @@ use std::collections::BTreeMap;
use chrono::{DateTime, Utc}; use chrono::{DateTime, Utc};
use picloud_shared::{ use picloud_shared::{
AppId, ExecutionId, Principal, RequestId, ScriptId, ScriptSandbox, TriggerEvent, AppId, ExecutionId, ExecutionLog, ExecutionSource, ExecutionStatus, Principal, RequestId,
ScriptId, ScriptSandbox, TriggerEvent,
}; };
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use thiserror::Error; use thiserror::Error;
use uuid::Uuid;
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)] #[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)]
#[serde(rename_all = "lowercase")] #[serde(rename_all = "lowercase")]
@@ -167,3 +169,77 @@ pub enum ExecError {
#[error("execution declined: server at capacity (retry after {retry_after_secs}s)")] #[error("execution declined: server at capacity (retry after {retry_after_secs}s)")]
Overloaded { retry_after_secs: u32 }, Overloaded { retry_after_secs: u32 },
} }
/// Build an `ExecutionLog` row from one invocation's outcome.
///
/// Shared by every execution path so the log shape stays identical
/// regardless of who ran the script: the orchestrator's sync/direct HTTP
/// handlers pass `ExecutionSource::Http`, while the manager's trigger
/// dispatcher passes the trigger's kind (`Kv`, `Cron`, `Invoke`, …). That
/// `source` is what lets `pic logs` surface background runs that were
/// previously invisible. `Overloaded` is never logged — admission is
/// refused before a row exists — but it maps to `Error` defensively.
#[allow(clippy::too_many_arguments)]
#[must_use]
pub fn build_execution_log(
app_id: AppId,
script_id: ScriptId,
request_id: RequestId,
request_path: String,
request_headers: BTreeMap<String, String>,
request_body: serde_json::Value,
source: ExecutionSource,
outcome: &Result<ExecResponse, ExecError>,
started: DateTime<Utc>,
finished: DateTime<Utc>,
) -> ExecutionLog {
let duration_ms = u64::try_from(
finished
.signed_duration_since(started)
.num_milliseconds()
.max(0),
)
.unwrap_or(0);
let (status, response_code, response_body, script_logs) = match outcome {
Ok(resp) => {
let logs = serde_json::to_value(&resp.logs).unwrap_or(serde_json::Value::Array(vec![]));
(
ExecutionStatus::Success,
Some(resp.status_code),
Some(resp.body.clone()),
logs,
)
}
Err(e) => {
let status = match e {
ExecError::Timeout(_) => ExecutionStatus::Timeout,
ExecError::OperationBudgetExceeded => ExecutionStatus::BudgetExceeded,
_ => ExecutionStatus::Error,
};
(
status,
None,
Some(serde_json::json!({ "error": e.to_string() })),
serde_json::Value::Array(vec![]),
)
}
};
ExecutionLog {
id: Uuid::new_v4(),
app_id,
script_id,
request_id,
request_path,
request_headers,
request_body,
response_code,
response_body,
script_logs,
duration_ms,
status,
source,
created_at: started,
}
}

View File

@@ -0,0 +1,20 @@
-- G1 (E2E #2 "Stash"): trigger executions were invisible to `pic logs`.
--
-- Only HTTP-route executions ever wrote an `execution_logs` row; queue,
-- cron, dead-letter, and `invoke()` runs left no trace, so background
-- workers were observable only via dead-letters (failures) or their own
-- side effects. The dispatcher now logs every trigger run too — this
-- column records which kind of event dispatched each execution so the
-- logs surface can show, and filter by, the origin.
--
-- DEFAULT 'http' backfills every pre-existing row: before this change the
-- only thing that logged was the HTTP path, so 'http' is correct history.
-- The CHECK list mirrors `manager-core::OutboxSourceKind` /
-- `shared::ExecutionSource`; keep all three in sync.
ALTER TABLE execution_logs
ADD COLUMN source TEXT NOT NULL DEFAULT 'http'
CHECK (source IN (
'http', 'kv', 'docs', 'dead_letter', 'cron',
'files', 'pubsub', 'email', 'invoke', 'queue'
));

View File

@@ -0,0 +1,22 @@
-- H1 (re-review of the S6 reserved-path fix): the reserved-prefix check is
-- now case-insensitive, both at route creation AND when the route table is
-- compiled at boot. Routes created before the fix — while validation was
-- case-sensitive — could hold paths like `/API/v2/x`, `/Admin/x`, or
-- `/HEALTHZ`. `compile_routes` now skips such rows with a warning instead of
-- aborting startup (so an un-upgraded boot can't be bricked), but those
-- routes violate the reserved namespace and can never be served safely, so
-- sweep them here on upgrade.
--
-- Mirrors `orchestrator-core::routing::pattern::check_reserved` exactly,
-- case-insensitively: a path is reserved if its lowercased form equals one
-- of the bare names (`/api` `/admin` `/healthz` `/version`) or starts with
-- one of the prefixes. Note `/api/` and `/admin/` reserve on the trailing
-- slash, while `/healthz` and `/version` reserve on bare prefix — matching
-- the RESERVED_PATH_PREFIXES list. (Idempotent: a no-op once swept.)
DELETE FROM routes
WHERE lower(path) IN ('/api', '/admin', '/healthz', '/version')
OR lower(path) LIKE '/api/%'
OR lower(path) LIKE '/admin/%'
OR lower(path) LIKE '/healthz%'
OR lower(path) LIKE '/version%';

View File

@@ -12,8 +12,8 @@ use axum::{
Extension, Json, Router, Extension, Json, Router,
}; };
use picloud_shared::{ use picloud_shared::{
AppId, ExecutionLog, InstanceRole, Principal, Script, ScriptId, ScriptKind, ScriptSandbox, AppId, ExecutionLog, ExecutionSource, InstanceRole, Principal, Script, ScriptId, ScriptKind,
ScriptValidator, ValidatedScript, ValidationError, ScriptSandbox, ScriptValidator, ValidatedScript, ValidationError,
}; };
use serde::Deserialize; use serde::Deserialize;
@@ -385,6 +385,10 @@ pub struct LogsQuery {
#[serde(default, rename = "offset")] #[serde(default, rename = "offset")]
#[allow(dead_code)] #[allow(dead_code)]
pub legacy_offset: Option<i64>, pub legacy_offset: Option<i64>,
/// Optional origin filter (`http`, `kv`, `cron`, `invoke`, …). Absent
/// → all sources. An unrecognized value is a 422 (see `list_logs`).
#[serde(default)]
pub source: Option<String>,
} }
const fn default_limit() -> i64 { const fn default_limit() -> i64 {
@@ -411,7 +415,17 @@ async fn list_logs<R: ScriptRepository, L: ExecutionLogRepository>(
.cursor .cursor
.as_deref() .as_deref()
.and_then(crate::repo::ExecutionLogCursor::decode); .and_then(crate::repo::ExecutionLogCursor::decode);
let logs = state.logs.list_for_script(id, limit, cursor).await?; let source = match q.source.as_deref() {
None | Some("" | "all") => None,
Some(s) => Some(
ExecutionSource::from_wire(s)
.ok_or_else(|| ApiError::BadRequest(format!("unknown log source: {s:?}")))?,
),
};
let logs = state
.logs
.list_for_script(id, limit, cursor, source)
.await?;
Ok(Json(logs)) Ok(Json(logs))
} }
@@ -427,6 +441,9 @@ pub enum ApiError {
#[error("app not found: {0}")] #[error("app not found: {0}")]
AppNotFound(String), AppNotFound(String),
#[error("bad request: {0}")]
BadRequest(String),
#[error("conflict: {0}")] #[error("conflict: {0}")]
Conflict(String), Conflict(String),
@@ -459,11 +476,11 @@ impl IntoResponse for ApiError {
fn into_response(self) -> Response { fn into_response(self) -> Response {
let (status, message) = match &self { let (status, message) = match &self {
Self::NotFound(_) => (StatusCode::NOT_FOUND, self.to_string()), Self::NotFound(_) => (StatusCode::NOT_FOUND, self.to_string()),
Self::AppNotFound(_) => (StatusCode::UNPROCESSABLE_ENTITY, self.to_string()), Self::AppNotFound(_)
| Self::BadRequest(_)
| Self::Invalid(_)
| Self::Ceiling(_) => (StatusCode::UNPROCESSABLE_ENTITY, self.to_string()),
Self::Conflict(_) => (StatusCode::CONFLICT, self.to_string()), Self::Conflict(_) => (StatusCode::CONFLICT, self.to_string()),
Self::Invalid(_) | Self::Ceiling(_) => {
(StatusCode::UNPROCESSABLE_ENTITY, self.to_string())
}
Self::Forbidden => (StatusCode::FORBIDDEN, self.to_string()), Self::Forbidden => (StatusCode::FORBIDDEN, self.to_string()),
Self::AuthzRepo(e) => { Self::AuthzRepo(e) => {
tracing::error!(error = %e, "authz repo error"); tracing::error!(error = %e, "authz repo error");

View File

@@ -0,0 +1,160 @@
//! Admin HTTP surface for the declarative reconcile engine.
//!
//! `POST /api/v1/admin/apps/{id}/plan` — diff a desired-state bundle
//! against the app's live state and return the plan. Read-only; requires
//! `AppRead`. The `apply` route (write path) lands in the next milestone.
use axum::{
extract::{Path, State},
http::StatusCode,
response::{IntoResponse, Response},
routing::post,
Extension, Json, Router,
};
use picloud_shared::{AppId, Principal};
use serde::Deserialize;
use serde_json::json;
use crate::app_repo::AppRepository;
use crate::apply_service::{ApplyError, ApplyReport, ApplyService, Bundle, BundleTrigger, Plan};
use crate::authz::{require, AuthzDenied, Capability};
/// Build the apply/plan router. Mounted under `/api/v1/admin`.
pub fn apply_router(service: ApplyService) -> Router {
Router::new()
.route("/apps/{id}/plan", post(plan_handler))
.route("/apps/{id}/apply", post(apply_handler))
.with_state(service)
}
#[derive(Deserialize)]
pub struct ApplyRequest {
pub bundle: Bundle,
#[serde(default)]
pub prune: bool,
}
async fn apply_handler(
State(svc): State<ApplyService>,
Extension(principal): Extension<Principal>,
Path(id_or_slug): Path<String>,
Json(req): Json<ApplyRequest>,
) -> Result<Json<ApplyReport>, ApplyError> {
let app_id = resolve_app_id(svc.apps.as_ref(), &id_or_slug).await?;
// Read is always needed; write caps are required for the resource kinds
// the bundle touches — and for ALL kinds when `prune` is set, since
// pruning deletes resources whose bundle section is empty (and a script
// delete cascades its routes/triggers).
require(svc.authz.as_ref(), &principal, Capability::AppRead(app_id))
.await
.map_err(map_authz)?;
if req.prune || !req.bundle.scripts.is_empty() {
require(
svc.authz.as_ref(),
&principal,
Capability::AppWriteScript(app_id),
)
.await
.map_err(map_authz)?;
}
if req.prune || !req.bundle.routes.is_empty() {
require(
svc.authz.as_ref(),
&principal,
Capability::AppWriteRoute(app_id),
)
.await
.map_err(map_authz)?;
}
if req.prune || !req.bundle.triggers.is_empty() {
require(
svc.authz.as_ref(),
&principal,
Capability::AppManageTriggers(app_id),
)
.await
.map_err(map_authz)?;
}
// Email triggers resolve and decrypt a stored secret by name server-side,
// which the secrets API guards with `AppSecretsRead`. Require it here too
// so apply can't bind a secret a principal couldn't otherwise read — the
// caps aren't strictly nested on the API-key scope path.
if req.bundle.triggers.iter().any(BundleTrigger::is_email) {
require(
svc.authz.as_ref(),
&principal,
Capability::AppSecretsRead(app_id),
)
.await
.map_err(map_authz)?;
}
let report = svc
.apply(app_id, &req.bundle, req.prune, principal.user_id)
.await?;
Ok(Json(report))
}
async fn plan_handler(
State(svc): State<ApplyService>,
Extension(principal): Extension<Principal>,
Path(id_or_slug): Path<String>,
Json(bundle): Json<Bundle>,
) -> Result<Json<Plan>, ApplyError> {
let app_id = resolve_app_id(svc.apps.as_ref(), &id_or_slug).await?;
// NOTE: the returned `Plan` discloses live secret NAMES (not values). That
// is safe today only because `AppRead` and `AppSecretsRead` are co-granted
// at every tier (same `script:read` scope, both in the viewer role). If a
// future authz split puts `AppSecretsRead` on its own tier, this handler
// must additionally require it — otherwise it leaks names a principal
// couldn't enumerate via the secrets API.
require(svc.authz.as_ref(), &principal, Capability::AppRead(app_id))
.await
.map_err(map_authz)?;
let plan = svc.plan(app_id, &bundle).await?;
Ok(Json(plan))
}
/// Resolve a slug-or-id path param to an `AppId`, mapping miss → 404.
/// Mirrors the `triggers_api` helper of the same shape.
async fn resolve_app_id(apps: &dyn AppRepository, ident: &str) -> Result<AppId, ApplyError> {
crate::app_repo::resolve_app(apps, ident)
.await
.map_err(|e| ApplyError::Backend(e.to_string()))?
.map(|l| l.app.id)
.ok_or_else(|| ApplyError::AppNotFound(ident.to_string()))
}
fn map_authz(denied: AuthzDenied) -> ApplyError {
match denied {
AuthzDenied::Denied => ApplyError::Forbidden,
AuthzDenied::Repo(e) => ApplyError::AuthzRepo(e.to_string()),
}
}
impl IntoResponse for ApplyError {
fn into_response(self) -> Response {
let (status, body) = match &self {
Self::AppNotFound(_) => (StatusCode::NOT_FOUND, json!({ "error": self.to_string() })),
Self::Invalid(_) => (
StatusCode::UNPROCESSABLE_ENTITY,
json!({ "error": self.to_string() }),
),
Self::Forbidden => (StatusCode::FORBIDDEN, json!({ "error": self.to_string() })),
Self::AuthzRepo(e) => {
tracing::error!(error = %e, "apply authz repo error");
(
StatusCode::INTERNAL_SERVER_ERROR,
json!({ "error": "internal error" }),
)
}
Self::Backend(e) => {
tracing::error!(error = %e, "apply backend error");
(
StatusCode::INTERNAL_SERVER_ERROR,
json!({ "error": "internal error" }),
)
}
};
(status, Json(body)).into_response()
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,48 @@
//! `GET /api/v1/admin/dev/emails` — dev-only inspection of mail captured
//! by the in-memory email sink (G5).
//!
//! Mounted **only** when the email service is running in dev-capture mode
//! (`PICLOUD_DEV_MODE=true` and no SMTP relay configured). In every other
//! configuration the route does not exist, so there is no production
//! surface here. Capture is instance-wide (the SMTP transport seam can't
//! see a script's `app_id`), so the endpoint is instance-wide too and is
//! restricted to instance Owners/Admins.
use std::sync::Arc;
use axum::extract::State;
use axum::http::StatusCode;
use axum::response::Json;
use axum::routing::get;
use axum::{Extension, Router};
use picloud_shared::{InstanceRole, Principal};
use crate::email_service::{CapturedEmail, DevEmailSink};
#[derive(Clone)]
pub struct DevEmailState {
pub sink: Arc<DevEmailSink>,
}
/// Build the dev-email router. Callers mount this only when dev-capture
/// mode is active (i.e. they hold a `Some(sink)`).
pub fn dev_emails_router(state: DevEmailState) -> Router {
Router::new()
.route("/dev/emails", get(list_dev_emails))
.with_state(state)
}
async fn list_dev_emails(
Extension(principal): Extension<Principal>,
State(state): State<DevEmailState>,
) -> Result<Json<Vec<CapturedEmail>>, StatusCode> {
// Instance-wide data → require an instance Owner/Admin. A Member
// (app-scoped) principal has no business reading every app's mail.
if !matches!(
principal.instance_role,
InstanceRole::Owner | InstanceRole::Admin
) {
return Err(StatusCode::FORBIDDEN);
}
Ok(Json(state.sink.snapshot()))
}

View File

@@ -24,11 +24,14 @@ use std::sync::Arc;
use std::time::Duration; use std::time::Duration;
use chrono::{DateTime, Utc}; use chrono::{DateTime, Utc};
use picloud_executor_core::{ExecError, ExecRequest, ExecResponse, InvocationType}; use picloud_executor_core::{
build_execution_log, ExecError, ExecRequest, ExecResponse, InvocationType,
};
use picloud_orchestrator_core::{ExecutionGate, ExecutorClient}; use picloud_orchestrator_core::{ExecutionGate, ExecutorClient};
use picloud_shared::{ use picloud_shared::{
DeadLetterId, ExecResponseSummary, ExecutionId, HttpDispatchPayload, InboxDeliveryOutcome, DeadLetterId, ExecResponseSummary, ExecutionId, ExecutionLogSink, ExecutionSource,
InboxFailureKind, InboxResolver, InboxResult, RequestId, ScriptId, ScriptSandbox, TriggerEvent, HttpDispatchPayload, InboxDeliveryOutcome, InboxFailureKind, InboxResolver, InboxResult,
RequestId, ScriptId, ScriptSandbox, TriggerEvent,
}; };
use rand::Rng; use rand::Rng;
use uuid::Uuid; use uuid::Uuid;
@@ -53,6 +56,11 @@ pub struct Dispatcher {
pub principals: Arc<dyn PrincipalResolver>, pub principals: Arc<dyn PrincipalResolver>,
pub executor: Arc<dyn ExecutorClient>, pub executor: Arc<dyn ExecutorClient>,
pub gate: Arc<ExecutionGate>, pub gate: Arc<ExecutionGate>,
/// G1: records an `execution_logs` row for every trigger run so
/// background workers (queue / cron / dead-letter / invoke) show up
/// in `pic logs`, not just synchronous HTTP. Same sink the
/// orchestrator's data plane writes through.
pub log_sink: Arc<dyn ExecutionLogSink>,
pub inbox: Arc<dyn InboxResolver>, pub inbox: Arc<dyn InboxResolver>,
/// v1.1.9. Reads `queue_messages` for the queue arm + the reclaim /// v1.1.9. Reads `queue_messages` for the queue arm + the reclaim
/// task. None in tests / harnesses that don't exercise queues. /// task. None in tests / harnesses that don't exercise queues.
@@ -149,6 +157,39 @@ fn async_exec_timeout_from_env() -> Duration {
/// parallelism, not the dispatcher's serial-await. /// parallelism, not the dispatcher's serial-await.
const QUEUE_DISPATCH_PARALLELISM: usize = 32; const QUEUE_DISPATCH_PARALLELISM: usize = 32;
/// Map an outbox row's source kind to the execution-log `source`. The
/// wire strings are identical, so this is total in practice; an unknown
/// kind would default to `Http`.
fn exec_source(row: &OutboxRow) -> ExecutionSource {
ExecutionSource::from_wire(row.source_kind.as_str()).unwrap_or_default()
}
/// G1: the slice of an `ExecRequest` needed to write an execution-log
/// row, captured before the request is moved into the executor.
struct ExecLogContext {
app_id: picloud_shared::AppId,
script_id: ScriptId,
request_id: RequestId,
path: String,
headers: std::collections::BTreeMap<String, String>,
body: serde_json::Value,
source: ExecutionSource,
}
impl ExecLogContext {
fn from_request(req: &ExecRequest, source: ExecutionSource) -> Self {
Self {
app_id: req.app_id,
script_id: req.script_id,
request_id: req.request_id,
path: req.path.clone(),
headers: req.headers.clone(),
body: req.body.clone(),
source,
}
}
}
impl Dispatcher { impl Dispatcher {
/// Spawn the dispatcher loop as a detached `tokio::task`. Also /// Spawn the dispatcher loop as a detached `tokio::task`. Also
/// spawns the v1.1.9 queue visibility-timeout reclaim task. Both /// spawns the v1.1.9 queue visibility-timeout reclaim task. Both
@@ -376,6 +417,11 @@ impl Dispatcher {
script_id: consumer.script_id, script_id: consumer.script_id,
updated_at: script.updated_at, updated_at: script.updated_at,
}; };
// G1: queue consumers dispatch outside the outbox, so they need
// their own log write. Source is always `queue`; no `reply_to`
// here, so there's no double-logging concern.
let log_cx = ExecLogContext::from_request(&exec_req, ExecutionSource::Queue);
let started = Utc::now();
let outcome = self let outcome = self
.executor .executor
.execute_with_identity( .execute_with_identity(
@@ -385,8 +431,12 @@ impl Dispatcher {
async_exec_timeout_from_env(), async_exec_timeout_from_env(),
) )
.await; .await;
let finished = Utc::now();
drop(permit); drop(permit);
self.record_execution_log(&log_cx, &outcome, started, finished)
.await;
// Best-effort touch on last_fired_at (a failure here doesn't // Best-effort touch on last_fired_at (a failure here doesn't
// change ack/nack behavior). // change ack/nack behavior).
if let Err(e) = self if let Err(e) = self
@@ -585,18 +635,67 @@ impl Dispatcher {
script_id: resolved.script_id, script_id: resolved.script_id,
updated_at: resolved.script_updated_at, updated_at: resolved.script_updated_at,
}; };
// G1: capture the request context before `exec_req` is consumed so
// we can write an execution-log row for this run (see below).
let log_cx = ExecLogContext::from_request(&exec_req, exec_source(&row));
let started = Utc::now();
let outcome = self let outcome = self
.executor .executor
.execute_with_identity(identity, &source, exec_req, async_exec_timeout_from_env()) .execute_with_identity(identity, &source, exec_req, async_exec_timeout_from_env())
.await; .await;
let finished = Utc::now();
drop(permit); drop(permit);
// G1: persist an execution-log row so this run shows up in
// `pic logs`. Skip rows with a synchronous receiver (`reply_to`
// is set) — those are sync HTTP routes that the orchestrator's
// inbox path already logs, and logging here too would duplicate
// them. Trigger rows and fire-and-forget async HTTP (202) have no
// receiver, so the dispatcher is the only place that can log them.
if row.reply_to.is_none() {
self.record_execution_log(&log_cx, &outcome, started, finished)
.await;
}
match outcome { match outcome {
Ok(resp) => self.handle_success(&row, &resolved, resp).await, Ok(resp) => self.handle_success(&row, &resolved, resp).await,
Err(err) => self.handle_failure(&row, &resolved, err).await, Err(err) => self.handle_failure(&row, &resolved, err).await,
} }
} }
/// G1: best-effort persistence of an execution-log row for a
/// dispatcher-run script. A sink failure is logged but never changes
/// ack/nack/retry behavior — the audit trail is not on the hot path
/// of message-delivery correctness.
async fn record_execution_log(
&self,
cx: &ExecLogContext,
outcome: &Result<ExecResponse, ExecError>,
started: DateTime<Utc>,
finished: DateTime<Utc>,
) {
let log = build_execution_log(
cx.app_id,
cx.script_id,
cx.request_id,
cx.path.clone(),
cx.headers.clone(),
cx.body.clone(),
cx.source,
outcome,
started,
finished,
);
if let Err(e) = self.log_sink.record(log).await {
tracing::warn!(
error = %e,
script_id = %cx.script_id,
source = cx.source.as_str(),
"failed to persist trigger execution log"
);
}
}
async fn resolve_trigger(&self, row: &OutboxRow) -> Result<ResolvedTrigger, DispatcherError> { async fn resolve_trigger(&self, row: &OutboxRow) -> Result<ResolvedTrigger, DispatcherError> {
// For KV and DL kinds, the outbox carries `trigger_id`. Use it // For KV and DL kinds, the outbox carries `trigger_id`. Use it
// to look up the trigger row, then resolve the script. // to look up the trigger row, then resolve the script.

View File

@@ -248,6 +248,15 @@ fn non_empty_env(key: &str) -> Option<String> {
std::env::var(key).ok().filter(|v| !v.trim().is_empty()) std::env::var(key).ok().filter(|v| !v.trim().is_empty())
} }
/// `PICLOUD_DEV_MODE=true` (case-insensitive). Matches the detection in
/// `shared::crypto` so the dev email sink and the dev master key turn on
/// together.
fn dev_mode_enabled() -> bool {
std::env::var("PICLOUD_DEV_MODE")
.map(|v| v.trim().eq_ignore_ascii_case("true"))
.unwrap_or(false)
}
/// Internal transport seam so the service can be tested without a live /// Internal transport seam so the service can be tested without a live
/// SMTP server. The production impl is [`LettreEmailTransport`]; tests /// SMTP server. The production impl is [`LettreEmailTransport`]; tests
/// use a recording fake. /// use a recording fake.
@@ -299,6 +308,91 @@ impl EmailTransport for LettreEmailTransport {
} }
} }
/// G5: how many recently-captured dev emails the in-memory sink keeps.
/// Old entries are evicted FIFO; this is a debugging aid, not storage.
pub const DEV_EMAIL_CAPACITY: usize = 100;
/// One email captured by the dev sink instead of being relayed. Serialized
/// straight onto the dev-only inspection endpoint.
#[derive(Clone, serde::Serialize)]
pub struct CapturedEmail {
pub captured_at: chrono::DateTime<chrono::Utc>,
pub from: Option<String>,
pub to: Vec<String>,
/// The full RFC 5322 message (headers + body), exactly as it would
/// have hit the relay — enough to eyeball subject/body in dev.
pub raw: String,
}
/// In-memory ring buffer of captured dev emails. Shared (`Arc`) between
/// the [`DevEmailTransport`] that writes and the dev endpoint that reads.
pub struct DevEmailSink {
captured: std::sync::Mutex<std::collections::VecDeque<CapturedEmail>>,
capacity: usize,
}
impl DevEmailSink {
#[must_use]
pub fn new(capacity: usize) -> Self {
Self {
captured: std::sync::Mutex::new(std::collections::VecDeque::new()),
capacity: capacity.max(1),
}
}
fn push(&self, email: CapturedEmail) {
let mut q = self.captured.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
while q.len() >= self.capacity {
q.pop_front();
}
q.push_back(email);
}
/// Newest-first snapshot of the captured mail.
#[must_use]
pub fn snapshot(&self) -> Vec<CapturedEmail> {
let q = self.captured.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
q.iter().rev().cloned().collect()
}
}
/// Dev transport: instead of relaying, capture the message in memory and
/// log it. Wired only when `PICLOUD_DEV_MODE=true` and no SMTP relay is
/// configured, so `email::send` is exercisable locally without a relay.
/// NEVER constructed in production (no dev mode → disabled mode instead).
pub struct DevEmailTransport {
sink: Arc<DevEmailSink>,
}
impl DevEmailTransport {
#[must_use]
pub fn new(sink: Arc<DevEmailSink>) -> Self {
Self { sink }
}
}
#[async_trait]
impl EmailTransport for DevEmailTransport {
async fn send(&self, message: &Message) -> Result<(), EmailError> {
let envelope = message.envelope();
let from = envelope.from().map(ToString::to_string);
let to: Vec<String> = envelope.to().iter().map(ToString::to_string).collect();
let raw = String::from_utf8_lossy(&message.formatted()).into_owned();
tracing::info!(
?from,
?to,
"email DEV CAPTURE: message captured in memory (not relayed)"
);
self.sink.push(CapturedEmail {
captured_at: chrono::Utc::now(),
from,
to,
raw,
});
Ok(())
}
}
pub struct EmailServiceImpl { pub struct EmailServiceImpl {
/// `None` → disabled mode (every send returns `NotConfigured`). /// `None` → disabled mode (every send returns `NotConfigured`).
transport: Option<Arc<dyn EmailTransport>>, transport: Option<Arc<dyn EmailTransport>>,
@@ -328,16 +422,28 @@ impl EmailServiceImpl {
/// — email is non-critical and must not block startup. /// — email is non-critical and must not block startup.
#[must_use] #[must_use]
pub fn from_env(authz: Arc<dyn AuthzRepo>) -> Self { pub fn from_env(authz: Arc<dyn AuthzRepo>) -> Self {
let config = EmailConfig::from_env(); Self::from_env_with_dev_capture(authz).0
let transport: Option<Arc<dyn EmailTransport>> = match SmtpConfig::from_env() {
None => {
tracing::warn!(
"email is DISABLED: set PICLOUD_SMTP_HOST/USER/PASSWORD to enable \
email::send. Scripts calling email::send will get an error."
);
None
} }
Some(cfg) => match LettreEmailTransport::build(&cfg) {
/// Like [`from_env`](Self::from_env), but in **dev mode with no SMTP
/// relay** it wires a [`DevEmailTransport`] that captures mail in
/// memory instead of returning `NotConfigured` — so `email::send` is
/// exercisable locally (G5). Returns the sink handle (`Some`) when
/// capture mode is active, so the caller can expose it via the
/// dev-only inspection endpoint.
///
/// Production is unaffected: without `PICLOUD_DEV_MODE=true` an unset
/// relay still yields disabled mode (`NotConfigured`), never capture.
#[must_use]
pub fn from_env_with_dev_capture(
authz: Arc<dyn AuthzRepo>,
) -> (Self, Option<Arc<DevEmailSink>>) {
let config = EmailConfig::from_env();
match SmtpConfig::from_env() {
Some(cfg) => {
let transport: Option<Arc<dyn EmailTransport>> = match LettreEmailTransport::build(
&cfg,
) {
Ok(t) => { Ok(t) => {
tracing::info!(host = %cfg.host, port = cfg.port, "outbound email enabled"); tracing::info!(host = %cfg.host, port = cfg.port, "outbound email enabled");
Some(Arc::new(t)) Some(Arc::new(t))
@@ -346,9 +452,28 @@ impl EmailServiceImpl {
tracing::error!(error = %e, "failed to build SMTP transport; email DISABLED"); tracing::error!(error = %e, "failed to build SMTP transport; email DISABLED");
None None
} }
},
}; };
Self::new(transport, authz, config) (Self::new(transport, authz, config), None)
}
None if dev_mode_enabled() => {
tracing::warn!(
"email DEV CAPTURE: PICLOUD_DEV_MODE=true and no SMTP relay configured — \
email::send will SUCCEED and capture messages in memory (last {DEV_EMAIL_CAPACITY}, \
readable at GET /api/v1/admin/dev/emails). NEVER use this in production."
);
let sink = Arc::new(DevEmailSink::new(DEV_EMAIL_CAPACITY));
let transport: Arc<dyn EmailTransport> =
Arc::new(DevEmailTransport::new(sink.clone()));
(Self::new(Some(transport), authz, config), Some(sink))
}
None => {
tracing::warn!(
"email is DISABLED: set PICLOUD_SMTP_HOST/USER/PASSWORD to enable \
email::send. Scripts calling email::send will get an error."
);
(Self::new(None, authz, config), None)
}
}
} }
async fn check_send(&self, cx: &SdkCallCx) -> Result<(), EmailError> { async fn check_send(&self, cx: &SdkCallCx) -> Result<(), EmailError> {

View File

@@ -0,0 +1,162 @@
//! `/api/v1/admin/apps/{id}/kv*` — read-only KV inspection (G2).
//!
//! Mirrors the minimal `files_api` / `queues_api` admin surface so the
//! `pic kv` CLI (and a future dashboard tab) can browse stored keys
//! without a script. **Read-only by design** — KV writes go through
//! `kv::set` in scripts, which emit change events the trigger framework
//! depends on; an admin write would bypass that and could break app
//! invariants, so it is deliberately out of scope here (matching the
//! read-only queues precedent).
//!
//! Two operations:
//! * `GET /apps/{id}/kv?collection=<c>&cursor=&limit=` — list keys in a
//! collection (cursor-paginated).
//! * `GET /apps/{id}/kv/{collection}/{key}` — fetch one value.
//!
//! Capability: `AppKvRead`, resolved against the app loaded from the
//! path (same tier the SDK read path uses).
use std::sync::Arc;
use axum::extract::{Path, Query, State};
use axum::response::{IntoResponse, Json, Response};
use axum::routing::get;
use axum::{Extension, Router};
use picloud_shared::{AppId, Principal};
use serde::{Deserialize, Serialize};
use serde_json::json;
use crate::app_repo::AppRepository;
use crate::authz::{require, AuthzDenied, AuthzRepo, Capability};
use crate::kv_repo::KvRepo;
#[derive(Clone)]
pub struct KvAdminState {
pub kv: Arc<dyn KvRepo>,
pub apps: Arc<dyn AppRepository>,
pub authz: Arc<dyn AuthzRepo>,
}
pub fn kv_admin_router(state: KvAdminState) -> Router {
Router::new()
.route("/apps/{app_id}/kv", get(list_keys))
.route("/apps/{app_id}/kv/{collection}/{key}", get(get_value))
.with_state(state)
}
#[derive(Debug, Deserialize)]
pub struct ListKvQuery {
pub collection: String,
#[serde(default)]
pub cursor: Option<String>,
#[serde(default)]
pub limit: Option<u32>,
}
#[derive(Debug, Serialize)]
struct GetValueResponse {
value: serde_json::Value,
}
/// Serialize mirror of `shared::KvListPage` (which is not `Serialize`).
#[derive(Debug, Serialize)]
struct ListKeysResponse {
keys: Vec<String>,
next_cursor: Option<String>,
}
async fn list_keys(
State(s): State<KvAdminState>,
Extension(principal): Extension<Principal>,
Path(id_or_slug): Path<String>,
Query(q): Query<ListKvQuery>,
) -> Result<Json<ListKeysResponse>, KvApiError> {
let app_id = resolve_app(&*s.apps, &id_or_slug).await?;
require(s.authz.as_ref(), &principal, Capability::AppKvRead(app_id)).await?;
let page =
s.kv.list(
app_id,
&q.collection,
q.cursor.as_deref(),
q.limit.unwrap_or(0),
)
.await
.map_err(|e| KvApiError::Backend(e.to_string()))?;
Ok(Json(ListKeysResponse {
keys: page.keys,
next_cursor: page.next_cursor,
}))
}
async fn get_value(
State(s): State<KvAdminState>,
Extension(principal): Extension<Principal>,
Path((id_or_slug, collection, key)): Path<(String, String, String)>,
) -> Result<Json<GetValueResponse>, KvApiError> {
let app_id = resolve_app(&*s.apps, &id_or_slug).await?;
require(s.authz.as_ref(), &principal, Capability::AppKvRead(app_id)).await?;
let value =
s.kv.get(app_id, &collection, &key)
.await
.map_err(|e| KvApiError::Backend(e.to_string()))?
.ok_or(KvApiError::NotFound)?;
Ok(Json(GetValueResponse { value }))
}
async fn resolve_app(apps: &dyn AppRepository, ident: &str) -> Result<AppId, KvApiError> {
crate::app_repo::resolve_app(apps, ident)
.await
.map_err(|e| KvApiError::Backend(e.to_string()))?
.map(|l| l.app.id)
.ok_or(KvApiError::AppNotFound)
}
#[derive(Debug, thiserror::Error)]
pub enum KvApiError {
#[error("app not found")]
AppNotFound,
#[error("key not found")]
NotFound,
#[error("forbidden")]
Forbidden,
#[error("authorization repo error: {0}")]
AuthzRepo(String),
#[error("kv backend: {0}")]
Backend(String),
}
impl From<AuthzDenied> for KvApiError {
fn from(d: AuthzDenied) -> Self {
match d {
AuthzDenied::Denied => Self::Forbidden,
AuthzDenied::Repo(e) => Self::AuthzRepo(e.to_string()),
}
}
}
impl IntoResponse for KvApiError {
fn into_response(self) -> Response {
use axum::http::StatusCode;
let (status, body) = match &self {
Self::AppNotFound | Self::NotFound => {
(StatusCode::NOT_FOUND, json!({ "error": self.to_string() }))
}
Self::Forbidden => (StatusCode::FORBIDDEN, json!({ "error": self.to_string() })),
Self::AuthzRepo(e) => {
tracing::error!(error = %e, "kv admin authz error");
(
StatusCode::INTERNAL_SERVER_ERROR,
json!({ "error": "internal error" }),
)
}
Self::Backend(e) => {
tracing::error!(error = %e, "kv admin backend error");
(
StatusCode::INTERNAL_SERVER_ERROR,
json!({ "error": "internal error" }),
)
}
};
(status, Json(body)).into_response()
}
}

View File

@@ -23,6 +23,8 @@ pub mod app_user_repo;
pub mod app_user_role_repo; pub mod app_user_role_repo;
pub mod app_user_session_repo; pub mod app_user_session_repo;
pub mod app_user_verification_repo; pub mod app_user_verification_repo;
pub mod apply_api;
pub mod apply_service;
pub mod apps_api; pub mod apps_api;
pub mod auth; pub mod auth;
pub mod auth_api; pub mod auth_api;
@@ -33,6 +35,7 @@ pub mod cron_scheduler;
pub mod dead_letter_repo; pub mod dead_letter_repo;
pub mod dead_letter_service; pub mod dead_letter_service;
pub mod dead_letters_api; pub mod dead_letters_api;
pub mod dev_email_api;
pub mod dispatcher; pub mod dispatcher;
pub mod docs_filter; pub mod docs_filter;
pub mod docs_repo; pub mod docs_repo;
@@ -46,6 +49,7 @@ pub mod files_sweep;
pub mod gc; pub mod gc;
pub mod http_service; pub mod http_service;
pub mod invoke_service; pub mod invoke_service;
pub mod kv_api;
pub mod kv_repo; pub mod kv_repo;
pub mod kv_service; pub mod kv_service;
pub mod log_sink; pub mod log_sink;
@@ -126,6 +130,8 @@ pub use app_user_session_repo::{
pub use app_user_verification_repo::{ pub use app_user_verification_repo::{
AppUserVerificationRepo, AppUserVerificationRepoError, PostgresAppUserVerificationRepo, AppUserVerificationRepo, AppUserVerificationRepoError, PostgresAppUserVerificationRepo,
}; };
pub use apply_api::apply_router;
pub use apply_service::{ApplyError, ApplyService, Bundle, Plan};
pub use apps_api::{apps_router, AppsState}; pub use apps_api::{apps_router, AppsState};
pub use auth_api::auth_router; pub use auth_api::auth_router;
pub use auth_bootstrap::{ pub use auth_bootstrap::{
@@ -143,6 +149,7 @@ pub use dead_letter_repo::{
}; };
pub use dead_letter_service::PostgresDeadLetterService; pub use dead_letter_service::PostgresDeadLetterService;
pub use dead_letters_api::{dead_letters_router, DeadLettersApiError, DeadLettersState}; pub use dead_letters_api::{dead_letters_router, DeadLettersApiError, DeadLettersState};
pub use dev_email_api::{dev_emails_router, DevEmailState};
pub use dispatcher::{compute_backoff, Dispatcher, DispatcherError}; pub use dispatcher::{compute_backoff, Dispatcher, DispatcherError};
pub use docs_repo::{DocsRepo, DocsRepoError, PostgresDocsRepo}; pub use docs_repo::{DocsRepo, DocsRepoError, PostgresDocsRepo};
pub use docs_service::DocsServiceImpl; pub use docs_service::DocsServiceImpl;
@@ -150,8 +157,8 @@ pub use email_inbound_api::{
email_inbound_router, EmailInboundError, EmailInboundState, InboundNonceDedup, email_inbound_router, EmailInboundError, EmailInboundState, InboundNonceDedup,
}; };
pub use email_service::{ pub use email_service::{
EmailConfig, EmailServiceImpl, EmailTransport, LettreEmailTransport, SmtpConfig, SmtpTls, CapturedEmail, DevEmailSink, DevEmailTransport, EmailConfig, EmailServiceImpl, EmailTransport,
DEFAULT_EMAIL_MAX_MESSAGE_BYTES, LettreEmailTransport, SmtpConfig, SmtpTls, DEFAULT_EMAIL_MAX_MESSAGE_BYTES,
}; };
pub use files_api::{files_admin_router, FilesAdminState}; pub use files_api::{files_admin_router, FilesAdminState};
pub use files_repo::{FilesConfig, FilesRepo, FilesRepoError, FsFilesRepo}; pub use files_repo::{FilesConfig, FilesRepo, FilesRepoError, FsFilesRepo};
@@ -159,6 +166,7 @@ pub use files_service::FilesServiceImpl;
pub use files_sweep::{spawn_files_orphan_sweep, sweep_orphan_tmp_files, SweepStats}; pub use files_sweep::{spawn_files_orphan_sweep, sweep_orphan_tmp_files, SweepStats};
pub use gc::{spawn_abandoned_gc, spawn_app_user_token_gc, spawn_dead_letter_gc}; pub use gc::{spawn_abandoned_gc, spawn_app_user_token_gc, spawn_dead_letter_gc};
pub use http_service::{HttpConfig, HttpServiceImpl}; pub use http_service::{HttpConfig, HttpServiceImpl};
pub use kv_api::{kv_admin_router, KvAdminState};
pub use kv_repo::{KvRepo, KvRepoError, PostgresKvRepo}; pub use kv_repo::{KvRepo, KvRepoError, PostgresKvRepo};
pub use kv_service::KvServiceImpl; pub use kv_service::KvServiceImpl;
pub use log_sink::PostgresExecutionLogSink; pub use log_sink::PostgresExecutionLogSink;

View File

@@ -31,9 +31,9 @@ impl ExecutionLogSink for PostgresExecutionLogSink {
id, app_id, script_id, request_id, \ id, app_id, script_id, request_id, \
request_path, request_headers, request_body, \ request_path, request_headers, request_body, \
response_code, response_body, \ response_code, response_body, \
logs, duration_ms, status, created_at \ logs, duration_ms, status, source, created_at \
) VALUES ( \ ) VALUES ( \
$1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13 \ $1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14 \
)", )",
) )
.bind(log.id) .bind(log.id)
@@ -48,6 +48,7 @@ impl ExecutionLogSink for PostgresExecutionLogSink {
.bind(&log.script_logs) .bind(&log.script_logs)
.bind(duration_ms) .bind(duration_ms)
.bind(log.status.as_str()) .bind(log.status.as_str())
.bind(log.source.as_str())
.bind(log.created_at) .bind(log.created_at)
.execute(&self.pool) .execute(&self.pool)
.await .await

View File

@@ -3,8 +3,8 @@ use std::collections::BTreeMap;
use async_trait::async_trait; use async_trait::async_trait;
use picloud_orchestrator_core::{ResolverError, ScriptResolver}; use picloud_orchestrator_core::{ResolverError, ScriptResolver};
use picloud_shared::{ use picloud_shared::{
AdminUserId, AppId, ExecutionLog, ExecutionStatus, RequestId, Script, ScriptId, ScriptKind, AdminUserId, AppId, ExecutionLog, ExecutionSource, ExecutionStatus, RequestId, Script,
ScriptSandbox, ScriptId, ScriptKind, ScriptSandbox,
}; };
use sqlx::PgPool; use sqlx::PgPool;
@@ -273,42 +273,8 @@ impl ScriptRepository for PostgresScriptRepository {
} }
async fn create(&self, input: NewScript) -> Result<Script, ScriptRepositoryError> { async fn create(&self, input: NewScript) -> Result<Script, ScriptRepositoryError> {
let sandbox_json = serde_json::to_value(input.sandbox.unwrap_or_default())
.unwrap_or_else(|_| serde_json::json!({}));
let mut tx = self.pool.begin().await?; let mut tx = self.pool.begin().await?;
let res = sqlx::query_as::<_, ScriptRow>(&format!( let script = insert_script_tx(&mut tx, &input).await?;
"INSERT INTO scripts ( \
app_id, name, description, source, kind, \
timeout_seconds, memory_limit_mb, sandbox \
) VALUES ($1, $2, $3, $4, $5, COALESCE($6, 30), COALESCE($7, 256), $8) \
RETURNING {SCRIPT_SELECT_COLS}"
))
.bind(input.app_id.into_inner())
.bind(&input.name)
.bind(input.description.as_deref())
.bind(&input.source)
.bind(input.kind.as_str())
.bind(input.timeout_seconds)
.bind(input.memory_limit_mb)
.bind(sandbox_json)
.fetch_one(&mut *tx)
.await;
let script: Script = match res {
Ok(row) => row.into(),
Err(sqlx::Error::Database(e)) if e.is_unique_violation() => {
return Err(ScriptRepositoryError::Conflict(format!(
"a script named {:?} already exists in this app",
input.name
)));
}
Err(e) => return Err(e.into()),
};
// Dep-graph: write any literal-path imports declared in the
// source. Unresolved names (the referenced module doesn't
// exist yet) are silently skipped — best-effort.
replace_imports_tx(&mut tx, script.id, script.app_id, &input.imports).await?;
tx.commit().await?; tx.commit().await?;
Ok(script) Ok(script)
} }
@@ -318,62 +284,8 @@ impl ScriptRepository for PostgresScriptRepository {
id: ScriptId, id: ScriptId,
patch: ScriptPatch, patch: ScriptPatch,
) -> Result<Script, ScriptRepositoryError> { ) -> Result<Script, ScriptRepositoryError> {
// COALESCE-based partial update: `NULL` parameters leave columns
// untouched. Description is double-Optioned so callers can
// explicitly set it to NULL (Some(None)) vs leave it alone (None).
// Sandbox is replaced wholesale when present; per-field merging
// happens in the API layer (clearer semantics for a "PUT a new
// sandbox config" call). app_id is immutable — moving a script
// to another app is a copy-and-delete, not an in-place edit.
let sandbox_json = patch
.sandbox
.as_ref()
.map(|s| serde_json::to_value(s).unwrap_or_else(|_| serde_json::json!({})));
let mut tx = self.pool.begin().await?; let mut tx = self.pool.begin().await?;
let res = sqlx::query_as::<_, ScriptRow>(&format!( let script = update_script_tx(&mut tx, id, &patch).await?;
"UPDATE scripts SET \
name = COALESCE($2, name), \
description = CASE WHEN $3::bool THEN $4 ELSE description END, \
source = COALESCE($5, source), \
timeout_seconds = COALESCE($6, timeout_seconds), \
memory_limit_mb = COALESCE($7, memory_limit_mb), \
sandbox = COALESCE($8, sandbox), \
kind = COALESCE($9, kind), \
version = version + 1, \
updated_at = NOW() \
WHERE id = $1 \
RETURNING {SCRIPT_SELECT_COLS}"
))
.bind(id.into_inner())
.bind(patch.name.as_deref())
.bind(patch.description.is_some())
.bind(patch.description.as_ref().and_then(|d| d.as_deref()))
.bind(patch.source.as_deref())
.bind(patch.timeout_seconds)
.bind(patch.memory_limit_mb)
.bind(sandbox_json)
.bind(patch.kind.map(ScriptKind::as_str))
.fetch_optional(&mut *tx)
.await;
let script: Script = match res {
Ok(Some(row)) => row.into(),
Ok(None) => return Err(ScriptRepositoryError::NotFound(id)),
Err(sqlx::Error::Database(e)) if e.is_unique_violation() => {
return Err(ScriptRepositoryError::Conflict(
"a script with that name already exists in this app".into(),
));
}
Err(e) => return Err(e.into()),
};
// Replace imports only when the caller has a fresh list (i.e.
// the source actually changed and the validator re-extracted
// imports). A name-only or description-only edit leaves the
// dep graph alone.
if let Some(imports) = patch.imports.as_deref() {
replace_imports_tx(&mut tx, script.id, script.app_id, imports).await?;
}
tx.commit().await?; tx.commit().await?;
Ok(script) Ok(script)
} }
@@ -469,6 +381,114 @@ async fn replace_imports_tx(
Ok(()) Ok(())
} }
/// Insert a script within an existing transaction — the declarative
/// `apply` engine composes scripts + routes + triggers into one tx.
/// Mirrors `create` minus the `begin`/`commit`.
pub(crate) async fn insert_script_tx(
tx: &mut sqlx::Transaction<'_, sqlx::Postgres>,
input: &NewScript,
) -> Result<Script, ScriptRepositoryError> {
let sandbox_json = serde_json::to_value(input.sandbox.unwrap_or_default())
.unwrap_or_else(|_| serde_json::json!({}));
let res = sqlx::query_as::<_, ScriptRow>(&format!(
"INSERT INTO scripts ( \
app_id, name, description, source, kind, \
timeout_seconds, memory_limit_mb, sandbox \
) VALUES ($1, $2, $3, $4, $5, COALESCE($6, 30), COALESCE($7, 256), $8) \
RETURNING {SCRIPT_SELECT_COLS}"
))
.bind(input.app_id.into_inner())
.bind(&input.name)
.bind(input.description.as_deref())
.bind(&input.source)
.bind(input.kind.as_str())
.bind(input.timeout_seconds)
.bind(input.memory_limit_mb)
.bind(sandbox_json)
.fetch_one(&mut **tx)
.await;
let script: Script = match res {
Ok(row) => row.into(),
Err(sqlx::Error::Database(e)) if e.is_unique_violation() => {
return Err(ScriptRepositoryError::Conflict(format!(
"a script named {:?} already exists in this app",
input.name
)));
}
Err(e) => return Err(e.into()),
};
replace_imports_tx(tx, script.id, script.app_id, &input.imports).await?;
Ok(script)
}
/// Update a script within an existing transaction. Mirrors `update`
/// minus the `begin`/`commit`.
pub(crate) async fn update_script_tx(
tx: &mut sqlx::Transaction<'_, sqlx::Postgres>,
id: ScriptId,
patch: &ScriptPatch,
) -> Result<Script, ScriptRepositoryError> {
let sandbox_json = patch
.sandbox
.as_ref()
.map(|s| serde_json::to_value(s).unwrap_or_else(|_| serde_json::json!({})));
let res = sqlx::query_as::<_, ScriptRow>(&format!(
"UPDATE scripts SET \
name = COALESCE($2, name), \
description = CASE WHEN $3::bool THEN $4 ELSE description END, \
source = COALESCE($5, source), \
timeout_seconds = COALESCE($6, timeout_seconds), \
memory_limit_mb = COALESCE($7, memory_limit_mb), \
sandbox = COALESCE($8, sandbox), \
kind = COALESCE($9, kind), \
version = version + 1, \
updated_at = NOW() \
WHERE id = $1 \
RETURNING {SCRIPT_SELECT_COLS}"
))
.bind(id.into_inner())
.bind(patch.name.as_deref())
.bind(patch.description.is_some())
.bind(patch.description.as_ref().and_then(|d| d.as_deref()))
.bind(patch.source.as_deref())
.bind(patch.timeout_seconds)
.bind(patch.memory_limit_mb)
.bind(sandbox_json)
.bind(patch.kind.map(ScriptKind::as_str))
.fetch_optional(&mut **tx)
.await;
let script: Script = match res {
Ok(Some(row)) => row.into(),
Ok(None) => return Err(ScriptRepositoryError::NotFound(id)),
Err(sqlx::Error::Database(e)) if e.is_unique_violation() => {
return Err(ScriptRepositoryError::Conflict(
"a script with that name already exists in this app".into(),
));
}
Err(e) => return Err(e.into()),
};
if let Some(imports) = patch.imports.as_deref() {
replace_imports_tx(tx, script.id, script.app_id, imports).await?;
}
Ok(script)
}
/// Delete a script within an existing transaction (its routes/triggers
/// cascade via their FKs). Mirrors `delete` minus the pool.
pub(crate) async fn delete_script_tx(
tx: &mut sqlx::Transaction<'_, sqlx::Postgres>,
id: ScriptId,
) -> Result<(), ScriptRepositoryError> {
let res = sqlx::query("DELETE FROM scripts WHERE id = $1")
.bind(id.into_inner())
.execute(&mut **tx)
.await?;
if res.rows_affected() == 0 {
return Err(ScriptRepositoryError::NotFound(id));
}
Ok(())
}
/// Row shape mirroring the `scripts` table for sqlx FromRow. /// Row shape mirroring the `scripts` table for sqlx FromRow.
#[derive(sqlx::FromRow)] #[derive(sqlx::FromRow)]
struct ScriptRow { struct ScriptRow {
@@ -584,6 +604,7 @@ pub trait ExecutionLogRepository: Send + Sync {
script_id: ScriptId, script_id: ScriptId,
limit: i64, limit: i64,
cursor: Option<ExecutionLogCursor>, cursor: Option<ExecutionLogCursor>,
source: Option<ExecutionSource>,
) -> Result<Vec<ExecutionLog>, ScriptRepositoryError>; ) -> Result<Vec<ExecutionLog>, ScriptRepositoryError>;
} }
@@ -605,23 +626,30 @@ impl ExecutionLogRepository for PostgresExecutionLogRepository {
script_id: ScriptId, script_id: ScriptId,
limit: i64, limit: i64,
cursor: Option<ExecutionLogCursor>, cursor: Option<ExecutionLogCursor>,
source: Option<ExecutionSource>,
) -> Result<Vec<ExecutionLog>, ScriptRepositoryError> { ) -> Result<Vec<ExecutionLog>, ScriptRepositoryError> {
// The optional `source` filter is folded into one bind via
// `$N::text IS NULL OR source = $N` so we don't fan out into four
// query strings. `None` → the predicate is always true (no filter).
let source = source.map(ExecutionSource::as_str);
let rows = match cursor { let rows = match cursor {
Some(c) => { Some(c) => {
sqlx::query_as::<_, ExecutionLogRow>( sqlx::query_as::<_, ExecutionLogRow>(
"SELECT id, app_id, script_id, request_id, \ "SELECT id, app_id, script_id, request_id, \
request_path, request_headers, request_body, \ request_path, request_headers, request_body, \
response_code, response_body, \ response_code, response_body, \
logs, duration_ms, status, created_at \ logs, duration_ms, status, source, created_at \
FROM execution_logs \ FROM execution_logs \
WHERE script_id = $1 \ WHERE script_id = $1 \
AND (created_at, id) < ($2, $3) \ AND (created_at, id) < ($2, $3) \
AND ($4::text IS NULL OR source = $4) \
ORDER BY created_at DESC, id DESC \ ORDER BY created_at DESC, id DESC \
LIMIT $4", LIMIT $5",
) )
.bind(script_id.into_inner()) .bind(script_id.into_inner())
.bind(c.created_at) .bind(c.created_at)
.bind(c.id) .bind(c.id)
.bind(source)
.bind(limit) .bind(limit)
.fetch_all(&self.pool) .fetch_all(&self.pool)
.await? .await?
@@ -631,13 +659,15 @@ impl ExecutionLogRepository for PostgresExecutionLogRepository {
"SELECT id, app_id, script_id, request_id, \ "SELECT id, app_id, script_id, request_id, \
request_path, request_headers, request_body, \ request_path, request_headers, request_body, \
response_code, response_body, \ response_code, response_body, \
logs, duration_ms, status, created_at \ logs, duration_ms, status, source, created_at \
FROM execution_logs \ FROM execution_logs \
WHERE script_id = $1 \ WHERE script_id = $1 \
AND ($2::text IS NULL OR source = $2) \
ORDER BY created_at DESC, id DESC \ ORDER BY created_at DESC, id DESC \
LIMIT $2", LIMIT $3",
) )
.bind(script_id.into_inner()) .bind(script_id.into_inner())
.bind(source)
.bind(limit) .bind(limit)
.fetch_all(&self.pool) .fetch_all(&self.pool)
.await? .await?
@@ -662,6 +692,7 @@ struct ExecutionLogRow {
logs: serde_json::Value, logs: serde_json::Value,
duration_ms: i32, duration_ms: i32,
status: String, status: String,
source: String,
created_at: chrono::DateTime<chrono::Utc>, created_at: chrono::DateTime<chrono::Utc>,
} }
@@ -675,6 +706,9 @@ impl From<ExecutionLogRow> for ExecutionLog {
"budget_exceeded" => ExecutionStatus::BudgetExceeded, "budget_exceeded" => ExecutionStatus::BudgetExceeded,
_ => ExecutionStatus::Error, _ => ExecutionStatus::Error,
}; };
// Unknown values can't occur (CHECK constraint) but default to
// Http rather than panicking on a forward-compat surprise.
let source = ExecutionSource::from_wire(&r.source).unwrap_or_default();
Self { Self {
id: r.id, id: r.id,
app_id: r.app_id.into(), app_id: r.app_id.into(),
@@ -688,6 +722,7 @@ impl From<ExecutionLogRow> for ExecutionLog {
script_logs: r.logs, script_logs: r.logs,
duration_ms: u64::try_from(r.duration_ms).unwrap_or(0), duration_ms: u64::try_from(r.duration_ms).unwrap_or(0),
status, status,
source,
created_at: r.created_at, created_at: r.created_at,
} }
} }

View File

@@ -373,14 +373,45 @@ async fn refresh_table<RR: RouteRepository, SR: ScriptRepository>(
state: &RouteAdminState<RR, SR>, state: &RouteAdminState<RR, SR>,
) -> Result<(), RouteApiError> { ) -> Result<(), RouteApiError> {
let rows = state.routes.list_all().await?; let rows = state.routes.list_all().await?;
let compiled = compile_routes(&rows)?; let compiled = compile_routes(&rows);
state.table.replace_all(compiled); state.table.replace_all(compiled);
Ok(()) Ok(())
} }
pub fn compile_routes(rows: &[Route]) -> Result<Vec<CompiledRoute>, pattern::ParseError> { /// Compile stored route rows into the in-memory match table.
///
/// **Lenient by design (H1).** A row that fails to parse is *skipped with
/// a warning*, not propagated as an error. The motivating case: a path
/// that was valid when created but became reserved under a later, stricter
/// validation (e.g. the case-insensitive reserved-prefix check) — but this
/// also covers any other parse failure. A single un-compilable legacy row
/// must never take down the entire data plane: this function runs at
/// startup (where a hard error aborts boot) and on every table rebuild
/// after a route edit (where it would fail an unrelated CRUD op). A skipped
/// route simply doesn't match; the warning tells the operator to delete or
/// fix it (and migration 0044 sweeps the reserved-path offenders on
/// upgrade).
#[must_use]
pub fn compile_routes(rows: &[Route]) -> Vec<CompiledRoute> {
rows.iter() rows.iter()
.map(|r| { .filter_map(|r| match compile_route(r) {
Ok(compiled) => Some(compiled),
Err(e) => {
tracing::warn!(
route_id = %r.id,
app_id = %r.app_id,
path = %r.path,
error = %e,
"skipping un-compilable stored route — it will not match; \
delete or fix it"
);
None
}
})
.collect()
}
fn compile_route(r: &Route) -> Result<CompiledRoute, pattern::ParseError> {
Ok(CompiledRoute { Ok(CompiledRoute {
route_id: r.id, route_id: r.id,
app_id: r.app_id, app_id: r.app_id,
@@ -390,14 +421,12 @@ pub fn compile_routes(rows: &[Route]) -> Result<Vec<CompiledRoute>, pattern::Par
method: r.method.clone(), method: r.method.clone(),
dispatch_mode: r.dispatch_mode, dispatch_mode: r.dispatch_mode,
}) })
})
.collect()
} }
/// Validate that a new route's (host_kind, host) is consistent with at /// Validate that a new route's (host_kind, host) is consistent with at
/// least one of the parent app's domain claims. `HostKind::Any` is /// least one of the parent app's domain claims. `HostKind::Any` is
/// always permitted — it catches every host the app already owns. /// always permitted — it catches every host the app already owns.
async fn validate_route_host_against_app( pub(crate) async fn validate_route_host_against_app(
domains: &dyn AppDomainRepository, domains: &dyn AppDomainRepository,
app_id: AppId, app_id: AppId,
host_kind: HostKind, host_kind: HostKind,
@@ -577,3 +606,49 @@ impl IntoResponse for RouteApiError {
(status, Json(body)).into_response() (status, Json(body)).into_response()
} }
} }
#[cfg(test)]
mod tests {
use super::*;
use picloud_shared::DispatchMode;
use uuid::Uuid;
fn route_with_path(path: &str) -> Route {
Route {
id: Uuid::new_v4(),
app_id: AppId::from(Uuid::new_v4()),
script_id: ScriptId::from(Uuid::new_v4()),
host_kind: HostKind::Any,
host: String::new(),
host_param_name: None,
path_kind: PathKind::Exact,
path: path.to_string(),
method: None,
dispatch_mode: DispatchMode::default(),
created_at: chrono::Utc::now(),
}
}
#[test]
fn compile_routes_skips_uncompilable_rows_instead_of_failing() {
// H1 regression guard: a stored route whose path is now reserved
// (creatable before the case-insensitive reserved-prefix fix) must
// be skipped, not abort the whole compile — otherwise one legacy
// row bricks startup (`compile_routes` runs in `build_app`).
let good_a = route_with_path("/ok");
let bad = route_with_path("/API/v2/x"); // now reserved, case-insensitive
let good_b = route_with_path("/items");
let rows = vec![good_a.clone(), bad.clone(), good_b.clone()];
let compiled = compile_routes(&rows);
let ids: Vec<Uuid> = compiled.iter().map(|c| c.route_id).collect();
assert_eq!(compiled.len(), 2, "the reserved row must be dropped");
assert!(ids.contains(&good_a.id));
assert!(ids.contains(&good_b.id));
assert!(
!ids.contains(&bad.id),
"a reserved-path route must be skipped, never abort the compile"
);
}
}

View File

@@ -111,36 +111,10 @@ impl RouteRepository for PostgresRouteRepository {
} }
async fn create(&self, input: NewRoute) -> Result<Route, ScriptRepositoryError> { async fn create(&self, input: NewRoute) -> Result<Route, ScriptRepositoryError> {
let res = sqlx::query_as::<_, RouteRow>( let mut tx = self.pool.begin().await?;
"INSERT INTO routes ( \ let route = insert_route_tx(&mut tx, &input).await?;
app_id, script_id, host_kind, host, host_param_name, \ tx.commit().await?;
path_kind, path, method, dispatch_mode \ Ok(route)
) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9) \
RETURNING id, app_id, script_id, host_kind, host, host_param_name, \
path_kind, path, method, dispatch_mode, created_at",
)
.bind(input.app_id.into_inner())
.bind(input.script_id.into_inner())
.bind(host_kind_str(input.host_kind))
.bind(&input.host)
.bind(input.host_param_name.as_deref())
.bind(path_kind_str(input.path_kind))
.bind(&input.path)
.bind(input.method.as_deref())
.bind(input.dispatch_mode.as_str())
.fetch_one(&self.pool)
.await;
match res {
Ok(row) => Ok(row.into()),
Err(sqlx::Error::Database(e)) if e.is_unique_violation() => Err(
ScriptRepositoryError::Conflict("a route with this binding already exists".into()),
),
Err(sqlx::Error::Database(e)) if e.is_foreign_key_violation() => {
Err(ScriptRepositoryError::NotFound(input.script_id))
}
Err(e) => Err(e.into()),
}
} }
async fn delete(&self, route_id: Uuid) -> Result<(), ScriptRepositoryError> { async fn delete(&self, route_id: Uuid) -> Result<(), ScriptRepositoryError> {
@@ -189,6 +163,56 @@ const fn path_kind_str(k: PathKind) -> &'static str {
} }
} }
/// Insert a route within an existing transaction (declarative apply
/// composes scripts + routes + triggers into one tx). Mirrors `create`
/// minus the `begin`/`commit`.
pub(crate) async fn insert_route_tx(
tx: &mut sqlx::Transaction<'_, sqlx::Postgres>,
input: &NewRoute,
) -> Result<Route, ScriptRepositoryError> {
let res = sqlx::query_as::<_, RouteRow>(
"INSERT INTO routes ( \
app_id, script_id, host_kind, host, host_param_name, \
path_kind, path, method, dispatch_mode \
) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9) \
RETURNING id, app_id, script_id, host_kind, host, host_param_name, \
path_kind, path, method, dispatch_mode, created_at",
)
.bind(input.app_id.into_inner())
.bind(input.script_id.into_inner())
.bind(host_kind_str(input.host_kind))
.bind(&input.host)
.bind(input.host_param_name.as_deref())
.bind(path_kind_str(input.path_kind))
.bind(&input.path)
.bind(input.method.as_deref())
.bind(input.dispatch_mode.as_str())
.fetch_one(&mut **tx)
.await;
match res {
Ok(row) => Ok(row.into()),
Err(sqlx::Error::Database(e)) if e.is_unique_violation() => Err(
ScriptRepositoryError::Conflict("a route with this binding already exists".into()),
),
Err(sqlx::Error::Database(e)) if e.is_foreign_key_violation() => {
Err(ScriptRepositoryError::NotFound(input.script_id))
}
Err(e) => Err(e.into()),
}
}
/// Delete a route by id within an existing transaction.
pub(crate) async fn delete_route_tx(
tx: &mut sqlx::Transaction<'_, sqlx::Postgres>,
route_id: Uuid,
) -> Result<(), ScriptRepositoryError> {
sqlx::query("DELETE FROM routes WHERE id = $1")
.bind(route_id)
.execute(&mut **tx)
.await?;
Ok(())
}
#[derive(sqlx::FromRow)] #[derive(sqlx::FromRow)]
struct RouteRow { struct RouteRow {
id: Uuid, id: Uuid,

View File

@@ -502,6 +502,220 @@ impl PostgresTriggerRepo {
} }
} }
/// Insert a trigger (parent row + per-kind detail) within an existing
/// transaction — used by the declarative `apply` engine. Supports the
/// five settled kinds; `email`/`queue`/`dead_letter` have their own
/// create paths and are rejected here.
#[allow(clippy::too_many_arguments, clippy::too_many_lines)]
pub(crate) async fn insert_trigger_tx(
tx: &mut sqlx::Transaction<'_, sqlx::Postgres>,
app_id: AppId,
script_id: ScriptId,
registered_by: AdminUserId,
dispatch_mode: TriggerDispatchMode,
retry_max_attempts: u32,
retry_backoff: BackoffShape,
retry_base_ms: u32,
details: &TriggerDetails,
) -> Result<TriggerId, TriggerRepoError> {
let kind = match details {
TriggerDetails::Kv { .. } => "kv",
TriggerDetails::Docs { .. } => "docs",
TriggerDetails::Files { .. } => "files",
TriggerDetails::Cron { .. } => "cron",
TriggerDetails::Pubsub { .. } => "pubsub",
TriggerDetails::Queue { .. } => "queue",
TriggerDetails::DeadLetter { .. } | TriggerDetails::Email { .. } => {
return Err(TriggerRepoError::Invalid(
"trigger kind not supported by declarative apply".into(),
));
}
};
// Queue: enforce the one-consumer-per-(app_id, queue_name) invariant —
// the same advisory-lock + existence guard the interactive
// `create_queue_trigger` uses. Without this, a concurrent apply +
// interactive create on disjoint locks could double-register a queue
// consumer (there is no DB unique constraint backing the invariant).
if let TriggerDetails::Queue { queue_name, .. } = details {
sqlx::query("SELECT pg_advisory_xact_lock($1)")
.bind(advisory_lock_key(app_id, queue_name))
.execute(&mut **tx)
.await?;
let existing: Option<(Uuid,)> = sqlx::query_as(
"SELECT t.id FROM triggers t \
JOIN queue_trigger_details d ON d.trigger_id = t.id \
WHERE t.app_id = $1 AND t.kind = 'queue' AND d.queue_name = $2",
)
.bind(app_id.into_inner())
.bind(queue_name)
.fetch_optional(&mut **tx)
.await?;
if existing.is_some() {
return Err(TriggerRepoError::Invalid(format!(
"queue '{queue_name}' already has a consumer trigger; remove the existing one first"
)));
}
}
let row: (Uuid,) = sqlx::query_as(
"INSERT INTO triggers ( \
app_id, script_id, kind, enabled, dispatch_mode, \
retry_max_attempts, retry_backoff, retry_base_ms, \
registered_by_principal \
) VALUES ($1, $2, $3, TRUE, $4, $5, $6, $7, $8) RETURNING id",
)
.bind(app_id.into_inner())
.bind(script_id.into_inner())
.bind(kind)
.bind(dispatch_mode.as_str())
.bind(i32::try_from(retry_max_attempts).unwrap_or(3))
.bind(retry_backoff.as_str())
.bind(i32::try_from(retry_base_ms).unwrap_or(1000))
.bind(registered_by.into_inner())
.fetch_one(&mut **tx)
.await?;
let tid = row.0;
match details {
TriggerDetails::Kv {
collection_glob,
ops,
} => {
let ops_str: Vec<String> = ops.iter().map(|o| o.as_str().to_string()).collect();
sqlx::query(
"INSERT INTO kv_trigger_details (trigger_id, collection_glob, ops) \
VALUES ($1, $2, $3)",
)
.bind(tid)
.bind(collection_glob)
.bind(&ops_str)
.execute(&mut **tx)
.await?;
}
TriggerDetails::Docs {
collection_glob,
ops,
} => {
let ops_str: Vec<String> = ops.iter().map(|o| o.as_str().to_string()).collect();
sqlx::query(
"INSERT INTO docs_trigger_details (trigger_id, collection_glob, ops) \
VALUES ($1, $2, $3)",
)
.bind(tid)
.bind(collection_glob)
.bind(&ops_str)
.execute(&mut **tx)
.await?;
}
TriggerDetails::Files {
collection_glob,
ops,
} => {
let ops_str: Vec<String> = ops.iter().map(|o| o.as_str().to_string()).collect();
sqlx::query(
"INSERT INTO files_trigger_details (trigger_id, collection_glob, ops) \
VALUES ($1, $2, $3)",
)
.bind(tid)
.bind(collection_glob)
.bind(&ops_str)
.execute(&mut **tx)
.await?;
}
TriggerDetails::Cron {
schedule, timezone, ..
} => {
sqlx::query(
"INSERT INTO cron_trigger_details (trigger_id, schedule, timezone) \
VALUES ($1, $2, $3)",
)
.bind(tid)
.bind(schedule)
.bind(timezone)
.execute(&mut **tx)
.await?;
}
TriggerDetails::Pubsub { topic_pattern } => {
sqlx::query(
"INSERT INTO pubsub_trigger_details (trigger_id, topic_pattern) VALUES ($1, $2)",
)
.bind(tid)
.bind(topic_pattern)
.execute(&mut **tx)
.await?;
}
TriggerDetails::Queue {
queue_name,
visibility_timeout_secs,
..
} => {
sqlx::query(
"INSERT INTO queue_trigger_details \
(trigger_id, queue_name, visibility_timeout_secs) \
VALUES ($1, $2, $3)",
)
.bind(tid)
.bind(queue_name)
.bind(i32::try_from(*visibility_timeout_secs).unwrap_or(30))
.execute(&mut **tx)
.await?;
}
TriggerDetails::DeadLetter { .. } | TriggerDetails::Email { .. } => {
unreachable!("guarded above")
}
}
Ok(tid.into())
}
/// Insert an email trigger within a transaction. The inbound HMAC secret
/// is sealed by the apply engine (resolved from the app's secret store);
/// this writes the ciphertext. Parent retry settings match the
/// interactive `create_email_trigger` path (async, 3, exponential, 1000).
pub(crate) async fn insert_email_trigger_tx(
tx: &mut sqlx::Transaction<'_, sqlx::Postgres>,
app_id: AppId,
script_id: ScriptId,
registered_by: AdminUserId,
inbound_secret_encrypted: &[u8],
inbound_secret_nonce: &[u8],
) -> Result<TriggerId, TriggerRepoError> {
let row: (Uuid,) = sqlx::query_as(
"INSERT INTO triggers ( \
app_id, script_id, kind, enabled, dispatch_mode, \
retry_max_attempts, retry_backoff, retry_base_ms, \
registered_by_principal \
) VALUES ($1, $2, 'email', TRUE, 'async', 3, 'exponential', 1000, $3) RETURNING id",
)
.bind(app_id.into_inner())
.bind(script_id.into_inner())
.bind(registered_by.into_inner())
.fetch_one(&mut **tx)
.await?;
sqlx::query(
"INSERT INTO email_trigger_details \
(trigger_id, inbound_secret_encrypted, inbound_secret_nonce) \
VALUES ($1, $2, $3)",
)
.bind(row.0)
.bind(inbound_secret_encrypted)
.bind(inbound_secret_nonce)
.execute(&mut **tx)
.await?;
Ok(row.0.into())
}
/// Delete a trigger by id within an existing transaction (its detail row
/// cascades via the FK). Used by `apply --prune`.
pub(crate) async fn delete_trigger_tx(
tx: &mut sqlx::Transaction<'_, sqlx::Postgres>,
id: TriggerId,
) -> Result<(), TriggerRepoError> {
sqlx::query("DELETE FROM triggers WHERE id = $1")
.bind(id.into_inner())
.execute(&mut **tx)
.await?;
Ok(())
}
#[async_trait] #[async_trait]
impl TriggerRepo for PostgresTriggerRepo { impl TriggerRepo for PostgresTriggerRepo {
async fn create_kv_trigger( async fn create_kv_trigger(

View File

@@ -194,6 +194,7 @@ table: execution_logs
status: text NOT NULL status: text NOT NULL
created_at: timestamp with time zone NOT NULL default=now() created_at: timestamp with time zone NOT NULL default=now()
app_id: uuid NOT NULL app_id: uuid NOT NULL
source: text NOT NULL default='http'::text
table: files table: files
app_id: uuid NOT NULL app_id: uuid NOT NULL
@@ -590,6 +591,7 @@ constraints on email_trigger_details:
[PRIMARY KEY] email_trigger_details_pkey: PRIMARY KEY (trigger_id) [PRIMARY KEY] email_trigger_details_pkey: PRIMARY KEY (trigger_id)
constraints on execution_logs: constraints on execution_logs:
[CHECK] execution_logs_source_check: CHECK ((source = ANY (ARRAY['http'::text, 'kv'::text, 'docs'::text, 'dead_letter'::text, 'cron'::text, 'files'::text, 'pubsub'::text, 'email'::text, 'invoke'::text, 'queue'::text])))
[CHECK] execution_logs_status_check: CHECK ((status = ANY (ARRAY['success'::text, 'error'::text, 'timeout'::text, 'budget_exceeded'::text]))) [CHECK] execution_logs_status_check: CHECK ((status = ANY (ARRAY['success'::text, 'error'::text, 'timeout'::text, 'budget_exceeded'::text])))
[FOREIGN KEY] execution_logs_app_id_fk: FOREIGN KEY (app_id) REFERENCES apps(id) ON DELETE CASCADE [FOREIGN KEY] execution_logs_app_id_fk: FOREIGN KEY (app_id) REFERENCES apps(id) ON DELETE CASCADE
[FOREIGN KEY] execution_logs_script_id_fkey: FOREIGN KEY (script_id) REFERENCES scripts(id) ON DELETE SET NULL [FOREIGN KEY] execution_logs_script_id_fkey: FOREIGN KEY (script_id) REFERENCES scripts(id) ON DELETE SET NULL
@@ -711,3 +713,5 @@ constraints on triggers:
0040: execution logs keep history 0040: execution logs keep history
0041: dead letters composite idx 0041: dead letters composite idx
0042: secrets envelope version 0042: secrets envelope version
0043: execution logs source
0044: delete reserved path routes

View File

@@ -15,11 +15,13 @@ use axum::{
Extension, Json, Router, Extension, Json, Router,
}; };
use chrono::Utc; use chrono::Utc;
use picloud_executor_core::{ExecError, ExecRequest, ExecResponse, InvocationType}; use picloud_executor_core::{
build_execution_log, ExecError, ExecRequest, ExecResponse, InvocationType,
};
use picloud_shared::{ use picloud_shared::{
AppId, DispatchMode, ExecutionId, ExecutionLog, ExecutionLogSink, ExecutionStatus, AppId, DispatchMode, ExecutionId, ExecutionLog, ExecutionLogSink, ExecutionSource,
HttpDispatchPayload, InboxFailureKind, InboxResult, NewHttpOutbox, OutboxWriter, Principal, ExecutionStatus, HttpDispatchPayload, InboxFailureKind, InboxResult, NewHttpOutbox,
RequestId, ScriptId, OutboxWriter, Principal, RequestId, ScriptId,
}; };
use serde_json::Value as Json_; use serde_json::Value as Json_;
use uuid::Uuid; use uuid::Uuid;
@@ -150,6 +152,7 @@ where
request_path, request_path,
request_headers, request_headers,
request_body, request_body,
ExecutionSource::Http,
&outcome, &outcome,
started, started,
finished, finished,
@@ -530,6 +533,7 @@ fn build_inbox_execution_log(
script_logs: Json_::Array(vec![]), script_logs: Json_::Array(vec![]),
duration_ms, duration_ms,
status, status,
source: ExecutionSource::Http,
created_at: started, created_at: started,
} }
} }
@@ -630,67 +634,9 @@ fn exec_response_to_http(resp: ExecResponse) -> Response {
(status, http_headers, Json(resp.body)).into_response() (status, http_headers, Json(resp.body)).into_response()
} }
#[allow(clippy::too_many_arguments)] // `build_execution_log` moved to `picloud_executor_core` so the manager's
fn build_execution_log( // trigger dispatcher can build identically-shaped rows (G1 — trigger
app_id: AppId, // executions are now logged with their `ExecutionSource`).
script_id: ScriptId,
request_id: RequestId,
request_path: String,
request_headers: BTreeMap<String, String>,
request_body: Json_,
outcome: &Result<ExecResponse, ExecError>,
started: chrono::DateTime<Utc>,
finished: chrono::DateTime<Utc>,
) -> ExecutionLog {
let duration_ms = u64::try_from(
finished
.signed_duration_since(started)
.num_milliseconds()
.max(0),
)
.unwrap_or(0);
let (status, response_code, response_body, script_logs) = match outcome {
Ok(resp) => {
let logs = serde_json::to_value(&resp.logs).unwrap_or(Json_::Array(vec![]));
(
ExecutionStatus::Success,
Some(resp.status_code),
Some(resp.body.clone()),
logs,
)
}
Err(e) => {
let status = match e {
ExecError::Timeout(_) => ExecutionStatus::Timeout,
ExecError::OperationBudgetExceeded => ExecutionStatus::BudgetExceeded,
_ => ExecutionStatus::Error,
};
(
status,
None,
Some(serde_json::json!({ "error": e.to_string() })),
Json_::Array(vec![]),
)
}
};
ExecutionLog {
id: Uuid::new_v4(),
app_id,
script_id,
request_id,
request_path,
request_headers,
request_body,
response_code,
response_body,
script_logs,
duration_ms,
status,
created_at: started,
}
}
// ---------------------------------------------------------------------------- // ----------------------------------------------------------------------------
// Errors // Errors

View File

@@ -108,8 +108,15 @@ pub fn parse_path(kind: PathKind, raw: &str) -> Result<PathPattern, ParseError>
} }
fn check_reserved(raw: &str) -> Result<(), ParseError> { fn check_reserved(raw: &str) -> Result<(), ParseError> {
// Case-fold before comparing: request-time path matching is case-sensitive,
// but the reserved namespace must be rejected regardless of case so a tenant
// can't publish look-alikes like `/Admin/login` or `/API/v2/x`. Method/host
// matching are already case-insensitive; this keeps the validation guard
// durable even if path matching ever follows.
let lowered = raw.to_ascii_lowercase();
for r in RESERVED_PATH_PREFIXES { for r in RESERVED_PATH_PREFIXES {
if raw == r.trim_end_matches('/') || raw.starts_with(r) { if lowered == r.trim_end_matches('/') || lowered.starts_with(r) {
// Preserve the caller's original case in the error for clarity.
return Err(ParseError::ReservedPath(raw.to_string())); return Err(ParseError::ReservedPath(raw.to_string()));
} }
} }
@@ -456,6 +463,29 @@ mod tests {
} }
} }
#[test]
fn rejects_reserved_paths_case_insensitively() {
// Case variants of every reserved prefix must be rejected: the reserved
// namespace is a security boundary and must not be bypassable by casing.
for raw in [
"/API/v2/foo",
"/Api/v2/foo",
"/aPi/x",
"/Admin/dashboard",
"/ADMIN/x",
"/HEALTHZ",
"/HealthZ",
"/Version",
"/VERSION",
] {
let e = parse_path(PathKind::Exact, raw).unwrap_err();
assert!(
matches!(e, ParseError::ReservedPath(_)),
"expected reserved for {raw:?}, got {e:?}"
);
}
}
#[test] #[test]
fn rejects_missing_leading_slash() { fn rejects_missing_leading_slash() {
let e = parse_path(PathKind::Exact, "greet").unwrap_err(); let e = parse_path(PathKind::Exact, "greet").unwrap_err();

View File

@@ -12,7 +12,7 @@ use chrono::{DateTime, Utc};
use percent_encoding::{utf8_percent_encode, AsciiSet, CONTROLS}; use percent_encoding::{utf8_percent_encode, AsciiSet, CONTROLS};
use picloud_shared::{ use picloud_shared::{
AdminUserId, ApiKeyId, App, AppDomain, AppId, AppRole, AppUser, DispatchMode, ExecutionLog, AdminUserId, ApiKeyId, App, AppDomain, AppId, AppRole, AppUser, DispatchMode, ExecutionLog,
HostKind, InstanceRole, PathKind, Route, Scope, Script, ScriptId, HostKind, InstanceRole, PathKind, Route, Scope, Script, ScriptId, ScriptKind, ScriptSandbox,
}; };
use reqwest::{header, Method, RequestBuilder, StatusCode}; use reqwest::{header, Method, RequestBuilder, StatusCode};
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
@@ -171,10 +171,23 @@ impl Client {
} }
/// `PUT /api/v1/admin/scripts/{id}` — matches the dashboard, which /// `PUT /api/v1/admin/scripts/{id}` — matches the dashboard, which
/// uses PUT despite the field-level update semantics. /// uses PUT despite the field-level update semantics. `cfg` carries
pub async fn scripts_update_source(&self, id: &str, source: &str) -> Result<Script> { /// optional per-script runtime overrides (G3); unset fields are
/// omitted so they keep their stored value.
pub async fn scripts_update_source(
&self,
id: &str,
source: &str,
cfg: &ScriptConfig,
) -> Result<Script> {
let id = seg(id); let id = seg(id);
let body = UpdateScriptBody { source }; let body = UpdateScriptBody {
source,
timeout_seconds: cfg.timeout_seconds,
memory_limit_mb: cfg.memory_limit_mb,
kind: cfg.kind,
sandbox: cfg.sandbox,
};
let resp = self let resp = self
.request(Method::PUT, &format!("/api/v1/admin/scripts/{id}")) .request(Method::PUT, &format!("/api/v1/admin/scripts/{id}"))
.json(&body) .json(&body)
@@ -222,15 +235,19 @@ impl Client {
} }
/// `GET /api/v1/admin/scripts/{id}/logs?limit=N` /// `GET /api/v1/admin/scripts/{id}/logs?limit=N`
pub async fn logs_list(&self, script_id: &str, limit: u32) -> Result<Vec<ExecutionLog>> { pub async fn logs_list(
&self,
script_id: &str,
limit: u32,
source: Option<&str>,
) -> Result<Vec<ExecutionLog>> {
let script_id = seg(script_id); let script_id = seg(script_id);
let resp = self let mut path = format!("/api/v1/admin/scripts/{script_id}/logs?limit={limit}");
.request( if let Some(src) = source {
Method::GET, path.push_str("&source=");
&format!("/api/v1/admin/scripts/{script_id}/logs?limit={limit}"), path.push_str(&seg(src));
) }
.send() let resp = self.request(Method::GET, &path).send().await?;
.await?;
decode(resp).await decode(resp).await
} }
@@ -709,6 +726,210 @@ impl Client {
.await?; .await?;
decode(resp).await decode(resp).await
} }
// --- App membership (G2) ----------------------------------------------
/// `GET /api/v1/admin/apps/{id_or_slug}/members`
pub async fn members_list(&self, app: &str) -> Result<Vec<AppMemberDto>> {
let app = seg(app);
let resp = self
.request(Method::GET, &format!("/api/v1/admin/apps/{app}/members"))
.send()
.await?;
decode(resp).await
}
/// `POST /api/v1/admin/apps/{id_or_slug}/members`
pub async fn members_grant(
&self,
app: &str,
user_id: &str,
role: AppRole,
) -> Result<AppMemberDto> {
let app = seg(app);
let body = serde_json::json!({ "user_id": user_id, "role": role });
let resp = self
.request(Method::POST, &format!("/api/v1/admin/apps/{app}/members"))
.json(&body)
.send()
.await?;
decode(resp).await
}
/// `PATCH /api/v1/admin/apps/{id_or_slug}/members/{user_id}`
pub async fn members_set_role(
&self,
app: &str,
user_id: &str,
role: AppRole,
) -> Result<AppMemberDto> {
let (app, user_id) = (seg(app), seg(user_id));
let body = serde_json::json!({ "role": role });
let resp = self
.request(
Method::PATCH,
&format!("/api/v1/admin/apps/{app}/members/{user_id}"),
)
.json(&body)
.send()
.await?;
decode(resp).await
}
/// `DELETE /api/v1/admin/apps/{id_or_slug}/members/{user_id}`
pub async fn members_remove(&self, app: &str, user_id: &str) -> Result<()> {
let (app, user_id) = (seg(app), seg(user_id));
let resp = self
.request(
Method::DELETE,
&format!("/api/v1/admin/apps/{app}/members/{user_id}"),
)
.send()
.await?;
decode_status(resp).await
}
// --- Files (G2, read-only admin surface) ------------------------------
/// `GET /api/v1/admin/apps/{id_or_slug}/files?collection=&limit=`
pub async fn files_list(
&self,
app: &str,
collection: &str,
limit: u32,
) -> Result<ListFilesResponse> {
let app = seg(app);
let collection = seg(collection);
let resp = self
.request(
Method::GET,
&format!("/api/v1/admin/apps/{app}/files?collection={collection}&limit={limit}"),
)
.send()
.await?;
decode(resp).await
}
/// `GET /api/v1/admin/apps/{id_or_slug}/files/{collection}/{file_id}` —
/// streams the raw bytes (download). Returns the body verbatim.
pub async fn files_get_bytes(
&self,
app: &str,
collection: &str,
file_id: &str,
) -> Result<Vec<u8>> {
let (app, collection, file_id) = (seg(app), seg(collection), seg(file_id));
let resp = self
.request(
Method::GET,
&format!("/api/v1/admin/apps/{app}/files/{collection}/{file_id}"),
)
.send()
.await?;
if resp.status().is_success() {
Ok(resp.bytes().await.context("reading file bytes")?.to_vec())
} else {
Err(server_error(resp).await)
}
}
/// `DELETE /api/v1/admin/apps/{id_or_slug}/files/{collection}/{file_id}`
pub async fn files_delete(&self, app: &str, collection: &str, file_id: &str) -> Result<()> {
let (app, collection, file_id) = (seg(app), seg(collection), seg(file_id));
let resp = self
.request(
Method::DELETE,
&format!("/api/v1/admin/apps/{app}/files/{collection}/{file_id}"),
)
.send()
.await?;
decode_status(resp).await
}
// --- KV (G2, read-only admin surface) ---------------------------------
/// `GET /api/v1/admin/apps/{id_or_slug}/kv?collection=&limit=`
pub async fn kv_list(&self, app: &str, collection: &str, limit: u32) -> Result<KvListPageDto> {
let app = seg(app);
let collection = seg(collection);
let resp = self
.request(
Method::GET,
&format!("/api/v1/admin/apps/{app}/kv?collection={collection}&limit={limit}"),
)
.send()
.await?;
decode(resp).await
}
/// `GET /api/v1/admin/apps/{id_or_slug}/kv/{collection}/{key}`
pub async fn kv_get(&self, app: &str, collection: &str, key: &str) -> Result<Value> {
let (app, collection, key) = (seg(app), seg(collection), seg(key));
let resp = self
.request(
Method::GET,
&format!("/api/v1/admin/apps/{app}/kv/{collection}/{key}"),
)
.send()
.await?;
let wrapped: KvGetResponse = decode(resp).await?;
Ok(wrapped.value)
}
// --- Queues (G2, read-only admin surface) -----------------------------
/// `GET /api/v1/admin/apps/{id_or_slug}/queues`
pub async fn queues_list(&self, app: &str) -> Result<Vec<QueueSummaryDto>> {
let app = seg(app);
let resp = self
.request(Method::GET, &format!("/api/v1/admin/apps/{app}/queues"))
.send()
.await?;
decode(resp).await
}
/// `GET /api/v1/admin/apps/{id_or_slug}/queues/{queue_name}`
pub async fn queue_get(&self, app: &str, queue_name: &str) -> Result<QueueDetailDto> {
let (app, queue_name) = (seg(app), seg(queue_name));
let resp = self
.request(
Method::GET,
&format!("/api/v1/admin/apps/{app}/queues/{queue_name}"),
)
.send()
.await?;
decode(resp).await
}
/// `POST /api/v1/admin/apps/{id_or_slug}/plan` — diff a desired-state
/// bundle against the app's live state. Read-only.
pub async fn plan(&self, app: &str, bundle: &serde_json::Value) -> Result<PlanDto> {
let app = seg(app);
let resp = self
.request(Method::POST, &format!("/api/v1/admin/apps/{app}/plan"))
.json(bundle)
.send()
.await?;
decode(resp).await
}
/// `POST /api/v1/admin/apps/{id_or_slug}/apply` — reconcile the live
/// app to the bundle in one transaction.
pub async fn apply(
&self,
app: &str,
bundle: &serde_json::Value,
prune: bool,
) -> Result<ApplyReportDto> {
let app = seg(app);
let body = serde_json::json!({ "bundle": bundle, "prune": prune });
let resp = self
.request(Method::POST, &format!("/api/v1/admin/apps/{app}/apply"))
.json(&body)
.send()
.await?;
decode(resp).await
}
} }
/// `POST /api/v1/admin/auth/login` — sits outside the `Client` because /// `POST /api/v1/admin/auth/login` — sits outside the `Client` because
@@ -733,6 +954,50 @@ pub async fn auth_login(url: &str, username: &str, password: &str) -> Result<Log
// ---------- DTOs (CLI-local, wire-shape-matched) ---------- // ---------- DTOs (CLI-local, wire-shape-matched) ----------
/// Response of `POST .../plan`: per-resource diffs grouped by kind.
#[derive(Debug, Deserialize)]
pub struct PlanDto {
#[serde(default)]
pub scripts: Vec<ChangeDto>,
#[serde(default)]
pub routes: Vec<ChangeDto>,
#[serde(default)]
pub triggers: Vec<ChangeDto>,
#[serde(default)]
pub secrets: Vec<ChangeDto>,
}
#[derive(Debug, Deserialize)]
pub struct ChangeDto {
pub op: String,
pub key: String,
#[serde(default)]
pub detail: Option<String>,
}
/// Response of `POST .../apply`: counts of what changed.
#[derive(Debug, Default, Deserialize)]
pub struct ApplyReportDto {
#[serde(default)]
pub scripts_created: u32,
#[serde(default)]
pub scripts_updated: u32,
#[serde(default)]
pub scripts_deleted: u32,
#[serde(default)]
pub routes_created: u32,
#[serde(default)]
pub routes_updated: u32,
#[serde(default)]
pub routes_deleted: u32,
#[serde(default)]
pub triggers_created: u32,
#[serde(default)]
pub triggers_deleted: u32,
#[serde(default)]
pub warnings: Vec<String>,
}
#[allow(dead_code)] #[allow(dead_code)]
#[derive(Debug, Deserialize)] #[derive(Debug, Deserialize)]
pub struct AuthMeDto { pub struct AuthMeDto {
@@ -957,6 +1222,17 @@ pub struct SecretItemDto {
pub updated_at: DateTime<Utc>, pub updated_at: DateTime<Utc>,
} }
/// Per-script runtime config the CLI can now set (G3). All optional — an
/// unset field is omitted so the server applies its own default (and the
/// `PICLOUD_SANDBOX_MAX_*` admin ceilings still clamp overrides).
#[derive(Debug, Default, Clone)]
pub struct ScriptConfig {
pub timeout_seconds: Option<i32>,
pub memory_limit_mb: Option<i32>,
pub kind: Option<ScriptKind>,
pub sandbox: Option<ScriptSandbox>,
}
#[derive(Debug, Serialize)] #[derive(Debug, Serialize)]
pub struct CreateScriptBody<'a> { pub struct CreateScriptBody<'a> {
pub app_id: AppId, pub app_id: AppId,
@@ -964,11 +1240,104 @@ pub struct CreateScriptBody<'a> {
#[serde(skip_serializing_if = "Option::is_none")] #[serde(skip_serializing_if = "Option::is_none")]
pub description: Option<&'a str>, pub description: Option<&'a str>,
pub source: &'a str, pub source: &'a str,
#[serde(skip_serializing_if = "Option::is_none")]
pub timeout_seconds: Option<i32>,
#[serde(skip_serializing_if = "Option::is_none")]
pub memory_limit_mb: Option<i32>,
#[serde(skip_serializing_if = "Option::is_none")]
pub kind: Option<ScriptKind>,
#[serde(skip_serializing_if = "Option::is_none")]
pub sandbox: Option<ScriptSandbox>,
} }
#[derive(Debug, Serialize)] #[derive(Debug, Serialize)]
struct UpdateScriptBody<'a> { struct UpdateScriptBody<'a> {
source: &'a str, source: &'a str,
#[serde(skip_serializing_if = "Option::is_none")]
timeout_seconds: Option<i32>,
#[serde(skip_serializing_if = "Option::is_none")]
memory_limit_mb: Option<i32>,
#[serde(skip_serializing_if = "Option::is_none")]
kind: Option<ScriptKind>,
#[serde(skip_serializing_if = "Option::is_none")]
sandbox: Option<ScriptSandbox>,
}
// --- G2 response DTOs (deserialize-only mirrors of the server shapes) ---
// `#[allow(dead_code)]` on a couple of fields below: these structs mirror
// the full server response shape (so the surface is documented and future
// columns are a one-line add), but the TSV/KvBlock renderers don't print
// every field today.
#[derive(Debug, Deserialize)]
pub struct AppMemberDto {
pub user_id: AdminUserId,
pub username: String,
#[allow(dead_code)]
pub email: Option<String>,
pub instance_role: InstanceRole,
pub is_active: bool,
pub role: AppRole,
pub created_at: DateTime<Utc>,
}
#[derive(Debug, Deserialize)]
pub struct FileMetaDto {
pub id: String,
#[allow(dead_code)]
pub collection: String,
pub name: String,
pub content_type: String,
pub size: u64,
#[allow(dead_code)]
pub checksum: String,
#[allow(dead_code)]
pub created_at: String,
pub updated_at: String,
}
#[derive(Debug, Deserialize)]
pub struct ListFilesResponse {
pub files: Vec<FileMetaDto>,
pub next_cursor: Option<String>,
}
#[derive(Debug, Deserialize)]
pub struct KvListPageDto {
pub keys: Vec<String>,
pub next_cursor: Option<String>,
}
#[derive(Debug, Deserialize)]
struct KvGetResponse {
value: Value,
}
#[derive(Debug, Deserialize)]
pub struct QueueSummaryDto {
pub queue_name: String,
pub total: u64,
pub pending: u64,
pub claimed: u64,
}
#[derive(Debug, Deserialize)]
pub struct QueueConsumerDto {
pub trigger_id: String,
pub script_id: ScriptId,
pub script_name: String,
pub visibility_timeout_secs: u32,
pub last_fired_at: Option<DateTime<Utc>>,
}
#[derive(Debug, Deserialize)]
pub struct QueueDetailDto {
pub queue_name: String,
pub total: u64,
pub pending: u64,
pub claimed: u64,
pub consumer: Option<QueueConsumerDto>,
} }
#[derive(Debug, Serialize)] #[derive(Debug, Serialize)]

View File

@@ -0,0 +1,52 @@
//! `pic apply [--file picloud.toml]` — reconcile the live app to the
//! manifest's desired state in one server-side transaction. Additive in
//! this milestone (creates + updates); pruning of stale resources lands
//! next.
use std::path::Path;
use anyhow::Result;
use crate::client::Client;
use crate::cmds::plan::build_bundle;
use crate::config;
use crate::manifest::Manifest;
use crate::output::{KvBlock, OutputMode};
pub async fn run(manifest_path: &Path, prune: bool, mode: OutputMode) -> Result<()> {
let creds = config::resolve()?;
let client = Client::from_creds(&creds)?;
let manifest = Manifest::load(manifest_path)?;
let base_dir = manifest_path.parent().unwrap_or_else(|| Path::new("."));
let bundle = build_bundle(&manifest, base_dir)?;
let report = client.apply(&manifest.app.slug, &bundle, prune).await?;
let mut block = KvBlock::new();
block
.field("app", manifest.app.slug.clone())
.field(
"scripts",
format!(
"+{} ~{} -{}",
report.scripts_created, report.scripts_updated, report.scripts_deleted
),
)
.field(
"routes",
format!(
"+{} ~{} -{}",
report.routes_created, report.routes_updated, report.routes_deleted
),
)
.field(
"triggers",
format!("+{} -{}", report.triggers_created, report.triggers_deleted),
);
for w in &report.warnings {
block.field("warning", w.clone());
}
block.print(mode);
Ok(())
}

View File

@@ -0,0 +1,71 @@
//! `pic files ls | get | rm` — operator-facing files inspection (G2).
//!
//! Wraps the read-only `/api/v1/admin/apps/{id}/files*` surface. There is
//! no `set`/`upload` — blob writes go through scripts (`files::create`);
//! the admin surface is inspect + delete only, matching the dashboard.
use std::io::Write;
use std::path::Path;
use anyhow::{Context, Result};
use crate::client::Client;
use crate::config;
use crate::output::{OutputMode, Table};
pub async fn ls(app: &str, collection: &str, limit: u32, mode: OutputMode) -> Result<()> {
let creds = config::resolve()?;
let client = Client::from_creds(&creds)?;
let page = client.files_list(app, collection, limit).await?;
let mut table = Table::new(["id", "name", "content_type", "size", "updated_at"]);
for f in page.files {
table.row([
f.id,
f.name,
f.content_type,
f.size.to_string(),
f.updated_at,
]);
}
table.print(mode);
if page.next_cursor.is_some() {
let _ = writeln!(
std::io::stderr(),
"(more results available — raise --limit to see them)"
);
}
Ok(())
}
/// Download a file's bytes. With `--out <path>` writes to disk; otherwise
/// streams raw bytes to stdout (pipe to a file or `xxd`).
pub async fn get(app: &str, collection: &str, file_id: &str, out: Option<&Path>) -> Result<()> {
let creds = config::resolve()?;
let client = Client::from_creds(&creds)?;
let bytes = client.files_get_bytes(app, collection, file_id).await?;
match out {
Some(path) => {
std::fs::write(path, &bytes).with_context(|| format!("writing {}", path.display()))?;
let _ = writeln!(
std::io::stderr(),
"Wrote {} bytes to {}",
bytes.len(),
path.display()
);
}
None => {
std::io::stdout()
.write_all(&bytes)
.context("writing bytes to stdout")?;
}
}
Ok(())
}
pub async fn rm(app: &str, collection: &str, file_id: &str) -> Result<()> {
let creds = config::resolve()?;
let client = Client::from_creds(&creds)?;
client.files_delete(app, collection, file_id).await?;
println!("Deleted file {file_id} from {collection}");
Ok(())
}

View File

@@ -0,0 +1,42 @@
//! `pic kv ls | get` — read-only KV inspection (G2).
//!
//! Wraps the read-only `/api/v1/admin/apps/{id}/kv*` surface. There is no
//! `set`/`rm` on purpose: KV writes go through `kv::set` in scripts (which
//! emit the change events triggers depend on); an admin write would bypass
//! that, so the CLI stays read-only (matching the queues precedent).
use std::io::Write;
use anyhow::Result;
use crate::client::Client;
use crate::config;
use crate::output::{OutputMode, Table};
pub async fn ls(app: &str, collection: &str, limit: u32, mode: OutputMode) -> Result<()> {
let creds = config::resolve()?;
let client = Client::from_creds(&creds)?;
let page = client.kv_list(app, collection, limit).await?;
let mut table = Table::new(["key"]);
for k in page.keys {
table.row([k]);
}
table.print(mode);
if page.next_cursor.is_some() {
let _ = writeln!(
std::io::stderr(),
"(more keys available — raise --limit to see them)"
);
}
Ok(())
}
pub async fn get(app: &str, collection: &str, key: &str) -> Result<()> {
let creds = config::resolve()?;
let client = Client::from_creds(&creds)?;
let value = client.kv_get(app, collection, key).await?;
// Always emit the JSON value (pretty) so it pipes cleanly into jq.
let pretty = serde_json::to_string_pretty(&value).unwrap_or_else(|_| value.to_string());
println!("{pretty}");
Ok(())
}

View File

@@ -12,10 +12,15 @@ use crate::client::Client;
use crate::config; use crate::config;
use crate::output::{OutputMode, Table}; use crate::output::{OutputMode, Table};
pub async fn run(script_id: &str, limit: u32, mode: OutputMode) -> Result<()> { pub async fn run(
script_id: &str,
limit: u32,
source: Option<&str>,
mode: OutputMode,
) -> Result<()> {
let creds = config::resolve()?; let creds = config::resolve()?;
let client = Client::from_creds(&creds)?; let client = Client::from_creds(&creds)?;
let entries = client.logs_list(script_id, limit).await?; let entries = client.logs_list(script_id, limit, source).await?;
match mode { match mode {
OutputMode::Tsv => render_tsv(&entries), OutputMode::Tsv => render_tsv(&entries),
OutputMode::Json => render_json(&entries), OutputMode::Json => render_json(&entries),
@@ -24,11 +29,14 @@ pub async fn run(script_id: &str, limit: u32, mode: OutputMode) -> Result<()> {
} }
fn render_tsv(entries: &[ExecutionLog]) { fn render_tsv(entries: &[ExecutionLog]) {
let mut table = Table::new(["created_at", "status", "summary"]); // `source` is shown so background runs (queue/cron/invoke/…) are
// distinguishable from HTTP at a glance — the whole point of G1.
let mut table = Table::new(["created_at", "source", "status", "summary"]);
for e in entries { for e in entries {
let summary = summarize(&e.response_body, &e.script_logs); let summary = summarize(&e.response_body, &e.script_logs);
table.row([ table.row([
e.created_at.to_rfc3339(), e.created_at.to_rfc3339(),
e.source.as_str().to_string(),
status_label(&e.status).to_string(), status_label(&e.status).to_string(),
truncate(&summary, 120), truncate(&summary, 120),
]); ]);

View File

@@ -0,0 +1,86 @@
//! `pic members ls | add | set | rm` — manage app membership (G2).
//!
//! Wraps `/api/v1/admin/apps/{id}/members*`. All gated on
//! `AppAdmin(app)` server-side; Editors/Viewers get a 403.
use anyhow::Result;
use picloud_shared::AppRole;
use crate::client::Client;
use crate::config;
use crate::output::{KvBlock, OutputMode, Table};
pub async fn ls(app: &str, mode: OutputMode) -> Result<()> {
let creds = config::resolve()?;
let client = Client::from_creds(&creds)?;
let members = client.members_list(app).await?;
let mut table = Table::new(["user_id", "username", "role", "instance_role", "active"]);
for m in members {
table.row([
m.user_id.to_string(),
m.username,
app_role_str(m.role).to_string(),
format!("{:?}", m.instance_role).to_lowercase(),
m.is_active.to_string(),
]);
}
table.print(mode);
Ok(())
}
pub async fn add(app: &str, user_id: &str, role: &str, mode: OutputMode) -> Result<()> {
let creds = config::resolve()?;
let client = Client::from_creds(&creds)?;
let m = client
.members_grant(app, user_id, parse_role(role)?)
.await?;
print_member(&m, mode);
Ok(())
}
pub async fn set(app: &str, user_id: &str, role: &str, mode: OutputMode) -> Result<()> {
let creds = config::resolve()?;
let client = Client::from_creds(&creds)?;
let m = client
.members_set_role(app, user_id, parse_role(role)?)
.await?;
print_member(&m, mode);
Ok(())
}
pub async fn rm(app: &str, user_id: &str) -> Result<()> {
let creds = config::resolve()?;
let client = Client::from_creds(&creds)?;
client.members_remove(app, user_id).await?;
println!("Removed {user_id} from {app}");
Ok(())
}
fn print_member(m: &crate::client::AppMemberDto, mode: OutputMode) {
let mut block = KvBlock::new();
block
.field("user_id", m.user_id.to_string())
.field("username", m.username.clone())
.field("role", app_role_str(m.role))
.field("created_at", m.created_at.to_rfc3339());
block.print(mode);
}
fn parse_role(s: &str) -> Result<AppRole> {
match s.to_ascii_lowercase().as_str() {
"app_admin" | "admin" => Ok(AppRole::AppAdmin),
"editor" => Ok(AppRole::Editor),
"viewer" => Ok(AppRole::Viewer),
other => Err(anyhow::anyhow!(
"unknown role {other:?} (want app_admin | editor | viewer)"
)),
}
}
fn app_role_str(r: AppRole) -> &'static str {
match r {
AppRole::AppAdmin => "app_admin",
AppRole::Editor => "editor",
AppRole::Viewer => "viewer",
}
}

View File

@@ -1,11 +1,18 @@
pub mod admins; pub mod admins;
pub mod api_keys; pub mod api_keys;
pub mod apply;
pub mod apps; pub mod apps;
pub mod apps_domains; pub mod apps_domains;
pub mod dead_letters; pub mod dead_letters;
pub mod files;
pub mod kv;
pub mod login; pub mod login;
pub mod logout; pub mod logout;
pub mod logs; pub mod logs;
pub mod members;
pub mod plan;
pub mod pull;
pub mod queues;
pub mod routes; pub mod routes;
pub mod scripts; pub mod scripts;
pub mod secrets; pub mod secrets;

View File

@@ -0,0 +1,123 @@
//! `pic plan [--file picloud.toml]` — diff the manifest's desired state
//! against the live app and print the per-resource changes. Read-only:
//! builds a bundle (manifest + script sources) and POSTs it to the
//! server's plan endpoint, which computes the diff.
use std::path::Path;
use anyhow::{Context, Result};
use serde::Serialize;
use serde_json::{json, Map, Value};
use crate::client::{ChangeDto, Client, PlanDto};
use crate::config;
use crate::manifest::Manifest;
use crate::output::{OutputMode, Table};
pub async fn run(manifest_path: &Path, mode: OutputMode) -> Result<()> {
let creds = config::resolve()?;
let client = Client::from_creds(&creds)?;
let manifest = Manifest::load(manifest_path)?;
let base_dir = manifest_path.parent().unwrap_or_else(|| Path::new("."));
let bundle = build_bundle(&manifest, base_dir)?;
let plan = client.plan(&manifest.app.slug, &bundle).await?;
render(&plan, mode);
Ok(())
}
/// Assemble the wire bundle: scripts carry inlined source (read from
/// their `file`), routes pass through, triggers flatten into a tagged
/// array, secrets are names only.
pub fn build_bundle(manifest: &Manifest, base_dir: &Path) -> Result<Value> {
let mut scripts = Vec::with_capacity(manifest.scripts.len());
for s in &manifest.scripts {
let source = std::fs::read_to_string(base_dir.join(&s.file))
.with_context(|| format!("reading script source {}", s.file))?;
let mut obj = Map::new();
obj.insert("name".into(), json!(s.name));
obj.insert("source".into(), json!(source));
obj.insert("kind".into(), serde_json::to_value(s.kind)?);
if let Some(d) = &s.description {
obj.insert("description".into(), json!(d));
}
if let Some(t) = s.timeout_seconds {
obj.insert("timeout_seconds".into(), json!(t));
}
if let Some(m) = s.memory_limit_mb {
obj.insert("memory_limit_mb".into(), json!(m));
}
if let Some(sb) = &s.sandbox {
obj.insert("sandbox".into(), serde_json::to_value(sb)?);
}
scripts.push(Value::Object(obj));
}
let routes = manifest
.routes
.iter()
.map(serde_json::to_value)
.collect::<Result<Vec<_>, _>>()?;
let t = &manifest.triggers;
let mut triggers = Vec::new();
for s in &t.kv {
triggers.push(tagged("kv", s)?);
}
for s in &t.docs {
triggers.push(tagged("docs", s)?);
}
for s in &t.files {
triggers.push(tagged("files", s)?);
}
for s in &t.cron {
triggers.push(tagged("cron", s)?);
}
for s in &t.pubsub {
triggers.push(tagged("pubsub", s)?);
}
for s in &t.email {
triggers.push(tagged("email", s)?);
}
for s in &t.queue {
triggers.push(tagged("queue", s)?);
}
Ok(json!({
"scripts": scripts,
"routes": routes,
"triggers": triggers,
"secrets": manifest.secrets.names,
}))
}
/// Serialize a trigger spec and stamp its `kind` discriminator.
fn tagged(kind: &str, spec: impl Serialize) -> Result<Value> {
let mut v = serde_json::to_value(spec)?;
if let Value::Object(map) = &mut v {
map.insert("kind".into(), Value::String(kind.to_string()));
}
Ok(v)
}
fn render(plan: &PlanDto, mode: OutputMode) {
let mut table = Table::new(["kind", "op", "resource", "detail"]);
let groups: [(&str, &Vec<ChangeDto>); 4] = [
("script", &plan.scripts),
("route", &plan.routes),
("trigger", &plan.triggers),
("secret", &plan.secrets),
];
for (kind, changes) in groups {
for c in changes {
table.row([
kind.to_string(),
c.op.clone(),
c.key.clone(),
c.detail.clone().unwrap_or_default(),
]);
}
}
table.print(mode);
}

View File

@@ -0,0 +1,285 @@
//! `pic pull <app> [--dir .]` — export an app's current server state into
//! a `picloud.toml` manifest plus `scripts/<name>.rhai` source files, for
//! declarative management with `pic plan` / `pic apply`.
//!
//! Read-only: issues `GET`s only and writes local files. Every trigger
//! kind is exported except `email` — the server stores the *sealed secret
//! value*, not the secret name, so the manifest's `inbound_secret_ref`
//! can't be reconstructed (email triggers must be set up by hand).
use std::collections::HashMap;
use std::path::Path;
use anyhow::{Context, Result};
use picloud_shared::{DispatchMode, DocsEventOp, FilesEventOp, KvEventOp, ScriptId};
use serde::Deserialize;
use crate::client::Client;
use crate::config;
use crate::manifest::{
CronTriggerSpec, DocsTriggerSpec, FilesTriggerSpec, KvTriggerSpec, Manifest, ManifestApp,
ManifestRoute, ManifestScript, ManifestSecrets, ManifestTriggers, PubsubTriggerSpec,
QueueTriggerSpec, MANIFEST_FILE,
};
use crate::output::{KvBlock, OutputMode};
pub async fn run(app_ident: &str, dir: &Path, mode: OutputMode) -> Result<()> {
let creds = config::resolve()?;
let client = Client::from_creds(&creds)?;
// One GET per resource kind (routes are per-script, below).
let app = client.apps_get(app_ident).await?;
let scripts = client.scripts_list_by_app(app_ident).await?;
let triggers = client.triggers_list(app_ident).await?.triggers;
let secrets = client.secrets_list(app_ident).await?.secrets;
let name_by_id: HashMap<ScriptId, String> =
scripts.iter().map(|s| (s.id, s.name.clone())).collect();
// Routes: the admin surface lists them per script.
let mut routes = Vec::new();
for s in &scripts {
for r in client.routes_list_for_script(&s.id.to_string()).await? {
routes.push(ManifestRoute {
script: s.name.clone(),
method: r.method,
host_kind: r.host_kind,
host: r.host,
host_param_name: r.host_param_name,
path_kind: r.path_kind,
path: r.path,
dispatch_mode: r.dispatch_mode,
});
}
}
// The server does not constrain script names to a filesystem-safe
// charset, so a name containing a path separator or `..` would let `pull`
// write outside the project dir. Validate ALL names up front, before any
// file is written, so a single bad name can't leave a half-written dir.
// Reject rather than sanitize: a silent rename would desync the manifest
// `name` from its `file`.
for s in &scripts {
if !is_safe_filename(&s.name) {
anyhow::bail!(
"script name {:?} is not filesystem-safe (contains a path \
separator, `..`, or a leading dot); cannot pull",
s.name
);
}
}
// Scripts: write each source next to the manifest and record a path ref.
let scripts_dir = dir.join("scripts");
std::fs::create_dir_all(&scripts_dir)
.with_context(|| format!("creating {}", scripts_dir.display()))?;
let mut manifest_scripts = Vec::with_capacity(scripts.len());
for s in &scripts {
let rel = format!("scripts/{}.rhai", s.name);
std::fs::write(dir.join(&rel), &s.source).with_context(|| format!("writing {rel}"))?;
manifest_scripts.push(ManifestScript {
name: s.name.clone(),
file: rel,
kind: s.kind,
description: s.description.clone(),
timeout_seconds: i32::try_from(s.timeout_seconds).ok(),
memory_limit_mb: i32::try_from(s.memory_limit_mb).ok(),
sandbox: if s.sandbox.is_empty() {
None
} else {
Some(s.sandbox)
},
});
}
// Triggers: map the five settled kinds; warn + skip the rest.
let mut manifest_triggers = ManifestTriggers::default();
let mut skipped: Vec<String> = Vec::new();
for t in &triggers {
let script = name_by_id
.get(&t.script_id)
.cloned()
.unwrap_or_else(|| t.script_id.to_string());
let dispatch_mode = DispatchMode::from_wire(&t.dispatch_mode);
let retry_max_attempts = Some(t.retry_max_attempts);
match t.kind.as_str() {
"kv" => {
let d: CollectionDetails<KvEventOp> = decode_details(&t.details, &t.kind)?;
manifest_triggers.kv.push(KvTriggerSpec {
script,
collection_glob: d.collection_glob,
ops: d.ops,
dispatch_mode,
retry_max_attempts,
});
}
"docs" => {
let d: CollectionDetails<DocsEventOp> = decode_details(&t.details, &t.kind)?;
manifest_triggers.docs.push(DocsTriggerSpec {
script,
collection_glob: d.collection_glob,
ops: d.ops,
dispatch_mode,
retry_max_attempts,
});
}
"files" => {
let d: CollectionDetails<FilesEventOp> = decode_details(&t.details, &t.kind)?;
manifest_triggers.files.push(FilesTriggerSpec {
script,
collection_glob: d.collection_glob,
ops: d.ops,
dispatch_mode,
retry_max_attempts,
});
}
"cron" => {
let d: CronDetails = decode_details(&t.details, &t.kind)?;
manifest_triggers.cron.push(CronTriggerSpec {
script,
schedule: d.schedule,
timezone: d.timezone,
dispatch_mode,
retry_max_attempts,
});
}
"pubsub" => {
let d: PubsubDetails = decode_details(&t.details, &t.kind)?;
manifest_triggers.pubsub.push(PubsubTriggerSpec {
script,
topic_pattern: d.topic_pattern,
dispatch_mode,
retry_max_attempts,
});
}
"queue" => {
let d: QueueDetails = decode_details(&t.details, &t.kind)?;
manifest_triggers.queue.push(QueueTriggerSpec {
script,
queue_name: d.queue_name,
visibility_timeout_secs: Some(d.visibility_timeout_secs),
dispatch_mode,
retry_max_attempts,
});
}
// `email` is skipped: the server stores the sealed secret value,
// not the secret *name*, so the manifest's `inbound_secret_ref`
// can't be reconstructed — set it up by hand.
other => skipped.push(format!("{other} ({})", t.id)),
}
}
for s in &skipped {
eprintln!("warning: skipping {s} trigger — not yet representable in the manifest");
}
let manifest = Manifest {
app: ManifestApp {
slug: app.app.slug.clone(),
name: app.app.name.clone(),
description: app.app.description.clone(),
},
scripts: manifest_scripts,
routes,
triggers: manifest_triggers,
secrets: ManifestSecrets {
names: secrets.iter().map(|s| s.name.clone()).collect(),
},
};
let manifest_path = dir.join(MANIFEST_FILE);
std::fs::write(&manifest_path, manifest.to_toml()?)
.with_context(|| format!("writing {}", manifest_path.display()))?;
let mut block = KvBlock::new();
block
.field("manifest", manifest_path.display().to_string())
.field("app", manifest.app.slug.clone())
.field("scripts", manifest.scripts.len().to_string())
.field("routes", manifest.routes.len().to_string())
.field("triggers", trigger_count(&manifest.triggers).to_string())
.field("secrets", manifest.secrets.names.len().to_string());
block.print(mode);
Ok(())
}
fn trigger_count(t: &ManifestTriggers) -> usize {
t.kv.len() + t.docs.len() + t.files.len() + t.cron.len() + t.pubsub.len() + t.queue.len()
}
/// True if `name` is safe to use as a single path component in `scripts/`.
/// Rejects empty names, path separators, `.`/`..`, and leading dots.
fn is_safe_filename(name: &str) -> bool {
!name.is_empty()
&& !name.starts_with('.')
&& !name.contains('/')
&& !name.contains('\\')
&& name != ".."
&& !name.contains('\0')
}
/// Deserialize a trigger's `details` JSON, attributing failures to the kind.
/// The server tags details with a `kind` field which these structs ignore.
fn decode_details<T: for<'de> Deserialize<'de>>(
details: &serde_json::Value,
kind: &str,
) -> Result<T> {
serde_json::from_value(details.clone())
.with_context(|| format!("decoding {kind} trigger details"))
}
#[derive(Deserialize)]
struct CollectionDetails<Op> {
collection_glob: String,
#[serde(default = "Vec::new")]
ops: Vec<Op>,
}
#[derive(Deserialize)]
struct CronDetails {
schedule: String,
#[serde(default = "default_timezone")]
timezone: String,
}
#[derive(Deserialize)]
struct PubsubDetails {
topic_pattern: String,
}
#[derive(Deserialize)]
struct QueueDetails {
queue_name: String,
visibility_timeout_secs: u32,
}
fn default_timezone() -> String {
"UTC".to_string()
}
#[cfg(test)]
mod tests {
use super::is_safe_filename;
#[test]
fn rejects_traversal_and_separators() {
for bad in [
"",
".",
"..",
"../etc/passwd",
"a/b",
"a\\b",
".hidden",
"with\0nul",
] {
assert!(!is_safe_filename(bad), "expected {bad:?} to be rejected");
}
}
#[test]
fn accepts_normal_names() {
for ok in ["create-post", "nightly_digest", "Greet", "x", "a.b"] {
assert!(is_safe_filename(ok), "expected {ok:?} to be accepted");
}
}
}

View File

@@ -0,0 +1,62 @@
//! `pic queues ls | show` — read-only queue inspection (G2).
//!
//! Wraps the read-only `/api/v1/admin/apps/{id}/queues*` surface (no
//! purge/requeue — that is v1.2). Gated on `AppLogRead` server-side.
use anyhow::Result;
use crate::client::Client;
use crate::config;
use crate::output::{KvBlock, OutputMode, Table};
pub async fn ls(app: &str, mode: OutputMode) -> Result<()> {
let creds = config::resolve()?;
let client = Client::from_creds(&creds)?;
let queues = client.queues_list(app).await?;
let mut table = Table::new(["queue", "total", "pending", "claimed"]);
for q in queues {
table.row([
q.queue_name,
q.total.to_string(),
q.pending.to_string(),
q.claimed.to_string(),
]);
}
table.print(mode);
Ok(())
}
pub async fn show(app: &str, queue_name: &str, mode: OutputMode) -> Result<()> {
let creds = config::resolve()?;
let client = Client::from_creds(&creds)?;
let q = client.queue_get(app, queue_name).await?;
let mut block = KvBlock::new();
block
.field("queue", q.queue_name)
.field("total", q.total.to_string())
.field("pending", q.pending.to_string())
.field("claimed", q.claimed.to_string());
match q.consumer {
Some(c) => {
block
.field("consumer_script", c.script_name)
.field("consumer_script_id", c.script_id.to_string())
.field("consumer_trigger_id", c.trigger_id)
.field(
"visibility_timeout_secs",
c.visibility_timeout_secs.to_string(),
)
.field(
"last_fired_at",
c.last_fired_at
.map(|t| t.to_rfc3339())
.unwrap_or_else(|| "-".to_string()),
);
}
None => {
block.field("consumer", "(none registered)");
}
}
block.print(mode);
Ok(())
}

View File

@@ -8,7 +8,7 @@ use anyhow::{anyhow, Context, Result};
use picloud_shared::AppId; use picloud_shared::AppId;
use serde_json::Value; use serde_json::Value;
use crate::client::{Client, CreateScriptBody}; use crate::client::{Client, CreateScriptBody, ScriptConfig};
use crate::config; use crate::config;
use crate::output::{KvBlock, OutputMode, Table}; use crate::output::{KvBlock, OutputMode, Table};
@@ -62,6 +62,7 @@ pub async fn deploy(
app_ident: &str, app_ident: &str,
name_override: Option<&str>, name_override: Option<&str>,
description: Option<&str>, description: Option<&str>,
cfg: &ScriptConfig,
mode: OutputMode, mode: OutputMode,
) -> Result<()> { ) -> Result<()> {
let creds = config::resolve()?; let creds = config::resolve()?;
@@ -90,7 +91,7 @@ pub async fn deploy(
let existing = client.scripts_list_by_app(app_ident).await?; let existing = client.scripts_list_by_app(app_ident).await?;
let (script, action) = if let Some(s) = existing.into_iter().find(|s| s.name == name) { let (script, action) = if let Some(s) = existing.into_iter().find(|s| s.name == name) {
let updated = client let updated = client
.scripts_update_source(&s.id.to_string(), &source) .scripts_update_source(&s.id.to_string(), &source, cfg)
.await?; .await?;
(updated, "updated") (updated, "updated")
} else { } else {
@@ -99,6 +100,10 @@ pub async fn deploy(
name: &name, name: &name,
description, description,
source: &source, source: &source,
timeout_seconds: cfg.timeout_seconds,
memory_limit_mb: cfg.memory_limit_mb,
kind: cfg.kind,
sandbox: cfg.sandbox,
}; };
(client.scripts_create(&body).await?, "created") (client.scripts_create(&body).await?, "created")
}; };

View File

@@ -124,6 +124,114 @@ pub async fn create_dead_letter(
Ok(()) Ok(())
} }
/// docs/files share KV's `{collection_glob, ops?}` shape.
async fn create_collection_trigger(
kind: &str,
app: &str,
script_id: &str,
collection_glob: &str,
ops: &[String],
dispatch: &str,
mode: OutputMode,
) -> Result<()> {
let creds = config::resolve()?;
let client = Client::from_creds(&creds)?;
let mut body = base_body(script_id, dispatch);
body["collection_glob"] = json!(collection_glob);
if !ops.is_empty() {
body["ops"] = json!(ops);
}
let created = client.triggers_create(app, kind, &body).await?;
print_created(&created, mode);
Ok(())
}
pub async fn create_docs(
app: &str,
script_id: &str,
collection_glob: &str,
ops: &[String],
dispatch: &str,
mode: OutputMode,
) -> Result<()> {
create_collection_trigger("docs", app, script_id, collection_glob, ops, dispatch, mode).await
}
pub async fn create_files(
app: &str,
script_id: &str,
collection_glob: &str,
ops: &[String],
dispatch: &str,
mode: OutputMode,
) -> Result<()> {
create_collection_trigger(
"files",
app,
script_id,
collection_glob,
ops,
dispatch,
mode,
)
.await
}
pub async fn create_pubsub(
app: &str,
script_id: &str,
topic_pattern: &str,
dispatch: &str,
mode: OutputMode,
) -> Result<()> {
let creds = config::resolve()?;
let client = Client::from_creds(&creds)?;
let mut body = base_body(script_id, dispatch);
body["topic_pattern"] = json!(topic_pattern);
let created = client.triggers_create(app, "pubsub", &body).await?;
print_created(&created, mode);
Ok(())
}
pub async fn create_queue(
app: &str,
script_id: &str,
queue_name: &str,
visibility_timeout_secs: Option<u32>,
dispatch: &str,
mode: OutputMode,
) -> Result<()> {
let creds = config::resolve()?;
let client = Client::from_creds(&creds)?;
let mut body = base_body(script_id, dispatch);
body["queue_name"] = json!(queue_name);
if let Some(v) = visibility_timeout_secs {
body["visibility_timeout_secs"] = json!(v);
}
let created = client.triggers_create(app, "queue", &body).await?;
print_created(&created, mode);
Ok(())
}
pub async fn create_email(
app: &str,
script_id: &str,
inbound_secret: Option<&str>,
mode: OutputMode,
) -> Result<()> {
let creds = config::resolve()?;
let client = Client::from_creds(&creds)?;
// Email triggers take no dispatch_mode (inbound webhook only) and an
// optional shared HMAC secret the provider signs POSTs with.
let mut body = json!({ "script_id": script_id });
if let Some(secret) = inbound_secret {
body["inbound_secret"] = json!(secret);
}
let created = client.triggers_create(app, "email", &body).await?;
print_created(&created, mode);
Ok(())
}
pub async fn create_from_json(app: &str, kind: &str, body: &str, mode: OutputMode) -> Result<()> { pub async fn create_from_json(app: &str, kind: &str, body: &str, mode: OutputMode) -> Result<()> {
let creds = config::resolve()?; let creds = config::resolve()?;
let client = Client::from_creds(&creds)?; let client = Client::from_creds(&creds)?;

View File

@@ -12,6 +12,7 @@ use clap::{Args, Parser, Subcommand, ValueEnum};
mod client; mod client;
mod cmds; mod cmds;
mod config; mod config;
mod manifest;
mod output; mod output;
use crate::output::OutputMode; use crate::output::OutputMode;
@@ -128,6 +129,178 @@ enum Cmd {
#[command(subcommand)] #[command(subcommand)]
cmd: SecretsCmd, cmd: SecretsCmd,
}, },
/// App membership — list members and grant / change / revoke their
/// per-app role (app_admin | editor | viewer).
Members {
#[command(subcommand)]
cmd: MembersCmd,
},
/// Files inspection — list a collection's blobs, download bytes, or
/// delete a file. Read + delete only; writes go through scripts.
Files {
#[command(subcommand)]
cmd: FilesCmd,
},
/// Queue inspection — list queues with depth counts, or drill into
/// one queue's stats + registered consumer. Read-only.
Queues {
#[command(subcommand)]
cmd: QueuesCmd,
},
/// KV inspection — list keys in a collection or fetch one value.
/// Read-only; writes go through `kv::set` in scripts.
Kv {
#[command(subcommand)]
cmd: KvCmd,
},
/// Reconcile the live app to a `picloud.toml` manifest in one
/// transaction (additive: creates + updates).
Apply(ApplyArgs),
/// Diff a `picloud.toml` manifest against the live app and print the
/// changes (create / update / no-op / delete). Read-only.
Plan(PlanArgs),
/// Export an app's current server state into a `picloud.toml` manifest
/// (+ `scripts/<name>.rhai` sources) for declarative management with
/// `pic plan` / `pic apply`.
Pull(PullArgs),
}
#[derive(Args)]
struct ApplyArgs {
/// Path to the manifest.
#[arg(long, default_value = "picloud.toml")]
file: PathBuf,
/// Delete live scripts/routes/triggers absent from the manifest.
/// Secrets are never pruned.
#[arg(long)]
prune: bool,
}
#[derive(Args)]
struct PlanArgs {
/// Path to the manifest.
#[arg(long, default_value = "picloud.toml")]
file: PathBuf,
}
#[derive(Args)]
struct PullArgs {
/// App slug or id to export.
app: String,
/// Directory to write `picloud.toml` + `scripts/` into.
#[arg(long, default_value = ".")]
dir: PathBuf,
}
#[derive(Subcommand)]
enum KvCmd {
/// List keys in a collection.
Ls {
#[arg(long)]
app: String,
#[arg(long)]
collection: String,
#[arg(long, default_value_t = 100)]
limit: u32,
},
/// Fetch one key's value (printed as JSON).
Get {
#[arg(long)]
app: String,
#[arg(long)]
collection: String,
key: String,
},
}
#[derive(Subcommand)]
enum MembersCmd {
/// List app members.
Ls {
#[arg(long)]
app: String,
},
/// Grant a user a role on the app.
Add {
#[arg(long)]
app: String,
#[arg(long = "user")]
user_id: String,
/// `app_admin` | `editor` | `viewer`.
#[arg(long)]
role: String,
},
/// Change an existing member's role.
Set {
#[arg(long)]
app: String,
#[arg(long = "user")]
user_id: String,
#[arg(long)]
role: String,
},
/// Remove a member.
Rm {
#[arg(long)]
app: String,
#[arg(long = "user")]
user_id: String,
},
}
#[derive(Subcommand)]
enum FilesCmd {
/// List files in a collection.
Ls {
#[arg(long)]
app: String,
#[arg(long)]
collection: String,
#[arg(long, default_value_t = 100)]
limit: u32,
},
/// Download a file's bytes (to `--out <path>` or stdout).
Get {
#[arg(long)]
app: String,
#[arg(long)]
collection: String,
#[arg(long = "id")]
file_id: String,
#[arg(long)]
out: Option<PathBuf>,
},
/// Delete a file.
Rm {
#[arg(long)]
app: String,
#[arg(long)]
collection: String,
#[arg(long = "id")]
file_id: String,
},
}
#[derive(Subcommand)]
enum QueuesCmd {
/// List queues with depth counts.
Ls {
#[arg(long)]
app: String,
},
/// Show one queue's stats + consumer.
Show {
#[arg(long)]
app: String,
queue_name: String,
},
} }
#[derive(Args)] #[derive(Args)]
@@ -234,6 +407,82 @@ struct DeployArgs {
name: Option<String>, name: Option<String>,
#[arg(long)] #[arg(long)]
description: Option<String>, description: Option<String>,
/// Per-script wall-clock timeout in seconds (overrides the instance
/// default; still clamped by `PICLOUD_SANDBOX_MAX_*` ceilings).
#[arg(long)]
timeout: Option<i32>,
/// Per-script memory ceiling in MB.
#[arg(long)]
memory: Option<i32>,
/// Script kind: `endpoint` (default) or `module` (importable, no route).
#[arg(long, value_enum)]
kind: Option<ScriptKindArg>,
/// Sandbox override as `key=value`, repeatable. Keys: `max_operations`,
/// `max_string_size`, `max_array_size`, `max_map_size`,
/// `max_call_levels`, `max_expr_depth`. E.g. `--sandbox max_operations=500000`.
#[arg(long = "sandbox", value_name = "KEY=VALUE")]
sandbox: Vec<String>,
}
#[derive(Copy, Clone, clap::ValueEnum)]
enum ScriptKindArg {
Endpoint,
Module,
}
impl From<ScriptKindArg> for picloud_shared::ScriptKind {
fn from(v: ScriptKindArg) -> Self {
match v {
ScriptKindArg::Endpoint => Self::Endpoint,
ScriptKindArg::Module => Self::Module,
}
}
}
impl DeployArgs {
/// Fold the runtime-config flags into a `ScriptConfig`, parsing and
/// validating the `--sandbox key=value` pairs.
fn script_config(&self) -> anyhow::Result<client::ScriptConfig> {
let sandbox = parse_sandbox_overrides(&self.sandbox)?;
Ok(client::ScriptConfig {
timeout_seconds: self.timeout,
memory_limit_mb: self.memory,
kind: self.kind.map(Into::into),
sandbox,
})
}
}
/// Parse repeatable `--sandbox key=value` flags into a `ScriptSandbox`.
/// Unknown keys and non-integer values are hard errors so a typo doesn't
/// silently deploy an unrestricted script.
fn parse_sandbox_overrides(
pairs: &[String],
) -> anyhow::Result<Option<picloud_shared::ScriptSandbox>> {
use anyhow::{anyhow, Context};
if pairs.is_empty() {
return Ok(None);
}
let mut sb = picloud_shared::ScriptSandbox::default();
for raw in pairs {
let (key, val) = raw
.split_once('=')
.ok_or_else(|| anyhow!("--sandbox expects key=value, got {raw:?}"))?;
let n: u64 = val
.trim()
.parse()
.with_context(|| format!("--sandbox {key}: {val:?} is not a non-negative integer"))?;
match key.trim() {
"max_operations" => sb.max_operations = Some(n),
"max_string_size" => sb.max_string_size = Some(n),
"max_array_size" => sb.max_array_size = Some(n),
"max_map_size" => sb.max_map_size = Some(n),
"max_call_levels" => sb.max_call_levels = Some(n),
"max_expr_depth" => sb.max_expr_depth = Some(n),
other => return Err(anyhow!("unknown --sandbox key: {other:?}")),
}
}
Ok(Some(sb))
} }
#[derive(Args)] #[derive(Args)]
@@ -275,6 +524,11 @@ struct LogsArgs {
script_id: String, script_id: String,
#[arg(long, default_value_t = 50)] #[arg(long, default_value_t = 50)]
limit: u32, limit: u32,
/// Filter by execution origin: `http`, `kv`, `cron`, `queue`,
/// `invoke`, `dead_letter`, `docs`, `files`, `pubsub`, `email`.
/// Omit (or `all`) to show every source.
#[arg(long)]
source: Option<String>,
} }
#[derive(Clone, Copy, ValueEnum)] #[derive(Clone, Copy, ValueEnum)]
@@ -498,10 +752,89 @@ enum TriggersCmd {
dispatch: DispatchModeArg, dispatch: DispatchModeArg,
}, },
/// Create a docs trigger — fires on document mutations in matching
/// collections.
#[command(name = "create-docs")]
CreateDocs {
#[arg(long)]
app: String,
#[arg(long)]
script: String,
/// Glob over collection names (`*`, `posts`, `events_*`, …).
#[arg(long)]
collection: String,
/// Repeat to filter ops: `--op insert --op delete`. Empty fires on any.
#[arg(long = "op")]
ops: Vec<String>,
#[arg(long, value_enum, default_value_t = DispatchModeArg::Async)]
dispatch: DispatchModeArg,
},
/// Create a files trigger — fires on blob create/update/delete in
/// matching collections.
#[command(name = "create-files")]
CreateFiles {
#[arg(long)]
app: String,
#[arg(long)]
script: String,
#[arg(long)]
collection: String,
#[arg(long = "op")]
ops: Vec<String>,
#[arg(long, value_enum, default_value_t = DispatchModeArg::Async)]
dispatch: DispatchModeArg,
},
/// Create a pub/sub trigger — fires on messages published to topics
/// matching the pattern.
#[command(name = "create-pubsub")]
CreatePubsub {
#[arg(long)]
app: String,
#[arg(long)]
script: String,
/// Topic glob (`*`, `orders.*`, `user.signup`, …).
#[arg(long = "topic")]
topic_pattern: String,
#[arg(long, value_enum, default_value_t = DispatchModeArg::Async)]
dispatch: DispatchModeArg,
},
/// Create a queue consumer trigger — fires per message claimed off
/// the named queue.
#[command(name = "create-queue")]
CreateQueue {
#[arg(long)]
app: String,
#[arg(long)]
script: String,
#[arg(long = "queue")]
queue_name: String,
/// Per-message visibility timeout in seconds (claim lease).
#[arg(long = "visibility-timeout")]
visibility_timeout_secs: Option<u32>,
#[arg(long, value_enum, default_value_t = DispatchModeArg::Async)]
dispatch: DispatchModeArg,
},
/// Create an email trigger — fires on inbound mail POSTed to the
/// webhook receiver. No dispatch mode (inbound webhook only).
#[command(name = "create-email")]
CreateEmail {
#[arg(long)]
app: String,
#[arg(long)]
script: String,
/// Shared HMAC secret the provider signs inbound POSTs with.
/// Omit to accept unsigned POSTs.
#[arg(long = "inbound-secret")]
inbound_secret: Option<String>,
},
/// Create a trigger of any kind from a raw JSON body — escape /// Create a trigger of any kind from a raw JSON body — escape
/// hatch for kinds the CLI doesn't expose per-kind wrappers for /// hatch for advanced retry/dispatch settings beyond the per-kind
/// (docs/files/pubsub/email/queue) and for advanced retry/dispatch /// wrappers' defaults.
/// settings beyond the per-kind defaults.
#[command(name = "create-from-json")] #[command(name = "create-from-json")]
CreateFromJson { CreateFromJson {
#[arg(long)] #[arg(long)]
@@ -753,6 +1086,9 @@ async fn main() -> ExitCode {
} }
Cmd::Logout => cmds::logout::run().await, Cmd::Logout => cmds::logout::run().await,
Cmd::Whoami => cmds::whoami::run(mode).await, Cmd::Whoami => cmds::whoami::run(mode).await,
Cmd::Apply(args) => cmds::apply::run(&args.file, args.prune, mode).await,
Cmd::Plan(args) => cmds::plan::run(&args.file, mode).await,
Cmd::Pull(args) => cmds::pull::run(&args.app, &args.dir, mode).await,
Cmd::Apps { cmd: AppsCmd::Ls } => cmds::apps::ls(mode).await, Cmd::Apps { cmd: AppsCmd::Ls } => cmds::apps::ls(mode).await,
Cmd::Apps { Cmd::Apps {
cmd: cmd:
@@ -790,16 +1126,20 @@ async fn main() -> ExitCode {
} => cmds::scripts::ls(app.as_deref(), mode).await, } => cmds::scripts::ls(app.as_deref(), mode).await,
Cmd::Scripts { Cmd::Scripts {
cmd: ScriptsCmd::Deploy(args), cmd: ScriptsCmd::Deploy(args),
} => { } => match args.script_config() {
Ok(cfg) => {
cmds::scripts::deploy( cmds::scripts::deploy(
&args.file, &args.file,
&args.app, &args.app,
args.name.as_deref(), args.name.as_deref(),
args.description.as_deref(), args.description.as_deref(),
&cfg,
mode, mode,
) )
.await .await
} }
Err(e) => Err(e),
},
Cmd::Scripts { Cmd::Scripts {
cmd: ScriptsCmd::Invoke(args), cmd: ScriptsCmd::Invoke(args),
} => cmds::scripts::invoke(&args.id, args.body.as_deref(), &args.headers).await, } => cmds::scripts::invoke(&args.id, args.body.as_deref(), &args.headers).await,
@@ -821,20 +1161,28 @@ async fn main() -> ExitCode {
Cmd::ApiKeys { Cmd::ApiKeys {
cmd: ApiKeysCmd::Rm { id }, cmd: ApiKeysCmd::Rm { id },
} => cmds::api_keys::rm(&id).await, } => cmds::api_keys::rm(&id).await,
Cmd::Logs(LogsArgs { script_id, limit }) => cmds::logs::run(&script_id, limit, mode).await, Cmd::Logs(LogsArgs {
script_id,
limit,
source,
}) => cmds::logs::run(&script_id, limit, source.as_deref(), mode).await,
Cmd::Invoke(args) => { Cmd::Invoke(args) => {
cmds::scripts::invoke(&args.id, args.body.as_deref(), &args.headers).await cmds::scripts::invoke(&args.id, args.body.as_deref(), &args.headers).await
} }
Cmd::Deploy(args) => { Cmd::Deploy(args) => match args.script_config() {
Ok(cfg) => {
cmds::scripts::deploy( cmds::scripts::deploy(
&args.file, &args.file,
&args.app, &args.app,
args.name.as_deref(), args.name.as_deref(),
args.description.as_deref(), args.description.as_deref(),
&cfg,
mode, mode,
) )
.await .await
} }
Err(e) => Err(e),
},
Cmd::Routes { Cmd::Routes {
cmd: RoutesCmd::Ls { script_id }, cmd: RoutesCmd::Ls { script_id },
} => cmds::routes::ls(&script_id, mode).await, } => cmds::routes::ls(&script_id, mode).await,
@@ -1004,6 +1352,92 @@ async fn main() -> ExitCode {
) )
.await .await
} }
Cmd::Triggers {
cmd:
TriggersCmd::CreateDocs {
app,
script,
collection,
ops,
dispatch,
},
} => {
cmds::triggers::create_docs(
&app,
&script,
&collection,
&ops,
dispatch_wire(dispatch),
mode,
)
.await
}
Cmd::Triggers {
cmd:
TriggersCmd::CreateFiles {
app,
script,
collection,
ops,
dispatch,
},
} => {
cmds::triggers::create_files(
&app,
&script,
&collection,
&ops,
dispatch_wire(dispatch),
mode,
)
.await
}
Cmd::Triggers {
cmd:
TriggersCmd::CreatePubsub {
app,
script,
topic_pattern,
dispatch,
},
} => {
cmds::triggers::create_pubsub(
&app,
&script,
&topic_pattern,
dispatch_wire(dispatch),
mode,
)
.await
}
Cmd::Triggers {
cmd:
TriggersCmd::CreateQueue {
app,
script,
queue_name,
visibility_timeout_secs,
dispatch,
},
} => {
cmds::triggers::create_queue(
&app,
&script,
&queue_name,
visibility_timeout_secs,
dispatch_wire(dispatch),
mode,
)
.await
}
Cmd::Triggers {
cmd:
TriggersCmd::CreateEmail {
app,
script,
inbound_secret,
},
} => cmds::triggers::create_email(&app, &script, inbound_secret.as_deref(), mode).await,
Cmd::Triggers { Cmd::Triggers {
cmd: TriggersCmd::CreateFromJson { app, kind, body }, cmd: TriggersCmd::CreateFromJson { app, kind, body },
} => cmds::triggers::create_from_json(&app, &kind, &body, mode).await, } => cmds::triggers::create_from_json(&app, &kind, &body, mode).await,
@@ -1074,6 +1508,65 @@ async fn main() -> ExitCode {
Cmd::Secrets { Cmd::Secrets {
cmd: SecretsCmd::Rm { app, name }, cmd: SecretsCmd::Rm { app, name },
} => cmds::secrets::rm(&app, &name).await, } => cmds::secrets::rm(&app, &name).await,
Cmd::Members {
cmd: MembersCmd::Ls { app },
} => cmds::members::ls(&app, mode).await,
Cmd::Members {
cmd: MembersCmd::Add { app, user_id, role },
} => cmds::members::add(&app, &user_id, &role, mode).await,
Cmd::Members {
cmd: MembersCmd::Set { app, user_id, role },
} => cmds::members::set(&app, &user_id, &role, mode).await,
Cmd::Members {
cmd: MembersCmd::Rm { app, user_id },
} => cmds::members::rm(&app, &user_id).await,
Cmd::Files {
cmd:
FilesCmd::Ls {
app,
collection,
limit,
},
} => cmds::files::ls(&app, &collection, limit, mode).await,
Cmd::Files {
cmd:
FilesCmd::Get {
app,
collection,
file_id,
out,
},
} => cmds::files::get(&app, &collection, &file_id, out.as_deref()).await,
Cmd::Files {
cmd:
FilesCmd::Rm {
app,
collection,
file_id,
},
} => cmds::files::rm(&app, &collection, &file_id).await,
Cmd::Queues {
cmd: QueuesCmd::Ls { app },
} => cmds::queues::ls(&app, mode).await,
Cmd::Queues {
cmd: QueuesCmd::Show { app, queue_name },
} => cmds::queues::show(&app, &queue_name, mode).await,
Cmd::Kv {
cmd:
KvCmd::Ls {
app,
collection,
limit,
},
} => cmds::kv::ls(&app, &collection, limit, mode).await,
Cmd::Kv {
cmd:
KvCmd::Get {
app,
collection,
key,
},
} => cmds::kv::get(&app, &collection, &key).await,
}; };
match result { match result {

View File

@@ -0,0 +1,362 @@
//! Declarative project manifest (`picloud.toml`).
//!
//! One manifest describes the desired state of a **single app** — its
//! scripts, routes, triggers, and the *names* of the secrets it expects
//! (values are pushed out-of-band via `pic secret set`, never committed).
//!
//! This is the foundation of the declarative project tool (`pic pull` /
//! `pic plan` / `pic apply`). The types deliberately reuse `picloud_shared`
//! enums (`HostKind`, `PathKind`, `DispatchMode`, `ScriptKind`,
//! `ScriptSandbox`, the event-op enums) so the manifest's wire shape stays
//! identical to the admin API — the CLI never depends on `manager-core`.
//!
//! All eight trigger kinds are representable except `dead_letter` (not
//! exposed declaratively). `email` triggers carry an `inbound_secret_ref`
//! (a secret name) resolved server-side at apply.
use std::fs;
use std::path::Path;
use anyhow::{Context, Result};
use picloud_shared::{
DispatchMode, DocsEventOp, FilesEventOp, HostKind, KvEventOp, PathKind, ScriptKind,
ScriptSandbox,
};
use serde::{Deserialize, Serialize};
/// Conventional manifest filename at a project root.
pub const MANIFEST_FILE: &str = "picloud.toml";
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct Manifest {
pub app: ManifestApp,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub scripts: Vec<ManifestScript>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub routes: Vec<ManifestRoute>,
#[serde(default, skip_serializing_if = "ManifestTriggers::is_empty")]
pub triggers: ManifestTriggers,
#[serde(default, skip_serializing_if = "ManifestSecrets::is_empty")]
pub secrets: ManifestSecrets,
}
impl Manifest {
/// Parse a manifest from TOML text.
pub fn parse(text: &str) -> Result<Self> {
toml::from_str(text).context("parsing manifest TOML")
}
/// Load and parse the manifest at `path`.
pub fn load(path: &Path) -> Result<Self> {
let body =
fs::read_to_string(path).with_context(|| format!("reading {}", path.display()))?;
Self::parse(&body)
}
/// Render to TOML text. Tables are emitted after scalars (the struct
/// field order already satisfies TOML's "values before tables" rule).
pub fn to_toml(&self) -> Result<String> {
toml::to_string_pretty(self).context("serializing manifest TOML")
}
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct ManifestApp {
pub slug: String,
pub name: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub description: Option<String>,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct ManifestScript {
pub name: String,
/// Path to the `.rhai` source, relative to the manifest's directory.
pub file: String,
#[serde(default, skip_serializing_if = "is_endpoint")]
pub kind: ScriptKind,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub description: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub timeout_seconds: Option<i32>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub memory_limit_mb: Option<i32>,
/// Per-script sandbox overrides; omitted entirely when no knob is set.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub sandbox: Option<ScriptSandbox>,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct ManifestRoute {
/// Name of the script this route binds to.
pub script: String,
/// HTTP method; omit for ANY.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub method: Option<String>,
pub host_kind: HostKind,
#[serde(default, skip_serializing_if = "String::is_empty")]
pub host: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub host_param_name: Option<String>,
pub path_kind: PathKind,
pub path: String,
#[serde(default, skip_serializing_if = "is_sync")]
pub dispatch_mode: DispatchMode,
}
/// Triggers grouped by kind (arrays-of-tables: `[[triggers.cron]]`, …).
#[derive(Debug, Clone, Default, PartialEq, Serialize, Deserialize)]
pub struct ManifestTriggers {
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub kv: Vec<KvTriggerSpec>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub docs: Vec<DocsTriggerSpec>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub files: Vec<FilesTriggerSpec>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub cron: Vec<CronTriggerSpec>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub pubsub: Vec<PubsubTriggerSpec>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub email: Vec<EmailTriggerSpec>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub queue: Vec<QueueTriggerSpec>,
}
impl ManifestTriggers {
#[must_use]
pub fn is_empty(&self) -> bool {
self.kv.is_empty()
&& self.docs.is_empty()
&& self.files.is_empty()
&& self.cron.is_empty()
&& self.pubsub.is_empty()
&& self.email.is_empty()
&& self.queue.is_empty()
}
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct KvTriggerSpec {
pub script: String,
pub collection_glob: String,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub ops: Vec<KvEventOp>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub dispatch_mode: Option<DispatchMode>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub retry_max_attempts: Option<u32>,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct DocsTriggerSpec {
pub script: String,
pub collection_glob: String,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub ops: Vec<DocsEventOp>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub dispatch_mode: Option<DispatchMode>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub retry_max_attempts: Option<u32>,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct FilesTriggerSpec {
pub script: String,
pub collection_glob: String,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub ops: Vec<FilesEventOp>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub dispatch_mode: Option<DispatchMode>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub retry_max_attempts: Option<u32>,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct CronTriggerSpec {
pub script: String,
/// 6-field cron expression (with seconds).
pub schedule: String,
#[serde(default = "default_timezone")]
pub timezone: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub dispatch_mode: Option<DispatchMode>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub retry_max_attempts: Option<u32>,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct PubsubTriggerSpec {
pub script: String,
pub topic_pattern: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub dispatch_mode: Option<DispatchMode>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub retry_max_attempts: Option<u32>,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct EmailTriggerSpec {
pub script: String,
/// Name of the secret (set via `pic secret set`) holding the inbound
/// HMAC value — resolved + sealed server-side at apply. Never the value.
pub inbound_secret_ref: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub dispatch_mode: Option<DispatchMode>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub retry_max_attempts: Option<u32>,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct QueueTriggerSpec {
pub script: String,
pub queue_name: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub visibility_timeout_secs: Option<u32>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub dispatch_mode: Option<DispatchMode>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub retry_max_attempts: Option<u32>,
}
/// `[secrets] names = [...]` — declares which secrets the app expects.
/// Values are never in the manifest; `pic secret set` pushes them.
#[derive(Debug, Clone, Default, PartialEq, Serialize, Deserialize)]
pub struct ManifestSecrets {
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub names: Vec<String>,
}
impl ManifestSecrets {
#[must_use]
pub fn is_empty(&self) -> bool {
self.names.is_empty()
}
}
// ---- serde skip/default helpers ----
fn is_endpoint(kind: &ScriptKind) -> bool {
*kind == ScriptKind::Endpoint
}
fn is_sync(mode: &DispatchMode) -> bool {
*mode == DispatchMode::Sync
}
fn default_timezone() -> String {
"UTC".to_string()
}
#[cfg(test)]
mod tests {
use super::*;
fn sample() -> Manifest {
Manifest {
app: ManifestApp {
slug: "blog".into(),
name: "My Blog".into(),
description: Some("demo".into()),
},
scripts: vec![
ManifestScript {
name: "create-post".into(),
file: "scripts/create-post.rhai".into(),
kind: ScriptKind::Endpoint,
description: None,
timeout_seconds: Some(10),
memory_limit_mb: Some(256),
sandbox: None,
},
ManifestScript {
name: "lib".into(),
file: "scripts/lib.rhai".into(),
kind: ScriptKind::Module,
description: None,
timeout_seconds: None,
memory_limit_mb: None,
sandbox: Some(ScriptSandbox {
max_operations: Some(5_000_000),
..ScriptSandbox::empty()
}),
},
],
routes: vec![ManifestRoute {
script: "create-post".into(),
method: Some("POST".into()),
host_kind: HostKind::Any,
host: String::new(),
host_param_name: None,
path_kind: PathKind::Exact,
path: "/posts".into(),
dispatch_mode: DispatchMode::Sync,
}],
triggers: ManifestTriggers {
cron: vec![CronTriggerSpec {
script: "create-post".into(),
schedule: "0 6 * * * *".into(),
timezone: "UTC".into(),
dispatch_mode: None,
retry_max_attempts: None,
}],
kv: vec![KvTriggerSpec {
script: "create-post".into(),
collection_glob: "users".into(),
ops: vec![KvEventOp::Insert, KvEventOp::Update],
dispatch_mode: Some(DispatchMode::Async),
retry_max_attempts: Some(5),
}],
..ManifestTriggers::default()
},
secrets: ManifestSecrets {
names: vec!["STRIPE_KEY".into()],
},
}
}
#[test]
fn round_trips_through_toml() {
let m = sample();
let text = m.to_toml().expect("serialize");
let back = Manifest::parse(&text).expect("parse");
assert_eq!(m, back, "manifest must survive a TOML round-trip");
}
#[test]
fn omits_defaulted_fields() {
let text = sample().to_toml().unwrap();
// Endpoint kind + sync dispatch are defaults → not emitted.
assert!(
!text.contains("kind = \"endpoint\""),
"default kind should be omitted:\n{text}"
);
assert!(
!text.contains("dispatch_mode = \"sync\""),
"default route dispatch should be omitted:\n{text}"
);
// Module kind IS non-default → emitted.
assert!(text.contains("kind = \"module\""), "got:\n{text}");
}
#[test]
fn empty_optional_sections_omitted() {
let m = Manifest {
app: ManifestApp {
slug: "x".into(),
name: "X".into(),
description: None,
},
scripts: vec![],
routes: vec![],
triggers: ManifestTriggers::default(),
secrets: ManifestSecrets::default(),
};
let text = m.to_toml().unwrap();
assert!(!text.contains("[[scripts]]"), "got:\n{text}");
assert!(!text.contains("triggers"), "got:\n{text}");
assert!(!text.contains("secrets"), "got:\n{text}");
// Still round-trips.
assert_eq!(m, Manifest::parse(&text).unwrap());
}
}

View File

@@ -0,0 +1,153 @@
//! `pic apply` journey: apply a manifest to an empty app (atomic create),
//! re-apply is an idempotent no-op, and a bundle containing any invalid
//! resource applies nothing (all-or-nothing).
use std::fs;
use tempfile::TempDir;
use crate::common;
use crate::common::cleanup::AppGuard;
fn manifest_dir() -> TempDir {
let dir = TempDir::new().expect("tempdir");
fs::create_dir_all(dir.path().join("scripts")).expect("scripts dir");
dir
}
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
#[test]
fn apply_creates_then_noop() {
let Some(fx) = common::fixture_or_skip() else {
return;
};
let env = common::admin_env(fx);
let slug = common::unique_slug("apply");
common::pic_as(&env)
.args(["apps", "create", &slug])
.assert()
.success();
let _guard = AppGuard::new(&env.url, &env.token, &slug);
let dir = manifest_dir();
fs::write(
dir.path().join("scripts/greet.rhai"),
"let body = #{ ok: true }; body",
)
.unwrap();
let manifest = format!(
"[app]\nslug = \"{slug}\"\nname = \"Apply Test\"\n\n\
[[scripts]]\nname = \"greet\"\nfile = \"scripts/greet.rhai\"\n\n\
[[routes]]\nscript = \"greet\"\nmethod = \"POST\"\n\
host_kind = \"any\"\npath_kind = \"exact\"\npath = \"/greet\"\n\n\
[[triggers.cron]]\nscript = \"greet\"\nschedule = \"0 0 * * * *\"\ntimezone = \"UTC\"\n"
);
let manifest_path = dir.path().join("picloud.toml");
fs::write(&manifest_path, &manifest).unwrap();
// First apply: creates script + route + trigger.
let out = common::pic_as(&env)
.args(["apply", "--file"])
.arg(&manifest_path)
.output()
.expect("apply");
assert!(
out.status.success(),
"apply failed: {}",
String::from_utf8_lossy(&out.stderr)
);
let stdout = String::from_utf8(out.stdout).unwrap();
assert!(
stdout.contains("+1"),
"expected creations in report:\n{stdout}"
);
// The resources now exist.
let s = String::from_utf8(
common::pic_as(&env)
.args(["scripts", "ls", "--app", &slug])
.output()
.unwrap()
.stdout,
)
.unwrap();
assert!(s.contains("greet"), "script not created:\n{s}");
// Plan is now clean (apply reached desired state).
let p = String::from_utf8(
common::pic_as(&env)
.args(["plan", "--file"])
.arg(&manifest_path)
.output()
.unwrap()
.stdout,
)
.unwrap();
assert!(
!p.contains("create") && !p.contains("update"),
"expected clean plan after apply:\n{p}"
);
// Re-apply: idempotent — nothing created/updated.
let r = String::from_utf8(
common::pic_as(&env)
.args(["apply", "--file"])
.arg(&manifest_path)
.output()
.unwrap()
.stdout,
)
.unwrap();
assert!(!r.contains("+1"), "re-apply should be a no-op:\n{r}");
}
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
#[test]
fn apply_rejects_bad_bundle_atomically() {
let Some(fx) = common::fixture_or_skip() else {
return;
};
let env = common::admin_env(fx);
let slug = common::unique_slug("apply-atomic");
common::pic_as(&env)
.args(["apps", "create", &slug])
.assert()
.success();
let _guard = AppGuard::new(&env.url, &env.token, &slug);
let dir = manifest_dir();
fs::write(dir.path().join("scripts/good.rhai"), "let x = 1; x").unwrap();
// Invalid Rhai — fails validation, so the whole apply must abort.
fs::write(dir.path().join("scripts/bad.rhai"), "let x = ;").unwrap();
let manifest = format!(
"[app]\nslug = \"{slug}\"\nname = \"Atomic Test\"\n\n\
[[scripts]]\nname = \"good\"\nfile = \"scripts/good.rhai\"\n\n\
[[scripts]]\nname = \"bad\"\nfile = \"scripts/bad.rhai\"\n"
);
let manifest_path = dir.path().join("picloud.toml");
fs::write(&manifest_path, &manifest).unwrap();
let out = common::pic_as(&env)
.args(["apply", "--file"])
.arg(&manifest_path)
.output()
.expect("apply");
assert!(
!out.status.success(),
"apply with an invalid script should fail"
);
// Atomic: the valid script must NOT have been created.
let s = String::from_utf8(
common::pic_as(&env)
.args(["scripts", "ls", "--app", &slug])
.output()
.unwrap()
.stdout,
)
.unwrap();
assert!(
!s.contains("good"),
"a failed apply must leave nothing behind:\n{s}"
);
}

View File

@@ -15,12 +15,17 @@ mod common;
mod admins; mod admins;
mod api_keys; mod api_keys;
mod apply;
mod apps; mod apps;
mod auth; mod auth;
mod dead_letters; mod dead_letters;
mod email_queue;
mod invoke; mod invoke;
mod logs; mod logs;
mod output; mod output;
mod plan;
mod prune;
mod pull;
mod roles; mod roles;
mod routes; mod routes;
mod scripts; mod scripts;

View File

@@ -0,0 +1,222 @@
//! M5: `pic apply` creates email + queue triggers. The email trigger's
//! inbound secret is referenced by name (pushed via `pic secret set`) and
//! resolved + re-sealed server-side — never written into the manifest.
use std::fs;
use tempfile::TempDir;
use crate::common;
use crate::common::cleanup::AppGuard;
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
#[test]
fn apply_email_and_queue_triggers() {
let Some(fx) = common::fixture_or_skip() else {
return;
};
let env = common::admin_env(fx);
let slug = common::unique_slug("m5");
common::pic_as(&env)
.args(["apps", "create", &slug])
.assert()
.success();
let _guard = AppGuard::new(&env.url, &env.token, &slug);
// The email trigger references this secret by name; push its value
// out-of-band first.
common::pic_as(&env)
.args(["secrets", "set", "--app", &slug, "email-hmac"])
.write_stdin("super-secret-hmac")
.assert()
.success();
let dir = TempDir::new().unwrap();
fs::create_dir_all(dir.path().join("scripts")).unwrap();
fs::write(dir.path().join("scripts/handler.rhai"), "let x = 1; x").unwrap();
let manifest = format!(
"[app]\nslug = \"{slug}\"\nname = \"M5\"\n\n\
[secrets]\nnames = [\"email-hmac\"]\n\n\
[[scripts]]\nname = \"handler\"\nfile = \"scripts/handler.rhai\"\n\n\
[[triggers.queue]]\nscript = \"handler\"\nqueue_name = \"jobs\"\n\n\
[[triggers.email]]\nscript = \"handler\"\ninbound_secret_ref = \"email-hmac\"\n"
);
let manifest_path = dir.path().join("picloud.toml");
fs::write(&manifest_path, &manifest).unwrap();
let out = common::pic_as(&env)
.args(["apply", "--file"])
.arg(&manifest_path)
.output()
.expect("apply");
assert!(
out.status.success(),
"apply failed: {}",
String::from_utf8_lossy(&out.stderr)
);
// Both triggers exist.
let s = String::from_utf8(
common::pic_as(&env)
.args(["triggers", "ls", "--app", &slug])
.output()
.unwrap()
.stdout,
)
.unwrap();
assert!(
s.lines().any(|l| l.contains("queue")),
"queue trigger missing:\n{s}"
);
assert!(
s.lines().any(|l| l.contains("email")),
"email trigger missing:\n{s}"
);
// Re-apply is a no-op (both triggers match by identity).
let r = String::from_utf8(
common::pic_as(&env)
.args(["apply", "--file"])
.arg(&manifest_path)
.output()
.unwrap()
.stdout,
)
.unwrap();
assert!(!r.contains("+1"), "re-apply should be a no-op:\n{r}");
}
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
#[test]
fn prune_refuses_to_orphan_email_trigger() {
// `pull` can't represent email triggers, so a manifest that omits the
// script owning one would, under `--prune`, cascade-delete the trigger
// (and its sealed secret) when the script is dropped. Apply must refuse.
let Some(fx) = common::fixture_or_skip() else {
return;
};
let env = common::admin_env(fx);
let slug = common::unique_slug("m5-orphan");
common::pic_as(&env)
.args(["apps", "create", &slug])
.assert()
.success();
let _guard = AppGuard::new(&env.url, &env.token, &slug);
common::pic_as(&env)
.args(["secrets", "set", "--app", &slug, "email-hmac"])
.write_stdin("super-secret-hmac")
.assert()
.success();
let dir = TempDir::new().unwrap();
fs::create_dir_all(dir.path().join("scripts")).unwrap();
fs::write(dir.path().join("scripts/handler.rhai"), "let x = 1; x").unwrap();
let manifest_path = dir.path().join("picloud.toml");
// v1: a script with an email trigger.
let v1 = format!(
"[app]\nslug = \"{slug}\"\nname = \"M5\"\n\n\
[secrets]\nnames = [\"email-hmac\"]\n\n\
[[scripts]]\nname = \"handler\"\nfile = \"scripts/handler.rhai\"\n\n\
[[triggers.email]]\nscript = \"handler\"\ninbound_secret_ref = \"email-hmac\"\n"
);
fs::write(&manifest_path, &v1).unwrap();
common::pic_as(&env)
.args(["apply", "--file"])
.arg(&manifest_path)
.assert()
.success();
// v2: drop the script (and, implicitly, its un-representable email
// trigger). A prune apply must REFUSE rather than cascade-destroy it.
let v2 = format!("[app]\nslug = \"{slug}\"\nname = \"M5\"\n");
fs::write(&manifest_path, &v2).unwrap();
let out = common::pic_as(&env)
.args(["apply", "--file"])
.arg(&manifest_path)
.arg("--prune")
.output()
.expect("apply --prune");
assert!(
!out.status.success(),
"prune must refuse to orphan an email trigger"
);
// The script and its email trigger both survive the refused apply.
let scripts = String::from_utf8(
common::pic_as(&env)
.args(["scripts", "ls", "--app", &slug])
.output()
.unwrap()
.stdout,
)
.unwrap();
assert!(
scripts.contains("handler"),
"script must survive:\n{scripts}"
);
let triggers = String::from_utf8(
common::pic_as(&env)
.args(["triggers", "ls", "--app", &slug])
.output()
.unwrap()
.stdout,
)
.unwrap();
assert!(
triggers.lines().any(|l| l.contains("email")),
"email trigger must survive:\n{triggers}"
);
}
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
#[test]
fn apply_email_unset_secret_fails() {
let Some(fx) = common::fixture_or_skip() else {
return;
};
let env = common::admin_env(fx);
let slug = common::unique_slug("m5-nosecret");
common::pic_as(&env)
.args(["apps", "create", &slug])
.assert()
.success();
let _guard = AppGuard::new(&env.url, &env.token, &slug);
let dir = TempDir::new().unwrap();
fs::create_dir_all(dir.path().join("scripts")).unwrap();
fs::write(dir.path().join("scripts/handler.rhai"), "let x = 1; x").unwrap();
let manifest = format!(
"[app]\nslug = \"{slug}\"\nname = \"M5\"\n\n\
[[scripts]]\nname = \"handler\"\nfile = \"scripts/handler.rhai\"\n\n\
[[triggers.email]]\nscript = \"handler\"\ninbound_secret_ref = \"never-set\"\n"
);
let manifest_path = dir.path().join("picloud.toml");
fs::write(&manifest_path, &manifest).unwrap();
// The referenced secret was never set → apply must fail atomically.
let out = common::pic_as(&env)
.args(["apply", "--file"])
.arg(&manifest_path)
.output()
.expect("apply");
assert!(
!out.status.success(),
"apply must fail when an email secret is unset"
);
// Atomic: neither the script nor the email trigger was created.
let s = String::from_utf8(
common::pic_as(&env)
.args(["scripts", "ls", "--app", &slug])
.output()
.unwrap()
.stdout,
)
.unwrap();
assert!(
!s.contains("handler"),
"failed apply must leave nothing behind:\n{s}"
);
}

View File

@@ -0,0 +1,81 @@
//! `pic plan` journey: a freshly-pulled manifest must diff to all-no-op
//! (pull→plan is idempotent), and editing a script source must surface
//! as an update.
use tempfile::TempDir;
use crate::common;
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
#[test]
fn plan_roundtrips_then_detects_change() {
let Some(fx) = common::fixture_or_skip() else {
return;
};
let env = common::admin_env(fx);
let (script_id, guard) = common::deploy_fixture(&env, "plan", "hello.rhai");
let app = guard.slug().to_string();
common::pic_as(&env)
.args([
"routes", "create", "--script", &script_id, "--path", "/p", "--method", "GET",
])
.assert()
.success();
// Pull the live state, then plan it back — must be a clean no-op.
let dir = TempDir::new().expect("tempdir");
common::pic_as(&env)
.args(["pull", &app, "--dir"])
.arg(dir.path())
.assert()
.success();
let manifest = dir.path().join("picloud.toml");
let out = common::pic_as(&env)
.args(["plan", "--file"])
.arg(&manifest)
.output()
.expect("plan");
assert!(
out.status.success(),
"plan failed: {}",
String::from_utf8_lossy(&out.stderr)
);
let stdout = String::from_utf8(out.stdout).unwrap();
let hello = stdout
.lines()
.find(|l| l.contains("hello"))
.unwrap_or_else(|| panic!("no hello row in plan:\n{stdout}"));
assert!(
hello.contains("noop"),
"expected hello no-op, got:\n{stdout}"
);
assert!(
!stdout.contains("create") && !stdout.contains("delete"),
"fresh pull should diff clean, got:\n{stdout}"
);
// Edit the script source on disk → plan must report an update.
std::fs::write(
dir.path().join("scripts/hello.rhai"),
"let body = #{ ok: false }; body",
)
.expect("rewrite source");
let out = common::pic_as(&env)
.args(["plan", "--file"])
.arg(&manifest)
.output()
.expect("plan after edit");
let stdout = String::from_utf8(out.stdout).unwrap();
let hello = stdout
.lines()
.find(|l| l.contains("hello"))
.unwrap_or_else(|| panic!("no hello row in plan:\n{stdout}"));
assert!(
hello.contains("update"),
"expected hello update after source edit, got:\n{stdout}"
);
drop(guard);
}

View File

@@ -0,0 +1,111 @@
//! `pic apply --prune` journey: a resource dropped from the manifest
//! survives a plain (additive) apply but is deleted with `--prune`.
use std::fs;
use tempfile::TempDir;
use crate::common;
use crate::common::cleanup::AppGuard;
fn scripts_ls(env: &common::TestEnv, slug: &str) -> String {
String::from_utf8(
common::pic_as(env)
.args(["scripts", "ls", "--app", slug])
.output()
.unwrap()
.stdout,
)
.unwrap()
}
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
#[test]
fn prune_deletes_stale_resources() {
let Some(fx) = common::fixture_or_skip() else {
return;
};
let env = common::admin_env(fx);
let slug = common::unique_slug("prune");
common::pic_as(&env)
.args(["apps", "create", &slug])
.assert()
.success();
let _guard = AppGuard::new(&env.url, &env.token, &slug);
let dir = TempDir::new().unwrap();
fs::create_dir_all(dir.path().join("scripts")).unwrap();
fs::write(dir.path().join("scripts/keep.rhai"), "let x = 1; x").unwrap();
fs::write(dir.path().join("scripts/drop.rhai"), "let y = 2; y").unwrap();
let manifest_path = dir.path().join("picloud.toml");
// v1: two scripts + a route on `drop`.
let v1 = format!(
"[app]\nslug = \"{slug}\"\nname = \"Prune Test\"\n\n\
[[scripts]]\nname = \"keep\"\nfile = \"scripts/keep.rhai\"\n\n\
[[scripts]]\nname = \"drop\"\nfile = \"scripts/drop.rhai\"\n\n\
[[routes]]\nscript = \"drop\"\nmethod = \"GET\"\n\
host_kind = \"any\"\npath_kind = \"exact\"\npath = \"/drop\"\n"
);
fs::write(&manifest_path, &v1).unwrap();
common::pic_as(&env)
.args(["apply", "--file"])
.arg(&manifest_path)
.assert()
.success();
// v2: drop `drop` and its route.
let v2 = format!(
"[app]\nslug = \"{slug}\"\nname = \"Prune Test\"\n\n\
[[scripts]]\nname = \"keep\"\nfile = \"scripts/keep.rhai\"\n"
);
fs::write(&manifest_path, &v2).unwrap();
// Plain apply is additive — `drop` survives.
common::pic_as(&env)
.args(["apply", "--file"])
.arg(&manifest_path)
.assert()
.success();
assert!(
scripts_ls(&env, &slug).contains("drop"),
"additive apply must not delete"
);
// Prune apply removes `drop` and its route.
let out = common::pic_as(&env)
.args(["apply", "--file"])
.arg(&manifest_path)
.arg("--prune")
.output()
.expect("apply --prune");
assert!(
out.status.success(),
"prune failed: {}",
String::from_utf8_lossy(&out.stderr)
);
let report = String::from_utf8(out.stdout).unwrap();
assert!(
report.contains("-1"),
"expected deletions in report:\n{report}"
);
let s = scripts_ls(&env, &slug);
assert!(!s.contains("drop"), "prune should delete `drop`:\n{s}");
assert!(s.contains("keep"), "prune must keep `keep`:\n{s}");
// Plan is clean after prune.
let p = String::from_utf8(
common::pic_as(&env)
.args(["plan", "--file"])
.arg(&manifest_path)
.output()
.unwrap()
.stdout,
)
.unwrap();
assert!(
!p.contains("delete"),
"plan should be clean after prune:\n{p}"
);
}

View File

@@ -0,0 +1,91 @@
//! `pic pull` journey: stand up an app with a script, route, cron trigger,
//! and a secret, then export it and assert the manifest + script file.
use tempfile::TempDir;
use crate::common;
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
#[test]
fn pull_exports_manifest_and_sources() {
let Some(fx) = common::fixture_or_skip() else {
return;
};
let env = common::admin_env(fx);
// App + script "hello" (deploy derives the name from the file stem).
let (script_id, guard) = common::deploy_fixture(&env, "pull", "hello.rhai");
let app = guard.slug().to_string();
// Route → script.
common::pic_as(&env)
.args([
"routes", "create", "--script", &script_id, "--path", "/hook", "--method", "POST",
])
.assert()
.success();
// Cron trigger → script.
common::pic_as(&env)
.args([
"triggers",
"create-cron",
"--app",
&app,
"--script",
&script_id,
"--schedule",
"0 0 * * * *",
])
.assert()
.success();
// Secret (name only ends up in the manifest; value stays server-side).
common::pic_as(&env)
.args(["secrets", "set", "--app", &app, "api_key"])
.write_stdin("xyzzy")
.assert()
.success();
// Pull into a scratch dir.
let out_dir = TempDir::new().expect("pull tempdir");
common::pic_as(&env)
.args(["pull", &app, "--dir"])
.arg(out_dir.path())
.assert()
.success();
// Manifest exists and captures every resource.
let manifest = std::fs::read_to_string(out_dir.path().join("picloud.toml"))
.expect("picloud.toml should be written");
assert!(
manifest.contains(&format!("slug = \"{app}\"")),
"manifest missing app slug:\n{manifest}"
);
assert!(
manifest.contains("name = \"hello\"") && manifest.contains("scripts/hello.rhai"),
"manifest missing script entry:\n{manifest}"
);
assert!(
manifest.contains("[[routes]]") && manifest.contains("path = \"/hook\""),
"manifest missing route:\n{manifest}"
);
assert!(
manifest.contains("[[triggers.cron]]") && manifest.contains("schedule = \"0 0 * * * *\""),
"manifest missing cron trigger:\n{manifest}"
);
assert!(
manifest.contains("api_key"),
"manifest missing secret name:\n{manifest}"
);
// Script source was written out faithfully.
let src = std::fs::read_to_string(out_dir.path().join("scripts/hello.rhai"))
.expect("scripts/hello.rhai should be written");
assert!(
src.contains("hello from pic"),
"exported source mismatch:\n{src}"
);
drop(guard);
}

View File

@@ -10,28 +10,28 @@ use axum::middleware::from_fn_with_state;
use axum::{routing::get, Json, Router}; use axum::{routing::get, Json, Router};
use picloud_executor_core::{Engine, Limits}; use picloud_executor_core::{Engine, Limits};
use picloud_manager_core::{ use picloud_manager_core::{
admin_router, admins_router, api_keys_router, app_members_router, apps_api, apps_router, admin_router, admins_router, api_keys_router, app_members_router, apply_router, apps_api,
attach_principal_if_present, auth_router, compile_routes, dead_letters_router, apps_router, attach_principal_if_present, auth_router, compile_routes, dead_letters_router,
email_inbound_router, files_admin_router, migrations, require_authenticated, dev_emails_router, email_inbound_router, files_admin_router, kv_admin_router, migrations,
route_admin_router, secrets_router, topics_router, triggers_router, AbandonedRepo, require_authenticated, route_admin_router, secrets_router, topics_router, triggers_router,
AdminPrincipalResolver, AdminSessionRepository, AdminState, AdminUserRepository, AdminsState, AbandonedRepo, AdminPrincipalResolver, AdminSessionRepository, AdminState, AdminUserRepository,
ApiKeyRepository, ApiKeysState, AppDomainRepository, AppMembersRepository, AppMembersState, AdminsState, ApiKeyRepository, ApiKeysState, AppDomainRepository, AppMembersRepository,
AppRepository, AppsState, AuthState, AuthzRepo, DeadLetterRepo, DeadLettersState, Dispatcher, AppMembersState, AppRepository, ApplyService, AppsState, AuthState, AuthzRepo, DeadLetterRepo,
DocsServiceImpl, EmailInboundState, EmailServiceImpl, FilesAdminState, FilesConfig, DeadLettersState, DevEmailState, Dispatcher, DocsServiceImpl, EmailInboundState,
FilesServiceImpl, FsFilesRepo, HttpConfig, HttpServiceImpl, InboundNonceDedup, KvServiceImpl, EmailServiceImpl, FilesAdminState, FilesConfig, FilesServiceImpl, FsFilesRepo, HttpConfig,
OutboxEventEmitter, OutboxRepo, PostgresAbandonedRepo, PostgresAdminSessionRepository, HttpServiceImpl, InboundNonceDedup, KvAdminState, KvServiceImpl, OutboxEventEmitter,
PostgresAdminUserRepository, PostgresApiKeyRepository, PostgresAppDomainRepository, OutboxRepo, PostgresAbandonedRepo, PostgresAdminSessionRepository, PostgresAdminUserRepository,
PostgresAppMembersRepository, PostgresAppRepository, PostgresAppSecretsRepo, PostgresApiKeyRepository, PostgresAppDomainRepository, PostgresAppMembersRepository,
PostgresAppUserInvitationRepo, PostgresAppUserPasswordResetRepo, PostgresAppUserRepository, PostgresAppRepository, PostgresAppSecretsRepo, PostgresAppUserInvitationRepo,
PostgresAppUserRoleRepo, PostgresAppUserSessionRepository, PostgresAppUserVerificationRepo, PostgresAppUserPasswordResetRepo, PostgresAppUserRepository, PostgresAppUserRoleRepo,
PostgresDeadLetterRepo, PostgresDeadLetterService, PostgresDocsRepo, PostgresAppUserSessionRepository, PostgresAppUserVerificationRepo, PostgresDeadLetterRepo,
PostgresExecutionLogRepository, PostgresExecutionLogSink, PostgresKvRepo, PostgresOutboxRepo, PostgresDeadLetterService, PostgresDocsRepo, PostgresExecutionLogRepository,
PostgresPubsubRepo, PostgresRouteRepository, PostgresScriptRepository, PostgresSecretsRepo, PostgresExecutionLogSink, PostgresKvRepo, PostgresOutboxRepo, PostgresPubsubRepo,
PostgresTopicRepo, PostgresTriggerRepo, PrincipalResolver, PubsubServiceImpl, PostgresRouteRepository, PostgresScriptRepository, PostgresSecretsRepo, PostgresTopicRepo,
RealtimeAuthorityImpl, RepoResolver, RouteAdminState, RouteRepository, SandboxCeiling, PostgresTriggerRepo, PrincipalResolver, PubsubServiceImpl, RealtimeAuthorityImpl, RepoResolver,
ScriptRepository, SecretsConfig, SecretsServiceImpl, SecretsState, SubscriberTokenConfig, RouteAdminState, RouteRepository, SandboxCeiling, ScriptRepository, SecretsConfig,
TopicRepo, TopicsState, TriggerConfig, TriggerRepo, TriggersState, UsersServiceConfig, SecretsServiceImpl, SecretsState, SubscriberTokenConfig, TopicRepo, TopicsState, TriggerConfig,
UsersServiceImpl, TriggerRepo, TriggersState, UsersServiceConfig, UsersServiceImpl,
}; };
use picloud_orchestrator_core::realtime::DEFAULT_GC_INTERVAL_SECS; use picloud_orchestrator_core::realtime::DEFAULT_GC_INTERVAL_SECS;
use picloud_orchestrator_core::routing::{AppDomainTable, RouteTable}; use picloud_orchestrator_core::routing::{AppDomainTable, RouteTable};
@@ -42,8 +42,8 @@ use picloud_orchestrator_core::{
use picloud_shared::{ use picloud_shared::{
DeadLetterService, DocsService, EmailService, ExecutionLogSink, FilesService, HttpService, DeadLetterService, DocsService, EmailService, ExecutionLogSink, FilesService, HttpService,
InboxResolver, KvService, MasterKey, OutboxWriter, PubsubService, RealtimeAuthority, InboxResolver, KvService, MasterKey, OutboxWriter, PubsubService, RealtimeAuthority,
RealtimeBroadcaster, ScriptValidator, SecretsService, ServiceEventEmitter, Services, RealtimeBroadcaster, SecretsService, ServiceEventEmitter, Services, UsersService, API_VERSION,
UsersService, API_VERSION, PRODUCT_VERSION, SDK_VERSION, WIRE_VERSION, PRODUCT_VERSION, SDK_VERSION, WIRE_VERSION,
}; };
use sqlx::postgres::PgPoolOptions; use sqlx::postgres::PgPoolOptions;
use sqlx::PgPool; use sqlx::PgPool;
@@ -146,7 +146,7 @@ pub async fn build_app(
outbox_repo.clone(), outbox_repo.clone(),
)); ));
let kv: Arc<dyn KvService> = Arc::new(KvServiceImpl::with_max_value_bytes( let kv: Arc<dyn KvService> = Arc::new(KvServiceImpl::with_max_value_bytes(
kv_repo, kv_repo.clone(),
authz.clone(), authz.clone(),
events.clone(), events.clone(),
picloud_manager_core::kv_service::kv_max_value_bytes_from_env(), picloud_manager_core::kv_service::kv_max_value_bytes_from_env(),
@@ -239,8 +239,12 @@ pub async fn build_app(
secrets_config, secrets_config,
)); ));
// v1.1.7 outbound email. Builds a lettre SMTP transport from // v1.1.7 outbound email. Builds a lettre SMTP transport from
// PICLOUD_SMTP_* env (disabled mode + warning if unconfigured). // PICLOUD_SMTP_* env (disabled mode + warning if unconfigured). G5: in
let email: Arc<dyn EmailService> = Arc::new(EmailServiceImpl::from_env(authz.clone())); // dev mode with no relay, captures mail in memory instead of erroring;
// `dev_email_sink` is `Some` then, and we mount the dev inspection
// endpoint below.
let (email_impl, dev_email_sink) = EmailServiceImpl::from_env_with_dev_capture(authz.clone());
let email: Arc<dyn EmailService> = Arc::new(email_impl);
// v1.1.8 data-plane user management. Wires Argon2id-hashed user // v1.1.8 data-plane user management. Wires Argon2id-hashed user
// rows + SHA-256-hashed sliding-window sessions to the Rhai // rows + SHA-256-hashed sliding-window sessions to the Rhai
// `users::*` namespace and the admin /apps/{id}/users HTTP surface. // `users::*` namespace and the admin /apps/{id}/users HTTP surface.
@@ -290,8 +294,10 @@ pub async fn build_app(
// routes. // routes.
let route_table = Arc::new(RouteTable::new()); let route_table = Arc::new(RouteTable::new());
let initial = route_repo.list_all().await?; let initial = route_repo.list_all().await?;
let compiled = compile_routes(&initial) // Lenient: a single un-compilable stored route (e.g. one whose path
.map_err(|e| anyhow::anyhow!("failed to compile stored routes: {e}"))?; // became reserved under stricter validation) is skipped-with-warning
// inside compile_routes, never aborting boot. (H1)
let compiled = compile_routes(&initial);
route_table.replace_all(compiled); route_table.replace_all(compiled);
// v1.1.9 function composition. InvokeServiceImpl resolves targets // v1.1.9 function composition. InvokeServiceImpl resolves targets
@@ -380,6 +386,7 @@ pub async fn build_app(
principals, principals,
executor: executor.clone(), executor: executor.clone(),
gate, gate,
log_sink: log_sink.clone(),
inbox: inbox_resolver, inbox: inbox_resolver,
queue: queue_repo.clone(), queue: queue_repo.clone(),
config: trigger_config, config: trigger_config,
@@ -392,7 +399,7 @@ pub async fn build_app(
logs: log_repo, logs: log_repo,
apps: apps_repo.clone(), apps: apps_repo.clone(),
authz: authz.clone(), authz: authz.clone(),
validator: engine as Arc<dyn ScriptValidator>, validator: engine.clone(),
sandbox_ceiling: SandboxCeiling::from_env(), sandbox_ceiling: SandboxCeiling::from_env(),
}; };
let route_admin = RouteAdminState { let route_admin = RouteAdminState {
@@ -407,7 +414,7 @@ pub async fn build_app(
resolver, resolver,
log_sink, log_sink,
app_domains: app_domain_table.clone(), app_domains: app_domain_table.clone(),
routes: route_table, routes: route_table.clone(),
inbox: inbox_registry, inbox: inbox_registry,
outbox: outbox_writer, outbox: outbox_writer,
}; };
@@ -433,7 +440,7 @@ pub async fn build_app(
// v1.1.4: cron scheduler. Polls cron_trigger_details on a tick and // v1.1.4: cron scheduler. Polls cron_trigger_details on a tick and
// enqueues due triggers into the outbox; the dispatcher above // enqueues due triggers into the outbox; the dispatcher above
// delivers them like any other async trigger. // delivers them like any other async trigger.
picloud_manager_core::spawn_cron_scheduler(pool, trigger_config.cron_tick_interval_ms); picloud_manager_core::spawn_cron_scheduler(pool.clone(), trigger_config.cron_tick_interval_ms);
// v1.1.6: GC empty realtime broadcast channels (one-shot subscribers) // v1.1.6: GC empty realtime broadcast channels (one-shot subscribers)
// and sweep orphaned `*.tmp.*` blobs left by crashed file writes. // and sweep orphaned `*.tmp.*` blobs left by crashed file writes.
spawn_realtime_gc(broadcaster_concrete, DEFAULT_GC_INTERVAL_SECS); spawn_realtime_gc(broadcaster_concrete, DEFAULT_GC_INTERVAL_SECS);
@@ -446,6 +453,23 @@ pub async fn build_app(
config: trigger_config, config: trigger_config,
master_key: master_key.clone(), master_key: master_key.clone(),
}; };
// Declarative reconcile engine (pic plan / apply). Trait-object repos
// for the read/diff path; shares the same handles as the CRUD routers.
let apply_service = ApplyService {
pool: pool.clone(),
scripts: script_repo.clone(),
routes: route_repo.clone(),
triggers: trigger_repo.clone(),
secrets: secrets_repo.clone(),
apps: apps_repo.clone(),
domains: domains_repo.clone(),
authz: authz.clone(),
validator: engine.clone(),
sandbox_ceiling: SandboxCeiling::from_env(),
trigger_config,
route_table: route_table.clone(),
master_key: master_key.clone(),
};
// v1.1.9: keep a clone for the queues-api state (built later). // v1.1.9: keep a clone for the queues-api state (built later).
let trigger_repo_for_queues = trigger_repo.clone(); let trigger_repo_for_queues = trigger_repo.clone();
// v1.1.7 public inbound-email receiver. Outside the admin auth layer // v1.1.7 public inbound-email receiver. Outside the admin auth layer
@@ -544,7 +568,7 @@ pub async fn build_app(
// else under /admin gets the require_authenticated layer; capability // else under /admin gets the require_authenticated layer; capability
// checks live in each handler (after the resource is loaded so the // checks live in each handler (after the resource is loaded so the
// capability binds to the resource's actual app_id). // capability binds to the resource's actual app_id).
let guarded_admin = Router::new() let mut guarded_admin = Router::new()
.merge(admin_router(admin)) .merge(admin_router(admin))
.merge(route_admin_router(route_admin)) .merge(route_admin_router(route_admin))
.merge(admins_router(admins_state)) .merge(admins_router(admins_state))
@@ -555,6 +579,7 @@ pub async fn build_app(
)) ))
.merge(api_keys_router(api_keys_state)) .merge(api_keys_router(api_keys_state))
.merge(triggers_router(triggers_state)) .merge(triggers_router(triggers_state))
.merge(apply_router(apply_service))
.merge(picloud_manager_core::queues_api::queues_router( .merge(picloud_manager_core::queues_api::queues_router(
picloud_manager_core::queues_api::QueuesState { picloud_manager_core::queues_api::QueuesState {
queues: queue_repo.clone(), queues: queue_repo.clone(),
@@ -565,10 +590,21 @@ pub async fn build_app(
}, },
)) ))
.merge(files_admin_router(files_admin_state)) .merge(files_admin_router(files_admin_state))
.merge(kv_admin_router(KvAdminState {
kv: kv_repo.clone(),
apps: apps_repo.clone(),
authz: authz.clone(),
}))
.merge(topics_router(topics_state)) .merge(topics_router(topics_state))
.merge(secrets_router(secrets_state)) .merge(secrets_router(secrets_state))
.merge(dead_letters_router(dead_letters_state)) .merge(dead_letters_router(dead_letters_state));
.layer(from_fn_with_state( // G5: dev-only mail inspection — mounted exactly when the email
// service is capturing in memory (dev mode + no relay). Same
// `require_authenticated` layer as the rest of /admin applies below.
if let Some(sink) = dev_email_sink {
guarded_admin = guarded_admin.merge(dev_emails_router(DevEmailState { sink }));
}
let guarded_admin = guarded_admin.layer(from_fn_with_state(
auth_state.clone(), auth_state.clone(),
require_authenticated, require_authenticated,
)); ));

View File

@@ -30,9 +30,77 @@ pub struct ExecutionLog {
pub duration_ms: u64, pub duration_ms: u64,
pub status: ExecutionStatus, pub status: ExecutionStatus,
/// What dispatched this execution: `http` for direct data-plane
/// ingress, or one of the trigger kinds (`kv`, `cron`, `queue`,
/// `invoke`, …) for background runs. Materialized so `pic logs`
/// can surface — and filter by — the origin of every run, not just
/// the HTTP ones. Defaults to `http` for rows written before the
/// column existed (migration 0043).
#[serde(default)]
pub source: ExecutionSource,
pub created_at: DateTime<Utc>, pub created_at: DateTime<Utc>,
} }
/// Origin of an execution. Wire strings mirror
/// `manager-core::OutboxSourceKind` (plus `Queue`, which the queue
/// consumer dispatches outside the outbox) so a trigger's source kind
/// maps straight through to its execution-log row. Keep the variants and
/// the `source` CHECK constraint in migration 0043 in sync.
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq, Default)]
#[serde(rename_all = "snake_case")]
pub enum ExecutionSource {
#[default]
Http,
Kv,
Docs,
DeadLetter,
Cron,
Files,
Pubsub,
Email,
Invoke,
Queue,
}
impl ExecutionSource {
#[must_use]
pub fn as_str(self) -> &'static str {
match self {
Self::Http => "http",
Self::Kv => "kv",
Self::Docs => "docs",
Self::DeadLetter => "dead_letter",
Self::Cron => "cron",
Self::Files => "files",
Self::Pubsub => "pubsub",
Self::Email => "email",
Self::Invoke => "invoke",
Self::Queue => "queue",
}
}
/// Parse a wire string back into a variant. Returns `None` for an
/// unknown value (callers treat that as "no filter" or a 422).
#[must_use]
pub fn from_wire(s: &str) -> Option<Self> {
match s {
"http" => Some(Self::Http),
"kv" => Some(Self::Kv),
"docs" => Some(Self::Docs),
"dead_letter" => Some(Self::DeadLetter),
"cron" => Some(Self::Cron),
"files" => Some(Self::Files),
"pubsub" => Some(Self::Pubsub),
"email" => Some(Self::Email),
"invoke" => Some(Self::Invoke),
"queue" => Some(Self::Queue),
_ => None,
}
}
}
/// Matches the CHECK constraint on `execution_logs.status`. Keep the /// Matches the CHECK constraint on `execution_logs.status`. Keep the
/// serde rename in sync with the migration. /// serde rename in sync with the migration.
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)] #[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)]

View File

@@ -48,7 +48,7 @@ pub use email::{EmailError, EmailService, NoopEmailService, OutboundEmail};
pub use error::Error; pub use error::Error;
pub use events::{EmitError, NoopEventEmitter, ServiceEvent, ServiceEventEmitter}; pub use events::{EmitError, NoopEventEmitter, ServiceEvent, ServiceEventEmitter};
pub use exec_summary::ExecResponseSummary; pub use exec_summary::ExecResponseSummary;
pub use execution_log::{ExecutionLog, ExecutionStatus}; pub use execution_log::{ExecutionLog, ExecutionSource, ExecutionStatus};
pub use files::{ pub use files::{
sanitize_stored_content_type, validate_collection as validate_files_collection, FileMeta, sanitize_stored_content_type, validate_collection as validate_files_collection, FileMeta,
FileUpdate, FilesError, FilesListPage, FilesService, NewFile, NoopFilesService, FileUpdate, FilesError, FilesListPage, FilesService, NewFile, NoopFilesService,

View File

@@ -0,0 +1,849 @@
# Groups, Projects & the Declarative Project Tool
> **Status:** Draft — design discussion captured 2026-06-18, revised 2026-06-20 with resolutions
> grounded in precedent (GitLab, Kustomize, Helm, Terraform, Kubernetes Server-Side Apply, Pulumi)
> and corrected against the codebase after three independent review passes (consistency, gaps,
> feasibility). Not yet scheduled into a phase.
>
> **Scope:** turns the imperative `pic` CLI into a declarative, file-based project tool, and
> introduces a server-side **groups** hierarchy so apps can share and inherit config/scripts
> without duplication.
>
> **Blueprint impact:** this reverses the §11.5 *snapshot-copy, not live-link* stance — top-down
> hierarchical inheritance (group → app) is now in scope, implemented as a *materialized, auto-
> invalidated* view (§5.1). Fold the outcome back into the blueprint before it drifts.
---
## 1. Motivation
Today `pic` is **imperative**: every command (`pic deploy`, `pic routes create`, `pic triggers
create-cron`, …) maps 1:1 to an admin API call. There is no memory of desired state, no project
file, and no way to express "these N apps are the staging/prod/tenant variants of one thing."
Two gaps follow:
1. **No declarative project layer.** Developers re-run commands by hand; nothing reconciles a repo
to an instance.
2. **No server-side notion of a project/group.** If we model environments as separate apps, the
*shared* base (config, scripts) is duplicated across every deployed app. The server sees N
unrelated apps.
This document designs both halves: a **declarative manifest + project tool**, and a **groups
hierarchy** on the server that the project tool projects onto the filesystem.
`manager-core` is the single writer to Postgres. We lean on that for two concrete things — an
**atomic desired-state write** (one DB transaction per apply, §4.2) and a **server-computed diff**
(§4.2) — *not* for continuous reconciliation, which we deliberately decline (drift handling is
detect-and-surface, §4.2). This is a narrower and more honest claim than "we can reconcile live
state": most comparable tools cannot do an all-or-nothing apply because their effects are
un-undoable cloud resources; ours are Postgres rows, so we can — within the boundary described in
§4.2.
---
## 2. Vocabulary
| Term | Meaning |
|---|---|
| **Group** | Server-side hierarchy node. Single-parent tree. Owns shared **code/config** definitions and members/roles. Nestable. |
| **App** | A deployable unit. The data-isolation boundary (`app_id`). Lives under a group. |
| **Environment** | A deployment variant (staging/production/…). **An environment *is* an app.** Also a *scope dimension* on config (§3). |
| **Tenant** | Modeled like an environment — a scope dimension / overlay axis, **not** a second parent (§5.4). |
| **Project** | *Not* a server entity. A CLI view over a repo-managed **subtree** of groups. |
| **Definition** | Entities that can be group-owned: **scripts/modules, vars, secret-refs only**. Routes, triggers, collections, topics are **app-scoped** (§3, §5.1). |
| **Data** | KV/docs/files collections + pub/sub topics. **Always app-owned** (`app_id`). The isolation boundary never moves. |
| **Manifest** | TOML file describing desired state for one node (group base) + per-env overlays. |
| **Effective view** | The materialized, per-app resolved set of definitions, served to the runtime and recomputed on any ancestor change (§5.1). |
| **Attach point** | Where a local working tree's root binds into the server group tree. Its ceiling — you cannot apply above it. |
Relationship: **the base + per-env-overlay "project" is the degenerate one-group subtree** of the
general groups model. Nesting generalizes it to multi-group subtrees. Nothing about the
single-project model is lost.
---
## 3. Configuration resolution
This is the single rule the whole design hangs off; everything else (env vars, secrets, `enabled`,
overrides) resolves through it. **All three precedent systems converge on the same shape, so we
adopt it wholesale:**
> ⚠️ **Net-new, not an extension (verified against the codebase).** PiCloud has **no env-agnostic
> config / `vars` layer today** — this resolution engine and the `vars` table are greenfield. Only
> `secrets` exists, and as a value-bearing table, not a ref. Read §3 as *new* infrastructure, not a
> modification of something present.
> **Resolution = sparse, per-field merge; environment is a *pre-filter*, not a precedence tier;
> proximity wins across levels.**
To resolve a key for an app in environment `E`:
1. **Filter by environment first, per level.** A value scoped `@E` is eligible; a value scoped `*`
(env-agnostic) is the fallback. At a *single* level, `@E` beats `*`. Environment scope decides
*eligibility*, it is **not** a competing precedence rank.
*Evidence:* GitLab CI/CD variables use `environment_scope` exactly this way — a `production`-scoped
row simply isn't visible to a `staging` job ([GitLab CI/CD variables](https://docs.gitlab.com/ci/variables/)).
2. **Then nearest level wins.** Walk up `acme → team-a → blog → app`; the closest level that defines
the (filtered) key wins. GitLab states this verbatim: *"if the same variable name exists in a
group and its subgroups, the job uses the value from the closest subgroup."*
3. **Merge granularity:** maps/vars deep-merge **per key** (set `title`, still inherit `region`);
entities (scripts) replace **by identity** (name); deletion is an explicit tombstone.
*Evidence:* Kustomize strategic-merge patches are sparse and deep-merge maps but **replace lists
unless keyed** ([Kustomize patches](https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/patches/));
Helm deep-merges nested maps and uses `null` to delete a defaulted key
([Helm values](https://helm.sh/docs/chart_template_guide/values_files/)).
This one rule resolves several issues at once: it makes **group config environment-scopable** (a
group sets `db_url@production` and `db_url@staging`; descendants inherit the right one — no per-leaf
duplication), it defines **merge granularity** (maps per-key, entities per-identity), and it makes
**`enabled` just another sparse field** (§4.3).
### 3.1 The env-scope manifest syntax
```toml
# group manifest (e.g. team-a/picloud.toml)
[vars]
region = "eu" # env-agnostic (scope *)
[vars.production] # scope @production
db_url = "postgres://prod/..."
[vars.staging]
db_url = "postgres://staging/..."
[secrets]
names = ["stripe_key"] # name-only; values pushed per env via CLI (§4.6)
```
### 3.2 The one deliberately-novel precedence call
With `db_url@production` at the root **and** a plain `db_url` at the leaf, the **leaf's env-agnostic
value wins** for a production app — proximity beats farther-level env-specificity. The research was
explicit that **no major system lets env-specificity override proximity across levels**; GitLab
structurally avoids the question by treating env as a per-level filter (above). So **proximity-first
is the evidence-backed default**, and we document it as a chosen rule: *inner scope shadows outer,
like lexical scoping.* To avoid this becoming a debugging nightmare at depth, `pic config
--effective --explain` must show **why** a value resolved (which level/scope it came from).
> **Residual risk (carried, not solved):** multi-level + environment scopes is genuinely novel
> territory — GitLab is two-level (group→project). Proximity-first at arbitrary depth is consistent
> and defensible but not battle-tested. The `--explain` tooling is a hard requirement, not a nicety.
---
## 4. Manifest & apply
### 4.1 Manifest & layering
- **TOML, data not code.** PiCloud runs untrusted scripts; the deployment descriptor stays inert,
diffable, server-validatable, and dashboard-renderable. Turing-completeness belongs in the Rhai
script, never the manifest.
- **Base + overlays.** A node's `picloud.toml` holds the shared base; `picloud.<env>.toml` overlays
add per-env vars/secrets/slug. Shared config is written once; overlays only carry deltas (the
Kustomize base+overlay model — overlays are sparse patches, not replacements).
- **Routes are top-level**, referencing scripts by name (one place to see all routing).
- **`pull` is first-class** — see §4.6 for its inheritance/masking semantics.
### 4.2 The apply engine
The IaC research reshaped this section materially; the relevant precedents are cited inline.
**Plan/apply split with a bound plan artifact.** `pic plan` computes a reviewable diff
(`create / update / delete / replace`), the server persists it as a row, and `apply` executes
*exactly that stored plan***detecting and refusing if state moved underneath it** (state
version/serial check, cheap given the single writer). *Evidence:* Terraform's saved plan
*"reliably perform[s] an exact set of pre-approved changes, even if the configuration or state has
changed in the minutes since"* ([Terraform run](https://developer.hashicorp.com/terraform/cloud-docs/run)).
The "state moved" check covers **both** the per-node **content** version **and** the **tree-structure**
version (§6): a reparent or a new app created under the subtree between `plan` and `apply` changes the
true blast radius, so the bound plan must refuse if *either* counter moved. Both are **net-new** — the
existing `scripts.version` column is an unconditional write counter, **not** a compare-and-set guard,
and must not be mistaken for one.
**Atomicity — the corrected position.** No major IaC tool does transactional apply, *because their
effects are un-undoable cloud resources* (you cannot un-create a VM). **That constraint does not
bind us:** a PiCloud manifest apply is **almost entirely Postgres row writes** (script source,
route/trigger defs, config). Therefore:
- **The desired-state write is a single DB transaction** — all-or-nothing across the subtree. This
is achievable *because* our substrate is one transactional store (unlike Terraform's un-undoable
cloud resources). It removes the downstream-breakage hazard of an earlier draft (which proposed
per-node, stop-on-error apply — **now superseded**): there is no intermediate state where a parent
committed and a child did not.
- **Propagation is forward-convergent, not transactional.** The post-commit effective-view refresh +
cache invalidation (§5.1) live *outside* the transaction. So "atomic" means
**atomic-for-desired-state, eventually-consistent-for-effect**, with a bounded window where the DB
says new and a cache says old.
> **Invariant to enforce — and it does NOT hold today (verified).** The single-transaction model
> requires apply to write **only Postgres**, but the *current* admin write paths violate that: route
> and domain writes prime in-process caches **synchronously, interleaved with the write**
> (`route_admin.rs:234`, `apps_api.rs:396`), and files fsync to disk. So moving to "commit the
> transaction, *then* refresh views" is a deliberate **restructuring of manager-core**, not a
> description of the status quo. Additionally, **domains** are a non-DB effect (Caddy/filesystem, out
> of band) — they must be **excluded from the transactional core** and handled by the convergence
> model below. Treat "apply writes only Postgres" as an invariant to *establish and guard*, not one
> we already have.
**Idempotent, convergent recovery (for the non-transactional propagation and any future side
effects).** Operations are idempotent upserts keyed by stable identity; "re-apply converges" is the
recovery story; "rollback" means revert the declaration and re-apply forward. Per-node status
(applied/pending/failed/drifted) is recorded so a partial propagation failure is observable and
re-runnable. *Evidence:* every surveyed tool (Terraform, ArgoCD, Flux, Pulumi) chose idempotent
forward-only convergence over distributed rollback.
**Drift model — upgraded from pure last-write-wins.** An earlier draft chose model "(c)":
last-write-wins, just log. That is *exactly* the pre-SSA `kubectl apply` footgun — KEP-555 lists the
scenario verbatim: *"User does an apply, then `kubectl edit`, then applies again: surprise!"*
Kubernetes Server-Side Apply exists to convert that silent clobber into an explicit conflict
([K8s SSA](https://kubernetes.io/docs/reference/using-api/server-side-apply/)). Concrete risk for us:
CI runs `pic apply` and silently re-enables a route an operator killed in an emergency. Resolution:
- **Keep (c)'s simplicity for ordinary fields** (last-write-wins, change reported).
- **For the security-relevant subset only** — `enabled` and a secret's *reference/existence in
desired state* — track a "changed out-of-band" bit. If that subset was changed outside the manifest
(e.g. an operator disabled a route in the dashboard), `pic plan` surfaces it as a **labeled
conflict** and `apply` **refuses without `--force`**. This is a scoped version of SSA's field
ownership; we deliberately avoid full per-field managers (object bloat, complexity).
- **Secret *values* are explicitly out of scope of the conflict/state-version machinery.** Values
are never in the manifest or the plan (§4.6), so a routine `pic secret set` between `plan` and
`apply` is the *supported* workflow and does **not** trip the state-version refusal — the
state-version check covers manifest-managed *desired state* (definitions, including secret
*references* and `enabled`), not value rotation.
- Out-of-band changes render as a **distinct labeled diff** in `plan`, separate from intended
changes. *Evidence:* Terraform/Pulumi both make read-only drift detection default and label drift
distinctly ([Pulumi drift](https://www.pulumi.com/docs/iac/operations/stack-management/drift/)).
> **Residual risk:** the conflict bit reintroduces some of the complexity (c) was chosen to avoid.
> The cruder fallback, if even that is too much: keep pure (c) but add a non-revertible
> **operational lock** flag an operator sets in an emergency that `apply` won't touch. Either closes
> the silent-revert hole; neither is free.
**Gating high-stakes applies: trigger ≠ authorization.** Any CI trigger may `plan`, but applying to
a confirm-required env is a separate, default-off, explicitly-authorized step. A blanket `--yes`
covers ordinary confirms; **confirm-required envs require an explicit per-env `--approve <env>`**, so
CI must opt in per environment. "Override a gate" is its own audited capability (maps onto
`manager-core::authz::can`). *Evidence:* Terraform Cloud parks runs in *Needs Confirmation* and
separates the *apply runs* permission from *manage policy overrides*
([TFC run states](https://developer.hashicorp.com/terraform/cloud-docs/run/states)).
**Concurrency.** Start with a **coarse per-instance (or per-root-group) apply lock** — one apply at a
time — which is trivially correct for the single-node MVP and makes "last-commit-wins" hold. Refine
*later* to **per-blast-radius advisory locks** (lock the affected apps in `app_id` order so
overlapping radii serialize while disjoint ones proceed; queue triggers already use this advisory-lock
primitive) only if contention appears. Don't build the fine-grained version speculatively.
**Blast radius — defined and bounded.** The blast radius is **the set of descendant apps whose
materialized effective view would actually change** — a *diff*, not "all descendants." Per changed
definition at node N it is `subtree(N)` minus apps that override that key nearer. `pic plan`
enumerates it for small radii; for large ones it **summarizes (count + sample)** and a threshold
triggers extra confirmation (`"this changes 4,213 apps — confirm"`). Because only scripts/vars/secret-
refs inherit (§5.1), blast radius applies to *those* changes; an app-scoped change (a route or
trigger) has blast radius = that single app. Root-level changes are accepted as expensive, rare, and
high-confirmation; the computation is bounded by `subtree(N)` size, and the confirmed radius is
re-validated at apply against the tree-structure version (above) so it can't go stale.
**In-flight executions.** An apply that disables or replaces a script affects **new invocations
immediately** (via the `enabled` re-check + view invalidation; pending trigger outbox rows are dropped
at fire-time, §4.3 — so no separate outbox purge is needed). **In-flight *running* executions run to
completion**, and note (verified) the executor has **no external-cancel path today**: executions are
`spawn_blocking` Rhai calls interruptible only by their operation budget or a pre-set wall-clock
deadline self-checked in `engine.on_progress`. A true must-stop-now **kill-switch** is therefore a
*net-new* capability — buildable by having that same `on_progress` hook also poll a per-execution
cancel flag — **gated by an admin capability (`authz::can`) and audited**, scheduled as a later item,
not phase 1. Killing mid-run also risks partial side effects, so it stays the explicit extreme, never
default apply behavior.
**Apply flow (end to end):**
```
CLI builds bundle (manifests + script sources) for the subtree
→ server computes plan + blast radius (incl. descendant apps in OTHER repos)
→ server persists plan artifact; checks state version
→ dev reviews; confirm/approve per env policy
→ server applies desired state in ONE DB transaction (all-or-nothing)
→ server refreshes effective views + bumps generation + invalidates caches (convergent)
→ server returns change report; CLI logs created / updated / re-enabled / pruned / conflicts
```
### 4.3 The three-state `enabled` lifecycle
`enabled` is a real platform feature (DB + server runtime + UI badge/toggle), not CLI sugar, and — by
§3 — it is **just another sparse, proximity-resolved field**.
| State | Meaning | Pruned? |
|---|---|---|
| declared, `enabled = true` (or omitted) | deployed, active | kept |
| declared, `enabled = false` | deployed but **inert** (route short-circuits, trigger doesn't fire, script not invocable) | **kept** — still desired state |
| absent from merged manifest | stale | **deleted** by prune / `--prune` |
- Default `true`; last-write-wins on merge; a base `enabled = false` is inherited until an overlay
(env *or* a nearer group/app) explicitly sets `enabled = true`. Overriding a *different* field does
**not** implicitly re-enable.
- **Across the group axis (resolved):** a descendant disables an inherited (group-owned) script via a
**sparse override** — an override row that sets only `enabled = false`, inheriting the source. A
descendant can re-enable a parent-disabled entity because nearest-level wins. This is the same
sparse-field mechanism as everything else (Kustomize patches are sparse — you specify only what
changes).
- **Base = the superset across envs.** Per-env you may toggle `enabled` in *either* direction
(disable a base-active entity, or re-enable a base-disabled one — proximity wins, §3), but you
**cannot *remove* a base entity** in one env; true removal requires the entity not be in the base.
An entity unique to one env goes in that overlay; an entity present everywhere but off in staging
goes in base with `enabled = false` in the staging overlay.
- **Disabled = invisible.** External callers hitting a disabled route get **404** (indistinguishable
from absent — no info leak).
- **Schema note:** the `triggers` table **already** has `enabled` (+ `dispatch_mode`, retry columns)
and it is **honored at match/schedule time** (`trigger_repo.rs`, `cron_scheduler.rs` — verified).
New work is `enabled` on **scripts** and **routes** only, plus runtime honoring in the matcher /
invoker.
- **Outbox fire-time gap (verified):** the dispatcher does **not** re-check `enabled` on an
already-enqueued outbox row (`dispatcher.rs:699`), so disabling a trigger stops *new* matches while
a pending item still fires. **Fix:** add an `enabled` re-check (trigger *and* script) at fire time
in `resolve_trigger`, so a pending outbox row for a now-disabled trigger/script is dropped when it
comes up — closing the gap cheaply, with no cache or kill-switch involvement. This is the home for
the §5.1 security-disable guarantee on the trigger path.
- **Provenance caveat (accepted):** a single boolean carries no "manual vs manifest" provenance, so a
disabled entity looks the same however it got there — which is *why* the §4.2 conflict bit on the
`enabled`/secrets subset exists, to stop the next apply silently reverting an operational disable.
### 4.4 Identity & naming
- **Kebab everywhere.** One canonical identifier regex: `^[a-z0-9][a-z0-9-]{0,62}$` for project
names, env names, script names/slugs, trigger names — unified with the existing app-slug rule.
- **Scripts:** unique `name`/`slug` per app = merge/upsert key.
- **Routes:** identity = the triple **`<method> <host> <path>`**, e.g.
`ANY *.beta.example.com /hello/:name`. `dispatch_mode`, `host_param_name`, etc. are *attributes*
overridable without changing identity. The CLI infers `host_kind`/`path_kind` from the pattern
syntax (`*`, `{name}`, `:name`, exact), with an explicit `kind` key as override (mirrors the UI).
Needs a normalization rule (default method `ANY`, default host `*`, case-folding) so manifest ↔
server match exactly.
- **Slugs are instance-global, derived from the path.** Two identifiers coexist:
- **path** = `acme/team-a/blog` — hierarchical, group-scoped, display/organization/RBAC.
- **slug** = flat, instance-global, the deployment key.
The derived default is **`{flattened-path}-{env}`** (e.g. `team-a-blog-staging`) — unique by
construction in the multi-group world, unlike a bare `{leaf}-{env}` which collides whenever two
groups reuse a leaf name. On >63-char overflow, truncate + short hash suffix. Explicit override
allowed; the path never *is* the slug, it only *seeds* it.
### 4.5 Triggers
Triggers are **app-scoped, not group-inherited** (§5.1 explains why). Two senses of "app" are in play
and must not be conflated: a trigger is declared once in the **leaf (logical app)** base — an
authoring convenience — and **materializes into per-`app_id` rows**, one per env-app, where `app_id`
is the server-app isolation boundary (§5.2). It is never group-owned.
Two distinct constraints:
- **`name`** (explicit, kebab) = the merge/identity key + upsert target. Unique per app.
*(Triggers have no name column today — new.)*
- **Semantic uniqueness** = a *post-merge validation*: no two triggers may share their kind-specific
semantic key. Checked after merging, so overriding a base trigger per-env (reuse `name`, change a
field) is fine; two differently-named triggers with identical effect is an error.
| Kind | Semantic key | Note |
|---|---|---|
| kv / docs / files | `(script, collection_glob, ops)` | **canonicalize `ops`** (sort + dedupe) |
| cron | `(script, schedule, timezone)` | exact TEXT match |
| pubsub | `(script, topic_pattern)` | |
| dead-letter | `(script, source_filter, trigger_id_filter, script_id_filter)` | NULL = literal value (not wildcard) for equality |
| email | `(script)` | no filters — one per script |
| **queue** | `(queue_name)`**not** script-scoped | already server-enforced (advisory lock: one consumer per `(app_id, queue_name)`) |
- **Matching vs identity:** `[]`/`NULL`/globs mean **"any/wildcard" at dispatch time** (unchanged
runtime matching) but are compared as **literal structural values for dedup**. We dedup on
**structural identity, never on overlap/subsumption**`ops = []` and `ops = ["insert"]` are *not*
duplicates. Overlapping triggers coexist (multiple triggers firing on one event is already how
PiCloud works).
- **Name backfill** for existing nameless rows: `{kind}-{entity}-{n}`, where `{entity}` is the kind's
identity token (collection / queue / topic; for cron/email/dead-letter fall back to `{kind}-{n}`),
sanitized to the kebab regex, with `-{n}` (discovery order) guaranteeing per-app uniqueness.
> **Ergonomic debt (accepted, watch it):** because triggers don't inherit, 100 tenant apps each
> needing the same 5 triggers = 500 declarations. The fix is group trigger/route **templates** that
> fan out per descendant (a *template/instantiation* mechanism, not inheritance) — deferred, but it
> bites early if tenant cardinality is high. Pressure-test against real tenant counts before
> committing to the narrow-inheritance choice (§5.1).
### 4.6 Secrets & `pull`
- **Name-only in the manifest; value pushed via CLI** (`pic secret set`, reads stdin). Unanimous
across every comparable tool — never commit secret values.
- **Env-scoped** like any var (`[secrets]` names declared once; values set per env).
- **Warn (don't block) if a referenced secret is not set.** This requires an app dev to see that a
group secret **exists / is set** (a boolean) without reading its value — an accepted, explicit
authz boundary. GitLab surfaces inherited masked variable *keys* the same way.
- **Email's `inbound_secret` is a reference**, not inline — same rule; the server already encrypts it
at rest.
- **`pull` exports own-rows only** (this node's overrides), **never effective/inherited state.** A
separate read-only **`pic config --effective`** shows the inherited result with **masked secrets
rendered as `<set>` / `***`, never plaintext.** This makes pull-under-masking safe *by
construction* — you cannot pull a secret you cannot read, and you cannot accidentally duplicate
inherited config into a leaf. *Evidence:* GitLab shows inherited group variables in a project
read-only and separate from project-own variables.
- Flat pull for a new project; "smart" delta-pull (own-vs-effective diff) is server-computed since an
app dev's checkout lacks ancestor manifests.
### 4.7 Apply-time warnings
- Enabled route/trigger pointing at a **disabled script**.
- An **endpoint** script deployed with **no route and no trigger** (unreachable). Modules are exempt.
- Sandbox override exceeding the admin **ceiling**.
- Referenced secret not set.
- An out-of-band change to the `enabled`/secrets subset (surfaced as a conflict, §4.2).
---
## 5. Groups & inheritance
### 5.1 What inherits, and the runtime model
- **GitLab-like, nested, single-parent tree.** Single parent keeps inheritance acyclic and
resolution deterministic (no diamond precedence).
- **Inherit code/config — narrowly. Inherit data — no.** Group-inheritable = **scripts/modules,
vars, secret-refs only.** Routes, triggers, collections, topics, files are **app-scoped.**
- *Why narrow:* routes and triggers are **bindings**, not pure code/config. A group-owned trigger
has no app data to watch (triggers fire on app-owned collections/topics/queues); a group-owned
route has no host (routing is Host→app first). Inheriting them is incoherent. There are two
distinct sharing mechanisms and an earlier draft conflated them: the **leaf base+overlay** shares
routes/triggers across *one app's environments*; **group inheritance** shares *code/config across
many apps*. *Evidence:* Serverless/SAM keep `events:` per-function and share logic via *layers*
bindings local, code shared.
- *Why data stays app-owned:* a group script executes in the *inheriting app's* context, so its
`cx.app_id` still scopes data to that app. Group-level *collections/topics* would break `app_id`
as the isolation boundary — that is the v1.3 cross-app data-sharing problem and stays **out**.
- **Runtime model: a materialized effective view + versioned cache (not per-request live-resolve).**
An earlier draft said "live-resolve," which is underspecified and would fight the existing cache.
The real model:
- manager-core **resolves-at-write into a materialized per-app effective view** (§3 rule applied:
sparse merge, env filter, proximity, CoW, `enabled`).
- The orchestrator/executor serve from that view, **keyed by `app_id` + a generation/version**. The
app_id-keyed *shape* survives (today's route cache is already an `app_id`→routes map), **but the
substance is net-new (verified):** there is no generation counter anywhere today, the route cache
is rebuilt whole-table on every write rather than per-app, and script *bodies* are live-resolved
per request. Read "serve from it" as *extend the keying*, not *reuse the mechanism* — and note
that fanning out today's full-rescan invalidation to thousands of descendants would be a
regression, so per-app incremental invalidation is part of the build.
- On any write to a node, manager-core (single writer, knows the tree) **recomputes descendants'
views + bumps the generation + invalidates caches.**
- The materialized view is a **derived cache, not a second source of truth** — canonical config
still lives once at the owning node, so this does **not** reintroduce duplication. This dissolves
the long-running "snapshot vs. live" tension: no duplication *and* propagation *and* a fast hot
path.
> **Residual risk (relocated, not solved):** cache invalidation is now a **correctness/security
> requirement** — disabling a script for a security reason must stop it running *everywhere* within
> bounded time. This is the same class as the existing PrincipalCache revocation lag. Require
> **synchronous invalidation for the security-relevant subset** (`enabled=false`, secret rotation)
> and accept bounded eventual staleness elsewhere; the hard SLA at fan-out to thousands of
> descendants is genuinely unsolved.
### 5.2 Schema impact
- Inheritable definition kinds get a polymorphic owner (`owner_kind ∈ {group, app}` + `owner_id`):
**scripts** (modify the existing table — and note its `app_id` FK is `ON DELETE RESTRICT`, not
CASCADE, so a pruning apply needs explicit ordering), plus **`vars` and `secret-refs`, which are
net-new tables** (they do not exist today — §3). So this is *one table modified + two invented*,
not "three tables touched." **Routes, triggers, collections, topics, files stay strictly
`app_id`-owned**, so the runtime isolation boundary stays fixed. (The CLAUDE.md `app_id NOT NULL …
CASCADE` rule is itself not universal today — scripts already use RESTRICT.)
- New: a `groups` table (single-parent, `parent_id`), group `membership`/roles, an `owner_project`
column on group nodes (§7), and the materialized effective-view store keyed by `app_id` +
generation (§5.1). Env-scoped values carry an `environment_scope` column (`*` or a specific env).
### 5.3 RBAC
- **Hierarchy-aware capabilities.** `authz::can(principal, cap, on=node)` resolves by walking
ancestors and taking the highest effective role. Instance → group(s) → app.
- **Inherited membership** (GitLab-style): a group admin is implicitly admin of every subgroup/app
beneath it.
- **Masked group secrets:** a group secret is *used by* an app at runtime but *not human-readable* by
the app's developers. Two orthogonal gates: **runtime resolution** (engine injects plaintext) vs
**human-read authz** (admin API returns a value only to a principal with rights at the *owning
group*). An app-scoped admin call never returns group secrets; runtime injection bypasses the human
gate. App devs may see a group secret **exists** (for the unset-warning, §4.6) but not its value.
**An app can run with config its own developers cannot see.**
> **Crypto caveat (verified):** secrets today are AES-GCM sealed with AAD `secret:{app_id}:{name}`,
> and the decrypt path hard-codes `cx.app_id` (migration 0042). A **group-owned** secret is *not
> expressible* under this scheme — there is no group identity in the AAD. It needs a new AAD
> identity (e.g. `secret:group:{group_id}:{name}`) **and** an owner-aware decrypt path that resolves
> whether the inherited secret is group- or app-owned. This is the single hardest correctness detail
> of group secrets and gates phasing step 3.
### 5.4 Tenants & single-parent
Single-parent forbids an app combining two *sibling* groups' configs — which seems to threaten the
multi-tenant use case (shared-platform base + per-tenant overlay). It does not, because **the
shared base is an *ancestor*, not a sibling:** model tenants as leaf apps under a `tenants/`
subgroup that inherits the platform base up the chain (tenant leaf → `tenants` group → platform
group). The "two parents" intuition is satisfied by the *chain*. Better still, **a tenant is a scope
dimension like environment** (§3) — the same env-scope machinery generalizes to a `tenant` scope, so
multi-tenant needs no new hierarchy primitive. *Evidence:* GitLab is single-parent and serves
multi-tenant teams via subgroups; "combine two siblings" is handled by promoting shared config to a
common ancestor.
---
### 5.5 Module & import resolution under inheritance
A group-owned script may `import` modules, and inheritance makes "which module?" ambiguous (an
inherited script importing a module a leaf has shadowed could bind to app-dev code it never saw — a
trust inversion). The rule:
- **Lexical by default.** An inherited script's imports resolve against the module set visible at the
**script's own defining node** (walking up from there), **not** the inheriting app's effective
view. A leaf cannot shadow a module an inherited group script depends on, and a group script
behaves identically across every app that inherits it — preserving both **determinism** and the
**trust boundary** (a security-authored shared script is tamper-proof from below). Ordinary lexical
scoping; "sealed by default", like non-`open` classes in Kotlin / `final` in Java.
- **"Defining node" of a CoW-overridden script = the node that authored *that body*.** An
inherited-unchanged script's defining node is the ancestor that owns it (imports sealed to the
ancestor's modules). A script a leaf *overrides* (CoW, §3) has the **leaf** as its defining node, so
the override's imports resolve from the leaf — which is *not* a trust inversion, because the app
owner wrote that body and is trusted within their own app. Consequence (intended): the same module
name can resolve to **two different bodies** in one app, depending on the importing script's defining
node.
- **Explicit extension points for opt-in polymorphism.** A module is marked an extension point in the
**manifest** (an `[extension_points]` declaration — *not* in Rhai source, keeping code inert and
giving the `plan` checker something to read). Such a module is one descendants are *expected* to
provide or override; **only** these resolve against the inheriting app's effective view. Controlled
template-method customization (a shared `render` whose `theme` module each tenant supplies) without
the blanket trust inversion of dynamic resolution. When a name is declared at more than one level —
or is concrete on one path and an extension point on another — **the nearest declaration's kind
wins** (proximity, §3); its default body (if any) is the inherited fallback.
- **Apply-time checks:** a dangling import (inherited script → missing module) is a `plan` error; an
extension point with no provider in a given app is an error for that app (a hard failure, joining
§4.7).
> **Residual (verified):** executor-core's `PicloudModuleResolver` is app-scoped today and ignores the
> importing script's origin (`module_resolver.rs` passes `_source` unused). Rhai *does* expose that
> origin, so the lexical-vs-dynamic split is expressible — but it requires re-keying the resolver cache
> by owner identity and adding per-import policy (sealed vs. extension point), i.e. a real
> resolver+cache redesign, not a parameter tweak. Lands with phasing step 4.
### 5.6 Tree lifecycle: delete, reparent, rename
Structural mutations of the group tree, which the rest of the design depends on staying acyclic and
non-orphaning:
- **Delete = RESTRICT, never implicit CASCADE.** Deleting a non-empty group is refused — an implicit
cascade would destroy descendant apps and their isolated data. CLI `--recursive` expands a delete
into *ordered, explicit, confirmed* child deletions; the DB FK stays RESTRICT. Corollary: a
referenced ancestor **cannot vanish while it has descendants**, so cross-repo read-only references
(§7) can't be orphaned by deletion — the RESTRICT protects them automatically. **App *data* is
destroyed only on explicit opt-in:** an app delete refuses unless `--purge-data`, which then removes
its KV/docs rows *and* its files blob tree under `PICLOUD_FILES_ROOT/<app_id>/` — a non-DB,
non-undoable effect run outside the transaction and logged. So `--recursive` group delete requires
`--purge-data` to touch any descendant app's data; without it, a non-empty app blocks the delete.
- **Reparent / rename: the slug is frozen at creation.** The path only *seeds* the derived slug
(§4.4); a move or rename updates the **display path** but never rewrites the **instance-global
slug** — the deployment key stays stable, external references don't break. After a move the slug no
longer mirrors the path (cosmetic, accepted).
- **Reparent recomputes descendant effective views** (it changes the resolution chain — the same
fan-out invalidation as a node write, §5.1) and is **doubly capability-gated**: group-admin at
*both* the source and destination parent (you remove from one ancestor's domain and add to
another's). Because it changes the resolution chain, a reparent is **validated like a plan and
refused (unless forced)** if the recompute would orphan a sparse `enabled`-override (now shadowing
nothing) or leave an extension point with no provider (§5.5) — a structural move must not silently
produce a state `apply` would have rejected.
- **Cycle guard, under the apply lock.** Reparent runs an **ancestor-walk check in manager-core**
(walk from the destination up to root; reject if it reaches the node being moved). A Postgres
`CHECK` can't express this; the guard is what guarantees §9's "resolution always terminates."
Single-parent + this guard = acyclic. **All structural mutations (reparent/rename/delete) take the
same coarse apply-lock (§4.2)**, so the ancestor-walk + `parent_id` write run serialized — two
concurrent reparents can't race into a cycle, and a reparent's view-recompute can't collide with an
overlapping apply on the materialized-view store.
## 6. The CLI ↔ server projection
- **Directories = groups** (the hierarchy axis). Single-parent falls out of the filesystem for free.
- **Overlay files = environments/apps** (the deployment-variant axis) — *not* subdirectories,
because envs share scripts/structure and only diverge on vars/secrets/slug; files structurally
prevent per-env script drift.
- **`scripts/` at every level cascades** up the tree; nearer overrides farther by name (CoW).
- **A leaf group = one logical app; its environments = the actual server apps.** Multiple distinct
apps = multiple sibling leaf groups. **Intermediate groups may also bear apps** (a dir may have
both subdirs and overlay files) — allowed, no special-casing.
- **Environment registry lives at the app-bearing node**, but **confirm-policy is inheritable** (set
"production always confirms" once at root; it flows down via the §3 mechanism). Leaves may declare
independent environment sets; a tree-wide `pic apply --env production` simply skips a leaf that has
no `production`.
- **Attach point:** the local root manifest declares where it binds into the server tree
(`parent_group = "acme"` or instance root). Ancestors above it are inherited/referenced but **not
present locally** — which makes the sparse checkout enforce the RBAC masking for free; effective
config needs a server round-trip (`pic config --effective`).
- **Stable IDs in gitignored `.picloud/`** (group IDs, instance URL, token ref) so a directory
rename/move maps to a server **reparent**, not delete+create.
- **Local/server structural divergence is detected, not silently fought.** Alongside the per-node
content version (§4.2), each node carries a **per-subtree structure version** (covering its own
parentage/subtree — **not one global counter**, so a structural edit in one team's subtree never
force-refuses an unrelated repo's plan). `pic plan` compares the local parent-by-ID (from
`.picloud/`) against the server's; on a structural mismatch (someone reparented server-side, or a
dir moved locally) it **refuses**, requiring an explicit `--adopt-server-structure` or
`--force-local-structure` (Terraform's detect-and-refuse on stale state). The content-version check
alone would miss a pure structural move; reparent/rename/delete each bump the affected subtree's
structure version.
- **Mono-repo** = attach at instance root. **Per-team repo** = attach at a subgroup, contain only its
slice. Same model, different attach depth.
---
## 7. Ownership of shared nodes
**Single-owner-per-node, ceilinged by the attach point.**
1. Each group node is owned by exactly one **project-root** (the repo that *manages* it — contains
its manifest as something it applies, not merely references). The server records `owner_project`.
2. **Your attach point is your ceiling** — you cannot apply to anything above your local root.
3. **First apply claims; transfer is explicit and capability-gated.** A second repo applying to an
owned node is rejected (`owned by project X; use --takeover`); takeover needs group-admin
capability. This stops one team silently clobbering org-wide config — whose blast radius (via the
effective-view fan-out) is the whole subtree.
4. **Ownership ⟂ RBAC.** Ownership = *which manifest is authoritative*; RBAC = *whether this
principal may*. The owner still needs group-admin capability to apply.
5. A node with **no project claim is UI/API-owned** — the dashboard is its source of truth and no
manifest fights it. Every node is either manifest-owned (one repo) or UI-owned.
**Corollary:** don't co-own a node — split config downward. Shared config lives *higher* (owned by a
platform/shared repo attaching at root); team-specific bits go into subgroups each team owns.
---
## 8. Diagrams
### 8.1 Server ownership & containment
Only scripts/vars/secret-refs are group-ownable (polymorphic owner); routes/triggers/data are always
app-owned via `app_id`.
```mermaid
graph TD
INST([Instance])
INST --> RG["Group: acme (root)"]
RG --> SGA["Group: team-a"]
RG --> SGB["Group: team-b"]
SGA --> LGA["Group: blog (leaf)"]
SGA --> LGB["Group: shop (leaf)"]
LGA --> APP1["App: blog-staging"]
LGA --> APP2["App: blog-production"]
RG -.->|"owns (shared)"| D0["scripts, vars, secret-refs"]
SGA -.->|owns| D1["team-a scripts, vars"]
APP2 -.->|"owns / overrides"| D2["app scripts, vars, secrets"]
APP1 ==>|app_id| C1[("routes, triggers, KV/Docs/Files, Topics")]
APP2 ==>|app_id| C2[("routes, triggers, KV/Docs/Files, Topics")]
```
### 8.2 Entity identity & cardinalities
```mermaid
erDiagram
GROUP ||--o{ GROUP : "parent-of (single parent)"
GROUP ||--o{ APP : contains
GROUP ||--o{ DEFINITION : "owns (scripts/vars/secret-refs)"
APP ||--o{ DEFINITION : "owns (override)"
APP ||--o{ APPSCOPED : "owns (routes/triggers/data, app_id)"
GROUP ||--o{ MEMBERSHIP : "role (inherited down)"
APP ||--o{ MEMBERSHIP : "role"
```
### 8.3 Config resolution (sparse merge, env filter, proximity-first)
Effective value for one app+env, resolved by §3. Env-scope filters per level; nearest level wins;
maps merge per key.
```mermaid
graph TB
I["Instance defaults"] --> RG["Group acme<br/>secret stripe_key (ref)<br/>script auth.rhai<br/>db_url@production"]
RG --> SG["Group team-a<br/>var region = eu"]
SG --> LG["Group blog<br/>script render.rhai<br/>var title = Blog"]
LG --> EV["Env overlay: production<br/>var title = Blog PROD (override)<br/>secret stripe_key = prod value"]
EV --> EFF[["Materialized effective view (app=blog-production):<br/>auth.rhai (acme), render.rhai (blog)<br/>title = Blog PROD (leaf overlay)<br/>region = eu (team-a)<br/>db_url = prod (acme @production filter)<br/>stripe_key = prod"]]
```
### 8.4 Filesystem ↔ server mapping
```mermaid
graph LR
subgraph FS["Git working tree"]
direction TB
R["acme/<br/>picloud.toml<br/>scripts/"]
R --> TA["team-a/<br/>picloud.toml<br/>scripts/"]
TA --> BL["blog/<br/>picloud.toml (base)<br/>picloud.staging.toml<br/>picloud.production.toml<br/>scripts/render.rhai"]
LINK[".picloud/ (gitignored)<br/>group IDs, instance URL, token ref"]
end
subgraph SRV["PiCloud server"]
direction TB
G0["Group acme"] --> G1["Group team-a"]
G1 --> G2["Group blog"]
G2 --> A1["App blog-staging"]
G2 --> A2["App blog-production"]
end
R -.->|defines| G0
TA -.->|defines| G1
BL -.->|"base defines"| G2
BL -.->|"staging.toml"| A1
BL -.->|"production.toml"| A2
LINK -.->|"stable IDs"| SRV
```
### 8.5 Multi-repo subtree views & single-owner ownership
```mermaid
graph TD
subgraph Server["Server group tree (authoritative)"]
acme["acme (root)"]
acme --> ta["team-a"]
acme --> tb["team-b"]
ta --> blog["blog"]
tb --> shop["shop"]
end
subgraph PR["Repo: platform"]
pr["manages acme"]
end
subgraph AR["Repo: team-a"]
ar["attaches at acme<br/>manages team-a + blog"]
end
pr ==>|owns| acme
ar ==>|owns| ta
ar ==>|owns| blog
ar -.->|"reference, read-only"| acme
```
### 8.6 Apply pipeline (bound plan → DB-atomic write → convergent propagation)
```mermaid
sequenceDiagram
actor Dev
participant CLI as pic CLI
participant Mgr as manager-core (single writer)
participant DB as Postgres
participant View as effective views + caches
Dev->>CLI: pic plan --env production
CLI->>CLI: read manifests + .rhai sources, build subtree bundle
CLI->>Mgr: send bundle (whole subtree)
Mgr->>DB: read state + version of affected nodes
Mgr->>Mgr: diff + blast radius + persist plan artifact
Mgr-->>CLI: PLAN (changes, N descendant apps incl. other repos, conflicts)
CLI-->>Dev: show plan; confirm/approve per env policy
Dev->>CLI: pic apply (executes stored plan)
CLI->>Mgr: apply (plan id)
Mgr->>DB: refuse if content or structure version moved; else ONE transaction (all-or-nothing)
Mgr->>View: recompute effective views + bump generation + invalidate
Mgr-->>CLI: change report (created/updated/re-enabled/pruned/conflicts)
CLI-->>Dev: log changes
```
### 8.7 RBAC: masked group secrets
```mermaid
graph TD
GA["Group admin (team-a)"] -->|"sets + can read"| GS["Group secret: stripe_key<br/>owned by team-a, encrypted at rest"]
AD["App developer (blog)"] -- "cannot read value (sees exists)" --x GS
AD -->|"can edit"| AS["App script source + app vars"]
GS ==>|"runtime injects plaintext"| EX["Executor: running app script"]
AS --> EX
GATE["Two gates:<br/>human-read authz vs runtime resolution"]
GATE -.-> GS
GATE -.-> EX
```
### 8.8 The three-state `enabled` lifecycle
```mermaid
stateDiagram-v2
[*] --> Active: declared (enabled=true/omitted)
Active --> Disabled: set enabled=false (manifest or UI toggle)
Disabled --> Active: set enabled=true (manifest or UI toggle)
Active --> Pruned: removed from manifest + prune/--prune
Disabled --> Pruned: removed from manifest + prune/--prune
Pruned --> [*]
note right of Disabled
Still desired state, NOT pruned.
Route 404s, trigger inert, script not invocable.
Out-of-band toggle on this field is conflict-guarded (4.2).
end note
```
---
## 9. Adoption & backfill
Groups land onto a live instance with existing flat apps, so a migration is a prerequisite, not an
afterthought:
- Create a **root group** (and/or a per-owner personal namespace, GitLab-style) and **reparent every
existing app** under it. Every app must have a parent from day one so resolution always terminates.
- Existing apps have no group-owned definitions, so their effective view = their own rows — the
materialized-view store can be backfilled trivially (identity resolution).
- The trigger `name` backfill (§4.5) runs in the same migration window.
- Existing app slugs are already instance-global, so no slug rewrite is needed; the path is new
metadata layered on top.
---
## 10. Open questions & residual risks
Resolved items now live inline next to their topic. What genuinely remains:
- **Effective-view invalidation SLA (§5.1)** — the security-staleness guarantee at fan-out to many
descendants is unsolved; synchronous-for-security + eventual-elsewhere is the proposed shape, not a
proven one. Highest-risk open item.
- **Conflict bit vs. operational lock (§4.2)** — *decided:* phase 1 ships the `enabled` / secret-
reference **conflict bit**; the operational-lock flag is the documented fallback if the bit proves
too heavy. (Was listed as undecided; resolved here to match §11 phase 1.)
- **Multi-level env-scope precedence (§3.2)** — *decided default:* proximity-first with env as a
per-level filter. The open part is only *validation at depth*, which is why `pic config --effective
--explain` is a **phase-3 hard requirement** (when multi-level resolution first ships), not a
precondition to adopting the rule.
- **Inherited-membership revocation lag (§5.3)** — revoking a group admin must drop implicit admin on
every descendant app, but §5.1's synchronous-invalidation subset covers only `enabled`/secrets, not
**role revocation** — leaving an unbounded window. New residual risk; should get the same
synchronous-for-security bound, lands with phase 3's authz.
- **External execution cancel (§4.2 kill-switch)** — the executor has **no external-cancel path
today** (`spawn_blocking` + self-checked deadline, verified); the kill-switch is a net-new capability
(a cancel flag polled in `on_progress`), deferred past phase 1. Until it exists, the strongest stop
is op-budget/deadline + the dispatcher fire-time `enabled` re-check (§4.3) for the trigger path.
- **Narrow-inheritance vs. trigger/route templates (§4.5, §5.1)** — the per-app binding tax bites
early at high tenant cardinality. Decide whether templates are truly deferrable for your target.
- **`pull --factor`** — auto-extract a shared base by diffing two pulled envs (later nicety).
---
## 11. Suggested phasing
1. **Declarative project tool, single-app (no groups yet).** `init`, `pull`/`config --effective`,
manifest parse/validate, `plan` (bound artifact), `apply` (**atomic desired-state write — requires
the manager-core post-commit-refresh restructuring of §4.2, domains/files excluded from the
transactional core**), `prune`, secrets push, link state, env-scoped config. Adds `enabled` to
scripts/routes + the three-state runtime + the dispatcher fire-time `enabled` re-check (§4.3) +
trigger `name` column/backfill + the `enabled`/secrets conflict bit + the net-new content +
tree-structure version counters + a coarse per-instance apply lock; in-flight executions finish (no
kill).
2. **Groups as pure org/RBAC/UI container.** Nested groups (single-parent, `parent_id`, **delete =
RESTRICT**, reparent/rename with the **ancestor-walk cycle guard** + **slug-freeze** +
**tree-structure version**, §5.6), inherited membership, hierarchy-aware `can`, UI grouping, the §9
backfill. No shared resources yet — cheap, no data-plane schema change.
3. **Group-inherited config** (vars, secret-refs, env-scoped). The net-new `vars`/`secret-refs`
tables + polymorphic owner; the group-secret AAD scheme (§5.3 caveat); masked group secrets; the
effective-view resolver + materialization + invalidation; **`config --effective --explain`** (hard
requirement, since multi-level resolution first ships here).
4. **Group-inherited scripts/modules.** CoW overrides; the **scope-aware module/import resolver +
extension points** (§5.5); cache-invalidation fan-out hardening; versioning/pinning if needed.
5. **Project tool maps onto groups.** Nested manifests, attach point, single-owner, server-computed
tree plan, per-env approval gating.
6. **(Much later) group-level collections/topics** — the v1.3 cross-app data-sharing problem, with a
real shared-scope authz model. Optionally, trigger/route **templates** (§4.5) if cardinality
demands.
---
## 12. Contracts still to draft
- The **apply bundle / plan artifact / change-report** wire contract (what the CLI ships, what the
server persists and returns), including the conflict and blast-radius shapes.
- The **effective-view resolver** (the read primitive) — the §3 rule made executable, plus the
materialization + invalidation protocol (§5.1).
- The **full manifest schema** spelling every block (scripts, routes, the 8 trigger kinds, storage
config, env-scoped vars, secret-refs, domains, `[project.environments]` + confirm policy).

View File

@@ -122,6 +122,18 @@ unauthenticated by default — public HTTP scripts run with `None`.
Services that need an authenticated identity (e.g., `users::*`) check Services that need an authenticated identity (e.g., `users::*`) check
`cx.principal.is_some()` and throw if missing. `cx.principal.is_some()` and throw if missing.
> **A public route ≠ public *data*.** When a route is unauthenticated,
> the script runs with `principal: None`, and the capability gate
> (`authz::script_gate`) returns `Ok` immediately — the script holds
> **full app authority**. It can read and write *all* of the app's
> `kv`, `docs`, `files`, `secrets`, `queue`, and `pubsub`, because the
> platform only gates *authenticated* principals; for anonymous ingress
> **the script itself is the only access boundary**. If a public route
> must not expose every secret or every row in the app, the script must
> enforce that — check a token, scope the collection, gate the
> operation. "This route is public" does not mean "this code may only
> touch public data." See blueprint §11.6 (principal model).
## Sync ↔ async bridge ## Sync ↔ async bridge
Rhai is synchronous; service trait methods (KV writes, HTTP calls) are Rhai is synchronous; service trait methods (KV writes, HTTP calls) are

View File

@@ -37,16 +37,33 @@ These come with the Rhai engine itself. See the
`index_of`, `split`, `trim`, `to_lower`, `to_upper`, `replace`, `chars`, `index_of`, `split`, `trim`, `to_lower`, `to_upper`, `replace`, `chars`,
`pad`, `sub_string`, `crop`, `+` (concatenation). `pad`, `sub_string`, `crop`, `+` (concatenation).
> **Footgun — `replace` mutates in place and returns `()`.** Rhai's > **Footgun — several string methods mutate in place and return `()`.**
> `String.replace(from, to)` edits the receiver and returns unit, *not* > A whole family of Rhai string methods edit the receiver *in place* and
> a new string. `let t = auth.replace("Bearer ", "")` sets `t` to `()`, > return unit, **not** a new string. `let t = auth.replace("Bearer ", "")`
> so a downstream `users::verify(t)` fails with > (or `let t = text.trim()`) sets `t` to `()`, so a downstream
> `Function not found: users::verify (())`. To strip a known prefix, > `text.split(",")` or `users::verify(t)` fails with
> slice instead: > `Function not found: split (())` / `… (())`.
>
> | Method | Returns | Use it as |
> |---|---|---|
> | `replace`, `trim`, `make_upper`, `make_lower`, `crop`, `truncate`, `pad` | `()` — **mutates in place** | a statement: `text.trim();` then read `text` |
> | `to_upper`, `to_lower`, `sub_string`, `split`, `chars` | a **new value** (string / array / range) | an expression: `let u = name.to_upper();` |
>
> So `to_upper`/`to_lower` are the *copying* variants (safe in `let x = …`),
> while `make_upper`/`make_lower` are the in-place ones. To trim and keep
> the result, mutate then read the same binding:
> ```rhai
> let token = auth;
> token.trim(); // mutate in place
> if token.starts_with("Bearer ") { // now use `token`
> token.replace("Bearer ", ""); // again: statement, not `let x = …`
> }
> ```
> Or, to strip a known prefix without mutation, slice:
> ```rhai > ```rhai
> let token = if auth.starts_with("Bearer ") { auth.sub_string(7) } else { auth }; > let token = if auth.starts_with("Bearer ") { auth.sub_string(7) } else { auth };
> ``` > ```
> Or call `replace` purely for its side effect: `let t = auth; t.replace("Bearer ", "");`. > (Verified against the pinned Rhai `=1.24` build.)
**Array:** `push`, `pop`, `shift`, `insert`, `remove`, `len`, `clear`, **Array:** `push`, `pop`, `shift`, `insert`, `remove`, `len`, `clear`,
`truncate`, `extend`, `filter`, `map`, `reduce`, `reduce_rev`, `find`, `truncate`, `extend`, `filter`, `map`, `reduce`, `reduce_rev`, `find`,