feat: declarative project-tool foundation (pull/plan/apply/prune)

Add a server-side, atomic, declarative reconcile loop for a single app —
the foundation of the project-tool design. Developers describe an app's
scripts, routes, triggers, and secret-names in `picloud.toml`, then
`pic pull / plan / apply [--prune]` to converge live state to the manifest.

Server (manager-core):
- apply_service: a pure diff engine (compute_diff) shared by plan and
  apply, plus an ApplyService that composes the existing per-repo writes
  into ONE Postgres transaction. Identity keys mirror the DB UNIQUE
  constraints (script=lower(name); route=(method,host_kind,host,
  path_kind,path); trigger=per-kind semantic tuple; secret=name).
  Apply takes a per-app advisory lock, recomputes the diff in-tx, applies
  scripts -> routes -> triggers, prunes dependents-first, commits, then
  refreshes the route table once post-commit.
- apply_api: POST /apps/{id}/plan (AppRead) and /apps/{id}/apply.
  Apply requires the per-kind write caps the bundle exercises (all three
  when --prune), plus AppSecretsRead when it binds an email trigger.
- tx-accepting repo siblings (insert/update/delete *_tx) so the existing
  create/update/delete delegate to one SQL definition each.
- email triggers reference an inbound secret by NAME; the value is
  resolved, decrypted (AAD-bound), and re-sealed server-side at apply —
  it never travels in the manifest.

CLI (picloud-cli):
- manifest.rs (picloud.toml model), client plan/apply, and the pull/plan/
  apply commands. pull rejects filesystem-unsafe script names up front.

Safety properties enforced and tested:
- idempotent: a freshly-pulled manifest re-applies as all-NoOp.
- atomic: a mid-bundle failure rolls back with nothing written.
- routes delete-before-insert so a freed binding is reusable in one apply.
- queue one-consumer invariant held inside the shared tx.
- email triggers are never pruned, and a script that still owns an
  email/dead-letter trigger can't be pruned (the FK cascade would destroy
  the sealed secret) — refused with a pointer to `pic triggers rm`.
- plan and apply agree on unset email-secret references.

No migration: the existing schema's UNIQUE constraints serve as identity
keys. Groups, env-scoping, and the `enabled` toggle are later milestones.

Tested: manager-core lib (360) + CLI bins (27) + 8 project-tool journeys
(pull/plan/apply/prune/email+queue), all green; clippy -D warnings clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
MechaCat02
2026-06-20 21:52:21 +02:00
parent c600177fd6
commit 3b650a2b14
21 changed files with 4055 additions and 129 deletions

View File

@@ -0,0 +1,153 @@
//! `pic apply` journey: apply a manifest to an empty app (atomic create),
//! re-apply is an idempotent no-op, and a bundle containing any invalid
//! resource applies nothing (all-or-nothing).
use std::fs;
use tempfile::TempDir;
use crate::common;
use crate::common::cleanup::AppGuard;
fn manifest_dir() -> TempDir {
let dir = TempDir::new().expect("tempdir");
fs::create_dir_all(dir.path().join("scripts")).expect("scripts dir");
dir
}
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
#[test]
fn apply_creates_then_noop() {
let Some(fx) = common::fixture_or_skip() else {
return;
};
let env = common::admin_env(fx);
let slug = common::unique_slug("apply");
common::pic_as(&env)
.args(["apps", "create", &slug])
.assert()
.success();
let _guard = AppGuard::new(&env.url, &env.token, &slug);
let dir = manifest_dir();
fs::write(
dir.path().join("scripts/greet.rhai"),
"let body = #{ ok: true }; body",
)
.unwrap();
let manifest = format!(
"[app]\nslug = \"{slug}\"\nname = \"Apply Test\"\n\n\
[[scripts]]\nname = \"greet\"\nfile = \"scripts/greet.rhai\"\n\n\
[[routes]]\nscript = \"greet\"\nmethod = \"POST\"\n\
host_kind = \"any\"\npath_kind = \"exact\"\npath = \"/greet\"\n\n\
[[triggers.cron]]\nscript = \"greet\"\nschedule = \"0 0 * * * *\"\ntimezone = \"UTC\"\n"
);
let manifest_path = dir.path().join("picloud.toml");
fs::write(&manifest_path, &manifest).unwrap();
// First apply: creates script + route + trigger.
let out = common::pic_as(&env)
.args(["apply", "--file"])
.arg(&manifest_path)
.output()
.expect("apply");
assert!(
out.status.success(),
"apply failed: {}",
String::from_utf8_lossy(&out.stderr)
);
let stdout = String::from_utf8(out.stdout).unwrap();
assert!(
stdout.contains("+1"),
"expected creations in report:\n{stdout}"
);
// The resources now exist.
let s = String::from_utf8(
common::pic_as(&env)
.args(["scripts", "ls", "--app", &slug])
.output()
.unwrap()
.stdout,
)
.unwrap();
assert!(s.contains("greet"), "script not created:\n{s}");
// Plan is now clean (apply reached desired state).
let p = String::from_utf8(
common::pic_as(&env)
.args(["plan", "--file"])
.arg(&manifest_path)
.output()
.unwrap()
.stdout,
)
.unwrap();
assert!(
!p.contains("create") && !p.contains("update"),
"expected clean plan after apply:\n{p}"
);
// Re-apply: idempotent — nothing created/updated.
let r = String::from_utf8(
common::pic_as(&env)
.args(["apply", "--file"])
.arg(&manifest_path)
.output()
.unwrap()
.stdout,
)
.unwrap();
assert!(!r.contains("+1"), "re-apply should be a no-op:\n{r}");
}
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
#[test]
fn apply_rejects_bad_bundle_atomically() {
let Some(fx) = common::fixture_or_skip() else {
return;
};
let env = common::admin_env(fx);
let slug = common::unique_slug("apply-atomic");
common::pic_as(&env)
.args(["apps", "create", &slug])
.assert()
.success();
let _guard = AppGuard::new(&env.url, &env.token, &slug);
let dir = manifest_dir();
fs::write(dir.path().join("scripts/good.rhai"), "let x = 1; x").unwrap();
// Invalid Rhai — fails validation, so the whole apply must abort.
fs::write(dir.path().join("scripts/bad.rhai"), "let x = ;").unwrap();
let manifest = format!(
"[app]\nslug = \"{slug}\"\nname = \"Atomic Test\"\n\n\
[[scripts]]\nname = \"good\"\nfile = \"scripts/good.rhai\"\n\n\
[[scripts]]\nname = \"bad\"\nfile = \"scripts/bad.rhai\"\n"
);
let manifest_path = dir.path().join("picloud.toml");
fs::write(&manifest_path, &manifest).unwrap();
let out = common::pic_as(&env)
.args(["apply", "--file"])
.arg(&manifest_path)
.output()
.expect("apply");
assert!(
!out.status.success(),
"apply with an invalid script should fail"
);
// Atomic: the valid script must NOT have been created.
let s = String::from_utf8(
common::pic_as(&env)
.args(["scripts", "ls", "--app", &slug])
.output()
.unwrap()
.stdout,
)
.unwrap();
assert!(
!s.contains("good"),
"a failed apply must leave nothing behind:\n{s}"
);
}

View File

@@ -15,12 +15,17 @@ mod common;
mod admins;
mod api_keys;
mod apply;
mod apps;
mod auth;
mod dead_letters;
mod email_queue;
mod invoke;
mod logs;
mod output;
mod plan;
mod prune;
mod pull;
mod roles;
mod routes;
mod scripts;

View File

@@ -0,0 +1,222 @@
//! M5: `pic apply` creates email + queue triggers. The email trigger's
//! inbound secret is referenced by name (pushed via `pic secret set`) and
//! resolved + re-sealed server-side — never written into the manifest.
use std::fs;
use tempfile::TempDir;
use crate::common;
use crate::common::cleanup::AppGuard;
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
#[test]
fn apply_email_and_queue_triggers() {
let Some(fx) = common::fixture_or_skip() else {
return;
};
let env = common::admin_env(fx);
let slug = common::unique_slug("m5");
common::pic_as(&env)
.args(["apps", "create", &slug])
.assert()
.success();
let _guard = AppGuard::new(&env.url, &env.token, &slug);
// The email trigger references this secret by name; push its value
// out-of-band first.
common::pic_as(&env)
.args(["secrets", "set", "--app", &slug, "email-hmac"])
.write_stdin("super-secret-hmac")
.assert()
.success();
let dir = TempDir::new().unwrap();
fs::create_dir_all(dir.path().join("scripts")).unwrap();
fs::write(dir.path().join("scripts/handler.rhai"), "let x = 1; x").unwrap();
let manifest = format!(
"[app]\nslug = \"{slug}\"\nname = \"M5\"\n\n\
[secrets]\nnames = [\"email-hmac\"]\n\n\
[[scripts]]\nname = \"handler\"\nfile = \"scripts/handler.rhai\"\n\n\
[[triggers.queue]]\nscript = \"handler\"\nqueue_name = \"jobs\"\n\n\
[[triggers.email]]\nscript = \"handler\"\ninbound_secret_ref = \"email-hmac\"\n"
);
let manifest_path = dir.path().join("picloud.toml");
fs::write(&manifest_path, &manifest).unwrap();
let out = common::pic_as(&env)
.args(["apply", "--file"])
.arg(&manifest_path)
.output()
.expect("apply");
assert!(
out.status.success(),
"apply failed: {}",
String::from_utf8_lossy(&out.stderr)
);
// Both triggers exist.
let s = String::from_utf8(
common::pic_as(&env)
.args(["triggers", "ls", "--app", &slug])
.output()
.unwrap()
.stdout,
)
.unwrap();
assert!(
s.lines().any(|l| l.contains("queue")),
"queue trigger missing:\n{s}"
);
assert!(
s.lines().any(|l| l.contains("email")),
"email trigger missing:\n{s}"
);
// Re-apply is a no-op (both triggers match by identity).
let r = String::from_utf8(
common::pic_as(&env)
.args(["apply", "--file"])
.arg(&manifest_path)
.output()
.unwrap()
.stdout,
)
.unwrap();
assert!(!r.contains("+1"), "re-apply should be a no-op:\n{r}");
}
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
#[test]
fn prune_refuses_to_orphan_email_trigger() {
// `pull` can't represent email triggers, so a manifest that omits the
// script owning one would, under `--prune`, cascade-delete the trigger
// (and its sealed secret) when the script is dropped. Apply must refuse.
let Some(fx) = common::fixture_or_skip() else {
return;
};
let env = common::admin_env(fx);
let slug = common::unique_slug("m5-orphan");
common::pic_as(&env)
.args(["apps", "create", &slug])
.assert()
.success();
let _guard = AppGuard::new(&env.url, &env.token, &slug);
common::pic_as(&env)
.args(["secrets", "set", "--app", &slug, "email-hmac"])
.write_stdin("super-secret-hmac")
.assert()
.success();
let dir = TempDir::new().unwrap();
fs::create_dir_all(dir.path().join("scripts")).unwrap();
fs::write(dir.path().join("scripts/handler.rhai"), "let x = 1; x").unwrap();
let manifest_path = dir.path().join("picloud.toml");
// v1: a script with an email trigger.
let v1 = format!(
"[app]\nslug = \"{slug}\"\nname = \"M5\"\n\n\
[secrets]\nnames = [\"email-hmac\"]\n\n\
[[scripts]]\nname = \"handler\"\nfile = \"scripts/handler.rhai\"\n\n\
[[triggers.email]]\nscript = \"handler\"\ninbound_secret_ref = \"email-hmac\"\n"
);
fs::write(&manifest_path, &v1).unwrap();
common::pic_as(&env)
.args(["apply", "--file"])
.arg(&manifest_path)
.assert()
.success();
// v2: drop the script (and, implicitly, its un-representable email
// trigger). A prune apply must REFUSE rather than cascade-destroy it.
let v2 = format!("[app]\nslug = \"{slug}\"\nname = \"M5\"\n");
fs::write(&manifest_path, &v2).unwrap();
let out = common::pic_as(&env)
.args(["apply", "--file"])
.arg(&manifest_path)
.arg("--prune")
.output()
.expect("apply --prune");
assert!(
!out.status.success(),
"prune must refuse to orphan an email trigger"
);
// The script and its email trigger both survive the refused apply.
let scripts = String::from_utf8(
common::pic_as(&env)
.args(["scripts", "ls", "--app", &slug])
.output()
.unwrap()
.stdout,
)
.unwrap();
assert!(
scripts.contains("handler"),
"script must survive:\n{scripts}"
);
let triggers = String::from_utf8(
common::pic_as(&env)
.args(["triggers", "ls", "--app", &slug])
.output()
.unwrap()
.stdout,
)
.unwrap();
assert!(
triggers.lines().any(|l| l.contains("email")),
"email trigger must survive:\n{triggers}"
);
}
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
#[test]
fn apply_email_unset_secret_fails() {
let Some(fx) = common::fixture_or_skip() else {
return;
};
let env = common::admin_env(fx);
let slug = common::unique_slug("m5-nosecret");
common::pic_as(&env)
.args(["apps", "create", &slug])
.assert()
.success();
let _guard = AppGuard::new(&env.url, &env.token, &slug);
let dir = TempDir::new().unwrap();
fs::create_dir_all(dir.path().join("scripts")).unwrap();
fs::write(dir.path().join("scripts/handler.rhai"), "let x = 1; x").unwrap();
let manifest = format!(
"[app]\nslug = \"{slug}\"\nname = \"M5\"\n\n\
[[scripts]]\nname = \"handler\"\nfile = \"scripts/handler.rhai\"\n\n\
[[triggers.email]]\nscript = \"handler\"\ninbound_secret_ref = \"never-set\"\n"
);
let manifest_path = dir.path().join("picloud.toml");
fs::write(&manifest_path, &manifest).unwrap();
// The referenced secret was never set → apply must fail atomically.
let out = common::pic_as(&env)
.args(["apply", "--file"])
.arg(&manifest_path)
.output()
.expect("apply");
assert!(
!out.status.success(),
"apply must fail when an email secret is unset"
);
// Atomic: neither the script nor the email trigger was created.
let s = String::from_utf8(
common::pic_as(&env)
.args(["scripts", "ls", "--app", &slug])
.output()
.unwrap()
.stdout,
)
.unwrap();
assert!(
!s.contains("handler"),
"failed apply must leave nothing behind:\n{s}"
);
}

View File

@@ -0,0 +1,81 @@
//! `pic plan` journey: a freshly-pulled manifest must diff to all-no-op
//! (pull→plan is idempotent), and editing a script source must surface
//! as an update.
use tempfile::TempDir;
use crate::common;
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
#[test]
fn plan_roundtrips_then_detects_change() {
let Some(fx) = common::fixture_or_skip() else {
return;
};
let env = common::admin_env(fx);
let (script_id, guard) = common::deploy_fixture(&env, "plan", "hello.rhai");
let app = guard.slug().to_string();
common::pic_as(&env)
.args([
"routes", "create", "--script", &script_id, "--path", "/p", "--method", "GET",
])
.assert()
.success();
// Pull the live state, then plan it back — must be a clean no-op.
let dir = TempDir::new().expect("tempdir");
common::pic_as(&env)
.args(["pull", &app, "--dir"])
.arg(dir.path())
.assert()
.success();
let manifest = dir.path().join("picloud.toml");
let out = common::pic_as(&env)
.args(["plan", "--file"])
.arg(&manifest)
.output()
.expect("plan");
assert!(
out.status.success(),
"plan failed: {}",
String::from_utf8_lossy(&out.stderr)
);
let stdout = String::from_utf8(out.stdout).unwrap();
let hello = stdout
.lines()
.find(|l| l.contains("hello"))
.unwrap_or_else(|| panic!("no hello row in plan:\n{stdout}"));
assert!(
hello.contains("noop"),
"expected hello no-op, got:\n{stdout}"
);
assert!(
!stdout.contains("create") && !stdout.contains("delete"),
"fresh pull should diff clean, got:\n{stdout}"
);
// Edit the script source on disk → plan must report an update.
std::fs::write(
dir.path().join("scripts/hello.rhai"),
"let body = #{ ok: false }; body",
)
.expect("rewrite source");
let out = common::pic_as(&env)
.args(["plan", "--file"])
.arg(&manifest)
.output()
.expect("plan after edit");
let stdout = String::from_utf8(out.stdout).unwrap();
let hello = stdout
.lines()
.find(|l| l.contains("hello"))
.unwrap_or_else(|| panic!("no hello row in plan:\n{stdout}"));
assert!(
hello.contains("update"),
"expected hello update after source edit, got:\n{stdout}"
);
drop(guard);
}

View File

@@ -0,0 +1,111 @@
//! `pic apply --prune` journey: a resource dropped from the manifest
//! survives a plain (additive) apply but is deleted with `--prune`.
use std::fs;
use tempfile::TempDir;
use crate::common;
use crate::common::cleanup::AppGuard;
fn scripts_ls(env: &common::TestEnv, slug: &str) -> String {
String::from_utf8(
common::pic_as(env)
.args(["scripts", "ls", "--app", slug])
.output()
.unwrap()
.stdout,
)
.unwrap()
}
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
#[test]
fn prune_deletes_stale_resources() {
let Some(fx) = common::fixture_or_skip() else {
return;
};
let env = common::admin_env(fx);
let slug = common::unique_slug("prune");
common::pic_as(&env)
.args(["apps", "create", &slug])
.assert()
.success();
let _guard = AppGuard::new(&env.url, &env.token, &slug);
let dir = TempDir::new().unwrap();
fs::create_dir_all(dir.path().join("scripts")).unwrap();
fs::write(dir.path().join("scripts/keep.rhai"), "let x = 1; x").unwrap();
fs::write(dir.path().join("scripts/drop.rhai"), "let y = 2; y").unwrap();
let manifest_path = dir.path().join("picloud.toml");
// v1: two scripts + a route on `drop`.
let v1 = format!(
"[app]\nslug = \"{slug}\"\nname = \"Prune Test\"\n\n\
[[scripts]]\nname = \"keep\"\nfile = \"scripts/keep.rhai\"\n\n\
[[scripts]]\nname = \"drop\"\nfile = \"scripts/drop.rhai\"\n\n\
[[routes]]\nscript = \"drop\"\nmethod = \"GET\"\n\
host_kind = \"any\"\npath_kind = \"exact\"\npath = \"/drop\"\n"
);
fs::write(&manifest_path, &v1).unwrap();
common::pic_as(&env)
.args(["apply", "--file"])
.arg(&manifest_path)
.assert()
.success();
// v2: drop `drop` and its route.
let v2 = format!(
"[app]\nslug = \"{slug}\"\nname = \"Prune Test\"\n\n\
[[scripts]]\nname = \"keep\"\nfile = \"scripts/keep.rhai\"\n"
);
fs::write(&manifest_path, &v2).unwrap();
// Plain apply is additive — `drop` survives.
common::pic_as(&env)
.args(["apply", "--file"])
.arg(&manifest_path)
.assert()
.success();
assert!(
scripts_ls(&env, &slug).contains("drop"),
"additive apply must not delete"
);
// Prune apply removes `drop` and its route.
let out = common::pic_as(&env)
.args(["apply", "--file"])
.arg(&manifest_path)
.arg("--prune")
.output()
.expect("apply --prune");
assert!(
out.status.success(),
"prune failed: {}",
String::from_utf8_lossy(&out.stderr)
);
let report = String::from_utf8(out.stdout).unwrap();
assert!(
report.contains("-1"),
"expected deletions in report:\n{report}"
);
let s = scripts_ls(&env, &slug);
assert!(!s.contains("drop"), "prune should delete `drop`:\n{s}");
assert!(s.contains("keep"), "prune must keep `keep`:\n{s}");
// Plan is clean after prune.
let p = String::from_utf8(
common::pic_as(&env)
.args(["plan", "--file"])
.arg(&manifest_path)
.output()
.unwrap()
.stdout,
)
.unwrap();
assert!(
!p.contains("delete"),
"plan should be clean after prune:\n{p}"
);
}

View File

@@ -0,0 +1,91 @@
//! `pic pull` journey: stand up an app with a script, route, cron trigger,
//! and a secret, then export it and assert the manifest + script file.
use tempfile::TempDir;
use crate::common;
#[ignore = "needs DATABASE_URL pointing at a running Postgres"]
#[test]
fn pull_exports_manifest_and_sources() {
let Some(fx) = common::fixture_or_skip() else {
return;
};
let env = common::admin_env(fx);
// App + script "hello" (deploy derives the name from the file stem).
let (script_id, guard) = common::deploy_fixture(&env, "pull", "hello.rhai");
let app = guard.slug().to_string();
// Route → script.
common::pic_as(&env)
.args([
"routes", "create", "--script", &script_id, "--path", "/hook", "--method", "POST",
])
.assert()
.success();
// Cron trigger → script.
common::pic_as(&env)
.args([
"triggers",
"create-cron",
"--app",
&app,
"--script",
&script_id,
"--schedule",
"0 0 * * * *",
])
.assert()
.success();
// Secret (name only ends up in the manifest; value stays server-side).
common::pic_as(&env)
.args(["secrets", "set", "--app", &app, "api_key"])
.write_stdin("xyzzy")
.assert()
.success();
// Pull into a scratch dir.
let out_dir = TempDir::new().expect("pull tempdir");
common::pic_as(&env)
.args(["pull", &app, "--dir"])
.arg(out_dir.path())
.assert()
.success();
// Manifest exists and captures every resource.
let manifest = std::fs::read_to_string(out_dir.path().join("picloud.toml"))
.expect("picloud.toml should be written");
assert!(
manifest.contains(&format!("slug = \"{app}\"")),
"manifest missing app slug:\n{manifest}"
);
assert!(
manifest.contains("name = \"hello\"") && manifest.contains("scripts/hello.rhai"),
"manifest missing script entry:\n{manifest}"
);
assert!(
manifest.contains("[[routes]]") && manifest.contains("path = \"/hook\""),
"manifest missing route:\n{manifest}"
);
assert!(
manifest.contains("[[triggers.cron]]") && manifest.contains("schedule = \"0 0 * * * *\""),
"manifest missing cron trigger:\n{manifest}"
);
assert!(
manifest.contains("api_key"),
"manifest missing secret name:\n{manifest}"
);
// Script source was written out faithfully.
let src = std::fs::read_to_string(out_dir.path().join("scripts/hello.rhai"))
.expect("scripts/hello.rhai should be written");
assert!(
src.contains("hello from pic"),
"exported source mismatch:\n{src}"
);
drop(guard);
}