fabi/PiCloud
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: declarative project-tool foundation (pull/plan/apply/prune)

Browse Source 
Add a server-side, atomic, declarative reconcile loop for a single app —
the foundation of the project-tool design. Developers describe an app's
scripts, routes, triggers, and secret-names in `picloud.toml`, then
`pic pull / plan / apply [--prune]` to converge live state to the manifest.

Server (manager-core):
- apply_service: a pure diff engine (compute_diff) shared by plan and
  apply, plus an ApplyService that composes the existing per-repo writes
  into ONE Postgres transaction. Identity keys mirror the DB UNIQUE
  constraints (script=lower(name); route=(method,host_kind,host,
  path_kind,path); trigger=per-kind semantic tuple; secret=name).
  Apply takes a per-app advisory lock, recomputes the diff in-tx, applies
  scripts -> routes -> triggers, prunes dependents-first, commits, then
  refreshes the route table once post-commit.
- apply_api: POST /apps/{id}/plan (AppRead) and /apps/{id}/apply.
  Apply requires the per-kind write caps the bundle exercises (all three
  when --prune), plus AppSecretsRead when it binds an email trigger.
- tx-accepting repo siblings (insert/update/delete *_tx) so the existing
  create/update/delete delegate to one SQL definition each.
- email triggers reference an inbound secret by NAME; the value is
  resolved, decrypted (AAD-bound), and re-sealed server-side at apply —
  it never travels in the manifest.

CLI (picloud-cli):
- manifest.rs (picloud.toml model), client plan/apply, and the pull/plan/
  apply commands. pull rejects filesystem-unsafe script names up front.

Safety properties enforced and tested:
- idempotent: a freshly-pulled manifest re-applies as all-NoOp.
- atomic: a mid-bundle failure rolls back with nothing written.
- routes delete-before-insert so a freed binding is reusable in one apply.
- queue one-consumer invariant held inside the shared tx.
- email triggers are never pruned, and a script that still owns an
  email/dead-letter trigger can't be pruned (the FK cascade would destroy
  the sealed secret) — refused with a pointer to `pic triggers rm`.
- plan and apply agree on unset email-secret references.

No migration: the existing schema's UNIQUE constraints serve as identity
keys. Groups, env-scoping, and the `enabled` toggle are later milestones.

Tested: manager-core lib (360) + CLI bins (27) + 8 project-tool journeys
(pull/plan/apply/prune/email+queue), all green; clippy -D warnings clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
MechaCat02
2026-06-20 21:52:21 +02:00
parent c600177fd6
commit 3b650a2b14
21 changed files with 4055 additions and 129 deletions
Download Patch File Download Diff File Expand all files Collapse all files

200
crates/manager-core/src/repo.rs
View File

@@ -273,42 +273,8 @@ impl ScriptRepository for PostgresScriptRepository {
}
async fn create(&self, input: NewScript) -> Result<Script, ScriptRepositoryError> {
let sandbox_json = serde_json::to_value(input.sandbox.unwrap_or_default())
.unwrap_or_else(|_| serde_json::json!({}));
let mut tx = self.pool.begin().await?;
let res = sqlx::query_as::<_, ScriptRow>(&format!(
"INSERT INTO scripts ( \
app_id, name, description, source, kind, \
timeout_seconds, memory_limit_mb, sandbox \
) VALUES ($1, $2, $3, $4, $5, COALESCE($6, 30), COALESCE($7, 256), $8) \
RETURNING {SCRIPT_SELECT_COLS}"
))
.bind(input.app_id.into_inner())
.bind(&input.name)
.bind(input.description.as_deref())
.bind(&input.source)
.bind(input.kind.as_str())
.bind(input.timeout_seconds)
.bind(input.memory_limit_mb)
.bind(sandbox_json)
.fetch_one(&mut *tx)
.await;
let script: Script = match res {
Ok(row) => row.into(),
Err(sqlx::Error::Database(e)) if e.is_unique_violation() => {
return Err(ScriptRepositoryError::Conflict(format!(
"a script named {:?} already exists in this app",
input.name
)));
}
Err(e) => return Err(e.into()),
};
// Dep-graph: write any literal-path imports declared in the
// source. Unresolved names (the referenced module doesn't
// exist yet) are silently skipped — best-effort.
replace_imports_tx(&mut tx, script.id, script.app_id, &input.imports).await?;
let script = insert_script_tx(&mut tx, &input).await?;
tx.commit().await?;
Ok(script)
}
@@ -318,62 +284,8 @@ impl ScriptRepository for PostgresScriptRepository {
id: ScriptId,
patch: ScriptPatch,
) -> Result<Script, ScriptRepositoryError> {
// COALESCE-based partial update: `NULL` parameters leave columns
// untouched. Description is double-Optioned so callers can
// explicitly set it to NULL (Some(None)) vs leave it alone (None).
// Sandbox is replaced wholesale when present; per-field merging
// happens in the API layer (clearer semantics for a "PUT a new
// sandbox config" call). app_id is immutable — moving a script
// to another app is a copy-and-delete, not an in-place edit.
let sandbox_json = patch
.sandbox
.as_ref()
.map(|s| serde_json::to_value(s).unwrap_or_else(|_| serde_json::json!({})));
let mut tx = self.pool.begin().await?;
let res = sqlx::query_as::<_, ScriptRow>(&format!(
"UPDATE scripts SET \
name = COALESCE($2, name), \
description = CASE WHEN $3::bool THEN $4 ELSE description END, \
source = COALESCE($5, source), \
timeout_seconds = COALESCE($6, timeout_seconds), \
memory_limit_mb = COALESCE($7, memory_limit_mb), \
sandbox = COALESCE($8, sandbox), \
kind = COALESCE($9, kind), \
version = version + 1, \
updated_at = NOW() \
WHERE id = $1 \
RETURNING {SCRIPT_SELECT_COLS}"
))
.bind(id.into_inner())
.bind(patch.name.as_deref())
.bind(patch.description.is_some())
.bind(patch.description.as_ref().and_then(|d| d.as_deref()))
.bind(patch.source.as_deref())
.bind(patch.timeout_seconds)
.bind(patch.memory_limit_mb)
.bind(sandbox_json)
.bind(patch.kind.map(ScriptKind::as_str))
.fetch_optional(&mut *tx)
.await;
let script: Script = match res {
Ok(Some(row)) => row.into(),
Ok(None) => return Err(ScriptRepositoryError::NotFound(id)),
Err(sqlx::Error::Database(e)) if e.is_unique_violation() => {
return Err(ScriptRepositoryError::Conflict(
"a script with that name already exists in this app".into(),
));
}
Err(e) => return Err(e.into()),
};
// Replace imports only when the caller has a fresh list (i.e.
// the source actually changed and the validator re-extracted
// imports). A name-only or description-only edit leaves the
// dep graph alone.
if let Some(imports) = patch.imports.as_deref() {
replace_imports_tx(&mut tx, script.id, script.app_id, imports).await?;
}
let script = update_script_tx(&mut tx, id, &patch).await?;
tx.commit().await?;
Ok(script)
}
@@ -469,6 +381,114 @@ async fn replace_imports_tx(
Ok(())
}
/// Insert a script within an existing transaction — the declarative
/// `apply` engine composes scripts + routes + triggers into one tx.
/// Mirrors `create` minus the `begin`/`commit`.
pub(crate) async fn insert_script_tx(
tx: &mut sqlx::Transaction<'_, sqlx::Postgres>,
input: &NewScript,
) -> Result<Script, ScriptRepositoryError> {
let sandbox_json = serde_json::to_value(input.sandbox.unwrap_or_default())
.unwrap_or_else(|_| serde_json::json!({}));
let res = sqlx::query_as::<_, ScriptRow>(&format!(
"INSERT INTO scripts ( \
app_id, name, description, source, kind, \
timeout_seconds, memory_limit_mb, sandbox \
) VALUES ($1, $2, $3, $4, $5, COALESCE($6, 30), COALESCE($7, 256), $8) \
RETURNING {SCRIPT_SELECT_COLS}"
))
.bind(input.app_id.into_inner())
.bind(&input.name)
.bind(input.description.as_deref())
.bind(&input.source)
.bind(input.kind.as_str())
.bind(input.timeout_seconds)
.bind(input.memory_limit_mb)
.bind(sandbox_json)
.fetch_one(&mut **tx)
.await;
let script: Script = match res {
Ok(row) => row.into(),
Err(sqlx::Error::Database(e)) if e.is_unique_violation() => {
return Err(ScriptRepositoryError::Conflict(format!(
"a script named {:?} already exists in this app",
input.name
)));
}
Err(e) => return Err(e.into()),
};
replace_imports_tx(tx, script.id, script.app_id, &input.imports).await?;
Ok(script)
}
/// Update a script within an existing transaction. Mirrors `update`
/// minus the `begin`/`commit`.
pub(crate) async fn update_script_tx(
tx: &mut sqlx::Transaction<'_, sqlx::Postgres>,
id: ScriptId,
patch: &ScriptPatch,
) -> Result<Script, ScriptRepositoryError> {
let sandbox_json = patch
.sandbox
.as_ref()
.map(|s| serde_json::to_value(s).unwrap_or_else(|_| serde_json::json!({})));
let res = sqlx::query_as::<_, ScriptRow>(&format!(
"UPDATE scripts SET \
name = COALESCE($2, name), \
description = CASE WHEN $3::bool THEN $4 ELSE description END, \
source = COALESCE($5, source), \
timeout_seconds = COALESCE($6, timeout_seconds), \
memory_limit_mb = COALESCE($7, memory_limit_mb), \
sandbox = COALESCE($8, sandbox), \
kind = COALESCE($9, kind), \
version = version + 1, \
updated_at = NOW() \
WHERE id = $1 \
RETURNING {SCRIPT_SELECT_COLS}"
))
.bind(id.into_inner())
.bind(patch.name.as_deref())
.bind(patch.description.is_some())
.bind(patch.description.as_ref().and_then(|d| d.as_deref()))
.bind(patch.source.as_deref())
.bind(patch.timeout_seconds)
.bind(patch.memory_limit_mb)
.bind(sandbox_json)
.bind(patch.kind.map(ScriptKind::as_str))
.fetch_optional(&mut **tx)
.await;
let script: Script = match res {
Ok(Some(row)) => row.into(),
Ok(None) => return Err(ScriptRepositoryError::NotFound(id)),
Err(sqlx::Error::Database(e)) if e.is_unique_violation() => {
return Err(ScriptRepositoryError::Conflict(
"a script with that name already exists in this app".into(),
));
}
Err(e) => return Err(e.into()),
};
if let Some(imports) = patch.imports.as_deref() {
replace_imports_tx(tx, script.id, script.app_id, imports).await?;
}
Ok(script)
}
/// Delete a script within an existing transaction (its routes/triggers
/// cascade via their FKs). Mirrors `delete` minus the pool.
pub(crate) async fn delete_script_tx(
tx: &mut sqlx::Transaction<'_, sqlx::Postgres>,
id: ScriptId,
) -> Result<(), ScriptRepositoryError> {
let res = sqlx::query("DELETE FROM scripts WHERE id = $1")
.bind(id.into_inner())
.execute(&mut **tx)
.await?;
if res.rows_affected() == 0 {
return Err(ScriptRepositoryError::NotFound(id));
}
Ok(())
}
/// Row shape mirroring the `scripts` table for sqlx FromRow.
#[derive(sqlx::FromRow)]
struct ScriptRow {