feat(workflows): M1 — schema + definition validation + declarative reconcile

First milestone of the v1.2 Workflows track (blueprint §9.1/§9.2): the durable
DAG engine's foundation — the definition model, its validation, and the
declarative `apply` reconcile path. No execution yet (the orchestrator is M2).

- Migration 0071_workflows: `workflows` (owner-polymorphic like scripts, app-
  owned in M1), `workflow_runs`, and `workflow_run_steps` (the per-step
  competing-consumer lease table the M2 orchestrator will claim). Schema golden
  reblessed.
- `shared::workflow`: the `WorkflowDefinition` / `WorkflowStepDef` DTOs (shared
  by the CLI, apply, and the future orchestrator) + run/step status enums.
- Pure, DB-free `workflow_template` (input `{{ input.x }}` / `{{ steps.a.output.y }}`
  resolution, type-preserving) and `workflow_expr` (a small safe JSON-predicate
  evaluator for `when` — keeps manager-core free of a scripting engine).
- `workflow_repo`: read trait + reconcile tx free-fns (insert/update/delete).
- apply_service: `BundleWorkflow` + `Plan.workflows` + `CurrentState.workflows`
  + `ApplyReport.workflows_*`; server-side `validate_workflow_definition`
  (unique/acyclic steps via topological sort, deps exist, function XOR workflow,
  `when`/template parse) rejecting on a group node; `diff_workflows` by
  lower(name) (Update on a definition change) + in-tx reconcile.
- CLI: `[[workflows]]` + `[[workflows.steps]]` manifest structs → wire bundle;
  plan/apply render + report counts.
- Tests: 16 manager-core lib tests (template, expr, definition validation) +
  the `workflows` CLI journey (apply → NoOp → prune; cyclic DAG rejected).

Deferred to later milestones: the orchestrator worker (M2), conditional/template
runtime wiring (M3), nested sub-workflows (M4), the SDK/API/CLI run surface
(M5), the dashboard (M6). The `group_id` column + run/step tables ship now so
those slot in without a migration churn.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
MechaCat02
2026-07-12 16:46:32 +02:00
parent c8c4f012ff
commit 44f992cbb0
18 changed files with 2059 additions and 3 deletions

View File

@@ -59,3 +59,6 @@ id_type!(InvitationId);
id_type!(QueueMessageId);
id_type!(GroupId);
id_type!(ProjectId);
id_type!(WorkflowId);
id_type!(WorkflowRunId);
id_type!(WorkflowRunStepId);

View File

@@ -46,6 +46,7 @@ pub mod users;
pub mod validator;
pub mod vars;
pub mod version;
pub mod workflow;
/// serde `default` for `enabled`-style boolean fields that should default to
/// `true` when absent from the wire (bool's own `Default` is `false`). Shared
@@ -80,7 +81,7 @@ pub use group_queue::{GroupQueueError, GroupQueueService, NoopGroupQueueService}
pub use http::{HttpError, HttpRequest, HttpResponse, HttpService, NoopHttpService};
pub use ids::{
AdminUserId, ApiKeyId, AppId, AppUserId, ExecutionId, GroupId, InvitationId, ProjectId,
QueueMessageId, RequestId, ScriptId, TriggerId,
QueueMessageId, RequestId, ScriptId, TriggerId, WorkflowId, WorkflowRunId, WorkflowRunStepId,
};
pub use inbox::{
InboxDeliveryOutcome, InboxFailureKind, InboxResolver, InboxResult, NoopInboxResolver,

View File

@@ -0,0 +1,202 @@
//! Workflow definition + status DTOs (v1.2 Workflows).
//!
//! Pure data shared across three consumers: the `pic` CLI (which builds a
//! [`WorkflowDefinition`] from the TOML manifest), the manager-core apply path
//! (which validates + stores it as JSONB), and the orchestrator (which executes
//! it). There is NO behavior here — the validator (`workflow_expr`), the input
//! resolver (`workflow_template`), and the durable orchestrator all live
//! server-side in `manager-core`, the only crate that touches Postgres.
//!
//! The `WorkflowService` trait (the injected SDK seam for `workflow::start`)
//! is added here in M5.
use serde::{Deserialize, Serialize};
/// A workflow's DAG: an ordered list of steps connected by `depends_on` edges.
/// Serialized to the `workflows.definition` JSONB column.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct WorkflowDefinition {
pub steps: Vec<WorkflowStepDef>,
}
/// One node in the DAG.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct WorkflowStepDef {
pub name: String,
/// A function step invokes a script by name (resolved in the app's scope).
/// Exactly one of `function` / `workflow` is set (enforced server-side).
#[serde(default, skip_serializing_if = "Option::is_none")]
pub function: Option<String>,
/// A workflow step starts a nested sub-workflow by name.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub workflow: Option<String>,
/// Input mapping — a JSON value whose string leaves may carry `{{ … }}`
/// references resolved at run time against the run input + prior step
/// outputs (see `workflow_template`). Absent = the run input is passed
/// through unchanged.
#[serde(default, skip_serializing_if = "serde_json::Value::is_null")]
pub input: serde_json::Value,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub depends_on: Vec<String>,
/// A JSON-predicate condition (see `workflow_expr`). When it evaluates
/// false the step is `skipped` (and counts as satisfied-but-empty for its
/// dependents). Absent = always run.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub when: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub retry: Option<WorkflowRetry>,
#[serde(default)]
pub on_error: OnError,
}
impl WorkflowStepDef {
/// The resolved target of this step, or `None` if neither/both are set
/// (a shape error caught by server-side validation).
#[must_use]
pub fn target(&self) -> Option<StepTarget<'_>> {
match (self.function.as_deref(), self.workflow.as_deref()) {
(Some(f), None) => Some(StepTarget::Function(f)),
(None, Some(w)) => Some(StepTarget::Workflow(w)),
_ => None,
}
}
/// Max attempts for this step (1 = no retry), derived from its retry policy.
#[must_use]
pub fn max_attempts(&self) -> u32 {
self.retry.as_ref().map_or(1, |r| r.max_attempts.max(1))
}
}
/// A step's target — a function (script by name) or a nested sub-workflow.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum StepTarget<'a> {
Function(&'a str),
Workflow(&'a str),
}
/// What to do when a step exhausts its attempts.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum OnError {
/// Fail the whole run (default).
#[default]
Fail,
/// Mark the step failed but let the run continue — dependents proceed and
/// the run can still succeed.
Continue,
}
/// Per-step retry policy.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct WorkflowRetry {
pub max_attempts: u32,
#[serde(default)]
pub backoff: WorkflowBackoff,
#[serde(default = "default_base_ms")]
pub base_ms: u32,
}
/// Backoff shape for step retries (self-contained so this DTO stays in `shared`;
/// the orchestrator maps it to its backoff computation).
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum WorkflowBackoff {
Constant,
Linear,
#[default]
Exponential,
}
fn default_base_ms() -> u32 {
500
}
/// Status of a workflow run (mirrors the `workflow_runs.status` CHECK).
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum RunStatus {
Pending,
Running,
Succeeded,
Failed,
Canceled,
}
impl RunStatus {
#[must_use]
pub fn as_str(self) -> &'static str {
match self {
Self::Pending => "pending",
Self::Running => "running",
Self::Succeeded => "succeeded",
Self::Failed => "failed",
Self::Canceled => "canceled",
}
}
#[must_use]
pub fn is_terminal(self) -> bool {
matches!(self, Self::Succeeded | Self::Failed | Self::Canceled)
}
#[must_use]
#[allow(clippy::should_implement_trait)]
pub fn from_str(s: &str) -> Option<Self> {
Some(match s {
"pending" => Self::Pending,
"running" => Self::Running,
"succeeded" => Self::Succeeded,
"failed" => Self::Failed,
"canceled" => Self::Canceled,
_ => return None,
})
}
}
/// Status of a single step within a run (mirrors `workflow_run_steps.status`).
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum StepStatus {
Pending,
Ready,
Running,
Succeeded,
Failed,
Skipped,
}
impl StepStatus {
#[must_use]
pub fn as_str(self) -> &'static str {
match self {
Self::Pending => "pending",
Self::Ready => "ready",
Self::Running => "running",
Self::Succeeded => "succeeded",
Self::Failed => "failed",
Self::Skipped => "skipped",
}
}
/// A dependency is "satisfied" (a dependent may proceed) once it has
/// succeeded OR been skipped (a false `when` — satisfied-but-empty).
#[must_use]
pub fn is_satisfied(self) -> bool {
matches!(self, Self::Succeeded | Self::Skipped)
}
#[must_use]
#[allow(clippy::should_implement_trait)]
pub fn from_str(s: &str) -> Option<Self> {
Some(match s {
"pending" => Self::Pending,
"ready" => Self::Ready,
"running" => Self::Running,
"succeeded" => Self::Succeeded,
"failed" => Self::Failed,
"skipped" => Self::Skipped,
_ => return None,
})
}
}