feat(workflows): M2 — durable orchestrator worker (claim/execute/advance)
The v1.2 Workflows durable DAG engine gains its runtime. A dedicated
background worker (`workflow_orchestrator.rs`, mirroring `cron_scheduler`,
NOT folded into the dispatcher) advances every in-flight run step-by-step,
durably, surviving restarts.
Per tick, two phases:
A. Claim + execute — up to a small batch of `ready`, due steps are claimed
with the same `FOR UPDATE SKIP LOCKED` competing-consumer lease the queue
uses (`claim_ready_step`), one execution-gate permit per step acquired
BEFORE the claim so the shared gate bounds real parallelism. Each step
resolves its function by name in the run's app scope (never a
script-passed arg — the isolation boundary), builds an `ExecRequest`, and
runs through the injected `ExecutorClient`. Claimed steps run concurrently.
B. Advance — the outcome is written and the DAG advanced in one token-gated
transaction (`complete_step_and_advance`): pending steps whose deps are
satisfied flip to `ready`, and the run's terminal status is recomputed.
Fan-in falls out (a join waits until its last dep flips it); a stale worker
matches zero rows and writes nothing.
The graph-advance decision is a pure, DB-free function (`compute_advance`) —
promotions + terminal run status, folding `on_error` fail-vs-continue and
skipped/failed dependency satisfaction — so it is unit-tested in isolation.
Retry uses the step's own policy via `compute_backoff`; a second, slower
cadence reclaims steps leased by a crashed worker (`reclaim_stale_steps`) — the
durability safety net. Steps run with no principal (like invoke_async), and log
under the new `ExecutionSource::Workflow` so `pic logs` surfaces them.
M2 executes function steps only; `when` + input templating land in M3, nested
sub-workflows in M4. Seams present (`StepTarget`, `run_input`, `workflow_depth`).
- migration 0072: widen the `execution_logs.source` CHECK with `workflow`
- shared: `ExecutionSource::Workflow`, `StepStatus::is_terminal`
- workflow_repo: run/step state — `start_run`, `claim_ready_step`,
`complete_step_and_advance`, `reclaim_stale_steps`, `get_run`,
`list_run_steps`, `compute_advance` (+ 7 pure unit tests)
- workflow_orchestrator: the worker + config (`PICLOUD_WORKFLOW_*` knobs),
spawned in `picloud/src/lib.rs` beside the dispatcher/cron scheduler
- tests: 8 DB-gated integration tests (linear, parallel fan-out/fan-in, retry,
on_error fail/continue, double-complete idempotency, stale-lease reclaim,
end-to-end tick with a fake executor)
Verified: cargo fmt, clippy -D warnings clean, 448 manager-core lib tests,
8 workflow_orchestrator DB tests, schema snapshot reblessed, M1 workflow CLI
journeys still pass (binary boots with the orchestrator wired in).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,18 @@
|
||||
-- v1.2 Workflows M2: the orchestrator executes each workflow step as a normal
|
||||
-- function invocation and logs it, so `pic logs` can surface (and filter by)
|
||||
-- workflow-driven runs — exactly as it does for cron/queue/invoke triggers.
|
||||
--
|
||||
-- Add 'workflow' to the execution_logs.source CHECK (extends 0043). The column
|
||||
-- default stays 'http'; this only widens the allowed set. Keep this list in
|
||||
-- sync with `shared::ExecutionSource` (which gains the matching `Workflow`
|
||||
-- variant) and `manager-core::OutboxSourceKind`.
|
||||
--
|
||||
-- Postgres has no ALTER … ALTER CONSTRAINT for a CHECK, so drop + re-add.
|
||||
|
||||
ALTER TABLE execution_logs DROP CONSTRAINT execution_logs_source_check;
|
||||
ALTER TABLE execution_logs
|
||||
ADD CONSTRAINT execution_logs_source_check
|
||||
CHECK (source IN (
|
||||
'http', 'kv', 'docs', 'dead_letter', 'cron',
|
||||
'files', 'pubsub', 'email', 'invoke', 'queue', 'workflow'
|
||||
));
|
||||
Reference in New Issue
Block a user