a55c2f112e
The v1.2 Workflows durable DAG engine gains its runtime. A dedicated
background worker (`workflow_orchestrator.rs`, mirroring `cron_scheduler`,
NOT folded into the dispatcher) advances every in-flight run step-by-step,
durably, surviving restarts.
Per tick, two phases:
A. Claim + execute — up to a small batch of `ready`, due steps are claimed
with the same `FOR UPDATE SKIP LOCKED` competing-consumer lease the queue
uses (`claim_ready_step`), one execution-gate permit per step acquired
BEFORE the claim so the shared gate bounds real parallelism. Each step
resolves its function by name in the run's app scope (never a
script-passed arg — the isolation boundary), builds an `ExecRequest`, and
runs through the injected `ExecutorClient`. Claimed steps run concurrently.
B. Advance — the outcome is written and the DAG advanced in one token-gated
transaction (`complete_step_and_advance`): pending steps whose deps are
satisfied flip to `ready`, and the run's terminal status is recomputed.
Fan-in falls out (a join waits until its last dep flips it); a stale worker
matches zero rows and writes nothing.
The graph-advance decision is a pure, DB-free function (`compute_advance`) —
promotions + terminal run status, folding `on_error` fail-vs-continue and
skipped/failed dependency satisfaction — so it is unit-tested in isolation.
Retry uses the step's own policy via `compute_backoff`; a second, slower
cadence reclaims steps leased by a crashed worker (`reclaim_stale_steps`) — the
durability safety net. Steps run with no principal (like invoke_async), and log
under the new `ExecutionSource::Workflow` so `pic logs` surfaces them.
M2 executes function steps only; `when` + input templating land in M3, nested
sub-workflows in M4. Seams present (`StepTarget`, `run_input`, `workflow_depth`).
- migration 0072: widen the `execution_logs.source` CHECK with `workflow`
- shared: `ExecutionSource::Workflow`, `StepStatus::is_terminal`
- workflow_repo: run/step state — `start_run`, `claim_ready_step`,
`complete_step_and_advance`, `reclaim_stale_steps`, `get_run`,
`list_run_steps`, `compute_advance` (+ 7 pure unit tests)
- workflow_orchestrator: the worker + config (`PICLOUD_WORKFLOW_*` knobs),
spawned in `picloud/src/lib.rs` beside the dispatcher/cron scheduler
- tests: 8 DB-gated integration tests (linear, parallel fan-out/fan-in, retry,
on_error fail/continue, double-complete idempotency, stale-lease reclaim,
end-to-end tick with a fake executor)
Verified: cargo fmt, clippy -D warnings clean, 448 manager-core lib tests,
8 workflow_orchestrator DB tests, schema snapshot reblessed, M1 workflow CLI
journeys still pass (binary boots with the orchestrator wired in).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
PiCloud
A lightweight, self-hosted, event-driven serverless compute platform. Upload a Rhai script, get an HTTP endpoint. Designed to run on a single modest server with no idle CPU cost, and to scale out to a small cluster when you need it.
Status: Phase 1 — MVP scaffolding in progress.
The authoritative design lives in
serverless_cloud_blueprint.md.
Why
Existing serverless platforms are either cloud-locked, heavyweight, or both. PiCloud aims for the opposite end of the spectrum: one binary, one database, one reverse proxy — running on hardware you already own.
Architecture (one paragraph)
PiCloud splits into three logical services — manager (control plane: scripts, schedules, dashboard), orchestrator (per-node event ingress and dispatch), and executor (per-node Rhai sandbox) — each backed by a *-core Rust library. In MVP they run in a single process; in cluster mode they run as three binaries with one manager and one orchestrator + executor per node. Caddy fronts everything; PostgreSQL is the single source of truth.
See CLAUDE.md for working notes and serverless_cloud_blueprint.md for the full design.
Quick Start
Coming as scaffolding lands. For now:
# Rust toolchain (pinned via rust-toolchain.toml)
cargo check --workspace
# Run the all-in-one MVP binary (once main.rs is wired up)
cargo run -p picloud
Repository Layout
crates/
shared/ cross-cutting types
executor-core/ Rhai engine + sandbox
orchestrator-core/ event ingress, dispatch
manager-core/ control plane
picloud/ MVP all-in-one binary
picloud-{manager,orchestrator,executor}/ cluster-mode binaries (skeleton)
dashboard/ SvelteKit
caddy/ Caddyfile
docker/ Dockerfiles
docs/
git-workflow.md Trunk-based workflow
Contributing
See docs/git-workflow.md for the branching and commit conventions. TL;DR: trunk-based, short-lived branches, Conventional Commits, no force-pushing main.
License
TBD.