feat: declarative project-tool foundation (pull/plan/apply/prune)

Add a server-side, atomic, declarative reconcile loop for a single app —
the foundation of the project-tool design. Developers describe an app's
scripts, routes, triggers, and secret-names in `picloud.toml`, then
`pic pull / plan / apply [--prune]` to converge live state to the manifest.

Server (manager-core):
- apply_service: a pure diff engine (compute_diff) shared by plan and
  apply, plus an ApplyService that composes the existing per-repo writes
  into ONE Postgres transaction. Identity keys mirror the DB UNIQUE
  constraints (script=lower(name); route=(method,host_kind,host,
  path_kind,path); trigger=per-kind semantic tuple; secret=name).
  Apply takes a per-app advisory lock, recomputes the diff in-tx, applies
  scripts -> routes -> triggers, prunes dependents-first, commits, then
  refreshes the route table once post-commit.
- apply_api: POST /apps/{id}/plan (AppRead) and /apps/{id}/apply.
  Apply requires the per-kind write caps the bundle exercises (all three
  when --prune), plus AppSecretsRead when it binds an email trigger.
- tx-accepting repo siblings (insert/update/delete *_tx) so the existing
  create/update/delete delegate to one SQL definition each.
- email triggers reference an inbound secret by NAME; the value is
  resolved, decrypted (AAD-bound), and re-sealed server-side at apply —
  it never travels in the manifest.

CLI (picloud-cli):
- manifest.rs (picloud.toml model), client plan/apply, and the pull/plan/
  apply commands. pull rejects filesystem-unsafe script names up front.

Safety properties enforced and tested:
- idempotent: a freshly-pulled manifest re-applies as all-NoOp.
- atomic: a mid-bundle failure rolls back with nothing written.
- routes delete-before-insert so a freed binding is reusable in one apply.
- queue one-consumer invariant held inside the shared tx.
- email triggers are never pruned, and a script that still owns an
  email/dead-letter trigger can't be pruned (the FK cascade would destroy
  the sealed secret) — refused with a pointer to `pic triggers rm`.
- plan and apply agree on unset email-secret references.

No migration: the existing schema's UNIQUE constraints serve as identity
keys. Groups, env-scoping, and the `enabled` toggle are later milestones.

Tested: manager-core lib (360) + CLI bins (27) + 8 project-tool journeys
(pull/plan/apply/prune/email+queue), all green; clippy -D warnings clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
MechaCat02
2026-06-20 21:52:21 +02:00
parent c600177fd6
commit 3b650a2b14
21 changed files with 4055 additions and 129 deletions

View File

@@ -10,13 +10,13 @@ use axum::middleware::from_fn_with_state;
use axum::{routing::get, Json, Router};
use picloud_executor_core::{Engine, Limits};
use picloud_manager_core::{
admin_router, admins_router, api_keys_router, app_members_router, apps_api, apps_router,
attach_principal_if_present, auth_router, compile_routes, dead_letters_router,
admin_router, admins_router, api_keys_router, app_members_router, apply_router, apps_api,
apps_router, attach_principal_if_present, auth_router, compile_routes, dead_letters_router,
dev_emails_router, email_inbound_router, files_admin_router, kv_admin_router, migrations,
require_authenticated, route_admin_router, secrets_router, topics_router, triggers_router,
AbandonedRepo, AdminPrincipalResolver, AdminSessionRepository, AdminState, AdminUserRepository,
AdminsState, ApiKeyRepository, ApiKeysState, AppDomainRepository, AppMembersRepository,
AppMembersState, AppRepository, AppsState, AuthState, AuthzRepo, DeadLetterRepo,
AppMembersState, AppRepository, ApplyService, AppsState, AuthState, AuthzRepo, DeadLetterRepo,
DeadLettersState, DevEmailState, Dispatcher, DocsServiceImpl, EmailInboundState,
EmailServiceImpl, FilesAdminState, FilesConfig, FilesServiceImpl, FsFilesRepo, HttpConfig,
HttpServiceImpl, InboundNonceDedup, KvAdminState, KvServiceImpl, OutboxEventEmitter,
@@ -42,8 +42,8 @@ use picloud_orchestrator_core::{
use picloud_shared::{
DeadLetterService, DocsService, EmailService, ExecutionLogSink, FilesService, HttpService,
InboxResolver, KvService, MasterKey, OutboxWriter, PubsubService, RealtimeAuthority,
RealtimeBroadcaster, ScriptValidator, SecretsService, ServiceEventEmitter, Services,
UsersService, API_VERSION, PRODUCT_VERSION, SDK_VERSION, WIRE_VERSION,
RealtimeBroadcaster, SecretsService, ServiceEventEmitter, Services, UsersService, API_VERSION,
PRODUCT_VERSION, SDK_VERSION, WIRE_VERSION,
};
use sqlx::postgres::PgPoolOptions;
use sqlx::PgPool;
@@ -399,7 +399,7 @@ pub async fn build_app(
logs: log_repo,
apps: apps_repo.clone(),
authz: authz.clone(),
validator: engine as Arc<dyn ScriptValidator>,
validator: engine.clone(),
sandbox_ceiling: SandboxCeiling::from_env(),
};
let route_admin = RouteAdminState {
@@ -414,7 +414,7 @@ pub async fn build_app(
resolver,
log_sink,
app_domains: app_domain_table.clone(),
routes: route_table,
routes: route_table.clone(),
inbox: inbox_registry,
outbox: outbox_writer,
};
@@ -440,7 +440,7 @@ pub async fn build_app(
// v1.1.4: cron scheduler. Polls cron_trigger_details on a tick and
// enqueues due triggers into the outbox; the dispatcher above
// delivers them like any other async trigger.
picloud_manager_core::spawn_cron_scheduler(pool, trigger_config.cron_tick_interval_ms);
picloud_manager_core::spawn_cron_scheduler(pool.clone(), trigger_config.cron_tick_interval_ms);
// v1.1.6: GC empty realtime broadcast channels (one-shot subscribers)
// and sweep orphaned `*.tmp.*` blobs left by crashed file writes.
spawn_realtime_gc(broadcaster_concrete, DEFAULT_GC_INTERVAL_SECS);
@@ -453,6 +453,23 @@ pub async fn build_app(
config: trigger_config,
master_key: master_key.clone(),
};
// Declarative reconcile engine (pic plan / apply). Trait-object repos
// for the read/diff path; shares the same handles as the CRUD routers.
let apply_service = ApplyService {
pool: pool.clone(),
scripts: script_repo.clone(),
routes: route_repo.clone(),
triggers: trigger_repo.clone(),
secrets: secrets_repo.clone(),
apps: apps_repo.clone(),
domains: domains_repo.clone(),
authz: authz.clone(),
validator: engine.clone(),
sandbox_ceiling: SandboxCeiling::from_env(),
trigger_config,
route_table: route_table.clone(),
master_key: master_key.clone(),
};
// v1.1.9: keep a clone for the queues-api state (built later).
let trigger_repo_for_queues = trigger_repo.clone();
// v1.1.7 public inbound-email receiver. Outside the admin auth layer
@@ -562,6 +579,7 @@ pub async fn build_app(
))
.merge(api_keys_router(api_keys_state))
.merge(triggers_router(triggers_state))
.merge(apply_router(apply_service))
.merge(picloud_manager_core::queues_api::queues_router(
picloud_manager_core::queues_api::QueuesState {
queues: queue_repo.clone(),