Files
PiCloud/crates/manager-core/src/lib.rs
MechaCat02 3b650a2b14 feat: declarative project-tool foundation (pull/plan/apply/prune)
Add a server-side, atomic, declarative reconcile loop for a single app —
the foundation of the project-tool design. Developers describe an app's
scripts, routes, triggers, and secret-names in `picloud.toml`, then
`pic pull / plan / apply [--prune]` to converge live state to the manifest.

Server (manager-core):
- apply_service: a pure diff engine (compute_diff) shared by plan and
  apply, plus an ApplyService that composes the existing per-repo writes
  into ONE Postgres transaction. Identity keys mirror the DB UNIQUE
  constraints (script=lower(name); route=(method,host_kind,host,
  path_kind,path); trigger=per-kind semantic tuple; secret=name).
  Apply takes a per-app advisory lock, recomputes the diff in-tx, applies
  scripts -> routes -> triggers, prunes dependents-first, commits, then
  refreshes the route table once post-commit.
- apply_api: POST /apps/{id}/plan (AppRead) and /apps/{id}/apply.
  Apply requires the per-kind write caps the bundle exercises (all three
  when --prune), plus AppSecretsRead when it binds an email trigger.
- tx-accepting repo siblings (insert/update/delete *_tx) so the existing
  create/update/delete delegate to one SQL definition each.
- email triggers reference an inbound secret by NAME; the value is
  resolved, decrypted (AAD-bound), and re-sealed server-side at apply —
  it never travels in the manifest.

CLI (picloud-cli):
- manifest.rs (picloud.toml model), client plan/apply, and the pull/plan/
  apply commands. pull rejects filesystem-unsafe script names up front.

Safety properties enforced and tested:
- idempotent: a freshly-pulled manifest re-applies as all-NoOp.
- atomic: a mid-bundle failure rolls back with nothing written.
- routes delete-before-insert so a freed binding is reusable in one apply.
- queue one-consumer invariant held inside the shared tx.
- email triggers are never pruned, and a script that still owns an
  email/dead-letter trigger can't be pruned (the FK cascade would destroy
  the sealed secret) — refused with a pointer to `pic triggers rm`.
- plan and apply agree on unset email-secret references.

No migration: the existing schema's UNIQUE constraints serve as identity
keys. Groups, env-scoping, and the `enabled` toggle are later milestones.

Tested: manager-core lib (360) + CLI bins (27) + 8 project-tool journeys
(pull/plan/apply/prune/email+queue), all green; clippy -D warnings clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 21:52:21 +02:00

210 lines
8.1 KiB
Rust

//! Control plane: script storage, scheduling, configuration.
//!
//! Single-writer to Postgres. The orchestrator may *read* scripts from
//! the same DB for now; once we add caching and per-node ingress, the
//! manager will publish change events.
pub mod abandoned_repo;
pub mod admin_session_repo;
pub mod admin_user_repo;
pub mod admin_users_api;
pub mod api;
pub mod api_key_repo;
pub mod api_keys_api;
pub mod app_bootstrap;
pub mod app_domain_repo;
pub mod app_members_api;
pub mod app_members_repo;
pub mod app_repo;
pub mod app_secrets_repo;
pub mod app_user_invitation_repo;
pub mod app_user_password_reset_repo;
pub mod app_user_repo;
pub mod app_user_role_repo;
pub mod app_user_session_repo;
pub mod app_user_verification_repo;
pub mod apply_api;
pub mod apply_service;
pub mod apps_api;
pub mod auth;
pub mod auth_api;
pub mod auth_bootstrap;
pub mod auth_middleware;
pub mod authz;
pub mod cron_scheduler;
pub mod dead_letter_repo;
pub mod dead_letter_service;
pub mod dead_letters_api;
pub mod dev_email_api;
pub mod dispatcher;
pub mod docs_filter;
pub mod docs_repo;
pub mod docs_service;
pub mod email_inbound_api;
pub mod email_service;
pub mod files_api;
pub mod files_repo;
pub mod files_service;
pub mod files_sweep;
pub mod gc;
pub mod http_service;
pub mod invoke_service;
pub mod kv_api;
pub mod kv_repo;
pub mod kv_service;
pub mod log_sink;
pub mod login_rate_limit;
pub mod migrations;
pub mod module_source;
pub mod outbox_event_emitter;
pub mod outbox_repo;
pub mod principal_resolver;
pub mod pubsub_repo;
pub mod pubsub_service;
pub mod queue_repo;
pub mod queue_service;
pub mod queues_api;
pub mod realtime_authority;
pub mod repo;
pub mod route_admin;
pub mod route_repo;
pub mod sandbox;
pub mod scheduler;
pub mod secrets_api;
pub mod secrets_repo;
pub mod secrets_service;
pub mod ssrf;
pub mod topic_repo;
pub mod topics_api;
pub mod trigger_config;
pub mod trigger_repo;
pub mod triggers_api;
pub mod users_admin_api;
pub mod users_service;
pub use abandoned_repo::{
AbandonedRepo, AbandonedRepoError, NewAbandonedExecution, PostgresAbandonedRepo,
};
pub use admin_session_repo::{
AdminSessionLookup, AdminSessionRepository, AdminSessionRepositoryError,
PostgresAdminSessionRepository,
};
pub use admin_user_repo::{
AdminUserCredentials, AdminUserRepository, AdminUserRepositoryError, AdminUserRow,
PostgresAdminUserRepository,
};
pub use admin_users_api::{admins_router, AdminsState};
pub use api::{admin_router, AdminState};
pub use api_key_repo::{
ApiKeyRepository, ApiKeyRepositoryError, ApiKeyRow, ApiKeyVerification, NewApiKey,
PostgresApiKeyRepository,
};
pub use api_keys_api::{api_keys_router, ApiKeysState};
pub use app_bootstrap::{seed_hello_world_if_fresh, HelloWorldOutcome};
pub use app_domain_repo::{AppDomainRepository, NewAppDomain, PostgresAppDomainRepository};
pub use app_members_api::{app_members_router, AppMembersApiError, AppMembersState};
pub use app_members_repo::{
AppMembersRepository, AppMembersRepositoryError, AppMembershipDetail, AppMembershipRow,
PostgresAppMembersRepository,
};
pub use app_repo::{resolve_app, AppLookup, AppRepository, PostgresAppRepository};
pub use app_secrets_repo::{
AppSecretsRepo, AppSecretsRepoError, PostgresAppSecretsRepo, SIGNING_KEY_LEN,
};
pub use app_user_invitation_repo::{
AppUserInvitationRepo, AppUserInvitationRepoError, ConsumedInvitation, InvitationRow,
PostgresAppUserInvitationRepo,
};
pub use app_user_password_reset_repo::{
AppUserPasswordResetRepo, AppUserPasswordResetRepoError, PostgresAppUserPasswordResetRepo,
};
pub use app_user_repo::{
AppUserCredentials, AppUserRepository, AppUserRepositoryError, AppUserRow,
ListOpts as AppUserListOpts, ListPage as AppUserListPage, PostgresAppUserRepository,
};
pub use app_user_role_repo::{AppUserRoleRepo, AppUserRoleRepoError, PostgresAppUserRoleRepo};
pub use app_user_session_repo::{
AppUserSessionLookup, AppUserSessionRepository, AppUserSessionRepositoryError,
PostgresAppUserSessionRepository,
};
pub use app_user_verification_repo::{
AppUserVerificationRepo, AppUserVerificationRepoError, PostgresAppUserVerificationRepo,
};
pub use apply_api::apply_router;
pub use apply_service::{ApplyError, ApplyService, Bundle, Plan};
pub use apps_api::{apps_router, AppsState};
pub use auth_api::auth_router;
pub use auth_bootstrap::{
bootstrap_first_admin, bootstrap_first_admin_with, BootstrapEnv, BootstrapError,
};
#[allow(deprecated)]
pub use auth_middleware::{
attach_principal_if_present, require_admin, require_authenticated, AuthState, AuthedAdmin,
API_KEY_PREFIX, API_KEY_PREFIX_LEN,
};
pub use authz::{can, require, AuthzDenied, AuthzError, AuthzRepo, Capability, Decision};
pub use cron_scheduler::spawn_cron_scheduler;
pub use dead_letter_repo::{
DeadLetterRepo, DeadLetterRepoError, DeadLetterRow, NewDeadLetter, PostgresDeadLetterRepo,
};
pub use dead_letter_service::PostgresDeadLetterService;
pub use dead_letters_api::{dead_letters_router, DeadLettersApiError, DeadLettersState};
pub use dev_email_api::{dev_emails_router, DevEmailState};
pub use dispatcher::{compute_backoff, Dispatcher, DispatcherError};
pub use docs_repo::{DocsRepo, DocsRepoError, PostgresDocsRepo};
pub use docs_service::DocsServiceImpl;
pub use email_inbound_api::{
email_inbound_router, EmailInboundError, EmailInboundState, InboundNonceDedup,
};
pub use email_service::{
CapturedEmail, DevEmailSink, DevEmailTransport, EmailConfig, EmailServiceImpl, EmailTransport,
LettreEmailTransport, SmtpConfig, SmtpTls, DEFAULT_EMAIL_MAX_MESSAGE_BYTES,
};
pub use files_api::{files_admin_router, FilesAdminState};
pub use files_repo::{FilesConfig, FilesRepo, FilesRepoError, FsFilesRepo};
pub use files_service::FilesServiceImpl;
pub use files_sweep::{spawn_files_orphan_sweep, sweep_orphan_tmp_files, SweepStats};
pub use gc::{spawn_abandoned_gc, spawn_app_user_token_gc, spawn_dead_letter_gc};
pub use http_service::{HttpConfig, HttpServiceImpl};
pub use kv_api::{kv_admin_router, KvAdminState};
pub use kv_repo::{KvRepo, KvRepoError, PostgresKvRepo};
pub use kv_service::KvServiceImpl;
pub use log_sink::PostgresExecutionLogSink;
pub use module_source::PostgresModuleSource;
pub use outbox_event_emitter::OutboxEventEmitter;
pub use outbox_repo::{
NewOutboxRow, OutboxRepo, OutboxRepoError, OutboxRow, OutboxSourceKind, PostgresOutboxRepo,
};
pub use principal_resolver::{AdminPrincipalResolver, PrincipalResolver, PrincipalResolverError};
pub use pubsub_repo::{PostgresPubsubRepo, PublishCtx, PubsubRepo, PubsubRepoError};
pub use pubsub_service::{PubsubServiceImpl, SubscriberTokenConfig};
pub use realtime_authority::RealtimeAuthorityImpl;
pub use repo::{
ExecutionLogRepository, NewScript, PostgresExecutionLogRepository, PostgresScriptRepository,
RepoResolver, ScriptPatch, ScriptRepository, ScriptRepositoryError,
};
pub use route_admin::{compile_routes, route_admin_router, RouteAdminState};
pub use route_repo::{NewRoute, PostgresRouteRepository, RouteRepository};
pub use sandbox::{CeilingError, SandboxCeiling};
pub use secrets_api::{secrets_router, SecretsApiError, SecretsState};
pub use secrets_repo::{
PostgresSecretsRepo, SecretMeta, SecretsMetaPage, SecretsNamePage, SecretsRepo,
SecretsRepoError, StoredSecret,
};
pub use secrets_service::{
open as open_secret, seal as seal_secret, SecretsConfig, SecretsServiceImpl,
DEFAULT_SECRET_MAX_VALUE_BYTES,
};
pub use topic_repo::{PostgresTopicRepo, Topic, TopicAuthMode, TopicRepo, TopicRepoError};
pub use topics_api::{topics_router, TopicsApiError, TopicsState};
pub use trigger_config::{BackoffShape, TriggerConfig};
pub use trigger_repo::{
collection_matches, CreateDeadLetterTrigger, CreateDocsTrigger, CreateEmailTrigger,
CreateFilesTrigger, CreateKvTrigger, CreatePubsubTrigger, DeadLetterTriggerMatch,
DocsTriggerMatch, EmailInboundTarget, FilesTriggerMatch, KvTriggerMatch, PostgresTriggerRepo,
Trigger, TriggerDetails, TriggerDispatchMode, TriggerKind, TriggerRepo, TriggerRepoError,
};
pub use triggers_api::{triggers_router, TriggersApiError, TriggersState};
pub use users_admin_api::{app_users_router, AppUsersApiError, AppUsersState};
pub use users_service::{UsersServiceConfig, UsersServiceImpl};