fabi/PiCloud
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: custom routing — bind scripts to your own URLs

Browse Source 
Scripts can now answer at user-chosen paths (e.g. /greet, /greet/:name,
/webhooks/*), on user-chosen hosts (strict or *.example.com wildcards),
on user-chosen methods. The internal /api/v1/execute/{id} endpoint
stays as the always-available ID-based bypass.

Routing rules (decided in design with the user; see chat history):

  Path kinds:
    exact   /greet              literal
    prefix  /greet/*            strict-subtree; stored as "/greet/";
                                does NOT match bare /greet (add an
                                exact route for that case)
    param   /users/:id          :name captures one whole segment;
                                mid-segment colons are rejected;
                                {name} is reserved for a future SDK

  Host kinds:
    any                         no Host header constraint
    strict  sub.example.com     literal match (case-insensitive)
    wildcard *.example.com      suffix match; multi-level subdomains OK

  Within-kind uniqueness:
    two routes of the same kind that could match the same request
    conflict at config time. Algorithm (orchestrator_core::routing::
    conflict):
      exact:  literal equality
      prefix: literal equality (longer-prefix coexists; longer wins
              at request time)
      param:  same segment count + same literals at every
              literal-vs-literal position (the user's example:
              :id vs :userId at same shape is a conflict)

  Request-time precedence:
    exact > param > prefix
    among non-exact: more leading-literal segments wins
    tie: param > prefix (more constrained)
    within prefix: longest matching prefix wins
    host bucket: strict > wildcard (longer suffix) > any; fall through
    to less specific buckets when path doesn't match

  Reserved path prefixes: /api/, /admin/, /healthz, /version

  Routes that look invalid at config time return 422 with the precise
  parse error; conflicting routes return 409 with the conflicting route
  in the body (so the dashboard can render the conflict inline).

What landed:

  * 0003_routes.sql — routes table (host_kind, host, host_param_name,
    path_kind, path, method, script_id) with UNIQUE index on the
    literal binding tuple. Schema 2 → 3.

  * shared::Route / HostKind / PathKind — flat storage shape that
    crosses wire boundaries cleanly.

  * orchestrator_core::routing — four sub-modules, all unit-tested:
      pattern.rs (16 tests)  parse + validate + display
      conflict.rs (12 tests) within-kind overlap predicate
      matcher.rs (12 tests)  runtime dispatch (specificity-aware)
      table.rs               Arc<RwLock<Vec<CompiledRoute>>>
                             shared by manager (writes) and
                             orchestrator (reads); atomic replace
                             after each admin write

  * manager-core::route_admin — five new admin endpoints under
    /api/v1/admin:
      POST   /scripts/{id}/routes      create
      GET    /scripts/{id}/routes      list per script
      DELETE /routes/{route_id}        delete (refreshes table)
      POST   /routes:check             pre-flight conflict check
                                       (powers the dashboard's
                                       live conflict warning)
      POST   /routes:match             synthetic URL → matched
                                       route + extracted params
                                       (powers the dashboard's
                                       match-preview tool)
    Stored path strings stay raw (user-typed); normalization
    happens only in the in-memory CompiledRoute so re-parses are
    idempotent.

  * orchestrator_core::api::user_routes_router — fallback handler
    mounted in picloud after the system routes. Reads Host /
    method / path / query from the request, dispatches via the
    table, builds an ExecRequest with params/query/rest filled,
    calls the executor, writes to the log sink. 10 MiB body cap.

  * executor-core::ctx (SDK 1.0 → 1.1) — adds
      ctx.request.params  (map of named-param captures)
      ctx.request.query   (parsed query string)
      ctx.request.rest    (suffix for prefix routes; "" otherwise)
    All three are always present (empty when not applicable) so
    scripts can read them unconditionally.

  * picloud::build_app — now async; loads routes at startup,
    populates the shared table, mounts route_admin_router under
    /api/v1/admin alongside the script CRUD, and the user-routes
    fallback at the app root.

  * caddy/Caddyfile + Caddyfile.prod widened: anything not
    /healthz, /version, /api/v1/admin/*, /api/v1/execute/*,
    /api/* (404 sunset), or /admin/* (dashboard) → picloud.

  * Dashboard moves to /admin/* via SvelteKit paths.base. Its
    internal Caddy strips the prefix and serves with SPA fallback.
    All in-app links use $app/paths. The dashboard URL is now
    http://localhost:8000/admin/ — one-time break for the new
    URL freedom users gained.

  * PICLOUD_PUBLIC_BASE_URL env var, exposed via /version so the
    dashboard renders full URLs for routes regardless of the
    operator's external port / TLS setup.

  * memory_limit_mb stays in the schema, still v1.3+ advisory.

Verified live through Caddy:
  /version              → schema 3, sdk 1.1, public_base_url
  GET /admin/           → 200, dashboard HTML containing "PiCloud"
  POST /api/v1/admin/scripts → 201
  POST .../scripts/{id}/routes (path=/greet/:name) → 201
  GET /greet/alice?lang=en → 200 {"name":"alice","q":"en"}
  POST conflicting route → 409 with conflicting_route body
  POST /admin/foo route → 422 "reserved"
  POST /api/v1/admin/routes:match → matched + params extracted
  GET /unbound-path → 404 JSON

Tests:
  * 40 routing unit tests (pattern + conflict + matcher tables)
  * 14 executor-core unit tests (one new for ctx.request.params/
    query/rest exposure)
  * 32 integration tests (10 new for routing CRUD + dispatch +
    conflict + reserved + specificity tie-break + match preview +
    delete invalidation + /version returns public_base_url)
  * default cargo test --workspace stays green; opt-in via
    DATABASE_URL + --include-ignored for the integration suite

Bumps: schema 2 → 3; SDK 1.0 → 1.1; product 0.3.0 → 0.4.0.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
MechaCat02
2026-05-23 18:18:16 +02:00
parent f51924fdbc
commit 07e2a62d98
36 changed files with 2449 additions and 111 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
crates/manager-core/src/lib.rs
View File

@@ -8,6 +8,8 @@ pub mod api;
pub mod log_sink;
pub mod migrations;
pub mod repo;
pub mod route_admin;
pub mod route_repo;
pub mod sandbox;
pub mod scheduler;
@@ -17,4 +19,6 @@ pub use repo::{
ExecutionLogRepository, NewScript, PostgresExecutionLogRepository, PostgresScriptRepository,
RepoResolver, ScriptPatch, ScriptRepository, ScriptRepositoryError,
};
pub use route_admin::{compile_routes, route_admin_router, RouteAdminState};
pub use route_repo::{NewRoute, PostgresRouteRepository, RouteRepository};
pub use sandbox::{CeilingError, SandboxCeiling};

347
crates/manager-core/src/route_admin.rs Normal file
View File

@@ -0,0 +1,347 @@
//! Admin endpoints for routes. Mounted under `/api/v1/admin` alongside
//! the script CRUD endpoints; the picloud binary wires the
//! `RouteTable` shared with the orchestrator dispatcher in here so
//! writes invalidate the in-memory snapshot.
use std::sync::Arc;
use axum::{
extract::{Path, State},
http::StatusCode,
response::{IntoResponse, Response},
routing::{delete, get, post},
Json, Router,
};
use picloud_orchestrator_core::routing::{conflict, matcher::CompiledRoute, pattern, RouteTable};
use picloud_shared::{HostKind, PathKind, Route, ScriptId};
use serde::{Deserialize, Serialize};
use uuid::Uuid;
use crate::repo::ScriptRepositoryError;
use crate::route_repo::{NewRoute, RouteRepository};
pub struct RouteAdminState<RR> {
pub routes: Arc<RR>,
pub table: Arc<RouteTable>,
}
impl<RR> Clone for RouteAdminState<RR> {
fn clone(&self) -> Self {
Self {
routes: self.routes.clone(),
table: self.table.clone(),
}
}
}
pub fn route_admin_router<RR>(state: RouteAdminState<RR>) -> Router
where
RR: RouteRepository + 'static,
{
Router::new()
.route(
"/scripts/{id}/routes",
get(list_routes::<RR>).post(create_route::<RR>),
)
.route("/routes/{route_id}", delete(delete_route::<RR>))
.route("/routes:check", post(check_route::<RR>))
.route("/routes:match", post(match_route::<RR>))
.with_state(state)
}
// ----------------------------------------------------------------------------
// DTOs
// ----------------------------------------------------------------------------
#[derive(Debug, Deserialize)]
pub struct CreateRouteRequest {
pub host_kind: HostKind,
#[serde(default)]
pub host: String,
#[serde(default)]
pub host_param_name: Option<String>,
pub path_kind: PathKind,
pub path: String,
pub method: Option<String>,
}
#[derive(Debug, Deserialize)]
pub struct CheckRouteRequest {
pub host_kind: HostKind,
#[serde(default)]
pub host: String,
pub path_kind: PathKind,
pub path: String,
pub method: Option<String>,
}
#[derive(Debug, Serialize)]
pub struct CheckRouteResponse {
pub ok: bool,
pub conflicting_route: Option<Route>,
pub conflict_reason: Option<String>,
}
#[derive(Debug, Deserialize)]
pub struct MatchRouteRequest {
pub url: String,
#[serde(default = "default_method")]
pub method: String,
}
fn default_method() -> String {
"GET".into()
}
#[derive(Debug, Serialize)]
pub struct MatchRouteResponse {
pub matched: Option<MatchedRoute>,
}
#[derive(Debug, Serialize)]
pub struct MatchedRoute {
pub route_id: Uuid,
pub script_id: ScriptId,
pub params: std::collections::BTreeMap<String, String>,
pub rest: Option<String>,
pub host_param: Option<(String, String)>,
}
// ----------------------------------------------------------------------------
// Handlers
// ----------------------------------------------------------------------------
async fn list_routes<RR: RouteRepository>(
State(state): State<RouteAdminState<RR>>,
Path(script_id): Path<ScriptId>,
) -> Result<Json<Vec<Route>>, RouteApiError> {
Ok(Json(state.routes.list_for_script(script_id).await?))
}
async fn create_route<RR: RouteRepository>(
State(state): State<RouteAdminState<RR>>,
Path(script_id): Path<ScriptId>,
Json(input): Json<CreateRouteRequest>,
) -> Result<(StatusCode, Json<Route>), RouteApiError> {
let normalized_path = parse_and_normalize_path(input.path_kind, &input.path)?;
pattern::parse_host(
input.host_kind,
&input.host,
input.host_param_name.as_deref(),
)?;
// Within-kind conflict check against existing routes.
let existing = state.routes.list_all().await?;
if let Some((conflicting, reason)) = first_conflict(
&existing,
input.host_kind,
&input.host,
input.path_kind,
&normalized_path,
input.method.as_deref(),
)? {
return Err(RouteApiError::Conflict {
conflicting_route: Box::new(conflicting),
reason,
});
}
let created = state
.routes
.create(NewRoute {
script_id,
host_kind: input.host_kind,
host: input.host,
host_param_name: input.host_param_name,
path_kind: input.path_kind,
path: normalized_path,
method: input.method,
})
.await?;
refresh_table(&state).await?;
Ok((StatusCode::CREATED, Json(created)))
}
async fn delete_route<RR: RouteRepository>(
State(state): State<RouteAdminState<RR>>,
Path(route_id): Path<Uuid>,
) -> Result<StatusCode, RouteApiError> {
state.routes.delete(route_id).await?;
refresh_table(&state).await?;
Ok(StatusCode::NO_CONTENT)
}
async fn check_route<RR: RouteRepository>(
State(state): State<RouteAdminState<RR>>,
Json(input): Json<CheckRouteRequest>,
) -> Result<Json<CheckRouteResponse>, RouteApiError> {
let normalized_path = parse_and_normalize_path(input.path_kind, &input.path)?;
pattern::parse_host(input.host_kind, &input.host, None)?;
let existing = state.routes.list_all().await?;
let conflict = first_conflict(
&existing,
input.host_kind,
&input.host,
input.path_kind,
&normalized_path,
input.method.as_deref(),
)?;
Ok(Json(match conflict {
None => CheckRouteResponse {
ok: true,
conflicting_route: None,
conflict_reason: None,
},
Some((route, reason)) => CheckRouteResponse {
ok: false,
conflicting_route: Some(route),
conflict_reason: Some(reason),
},
}))
}
async fn match_route<RR: RouteRepository>(
State(state): State<RouteAdminState<RR>>,
Json(input): Json<MatchRouteRequest>,
) -> Result<Json<MatchRouteResponse>, RouteApiError> {
let parsed = url::Url::parse(&input.url)
.map_err(|e| RouteApiError::BadRequest(format!("invalid url: {e}")))?;
let host = parsed.host_str().unwrap_or("").to_string();
let path = parsed.path().to_string();
let result = state.table.match_request(&host, &input.method, &path);
Ok(Json(MatchRouteResponse {
matched: result.map(|r| MatchedRoute {
route_id: r.matched.route_id,
script_id: r.matched.script_id,
params: r.params,
rest: r.rest,
host_param: r.host_param,
}),
}))
}
// ----------------------------------------------------------------------------
// Helpers
// ----------------------------------------------------------------------------
/// Validate the raw user-typed path string and return it verbatim if
/// it parses cleanly. Prefix normalization (`/echo/*` → `/echo/`)
/// happens only in memory at compile time; persisted strings stay in
/// the form the user submitted so re-parses are idempotent.
fn parse_and_normalize_path(kind: PathKind, raw: &str) -> Result<String, pattern::ParseError> {
pattern::parse_path(kind, raw)?;
Ok(raw.to_string())
}
#[allow(clippy::type_complexity)]
fn first_conflict(
existing: &[Route],
host_kind: HostKind,
host: &str,
path_kind: PathKind,
path: &str,
method: Option<&str>,
) -> Result<Option<(Route, String)>, RouteApiError> {
let new_host = pattern::parse_host(host_kind, host, None)?;
let new_path = pattern::parse_path(path_kind, path)?;
for r in existing {
let r_host = pattern::parse_host(r.host_kind, &r.host, r.host_param_name.as_deref())?;
if !conflict::hosts_overlap(&new_host, &r_host) {
continue;
}
if !conflict::methods_overlap(method, r.method.as_deref()) {
continue;
}
let r_path = pattern::parse_path(r.path_kind, &r.path)?;
if let Some(reason) = conflict::conflicts(&new_path, &r_path) {
return Ok(Some((r.clone(), format!("{reason:?}"))));
}
}
Ok(None)
}
async fn refresh_table<RR: RouteRepository>(
state: &RouteAdminState<RR>,
) -> Result<(), RouteApiError> {
let rows = state.routes.list_all().await?;
let compiled = compile_routes(&rows)?;
state.table.replace(compiled);
Ok(())
}
pub fn compile_routes(rows: &[Route]) -> Result<Vec<CompiledRoute>, pattern::ParseError> {
rows.iter()
.map(|r| {
Ok(CompiledRoute {
route_id: r.id,
script_id: r.script_id,
host: pattern::parse_host(r.host_kind, &r.host, r.host_param_name.as_deref())?,
path: pattern::parse_path(r.path_kind, &r.path)?,
method: r.method.clone(),
})
})
.collect()
}
// ----------------------------------------------------------------------------
// Errors
// ----------------------------------------------------------------------------
#[derive(Debug, thiserror::Error)]
pub enum RouteApiError {
#[error("route conflicts with existing route ({reason})")]
Conflict {
conflicting_route: Box<Route>,
reason: String,
},
#[error("invalid route: {0}")]
Pattern(#[from] pattern::ParseError),
#[error("bad request: {0}")]
BadRequest(String),
#[error("repository error: {0}")]
Repo(#[from] ScriptRepositoryError),
}
impl IntoResponse for RouteApiError {
fn into_response(self) -> Response {
let (status, body) = match &self {
Self::Conflict {
conflicting_route,
reason,
} => (
StatusCode::CONFLICT,
serde_json::json!({
"error": self.to_string(),
"reason": reason,
"conflicting_route": &**conflicting_route,
}),
),
Self::Pattern(_) | Self::BadRequest(_) => (
StatusCode::UNPROCESSABLE_ENTITY,
serde_json::json!({ "error": self.to_string() }),
),
Self::Repo(ScriptRepositoryError::NotFound(_)) => (
StatusCode::NOT_FOUND,
serde_json::json!({ "error": self.to_string() }),
),
Self::Repo(ScriptRepositoryError::Conflict(_)) => (
StatusCode::CONFLICT,
serde_json::json!({ "error": self.to_string() }),
),
Self::Repo(ScriptRepositoryError::Db(e)) => {
tracing::error!(error = %e, "route admin db error");
(
StatusCode::INTERNAL_SERVER_ERROR,
serde_json::json!({ "error": "internal error" }),
)
}
};
(status, Json(body)).into_response()
}
}

168
crates/manager-core/src/route_repo.rs Normal file
View File

@@ -0,0 +1,168 @@
//! CRUD over the `routes` table.
//!
//! The orchestrator's `RouteTable` is repopulated from this repo after
//! every write — see the route_admin module for the binding.
use async_trait::async_trait;
use picloud_shared::{HostKind, PathKind, Route, ScriptId};
use sqlx::PgPool;
use uuid::Uuid;
use crate::repo::ScriptRepositoryError;
#[derive(Debug, Clone)]
pub struct NewRoute {
pub script_id: ScriptId,
pub host_kind: HostKind,
pub host: String,
pub host_param_name: Option<String>,
pub path_kind: PathKind,
pub path: String,
pub method: Option<String>,
}
#[async_trait]
pub trait RouteRepository: Send + Sync {
async fn list_all(&self) -> Result<Vec<Route>, ScriptRepositoryError>;
async fn list_for_script(
&self,
script_id: ScriptId,
) -> Result<Vec<Route>, ScriptRepositoryError>;
async fn create(&self, input: NewRoute) -> Result<Route, ScriptRepositoryError>;
async fn delete(&self, route_id: Uuid) -> Result<(), ScriptRepositoryError>;
}
pub struct PostgresRouteRepository {
pool: PgPool,
}
impl PostgresRouteRepository {
#[must_use]
pub fn new(pool: PgPool) -> Self {
Self { pool }
}
}
#[async_trait]
impl RouteRepository for PostgresRouteRepository {
async fn list_all(&self) -> Result<Vec<Route>, ScriptRepositoryError> {
let rows = sqlx::query_as::<_, RouteRow>(
"SELECT id, script_id, host_kind, host, host_param_name, \
path_kind, path, method, created_at \
FROM routes ORDER BY created_at",
)
.fetch_all(&self.pool)
.await?;
Ok(rows.into_iter().map(Into::into).collect())
}
async fn list_for_script(
&self,
script_id: ScriptId,
) -> Result<Vec<Route>, ScriptRepositoryError> {
let rows = sqlx::query_as::<_, RouteRow>(
"SELECT id, script_id, host_kind, host, host_param_name, \
path_kind, path, method, created_at \
FROM routes WHERE script_id = $1 ORDER BY created_at",
)
.bind(script_id.into_inner())
.fetch_all(&self.pool)
.await?;
Ok(rows.into_iter().map(Into::into).collect())
}
async fn create(&self, input: NewRoute) -> Result<Route, ScriptRepositoryError> {
let res = sqlx::query_as::<_, RouteRow>(
"INSERT INTO routes ( \
script_id, host_kind, host, host_param_name, \
path_kind, path, method \
) VALUES ($1, $2, $3, $4, $5, $6, $7) \
RETURNING id, script_id, host_kind, host, host_param_name, \
path_kind, path, method, created_at",
)
.bind(input.script_id.into_inner())
.bind(host_kind_str(input.host_kind))
.bind(&input.host)
.bind(input.host_param_name.as_deref())
.bind(path_kind_str(input.path_kind))
.bind(&input.path)
.bind(input.method.as_deref())
.fetch_one(&self.pool)
.await;
match res {
Ok(row) => Ok(row.into()),
Err(sqlx::Error::Database(e)) if e.is_unique_violation() => Err(
ScriptRepositoryError::Conflict("a route with this binding already exists".into()),
),
Err(sqlx::Error::Database(e)) if e.is_foreign_key_violation() => {
Err(ScriptRepositoryError::NotFound(input.script_id))
}
Err(e) => Err(e.into()),
}
}
async fn delete(&self, route_id: Uuid) -> Result<(), ScriptRepositoryError> {
let res = sqlx::query("DELETE FROM routes WHERE id = $1")
.bind(route_id)
.execute(&self.pool)
.await?;
if res.rows_affected() == 0 {
return Err(ScriptRepositoryError::NotFound(ScriptId::from(route_id)));
}
Ok(())
}
}
const fn host_kind_str(k: HostKind) -> &'static str {
match k {
HostKind::Any => "any",
HostKind::Strict => "strict",
HostKind::Wildcard => "wildcard",
}
}
const fn path_kind_str(k: PathKind) -> &'static str {
match k {
PathKind::Exact => "exact",
PathKind::Prefix => "prefix",
PathKind::Param => "param",
}
}
#[derive(sqlx::FromRow)]
struct RouteRow {
id: Uuid,
script_id: Uuid,
host_kind: String,
host: String,
host_param_name: Option<String>,
path_kind: String,
path: String,
method: Option<String>,
created_at: chrono::DateTime<chrono::Utc>,
}
impl From<RouteRow> for Route {
fn from(r: RouteRow) -> Self {
Self {
id: r.id,
script_id: r.script_id.into(),
host_kind: match r.host_kind.as_str() {
"strict" => HostKind::Strict,
"wildcard" => HostKind::Wildcard,
_ => HostKind::Any,
},
host: r.host,
host_param_name: r.host_param_name,
path_kind: match r.path_kind.as_str() {
"prefix" => PathKind::Prefix,
"param" => PathKind::Param,
_ => PathKind::Exact,
},
path: r.path,
method: r.method,
created_at: r.created_at,
}
}
}