Files

MechaCat02 07e2a62d98 feat: custom routing — bind scripts to your own URLs

Scripts can now answer at user-chosen paths (e.g. /greet, /greet/:name,
/webhooks/*), on user-chosen hosts (strict or *.example.com wildcards),
on user-chosen methods. The internal /api/v1/execute/{id} endpoint
stays as the always-available ID-based bypass.

Routing rules (decided in design with the user; see chat history):

  Path kinds:
    exact   /greet              literal
    prefix  /greet/*            strict-subtree; stored as "/greet/";
                                does NOT match bare /greet (add an
                                exact route for that case)
    param   /users/:id          :name captures one whole segment;
                                mid-segment colons are rejected;
                                {name} is reserved for a future SDK

  Host kinds:
    any                         no Host header constraint
    strict  sub.example.com     literal match (case-insensitive)
    wildcard *.example.com      suffix match; multi-level subdomains OK

  Within-kind uniqueness:
    two routes of the same kind that could match the same request
    conflict at config time. Algorithm (orchestrator_core::routing::
    conflict):
      exact:  literal equality
      prefix: literal equality (longer-prefix coexists; longer wins
              at request time)
      param:  same segment count + same literals at every
              literal-vs-literal position (the user's example:
              :id vs :userId at same shape is a conflict)

  Request-time precedence:
    exact > param > prefix
    among non-exact: more leading-literal segments wins
    tie: param > prefix (more constrained)
    within prefix: longest matching prefix wins
    host bucket: strict > wildcard (longer suffix) > any; fall through
    to less specific buckets when path doesn't match

  Reserved path prefixes: /api/, /admin/, /healthz, /version

  Routes that look invalid at config time return 422 with the precise
  parse error; conflicting routes return 409 with the conflicting route
  in the body (so the dashboard can render the conflict inline).

What landed:

  * 0003_routes.sql — routes table (host_kind, host, host_param_name,
    path_kind, path, method, script_id) with UNIQUE index on the
    literal binding tuple. Schema 2 → 3.

  * shared::Route / HostKind / PathKind — flat storage shape that
    crosses wire boundaries cleanly.

  * orchestrator_core::routing — four sub-modules, all unit-tested:
      pattern.rs (16 tests)  parse + validate + display
      conflict.rs (12 tests) within-kind overlap predicate
      matcher.rs (12 tests)  runtime dispatch (specificity-aware)
      table.rs               Arc<RwLock<Vec<CompiledRoute>>>
                             shared by manager (writes) and
                             orchestrator (reads); atomic replace
                             after each admin write

  * manager-core::route_admin — five new admin endpoints under
    /api/v1/admin:
      POST   /scripts/{id}/routes      create
      GET    /scripts/{id}/routes      list per script
      DELETE /routes/{route_id}        delete (refreshes table)
      POST   /routes:check             pre-flight conflict check
                                       (powers the dashboard's
                                       live conflict warning)
      POST   /routes:match             synthetic URL → matched
                                       route + extracted params
                                       (powers the dashboard's
                                       match-preview tool)
    Stored path strings stay raw (user-typed); normalization
    happens only in the in-memory CompiledRoute so re-parses are
    idempotent.

  * orchestrator_core::api::user_routes_router — fallback handler
    mounted in picloud after the system routes. Reads Host /
    method / path / query from the request, dispatches via the
    table, builds an ExecRequest with params/query/rest filled,
    calls the executor, writes to the log sink. 10 MiB body cap.

  * executor-core::ctx (SDK 1.0 → 1.1) — adds
      ctx.request.params  (map of named-param captures)
      ctx.request.query   (parsed query string)
      ctx.request.rest    (suffix for prefix routes; "" otherwise)
    All three are always present (empty when not applicable) so
    scripts can read them unconditionally.

  * picloud::build_app — now async; loads routes at startup,
    populates the shared table, mounts route_admin_router under
    /api/v1/admin alongside the script CRUD, and the user-routes
    fallback at the app root.

  * caddy/Caddyfile + Caddyfile.prod widened: anything not
    /healthz, /version, /api/v1/admin/*, /api/v1/execute/*,
    /api/* (404 sunset), or /admin/* (dashboard) → picloud.

  * Dashboard moves to /admin/* via SvelteKit paths.base. Its
    internal Caddy strips the prefix and serves with SPA fallback.
    All in-app links use $app/paths. The dashboard URL is now
    http://localhost:8000/admin/ — one-time break for the new
    URL freedom users gained.

  * PICLOUD_PUBLIC_BASE_URL env var, exposed via /version so the
    dashboard renders full URLs for routes regardless of the
    operator's external port / TLS setup.

  * memory_limit_mb stays in the schema, still v1.3+ advisory.

Verified live through Caddy:
  /version              → schema 3, sdk 1.1, public_base_url
  GET /admin/           → 200, dashboard HTML containing "PiCloud"
  POST /api/v1/admin/scripts → 201
  POST .../scripts/{id}/routes (path=/greet/:name) → 201
  GET /greet/alice?lang=en → 200 {"name":"alice","q":"en"}
  POST conflicting route → 409 with conflicting_route body
  POST /admin/foo route → 422 "reserved"
  POST /api/v1/admin/routes:match → matched + params extracted
  GET /unbound-path → 404 JSON

Tests:
  * 40 routing unit tests (pattern + conflict + matcher tables)
  * 14 executor-core unit tests (one new for ctx.request.params/
    query/rest exposure)
  * 32 integration tests (10 new for routing CRUD + dispatch +
    conflict + reserved + specificity tie-break + match preview +
    delete invalidation + /version returns public_base_url)
  * default cargo test --workspace stays green; opt-in via
    DATABASE_URL + --include-ignored for the integration suite

Bumps: schema 2 → 3; SDK 1.0 → 1.1; product 0.3.0 → 0.4.0.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2026-05-23 18:18:16 +02:00

8.0 KiB

Raw Blame History

Versioning

PiCloud carries one product version for the build you install, and independent versions on the four contracts that actually break for users. The product version answers "which build do I have"; surface versions answer "which contracts does that build honor".

This split exists because crate-level SemVer between, say, picloud-shared and picloud-manager-core is fiction — they always ship together. The boundaries that matter are user-facing: scripts depending on the SDK, callers hitting the HTTP API, databases shared across deploys, and (later) executor nodes talking to a manager.

What gets a version

Lockstep — one number for the whole thing

All of these carry the same version and are bumped together:

Every crate in the Cargo workspace (via version.workspace = true)
The dashboard's package.json
Docker image tags (picloud:0.2.0)
Git tags (v0.2.0)

Defined once in Cargo.toml under [workspace.package]. There is no scenario where one crate is at a different version than another in the same build.

Independent — versioned at each surface

Surface	Where the version lives	Format	Bump rule
Rhai SDK	`shared::version::SDK_VERSION`, exposed to scripts as `ctx.sdk_version`	`"major.minor"` string	Minor: additions; Major: removals/renames/retyped
HTTP API	URL prefix `/api/v{N}/...`; `shared::version::API_VERSION` is the current major	integer	New integer when request/response shape, status semantics, or auth model changes
Database schema	Largest applied migration ID (`manager-core::migrations::latest_version()`)	integer, monotonic	One per forward migration; never edit a committed file
Inter-service wire (cluster mode, v1.3+)	`X-PiCloud-Wire` request header; `shared::version::WIRE_VERSION`	integer	New integer when RPC shape changes

All five live in one place so /version can return them honestly.

Per-surface compatibility rules

Rhai SDK (strictest)

Scripts run in production with no recompile. A wrong SDK bump silently breaks user code.

Patch (1.2.0 → 1.2.1) — doc fixes, internal optimizations. No script-observable change.
Minor (1.2 → 1.3) — added functions; added optional ctx.* fields; relaxed limits; new variants accepted alongside old ones. Every script written for 1.2 must still run unchanged on 1.3.
Major (1 → 2) — anything removed, renamed, retyped, restricted, or made required.

Scripts can detect available features at runtime:

if ctx.sdk_version >= "1.2" {
    // call kv.* (added in 1.2)
}

The contract test in crates/executor-core/tests/sdk_contract/ (coming alongside the first SDK additions) holds golden scripts that exercise every documented SDK surface. They must pass on every commit. A minor bump that breaks any of them is a build failure.

HTTP API

Path prefix is the version. Within a major, the following are non-breaking and welcome:

New endpoints
New optional request fields
New response fields (clients must ignore unknown fields)
New Deprecation: headers warning of upcoming removals

The following require a new major (/api/v2/...):

Removed endpoints, removed response fields, renamed fields
Changed request-field types or required-field additions
Changed status-code semantics for the same outcome
Auth model changes

When vN+1 ships, vN stays live for at least one product minor (so users have a release cycle to migrate). Deprecation is announced via the Deprecation: true and Sunset: <date> response headers on the old prefix before removal.

Database schema

Forward-only. Never edit a migration that has shipped. If a migration was wrong, write a new one that fixes it.
Migrations are numbered sequentially (0001_init.sql, 0002_*.sql, ...). The number is the schema version.
A given binary applies migrations strictly greater than the last-applied ID, then refuses to start if its embedded migrations are older than what's in the DB — that would imply a downgrade, which is never automatic.
This makes rolling deploys safe: the schema is always "ahead of or equal to" any running binary in the cluster.

Wire protocol (cluster mode, v1.3+)

Inter-service RPCs include X-PiCloud-Wire: N.
A peer that doesn't recognize N refuses the call and returns 426 Upgrade Required with the version it speaks.
Both versions must be live in the cluster during rolling upgrades — current and current-minus-one — until all nodes agree on the new one.

How we check and enforce

A versioning scheme without enforcement decays in months. Five cheap mechanical checks:

Compile-time uniformity. All workspace crates inherit version.workspace = true. Drift is impossible to introduce.
Runtime self-report. GET /version returns every surface version. Dashboards, monitoring, inter-service handshakes, and humans all read from one source. /healthz stays a plain "ok" string for k8s probes — version negotiation is a separate concern.
Golden SDK contract tests. tests/sdk_contract/ Rhai scripts exercise every SDK surface and must pass on every commit. The contract is the test.
Migration replay test. An integration test that boots a fresh Postgres, applies every migration in order, and asserts the resulting schema. Catches the most common mistake (edited-not-added migration).
CI guardrail script. A small diff-aware check that:
- Fails if SDK_VERSION's major changed without a CHANGELOG.md breaking-change entry
- Fails if a new file appeared in migrations/ that isn't the next sequential number
- Fails if a route handler removed or retyped a public field without a BREAKING: line in the commit message

(3) through (5) are wired in over the next few PRs; (1) and (2) land in the same commit as this document.

When to bump what

The product version follows SemVer applied pragmatically — we're pre-1.0, so the rules are looser:

Patch (0.2.0 → 0.2.1) — bug fixes, no surface change
Minor (0.2 → 0.3) — any surface bump, new features, or breaking changes (pre-1.0 license)
Major (0 → 1) — first stable release; SDK and API both committed to long-term compatibility

After 1.0, the product version follows strict SemVer based on the worst surface change:

Any surface major bump → product major bump
Any surface minor bump → product minor bump (at minimum)
No surface changes → product patch

A surface can hit its own 1.0 independently of the product. The SDK in particular is likely to stabilize before the platform does, since scripts in production demand it.

Current versions

	Version
Product	`0.4.0`
SDK	`1.1` (adds `ctx.request.params`, `ctx.request.query`, `ctx.request.rest`)
API	`1`
Schema	`3` (matches `migrations/0003_routes.sql`)
Wire	`1` (reserved; cluster mode not implemented)

Read live from GET /version on any running instance.

Examples

Adding a kv.* SDK in v1.1+:

Workspace bump: 0.2.0 → 0.3.0 (pre-1.0 minor)
SDK bump: "1.0" → "1.1" (added functions only)
API bump: none (no new endpoints affect existing API contract)
Schema bump: 1 → 2 (0002_kv_store.sql adds the kv_store table)

Renaming ctx.execution_id to ctx.exec_id:

SDK bump: "1.x" → "2.0" (breaking)
Product: minor bump pre-1.0, major bump post-1.0
Migration path: keep ctx.execution_id available in 1.x for a deprecation window, add ctx.exec_id alongside; flip to 2.0 only when both fields have shipped together for a release.

Adding pagination to GET /api/v1/admin/scripts:

New optional ?limit=&offset= query params with sensible defaults → no API bump
Response keeps the same shape; clients that don't pass limit see the old behavior → no API bump

Changing the response shape of GET /api/v1/admin/scripts/{id} to wrap in { script: {...} }:

Breaking. Ship as /api/v2/admin/scripts/{id}. Keep /api/v1 live until at least one product minor passes.

8.0 KiB Raw Blame History