80fbf2e64293f9c981196c98fd0b5d778d651e63
5 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
MechaCat02
|
b5ce4cec01 |
ci: run the DB-backed tests that CI was silently skipping
CI ran `cargo test --workspace` with no `--include-ignored`, so it executed 927 tests and skipped 237 — every DB-backed integration test is `#[ignore = "needs DATABASE_URL..."]`, which covers ALL of api.rs, ALL of authz.rs, and the entire CLI journey suite. The isolation and RBAC tests existed but never ran (AUDIT.md F-Q-014, logged and never remediated). CI provides Postgres, so it can run them. Three things had to be right for that to go green: - **`--all-targets`, not a bare workspace run.** `-- --include-ignored` un-ignores not just `#[ignore]` tests but also ` ```ignore ` DOCTESTS, which are illustrative pseudocode that does not compile. `--all-targets` runs lib/bins/ integration tests but excludes doctests (the same reason clippy uses it); a separate `--doc` step runs the doctests without the flag. Structural, so a future pseudocode doctest can't silently break CI either. - **The CLI journeys are their own step.** They spawn a real picloud whose dispatcher/orchestrator claim loops are global by design; run concurrently with the manager-core suites on the shared database they would claim those suites' outbox and workflow rows. Sequential steps keep the live server off the DB while the other suites use it. The step also rebuilds `-p picloud` first (the harness execs the prebuilt binary) and sets the dev-mode env the server needs. - **A higher `max_connections`.** `#[sqlx::test]` pools are lazy, but mass-parallel test startup briefly opens many at once (each test creates its own throwaway database); on a many-core box that transient spike exceeded the default 100 and Postgres answered "sorry, too many clients already". Steady-state peak is only ~26; 500 absorbs the spike with room to spare. Serving the app needs nothing like this many. (This is the local compose ceiling; a small CI runner's default 100 has ample headroom for its lower parallelism.) Also fixes the test the CI gap had let rot: api.rs asserted `v["schema"] == 66` with a hand-bump comment, and since nothing ran it, it sat broken from migration 0066 to 0073. It now asserts `/version` surfaces the live constants (`migrations::latest_version()`, `SDK_VERSION`) — the wiring — while value drift stays caught by schema_snapshot + check-versioning. A constant hand-synced to another constant is a chore, not a test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
MechaCat02
|
64799b73ff |
chore(docker): caddy restart unless-stopped
Other services in the prod overlay already have it. Without it, a `docker compose stop caddy` followed by `docker compose up -d` doesn't bring caddy back up. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
MechaCat02
|
ee0dbc428f |
chore(compose): require bootstrap admin env vars instead of defaulting to admin/admin
The previous interpolation used `${PICLOUD_ADMIN_USERNAME:-admin}` and
`${PICLOUD_ADMIN_PASSWORD:-admin}`, which made docker compose silently
bootstrap a production stack with `admin`/`admin` whenever the operator
forgot to set them. Flip to `${VAR:?…}` so an unset value aborts
`docker compose up` with a clear "set this var" message; dev still gets
the convenient default through the gitignored `.env` (documented in
`.env.example`).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
MechaCat02
|
07e2a62d98 |
feat: custom routing — bind scripts to your own URLs
Scripts can now answer at user-chosen paths (e.g. /greet, /greet/:name,
/webhooks/*), on user-chosen hosts (strict or *.example.com wildcards),
on user-chosen methods. The internal /api/v1/execute/{id} endpoint
stays as the always-available ID-based bypass.
Routing rules (decided in design with the user; see chat history):
Path kinds:
exact /greet literal
prefix /greet/* strict-subtree; stored as "/greet/";
does NOT match bare /greet (add an
exact route for that case)
param /users/:id :name captures one whole segment;
mid-segment colons are rejected;
{name} is reserved for a future SDK
Host kinds:
any no Host header constraint
strict sub.example.com literal match (case-insensitive)
wildcard *.example.com suffix match; multi-level subdomains OK
Within-kind uniqueness:
two routes of the same kind that could match the same request
conflict at config time. Algorithm (orchestrator_core::routing::
conflict):
exact: literal equality
prefix: literal equality (longer-prefix coexists; longer wins
at request time)
param: same segment count + same literals at every
literal-vs-literal position (the user's example:
:id vs :userId at same shape is a conflict)
Request-time precedence:
exact > param > prefix
among non-exact: more leading-literal segments wins
tie: param > prefix (more constrained)
within prefix: longest matching prefix wins
host bucket: strict > wildcard (longer suffix) > any; fall through
to less specific buckets when path doesn't match
Reserved path prefixes: /api/, /admin/, /healthz, /version
Routes that look invalid at config time return 422 with the precise
parse error; conflicting routes return 409 with the conflicting route
in the body (so the dashboard can render the conflict inline).
What landed:
* 0003_routes.sql — routes table (host_kind, host, host_param_name,
path_kind, path, method, script_id) with UNIQUE index on the
literal binding tuple. Schema 2 → 3.
* shared::Route / HostKind / PathKind — flat storage shape that
crosses wire boundaries cleanly.
* orchestrator_core::routing — four sub-modules, all unit-tested:
pattern.rs (16 tests) parse + validate + display
conflict.rs (12 tests) within-kind overlap predicate
matcher.rs (12 tests) runtime dispatch (specificity-aware)
table.rs Arc<RwLock<Vec<CompiledRoute>>>
shared by manager (writes) and
orchestrator (reads); atomic replace
after each admin write
* manager-core::route_admin — five new admin endpoints under
/api/v1/admin:
POST /scripts/{id}/routes create
GET /scripts/{id}/routes list per script
DELETE /routes/{route_id} delete (refreshes table)
POST /routes:check pre-flight conflict check
(powers the dashboard's
live conflict warning)
POST /routes:match synthetic URL → matched
route + extracted params
(powers the dashboard's
match-preview tool)
Stored path strings stay raw (user-typed); normalization
happens only in the in-memory CompiledRoute so re-parses are
idempotent.
* orchestrator_core::api::user_routes_router — fallback handler
mounted in picloud after the system routes. Reads Host /
method / path / query from the request, dispatches via the
table, builds an ExecRequest with params/query/rest filled,
calls the executor, writes to the log sink. 10 MiB body cap.
* executor-core::ctx (SDK 1.0 → 1.1) — adds
ctx.request.params (map of named-param captures)
ctx.request.query (parsed query string)
ctx.request.rest (suffix for prefix routes; "" otherwise)
All three are always present (empty when not applicable) so
scripts can read them unconditionally.
* picloud::build_app — now async; loads routes at startup,
populates the shared table, mounts route_admin_router under
/api/v1/admin alongside the script CRUD, and the user-routes
fallback at the app root.
* caddy/Caddyfile + Caddyfile.prod widened: anything not
/healthz, /version, /api/v1/admin/*, /api/v1/execute/*,
/api/* (404 sunset), or /admin/* (dashboard) → picloud.
* Dashboard moves to /admin/* via SvelteKit paths.base. Its
internal Caddy strips the prefix and serves with SPA fallback.
All in-app links use $app/paths. The dashboard URL is now
http://localhost:8000/admin/ — one-time break for the new
URL freedom users gained.
* PICLOUD_PUBLIC_BASE_URL env var, exposed via /version so the
dashboard renders full URLs for routes regardless of the
operator's external port / TLS setup.
* memory_limit_mb stays in the schema, still v1.3+ advisory.
Verified live through Caddy:
/version → schema 3, sdk 1.1, public_base_url
GET /admin/ → 200, dashboard HTML containing "PiCloud"
POST /api/v1/admin/scripts → 201
POST .../scripts/{id}/routes (path=/greet/:name) → 201
GET /greet/alice?lang=en → 200 {"name":"alice","q":"en"}
POST conflicting route → 409 with conflicting_route body
POST /admin/foo route → 422 "reserved"
POST /api/v1/admin/routes:match → matched + params extracted
GET /unbound-path → 404 JSON
Tests:
* 40 routing unit tests (pattern + conflict + matcher tables)
* 14 executor-core unit tests (one new for ctx.request.params/
query/rest exposure)
* 32 integration tests (10 new for routing CRUD + dispatch +
conflict + reserved + specificity tie-break + match preview +
delete invalidation + /version returns public_base_url)
* default cargo test --workspace stays green; opt-in via
DATABASE_URL + --include-ignored for the integration suite
Bumps: schema 2 → 3; SDK 1.0 → 1.1; product 0.3.0 → 0.4.0.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
MechaCat02
|
9efe678983 |
feat(compose): full-stack Caddy + docker-compose wiring
Brings up the whole platform behind a single Caddy entrypoint so the
routing topology can be exercised end-to-end before any feature code
lands. Same Caddyfile shape (admin / data plane / dashboard) maps to
single-process MVP today and will map to cluster mode later by
swapping the upstream lists, not by restructuring the proxy.
* caddy/Caddyfile — dev: HTTP only, picloud and dashboard upstreams
by service name. caddy/Caddyfile.prod — Let's Encrypt for
PICLOUD_DOMAIN with PICLOUD_ADMIN_EMAIL.
* docker/orchestrator.Dockerfile — multi-stage build of the
`picloud` all-in-one against the pinned 1.92 toolchain; debian
slim runtime, non-root user, /healthz HEALTHCHECK.
* docker/dashboard.Dockerfile — node:24-alpine builder + caddy
runtime that serves the static SPA with SPA fallback.
* docker-compose.yml — postgres + picloud + dashboard + caddy,
Caddy exposed on host :8000 (configurable), Postgres on :15432
(loopback only). Health-gated startup ordering.
* docker-compose.prod.yml — overlay: removes Postgres host
mapping, expands Caddy to 80/443/443udp, swaps Caddyfile.prod,
adds restart policy.
* .env.example documents every knob the compose stack reads.
Verified via `docker compose up -d`:
* `curl :8000/healthz` → 200 ok (orchestrator)
* `curl :8000/api/admin/scripts` → 404 (manager, routed correctly)
* `curl :8000/api/execute/<id>` → 404 (orchestrator, routed correctly)
* `curl :8000/` → SPA index served (dashboard via Caddy)
* `curl :8000/favicon.svg` → 200 image/svg+xml
* Postgres healthy and reachable on 127.0.0.1:15432.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|