Add a server-side, atomic, declarative reconcile loop for a single app —
the foundation of the project-tool design. Developers describe an app's
scripts, routes, triggers, and secret-names in `picloud.toml`, then
`pic pull / plan / apply [--prune]` to converge live state to the manifest.
Server (manager-core):
- apply_service: a pure diff engine (compute_diff) shared by plan and
apply, plus an ApplyService that composes the existing per-repo writes
into ONE Postgres transaction. Identity keys mirror the DB UNIQUE
constraints (script=lower(name); route=(method,host_kind,host,
path_kind,path); trigger=per-kind semantic tuple; secret=name).
Apply takes a per-app advisory lock, recomputes the diff in-tx, applies
scripts -> routes -> triggers, prunes dependents-first, commits, then
refreshes the route table once post-commit.
- apply_api: POST /apps/{id}/plan (AppRead) and /apps/{id}/apply.
Apply requires the per-kind write caps the bundle exercises (all three
when --prune), plus AppSecretsRead when it binds an email trigger.
- tx-accepting repo siblings (insert/update/delete *_tx) so the existing
create/update/delete delegate to one SQL definition each.
- email triggers reference an inbound secret by NAME; the value is
resolved, decrypted (AAD-bound), and re-sealed server-side at apply —
it never travels in the manifest.
CLI (picloud-cli):
- manifest.rs (picloud.toml model), client plan/apply, and the pull/plan/
apply commands. pull rejects filesystem-unsafe script names up front.
Safety properties enforced and tested:
- idempotent: a freshly-pulled manifest re-applies as all-NoOp.
- atomic: a mid-bundle failure rolls back with nothing written.
- routes delete-before-insert so a freed binding is reusable in one apply.
- queue one-consumer invariant held inside the shared tx.
- email triggers are never pruned, and a script that still owns an
email/dead-letter trigger can't be pruned (the FK cascade would destroy
the sealed secret) — refused with a pointer to `pic triggers rm`.
- plan and apply agree on unset email-secret references.
No migration: the existing schema's UNIQUE constraints serve as identity
keys. Groups, env-scoping, and the `enabled` toggle are later milestones.
Tested: manager-core lib (360) + CLI bins (27) + 8 project-tool journeys
(pull/plan/apply/prune/email+queue), all green; clippy -D warnings clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Closes the gaps and the one security finding from the second end-to-end
CLI test (E2E_STASH_REPORT.md), plus the H1 boot-regression found while
re-reviewing those fixes.
Security
- S6: reserved-path validation (`check_reserved`) now case-folds before
comparing, so `/API/v2/x`, `/HEALTHZ`, `/Admin/x` are rejected like
their lowercase forms. Request-time matching stays case-sensitive.
- S10: "public route != public data" callout in sdk-shape.md (script_gate
skips authz when the principal is anonymous).
Observability / features
- G1: trigger executions now write `execution_logs`. Migration 0043 adds
a `source` column (CHECK mirrors ExecutionSource/OutboxSourceKind,
DEFAULT 'http' backfills history); a shared `build_execution_log` helper
in executor-core; dispatcher logging for outbox triggers + queue
consumers (skips sync-HTTP rows the orchestrator already logs). `pic
logs` gains a source column + `--source` filter.
- G5: dev-only in-memory email capture under PICLOUD_DEV_MODE with no SMTP
(email::send succeeds locally), readable at GET /api/v1/admin/dev/emails
(Owner/Admin only; route mounted only in capture mode).
- G6: generalized the Rhai in-place-mutation footgun note (trim/replace/
make_upper/make_lower/crop/truncate/pad return ()).
- G2/G3/G4 (CLI): `pic members`, `pic files`, `pic queues`, read-only
`pic kv` (+ new kv_api.rs); `pic deploy --timeout/--memory/--kind/
--sandbox`; first-class `pic triggers create-{docs,files,pubsub,queue,
email}` wrappers. All new client path segments percent-encoded via seg().
H1 regression fix (found in re-review)
- The S6 change also runs in `compile_routes`, which compiles every stored
route at boot and on each route CRUD. A single stored route the new
validation rejects (creatable while the S6 gap existed) made the whole
compile Err and aborted startup. `compile_routes` is now lenient: it
skips an un-compilable row with a warning instead of bricking boot
(route creation still validates separately). Migration 0044 sweeps
pre-existing reserved-path routes on upgrade (WHERE mirrors
check_reserved exactly). Added regression tests for both.
Verified: cargo fmt, clippy --all-targets --all-features -D warnings, the
schema_snapshot test, and the new S6/lenient-compile unit tests all pass;
boot-resilience and G1/G5 confirmed live.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The login handler had no admission control before
`spawn_blocking(verify_password)`. On Pi-class hardware Argon2id is ~50-
100 ms per attempt, so a few dozen concurrent anonymous POSTs to
/auth/login park every blocking worker, wedging the entire admin API
and any other path that uses `spawn_blocking`.
Adds two cheap, in-process guards modeled on EmailRateLimiter:
* `LoginRateLimiter` — two sliding-window token buckets, one per
`(remote_ip, username)` (burst 5/60s) and one per `username`
(burst 10/15min). Per-(ip, user) defeats a single attacker pounding
one account; per-user defeats credential-stuffing distributed across
many IPs. Username is case-folded so case variants share a bucket.
Maps GC lazily when they cross a soft size cap.
* Per-process `tokio::sync::Semaphore` — caps concurrent Argon2 verifies
during login. Default 2 permits (Pi-class), overridable via
`PICLOUD_LOGIN_ARGON2_PARALLELISM`. Acquired AFTER the bucket check so
attackers can't queue.
Limit check fires before the DB credentials lookup, so even an unknown
username doesn't cost a query. Denied attempts return 429 + Retry-After.
Real client IP comes from the first X-Forwarded-For entry (Caddy is the
trusted single hop); falls back to "unknown" so the per-user bucket
still gates.
4 unit tests cover bucket burst exhaustion, per-user crossing IPs, case
folding, and per-user independence.
Audit ref: security_audit/08_dos_resource.md (H-2 / H-B1).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Same-origin CSRF: the platform co-hosts user-route HTML on the same
origin as /api/v1/admin/*; SameSite=Lax did not block a same-origin POST
from a malicious script at /<route>, riding the admin's picloud_session
cookie. The dashboard already uses Bearer (dashboard/src/lib/auth.ts +
api.ts:495), so cookie auth was dead weight on the admin side and
exploitable.
Cuts the cookie path entirely:
- auth_middleware::extract_token is Bearer-only; cookie branch removed.
- auth_api::login no longer sets Set-Cookie.
- auth_api::logout no longer clears the cookie (the bearer token is
still revoked by deleting the session row).
- extract_token_for_logout matches.
- SESSION_COOKIE const + PICLOUD_COOKIE_SECURE env var deleted.
Audit ref: security_audit/07_http_cors_csrf_xss.md#c07-01.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes 4 High-severity audit findings:
- describe_event broken for HTTP-async payloads: was reading source/op
JSON fields that HttpDispatchPayload doesn't carry, producing empty
source on DL rows. from_wire("") then fell back to Kv on replay, the
dispatcher tried to resolve a non-existent trigger, and replay no-op'd
silently. Now branches on OutboxSourceKind: HTTP rows emit
"<method> <path>" + source="http"; Invoke rows emit "invoke_async" +
source="invoke"; TriggerEvent payloads keep top-level field extraction.
- HTTP outbound Authorization leaked across cross-origin redirects.
Manual redirect loop (Policy::none) reused header_map on every hop
and only scrubbed Content-Type for POST->GET. Now compares
url::Url::origin() across hops and strips Authorization,
Proxy-Authorization, Cookie when origin changes — matching reqwest's
default policy.
- F-S-010 inbound email HMAC had no replay protection. Signature was
computed over body only with no timestamp or nonce, so captured POSTs
were replayable indefinitely. Adds X-Picloud-Timestamp header bound
into the HMAC input (signed string is ts || "." || body), 300s
tolerance window, and a process-local LRU keyed on
(ts, sha256(body)) with 600s TTL to catch within-window replays.
Breaking change: webhook senders must include the timestamp header.
- expected_schema.txt re-blessed for migrations 0036-0039 (the previous
snapshot stopped at 0035 so CI would have gone red on first run).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
manager-core/invoke_service.rs implements InvokeService over:
- ScriptRepository (for Id + Name resolution)
- RouteTable (for Path resolution via the orchestrator's matcher)
- OutboxRepo (for invoke_async)
Same-app guard runs at the service entry point: resolve_id() always
verifies resolved.app_id == cx.app_id and returns InvokeError::CrossApp
otherwise. resolve_name() reads cx.app_id when querying (so a wrong
app_id from somewhere else couldn't slip through), but defense-in-depth
won't hurt — future hardening if it surfaces.
resolve_path() runs the matcher against this app's routes only
(RouteTable::snapshot_for_app). Method assumed POST→GET fallback for
v1.1.9 — surfacing method via the SDK is a future addition.
enqueue_async() resolves the target, then writes one outbox row with
source_kind = 'invoke'. The payload carries script_id, script_name,
args, fresh execution_id, root_execution_id (inherited from caller),
trigger_depth + 1. caller principal recorded as origin_principal
(forensic only — the executing script runs with no principal). One
shot — no retry policy on the row; users wrap in retry::with() if
they want retries.
picloud/lib.rs:
- route_table construction moved BEFORE Services::new so the invoke
service can hold it. populates from route_repo as before.
- InvokeServiceImpl wired into Services::new in place of the
NoopInvokeService placeholder.
Unit tests cover: resolve by Id same-app, cross-app rejected, resolve
by Name finds + not-found, enqueue_async writes Invoke outbox row with
trigger_depth=1.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- shared/services.rs: Services::new gains queue: Arc<dyn QueueService>
positionally after users (mirrors v1.1.8's append of users).
with_noop_services adds NoopQueueService.
- manager-core/queue_service.rs: QueueServiceImpl wraps QueueRepo with
script-as-gate authz on AppQueueEnqueue. enqueue clamps max_attempts
to [1,20] and delay_ms to [0, 86_400_000ms]. depth/depth_pending are
read-only — no authz check (scripts in the app can see their own
queue depths). cx.principal threads through as enqueued_by_principal
(forensic only).
- manager-core/authz.rs: AppQueueEnqueue(AppId) capability — script:write
scope, granted to editor+ (same trust shape as AppPubsubPublish).
- picloud/lib.rs: wires PostgresQueueRepo + QueueServiceImpl into
Services::new alongside the existing v1.1.7+ services.
- 11 sdk test binaries + manager-core/realtime_authority.rs updated to
pass NoopQueueService to Services::new.
Unit tests cover empty queue_name reject, max_attempts clamping
(0/21 → invalid), delay_ms negative-reject, anonymous principal skips
authz, depth/depth_pending pass-through.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- shared/queue.rs: QueueService trait (enqueue / depth / depth_pending),
EnqueueOpts, QueueError, NoopQueueService. Methods derive app_id from
cx.app_id — no script-passed app_id. The handle-less surface mirrors
pubsub (queues are the grouping unit).
- shared/ids.rs: QueueMessageId.
- manager-core/queue_repo.rs: PostgresQueueRepo with:
- enqueue(NewQueueMessage) — single INSERT
- claim(app_id, queue_name) — atomic UPDATE WHERE id = (SELECT … FOR
UPDATE SKIP LOCKED LIMIT 1) RETURNING — the single-round-trip claim
from the design notes
- ack(id, claim_token) — DELETE WHERE id AND claim_token (lease check)
- nack(id, claim_token, retry_delay) — clear claim + set deliver_after
- reclaim_visibility_timeouts() — periodic UPDATE joining triggers +
queue_trigger_details, clears claims older than per-queue
visibility_timeout_secs
- depth / depth_pending / list_for_app (dashboard read-only)
- dead_letter(...) — atomic move to dead_letters + DELETE from
queue_messages, in one transaction. Uses a JSON payload shaped the
same as TriggerEvent::Queue so the existing fan_out_dead_letter
path delivers the original to registered dead_letter triggers
without special-casing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
8 files needed re-wrapping after the F2 clippy-fix pass landed
un-fmt'd. Pure cosmetic — no behavioral change. Re-runs of
`cargo fmt --all -- --check` now exit clean.
Reviewer-supplied per REVIEW.md §7; folded onto the branch
to avoid a NEEDS-CHANGES round-trip on a purely mechanical fix.
Extend the existing weekly retention sweeper with a new
spawn_app_user_token_gc function that prunes four tables in one
tick:
* app_user_sessions — expired (sliding window or absolute cap) or
explicitly revoked
* app_user_email_verifications — consumed or expired
* app_user_password_resets — consumed or expired
* app_user_invitations — accepted or expired
Each underlying repo's gc(batch_size) uses FOR UPDATE SKIP LOCKED so
concurrent sweepers don't fight (cluster mode v1.3+ ready). No
configurable retention — rows die when their TTL or terminal state
hits, not on an arbitrary clock.
Wired into picloud's build_app alongside spawn_dead_letter_gc and
spawn_abandoned_gc.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
users_admin_api router merged into the guarded admin tree at
/api/v1/admin/apps/{id_or_slug}/users/* + /invitations/*.
Endpoints (capability gates applied via the service layer):
GET /users AppUsersRead
GET /users/{user_id} AppUsersRead
POST /users AppUsersWrite
PATCH /users/{user_id} AppUsersWrite
DELETE /users/{user_id} AppUsersWrite (admins satisfy implicitly)
POST /users/{user_id}/reset-password AppUsersAdmin -> one-shot token
POST /users/{user_id}/revoke-sessions AppUsersAdmin
GET /invitations AppUsersAdmin
POST /invitations AppUsersAdmin
DELETE /invitations/{invite_id} AppUsersAdmin
DTOs never include password_hash, session tokens, or unrotated
reset tokens. The reset-password endpoint returns the raw one-shot
token exactly once in the response body so an admin can paste it
into a manual reset link (TTL 1h by default).
Synth_cx() factors the admin-side cx construction into one place
(marked) so the SDK and admin code paths share the service's authz
fan-out. Admin-mediated trait methods (admin_create_invitation,
admin_reset_password_token, admin_revoke_all_sessions,
list_invitations, revoke_invitation) take &Principal directly,
not the synthesized cx.
UsersServiceImpl: removed the NOT_YET_IMPL constant + unused
auth alias (every method has a real implementation now). Added
Serialize+Deserialize on the AppUser / Invitation shared DTOs so
the admin HTTP layer doesn't need a parallel set of types.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
migration 0031: app_user_roles table — composite PK (app_id, user_id,
role) so add is idempotent (ON CONFLICT DO NOTHING). v1.1.8 stores
strings only; permission matrices / hierarchies / role registry are
explicitly v1.2 work per the brief.
UsersServiceImpl wires roles: Arc<dyn AppUserRoleRepo>:
* fetch_roles() now actually queries the repo (replacing the empty
Vec stub from commit 4). Every AppUser returned from get /
find_by_email / list / update / verify / login now carries its
role list.
* users::add_role gated on AppUsersAdmin; first checks the user
exists in this app so a FK violation can't leak "no such user".
* users::remove_role gated on AppUsersAdmin; idempotent.
* users::has_role gated on AppUsersRead.
* accept_invite now applies pre-staged roles atomically with the
user creation; malformed role strings are skipped with a warn
rather than aborting the whole accept (the invitation was an
admin's promise — we honor as much of it as we can).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
migration 0030: app_user_invitations table — surrogate id PK + unique
token_hash, app_id FK cascading, pre-stages email + display_name +
roles for a user that doesn't exist yet. One-shot via atomic UPDATE
SET accepted_at = NOW() WHERE accepted_at IS NULL.
UsersServiceImpl gains invitations: Arc<dyn AppUserInvitationRepo>
plus a mint_session() helper factored from login() and reused by
accept_invite().
users::invite(email, opts) is gated on AppUsersAdmin (per brief —
the most senior of the three new capabilities). Optional
EmailTemplateOpts inside InviteOpts: omitting the template skips the
email send so an admin can stamp invitations for out-of-band
delivery (mailers, printed onboarding letters, etc.). If the template
is present and the email service isn't configured, surfaces as
NotConfigured; non-NotConfigured failures are logged but kept silent
so the invitation row remains valid for retry.
users::accept_invite(token, password, display_name?) atomically
consumes the invitation, validates the new password, creates the
user (returning () on DuplicateEmail — sign-up beat acceptance,
they'll log in normally), and mints a fresh session via mint_session
so the caller can return both the user and a working session token
in one round trip.
Pre-staged roles are stored on the invitation row but not yet
applied — the app_user_roles table arrives in commit 8 (migration
0031). For commit 7 the staged-but-not-applied case logs an info
record so an operator can audit the gap.
list_invitations + revoke_invitation (admin-mediated, gated on
AppUsersAdmin) ship in this commit and become reachable from the
HTTP surface later in the series.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
migration 0029: app_user_password_resets table — same shape as
verification (token_hash PK, app_id + user_id FKs, expires_at,
consumed_at). One-shot via atomic UPDATE WHERE consumed_at IS NULL.
Default TTL 1h (shorter than verification's 48h — reset tokens are
higher-risk).
UsersServiceImpl gains password_resets: Arc<dyn AppUserPasswordResetRepo>.
users::request_password_reset(email, opts):
* Returns Ok(()) regardless of whether the email matched — no
existence-leak signal in script-land (per brief).
* Email-not-configured surfaces as NotConfigured so scripts can
fall back to a synchronous reset path. Other email errors are
silently swallowed and logged server-side; surfacing them would
leak which addresses produced a "real send attempted" signal vs
a no-op.
users::complete_password_reset(token, new_password):
* Atomically consumes the token, updates the Argon2id hash, and
revokes EVERY active session for that user (anyone with a stale
token shouldn't be able to ride out the reset). Emits
users::password_changed.
* Returns the user on success, () on bad/expired/already-used.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
migration 0028: app_user_email_verifications table — token_hash PK,
app_id + user_id FKs cascading, expires_at, consumed_at. Single-use
via atomic UPDATE WHERE consumed_at IS NULL.
UsersServiceImpl gains:
* verifications: Arc<dyn AppUserVerificationRepo>
* email: Arc<dyn EmailService>
users::send_verification_email(user_id, opts) mints a 32-byte token,
stores SHA-256(token), and calls EmailService::send with the body
template's {link} placeholder substituted by link_base + ?token=raw.
EmailError::NotConfigured propagates as UsersError::EmailNotConfigured
so scripts already handling email-disabled mode (v1.1.7 email::send)
don't need new branches.
users::verify_email(token) atomically consumes the one-shot token via
the verifications repo, marks the user's email_verified_at = NOW(),
and emits a "users::email_verified" event for future triggers.
Internal email send uses a synthesized SdkCallCx with principal=None
so the email-service AppEmailSend authz check is skipped (the
users::* surface has already gated on AppUsersWrite — the internal
hop isn't the script's direct call). Documented in HANDBACK §7.
EmailTemplateOpts now requires `from` (the v1.1.7 email service needs
an envelope sender). The brief example omitted it; deviation logged
in HANDBACK §7.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
UsersService trait in shared::users + Postgres-backed impl in
manager-core::users_service. Wired into the Services bundle (new
`users` field) and the picloud binary's startup.
Commit 4 ships:
* CRUD: create / get / find_by_email / update / delete / list
(cursor-paged on created_at). create validates email shape,
8-char password minimum, optional display_name; throws
DuplicateEmail on (app_id, lower(email)) collision.
* Auth: login (timing-flat — runs verify_password against the
shared dummy Argon2id PHC even on email miss, so the
bad-email and good-email branches share wall-clock cost);
verify (sliding TTL bump capped at absolute_expires_at);
logout (revoke by token).
* verify_session_for_realtime — non-SDK method taking app_id
explicitly; used by the F3 realtime auth_mode='session' path
in a later commit.
* Cross-app isolation everywhere: cx.app_id (or explicit app_id)
is the only source of truth; a logout for a foreign-app token
is a silent no-op rather than a probe of session existence.
Email-tied flows (verification, password reset, invitations), roles,
and admin-mediated helpers are stubbed UsersError::Backend and ship
in later commits in this series.
Config (UsersServiceConfig) sources from env with documented
defaults:
PICLOUD_APP_USER_SESSION_TTL_HOURS (24)
PICLOUD_APP_USER_SESSION_ABSOLUTE_HOURS (720 = 30d)
PICLOUD_APP_USER_VERIFICATION_TTL_HOURS (48)
PICLOUD_APP_USER_PASSWORD_RESET_TTL_HOURS (1)
PICLOUD_APP_USER_INVITATION_TTL_DAYS (7)
Reserved a TIMING_FLAT_DUMMY_HASH constant on manager-core::auth so
the dummy PHC is one shared definition.
Added AppUserId + InvitationId id types in picloud-shared.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
App-user session storage (migration 0027) mirrors admin_sessions but
adds three things v1.1.8 needs:
* app_id column + FK cascade — every v1.1+ table starts with app_id
so cross-app isolation is bright at the SQL layer (lookup keys
off the hash only, but defense-in-depth: a leaked row's session
still scopes to its app on every read).
* absolute_expires_at — hard cap on the sliding window (default 30d
via PICLOUD_APP_USER_SESSION_ABSOLUTE_HOURS). Beyond this the
user must re-login regardless of recent activity.
* revoked_at — explicit revocation by token (logout) or per-user
(admin revoke-sessions button, password reset). Lookups reject
revoked rows immediately so revocation takes effect before the
weekly GC sweep runs.
The repo's gc() uses FOR UPDATE SKIP LOCKED matching the dead-letter
and abandoned-executions sweep patterns; the GC wiring lands in a
later commit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per-app end-user table for the v1.1.8 users::* SDK. Distinct from
admin_users (control-plane operators) — same Argon2id password hash
shape but everything else (uniqueness scope, ownership, lifecycle)
independent.
- Uniqueness on (app_id, lower(email)) — case-insensitive within an
app; same email may exist across two apps.
- AppUserRepository trait + Postgres impl; every method takes app_id
explicitly so cross-app reads are unmistakable at the call site
(matches v1.1.3 cross-app discipline).
- Public AppUserRow never includes the password hash; the credentials
shape is its own struct returned only by the login lookup.
- Cursor-based list keyed on (created_at, id).
- Reserved a timing-flat dummy Argon2id PHC constant in auth.rs for
the upcoming login path so the bad-email and good-email branches
share wall-clock cost.
- Added AppUserId + InvitationId id types in shared::ids.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Inbound email: a provider POSTs a normalized JSON message to
POST /api/v1/email-inbound/{app_id}/{trigger_id}; the public receiver
verifies the optional HMAC signature, builds a TriggerEvent::Email, and
enqueues an outbox row the dispatcher delivers like any async trigger.
Handlers see ctx.event.email = #{from,to,cc,subject,text,html,
received_at,message_id}.
- migration 0024: widen triggers.kind + outbox.source_kind CHECKs to
'email'; new email_trigger_details table.
- TriggerKind::Email, TriggerDetails::Email{has_inbound_secret},
OutboxSourceKind::Email, TriggerEvent::Email; dispatcher routes the
email row via the generic resolve_trigger path.
- Admin POST /apps/{id}/triggers/email (validate_trigger_target; module
+ cross-app rejection). inbound_secret is stored ENCRYPTED via the
master key (deviation from the brief's plaintext default; decrypted
per inbound request — see HANDBACK §7).
- Dashboard: email trigger form on the Triggers tab + webhook URL +
expected-payload help.
- 8 DB-gated e2e tests (202/401/404/422/cross-app/handler-fire) +
receiver unit tests (HMAC verify, secret round-trip, payload parse).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Outbound email reachable from scripts as email::send(#{...}) (plain
text) and email::send_html(#{...}) (multipart text + HTML). Backed by a
lettre SMTP relay configured from PICLOUD_SMTP_HOST/PORT/USER/PASSWORD/
TLS/TIMEOUT_SECS; if HOST/USER/PASSWORD aren't all set the service runs
in disabled mode (every send throws NotConfigured, warned at startup).
- EmailService trait + OutboundEmail DTO (picloud-shared);
EmailServiceImpl + EmailTransport seam + lettre transport
(manager-core), wired into the Services bundle and Rhai engine.
- Capability::AppEmailSend (→ script:write); seven-scope commitment held.
- Required-field + RFC5322-ish address validation; 25 MB per-message cap
(PICLOUD_EMAIL_MAX_MESSAGE_BYTES). reply_to defaults to from.
- Per-call connection (pooling deferred to v1.2); no per-app from
validation (operator's SMTP/SPF/DKIM concern).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Encrypted per-app secrets, reachable from scripts as
secrets::{get,set,delete,list}(name) and managed from the dashboard
Secrets tab. Values are AES-256-GCM-sealed with the process master key
(picloud_shared::crypto) before they touch Postgres; the repo only ever
sees ciphertext + nonce. JSON round-trip preserves Rhai types.
- migration 0023_secrets.sql (PRIMARY KEY (app_id, name)).
- SecretsService trait (picloud-shared) + SecretsServiceImpl + repo
(manager-core), wired into the Services bundle and Rhai engine.
- Capability::AppSecretsRead/Write (→ script:read / script:write); no
new Scope variants (seven-scope commitment).
- Admin API GET/POST/DELETE /apps/{id}/secrets (list returns names +
updated_at, never values).
- build_app now takes a MasterKey, sourced from PICLOUD_SECRET_KEY in
main.rs; test callers pass a fixed test key.
- 64 KB value cap (PICLOUD_SECRET_MAX_VALUE_BYTES); no ServiceEvent
emission (secret writes don't fire triggers, by design).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Durable pub/sub through the universal outbox — the sixth trigger kind.
- `pubsub::publish_durable(topic, message)` Rhai SDK (no handle; topics
ARE the grouping unit). Message JSON-encoded; Blobs base64 at any
depth.
- `PubsubService` trait in picloud-shared with the topic matcher +
validator (exact / `<prefix>.*` / `*`; mid-pattern wildcards
rejected). `PostgresPubsubRepo` + `PubsubServiceImpl` in manager-core.
- Publish-time fan-out: one outbox row per matching enabled pubsub
trigger, all in ONE transaction (no half-fan-out on crash). No
matching trigger → publish succeeds silently, zero rows.
- `pubsub:*` trigger kind via Layout-E (0020: widen both CHECKs +
pubsub_trigger_details + partial index), TriggerEvent::Pubsub +
ctx.event.pubsub, dispatcher arm, admin endpoint POST /triggers/pubsub
(validates topic pattern + reuses validate_trigger_target).
- AppPubsubPublish capability → script:write (seven-scope held).
- Dashboard Pub/Sub trigger form on the Triggers tab + list rendering.
publish_ephemeral stays deferred to v1.2. ~18 new tests (service
in-memory incl. transactional-rollback, shared matcher, bridge
encoding). No DB required for the suite.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Filesystem-backed blob storage as the fifth concrete trigger kind.
- `files::collection(c).{create,head,get,update,delete,list}` Rhai SDK
(blob in/out; metadata maps; missing-field throws naming the field).
- `FilesService` trait in picloud-shared; `FsFilesRepo` (atomic
write: temp→fsync→rename→fsync-dir→DB; single-pass SHA-256;
checksum-verified reads → Corrupted) + `FilesServiceImpl` in
manager-core. Metadata in Postgres (0018), bytes on disk under
PICLOUD_FILES_ROOT with 0o700 shard dirs.
- `files:*` trigger kind via the Layout-E pattern (0019: widen both
CHECKs + files_trigger_details), TriggerEvent::Files (metadata only,
no bytes), emit_files fan-out, dispatcher arm, admin endpoint
POST /triggers/files (reuses validate_trigger_target).
- AppFilesRead/AppFilesWrite capabilities → script:read/script:write
(seven-scope commitment held). AppPubsubPublish reserved for v1.1.6.
- Admin files API (list + delete) + dashboard Files view per app.
Cross-app isolation keyed on cx.app_id at every layer. ~45 new tests
(service in-memory, fs tempdir, bridge integration). No DB required
for the suite. publish_ephemeral and the orphan sweep stay deferred.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
HTTP (`http::*`):
- `HttpService` trait (picloud-shared) + reqwest-backed `HttpServiceImpl`
(manager-core), wired into the `Services` bundle.
- SSRF deny-list applied to the resolved IP via a custom reqwest
`dns_resolver` (covers every redirect hop + defeats DNS rebinding) plus
a literal-IP check at URL-parse time. Scheme/port restrictions, request
+ response body caps (stream-with-cap), layered timeout. Error reason is
a CIDR category, never the IP. `PICLOUD_HTTP_ALLOW_PRIVATE` dev override
(logs a startup warning).
- Rhai bridge with three-arg split `verb(url, body, opts)` (resolves the
brief's body-vs-opts contradiction; unknown opt keys throw). Body
dispatch by type; response `#{status,headers,body,body_raw}` with JSON
auto-parse; non-2xx does not throw.
- `Capability::AppHttpRequest` → existing `script:write` scope (no new
Scope variant). `SdkCallCx` gains `script_id` (attribution + User-Agent).
Cron triggers (4th trigger kind):
- Migration 0017 widens the kind/source_kind CHECKs and adds
`cron_trigger_details`. `cron`/`chrono-tz` parse + validate 6-field
schedules and IANA timezones.
- `spawn_cron_scheduler` polls due triggers and enqueues to the universal
outbox; the dispatcher delivers them (one-line match-arm extension).
Catch-up fires exactly once per trigger per tick, not once per missed
window. `ctx.event.cron` for handlers.
- `POST /api/v1/admin/apps/{id}/triggers/cron` reuses the v1.1.3
cross-app + kind!=module target check.
- Dashboard: admin-gated Triggers tab (cron create form + list).
Follow-ups: redact module backend errors at the resolver boundary (log
original at error level); pin `rhai = "=1.24"`; CHANGELOG incl. retroactive
v1.1.3 cross-app-trigger security note. Version bumps: workspace 1.1.4,
SDK 1.5, dashboard 0.10.0.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Lays down the v1.1.3 plumbing:
- `ScriptKind` enum in `picloud-shared` ('endpoint' | 'module').
- `ModuleSource` trait + `ModuleScript` DTO + `NoopModuleSource` in
`picloud-shared`. Resolver lives in `executor-core`; Postgres impl
in `manager-core` (`PostgresModuleSource`).
- `Services::new` grows a fifth `modules: Arc<dyn ModuleSource>` arg.
- `ScriptValidator` returns `ValidatedScript { imports }` so the
manager can populate the dep-graph table on save. New
`validate_module` method on the trait gates module-shape rules.
- `Engine::execute_ast(&Arc<rhai::AST>, req)` lets the orchestrator's
script cache reuse compiled ASTs. `Engine::execute(&str, req)` is
preserved as a convenience that compiles inline. `Engine::compile`
exposes the AST for callers that want to cache.
- `PicloudModuleResolver` replaces `DummyModuleResolver` per-call.
Bridges Rhai's sync `ModuleResolver::resolve` to async
`ModuleSource::lookup` via `Handle::block_on`. Enforces:
- cross-app isolation (resolver captures `Arc<SdkCallCx>`),
- circular import detection (in-progress stack on the resolver),
- import depth limit (default 8 via
`Limits::module_import_depth_max`).
- Module-shape validation walks `ast.statements()` via `rhai/internals`
and accepts only `Var { CONSTANT }`, `Import`, and `Noop`. The
manager admin endpoint runs `validate_module` at save (primary
gate); resolver re-runs it at load (defense in depth).
- LRU cache `(AppId, name) -> (updated_at, Arc<Module>)` owned by
`Engine`. Size from `PICLOUD_MODULE_CACHE_SIZE` (default 512).
- Migration `0015_scripts_kind.sql` adds `scripts.kind` + composite
index + module-name shape CHECK.
- Migration `0016_script_imports.sql` adds the dep-graph table with
FK CASCADE on both columns.
- Repo: `kind` threaded through SELECT/INSERT/UPDATE. New
`count_routes_for_script` / `count_triggers_for_script` /
`list_imports` methods. `create`/`update` open a transaction and
call `replace_imports_tx` to populate the dep-graph.
- Admin endpoint: accepts `kind`; rejects reserved module names;
rejects `endpoint → module` transitions when routes / triggers
exist.
- SDK_VERSION 1.3 → 1.4.
Workspace builds; full test suite (~440 tests) green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
DocsServiceImpl mirrors KvServiceImpl's script-as-gate authz pattern,
the empty-collection rejection, and the best-effort emitter call —
adding "data must be a JSON object" validation, NotFound on update of
a missing doc, and prev_data plumbing via repo.update returning the
prior data.
PostgresDocsRepo handles CRUD against the docs table. The find path
runs through the v1.1.2 query DSL parser (docs_filter::parse_filter)
before building parameterised SQL via sqlx::QueryBuilder:
* Every field-path segment + comparison value is bound as $N.
* jsonb_extract_path_text(data, $N1, $N2, ...) handles variable
depth without segment interpolation.
* Base WHERE is fixed: WHERE app_id = $1 AND collection = $2.
Filter conditions can only narrow, never widen. Load-bearing
test in sql_shape_tests pins this prefix on every emitted query
+ asserts no user string ever lands in the SQL text.
* $ne uses IS DISTINCT FROM (not <>) so missing paths + JSON nulls
are correctly included.
* $in binds the value list as TEXT[] via = ANY($N::text[]).
* $sort always appends a ", id ASC" tiebreaker for stable cursor
pagination semantics; $limit is clamped to MAX_FIND_LIMIT.
docs_filter is the AST + parser for the DSL. Operator allowlist is
explicit; any non-v1.1.2 operator throws UnsupportedOperator with a
v1.2 pointer. Snapshot tests pin the SDK-contract error strings so
changing them is a deliberate act.
Two new Capability variants — AppDocsRead and AppDocsWrite — map to
the existing Scope::ScriptRead and ScriptWrite per the seven-scope
commitment from v1.1.0. role_satisfies grants read at Viewer,
write at Editor (same trust shape as KV).
59 unit tests added across the three new files. All pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two tokio tasks spawned at startup that sweep their respective
tables on a weekly cadence (design notes §3 #9 + §4 retention).
Both use `FOR UPDATE SKIP LOCKED` on the claim query so concurrent
sweepers in cluster mode (v1.3+) don't fight each other.
Defaults: 30 days for dead_letters, 7 days for abandoned_executions.
Both env-overridable via `PICLOUD_DEAD_LETTER_RETENTION_DAYS` and
`PICLOUD_ABANDONED_EXECUTIONS_RETENTION_DAYS` (loaded into
`TriggerConfig::from_env` from commit 5).
Per-tick batch cap (5_000 rows) so a sweep can't lock up the table
in a single transaction; the inner loop continues until 0 rows
affected, after which the outer tick waits for the next week.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`PostgresDeadLetterService` lands as the real `DeadLetterService`
impl, replacing `NoopDeadLetterService` in the picloud binary's
`Services` bundle. Both methods are gated by
`Capability::AppDeadLetterManage(AppId)` — public-HTTP scripts with
`principal: None` fail the check, per design notes §4.
- `dead_letters::replay(id)` (Rhai SDK + admin endpoint): re-inserts
the original event payload into the outbox with attempt_count=0,
reply_to=None. The DL row is marked `resolution='replayed'`.
- `dead_letters::resolve(id, reason)` (Rhai SDK + admin endpoint):
closes the row with `resolved_at = NOW()` and the given reason.
CHECK constraint on the column enforces the 4-value vocabulary.
- `dead_letters::list(filter)` is intentionally NOT shipped —
design notes §4 defers it to v1.2 to align with the eventual
`docs::find()` query DSL.
Admin endpoints under `/api/v1/admin/apps/{id}/dead_letters/*`:
- `GET /` (with `?unresolved=true`) → list view
- `GET /count` → unresolved-count badge
- `GET /{dl_id}` → row detail (full payload + error)
- `POST /{dl_id}/replay` → re-enqueue
- `POST /{dl_id}/resolve` body `{reason}` → close out
All cross-app-aware: the row's `app_id` is compared against the path
param so a caller with rights on app A cannot manipulate app B's
dead letters by id alone.
The Rhai bridge for `dead_letters::*` follows the same sync↔async
pattern as the `kv::` bridge (`Handle::current().block_on(...)`
inside the spawn_blocking-wrapped Rhai engine).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`OutboxEventEmitter` replaces `NoopEventEmitter` in the picloud
binary's `Services` bundle. KV mutations now fan out to the outbox
via `TriggerRepo::list_matching_kv` — one row per matching trigger,
carrying the serialized `TriggerEvent` payload + the matching
trigger's retry policy.
`Dispatcher` is the single tokio task that polls the outbox every
100ms, claims due rows via FOR UPDATE SKIP LOCKED (with a batch cap),
and routes each to the executor. Shares the `ExecutionGate` with
sync HTTP per design notes §2 — gate saturation reschedules the
row instead of dropping it.
Outcome handling matches design notes §3 and §4:
- reply_to.is_some() (sync HTTP): never retry. Deliver via
`InboxResolver`; if the receiver was dropped, write an
`abandoned_executions` row.
- is_dead_letter_handler == true: never retry, never DL. On
failure, annotate the original DL row with
`resolution = 'handler_failed'`. Stops the recursion that would
otherwise re-fire a broken handler script.
- Otherwise async: bump attempt_count, reschedule with exponential
backoff + ±jitter; once max_attempts is reached, write a
`dead_letters` row and drop from outbox.
- Trigger-depth limit: `cx.trigger_depth > max_trigger_depth` skips
execution entirely (log + future metric), NEVER dead-letters.
Loops are not retried via the DL chain — they're terminated.
`InboxResolver` trait lands in `picloud-shared` with a
`NoopInboxResolver` bootstrap that flags every delivery as
`Abandoned`. Commit 6 replaces the noop with the real
in-process registry in `orchestrator-core`.
`AdminPrincipalResolver` builds a `Principal` from a trigger's
`registered_by_principal` user id so the dispatched script executes
as the trigger registrant (design notes §4).
Unit tests cover backoff math (exponential/linear/constant) +
jitter range + ExecError → InboxFailureKind classification + the
status-code table mapping. Integration tests for the full
dispatcher loop need a real Postgres + executor; reviewer runs them
via the manual smoke flow in the plan / HANDBACK.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`/api/v1/admin/apps/{id}/triggers/*` — separate POST endpoints per
kind (kv / dead_letter) so each request validates against the
correct shape. List and DELETE work across both kinds.
Gated on `Capability::AppManageTriggers(app_id)`, which maps onto
`Scope::AppAdmin` (no new scope variants — seven-scope commitment
held) and is granted at the per-app `AppAdmin` role.
Request payloads accept `dispatch_mode` (defaults to `async`) and
retry-override fields. Omitted retry fields fall back to
`TriggerConfig::from_env`, which the binary plumbs into
`TriggersState` so the row is auditable from itself (no lazy
resolution at dispatch time). `registered_by_principal` is taken
from the authenticated principal — design notes §4: "a trigger
execution runs as the principal that registered the trigger".
DELETE loads the trigger first and 404s if its `app_id` doesn't
match the path — prevents a caller with rights on app A from
deleting a trigger via app B's path (bound-key safety net).
In-memory tests cover: app-not-found, member-without-role 403,
default-fallback for retry settings when request omits them,
empty-glob rejection, cross-app delete is treated as not-found.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Migrations 0008-0011 lay down the triggers framework's storage:
- `triggers` + `kv_trigger_details` + `dead_letter_trigger_details`
(Layout E, design notes §2). Parent table carries common columns
including `registered_by_principal` — the dispatcher uses this to
run the trigger as the user that registered it (design notes §4).
- `outbox`: universal async dispatch substrate. KV/cron/pubsub/queue/
email/dead-letter all write rows in the same shape; the dispatcher
claims due rows via FOR UPDATE SKIP LOCKED. `reply_to` is the
NATS-style inbox id for sync HTTP (commit 6) — its presence flags
"don't retry" per the design.
- `dead_letters`: exact schema from design notes §4 with the four-
value `resolution` CHECK constraint (`replayed | ignored |
handled_by_script | handler_failed`) and partial index on
unresolved rows for the dashboard badge.
- `abandoned_executions`: forensic table for the dispatcher's
"tried to resolve a dropped inbox" edge case (design notes §3 #9).
Repo surfaces with Postgres impls behind traits so unit tests can
swap in-memory backings:
- `TriggerRepo` — CRUD + the `list_matching_kv` /
`list_matching_dead_letter` hot paths the dispatcher uses.
Includes a `collection_matches` helper that handles `*`, `prefix:*`,
and exact-name globs.
- `OutboxRepo` — insert + claim-due + delete + reschedule.
- `DeadLetterRepo` — insert + get + list + unresolved-count +
resolve + GC.
- `AbandonedRepo` — insert + GC.
`TriggerConfig::from_env` (new module) follows the existing
`SandboxCeiling` env-loading pattern for `PICLOUD_MAX_TRIGGER_DEPTH`,
`PICLOUD_TRIGGER_RETRY_*`, `PICLOUD_DEAD_LETTER_RETENTION_DAYS`, and
`PICLOUD_ABANDONED_EXECUTIONS_RETENTION_DAYS`.
`Capability::AppManageTriggers(AppId)` and `AppDeadLetterManage(AppId)`
join the enum. Both map onto the existing `Scope::AppAdmin` per the
seven-scope commitment; `role_satisfies` grants them at the
`AppAdmin` per-app role.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
First v1.1.1 commit. Adds the KV store the design notes commit to:
`(app_id, collection, key)` identity with JSONB value and a per-app
index. Trait lives in `picloud-shared` so the executor-core Rhai
bridge (next commit), the Postgres impl, and tests all depend on the
same surface without coupling crates.
The `Services` bundle grows from empty to three fields: `kv`,
`dead_letters` (NoopDeadLetterService stub — replaced by the
Postgres impl in commit 8), and `events` (NoopEventEmitter until the
outbox emitter lands with the dispatcher). Tests use
`Services::default()` for an all-noop bundle.
New capabilities `AppKvRead` / `AppKvWrite` join the Capability
enum. They map onto the existing seven-value `Scope` (script:read /
script:write) — the scope vocabulary stays locked per the
`docs/versioning.md` commitment.
Script-as-gate semantics in `KvServiceImpl`: capability check runs
when `cx.principal.is_some()`, skipped when None (public HTTP).
Cross-app isolation is enforced independently by deriving every
row's `app_id` from `cx.app_id` rather than a script-passed argument.
In-memory `KvRepo` impl + unit tests cover the round-trips, the
cross-app isolation property, empty-collection rejection,
script-as-gate behaviour for both anonymous and authed contexts,
and cursor-style pagination. Postgres impl exists; integration
testing waits for a real DB harness (see HANDBACK).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The data-plane (POST /execute/{id} + user-route fallback) is
unauthenticated by default — public scripts get hit by anonymous HTTP
traffic. But some calls are authed (dashboard test-runs, API-key
invocations) and v1.1.x services will want to see the caller via
`cx.principal` for audit / authz once those features land.
- New manager-core::attach_principal_if_present middleware. Always
inserts Extension<Option<Principal>>: Some on resolved bearer/cookie,
None on absent or malformed token. Fail-open on DB blip so a
transient infra failure can't 500 anonymous traffic.
- Wired in picloud build_app, scoped to the data-plane and user-routes
routers only. The admin path keeps using require_authenticated; no
double-resolve on the same token.
- orchestrator-core handlers (execute_by_id, user_route_handler) now
extract Extension<Option<Principal>> and pass it to build_exec_request.
Replaces the temporary `None` placeholders from the previous commit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
apps_api.rs and app_members_api.rs each grew a near-identical local
`resolve_app` that parses an id-or-slug param and translates None into
their own AppNotFound variant. Promote the lookup half to
`app_repo::resolve_app` (returns `Result<Option<AppLookup>, ...>`) and
let callers handle the None → not-found mapping.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds /api/v1/admin/apps/{id_or_slug}/members[/{user_id}]:
- GET list members (joined with admin_users via list_for_app_enriched)
- POST grant membership — 201 with enriched DTO
409 on duplicate (promotions go through PATCH on purpose so
the UI can surface "already a member" cleanly)
422 if the target user is deactivated
422 if the target's instance_role isn't `member` — owners and
admins already have implicit authority, so an explicit row
would be dead weight
- PATCH change role — 200 with enriched DTO
404 if no existing membership (use POST to create)
- DELETE remove — 204, idempotent (matches the repo's `remove`
contract; 204 also when the row never existed)
All four gated on `Capability::AppAdmin(app_id)`. Editors and viewers
get 403 from list and never see the dashboard's Members tab.
No last-app-admin guard: owners implicitly satisfy AppAdmin via
`role_grants`, so removing the last explicit app_admin row cannot
permanently orphan an app — an owner can always re-issue grants.
Wires through picloud/src/lib.rs by splitting the Postgres app_members
repo Arc into two trait views (AppMembersRepository for CRUD, AuthzRepo
for the existing capability lookups) without re-instantiating against
the pool.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds `AppMembershipDetail` (membership row + joined username, email,
instance_role, is_active) and `list_for_app_enriched` on
`AppMembersRepository`. The Postgres impl does a single JOIN on
admin_users ordered by username, so the upcoming `GET
/apps/{id}/members` handler can render its table without an N+1 fetch.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* auth: generate_api_key() mints pic_<base32(32 bytes)>, splits the
indexed 8-char prefix, and Argon2-hashes the body. Adds the
data-encoding workspace dep for unpadded base32.
* api_keys_api: POST /api/v1/admin/api-keys (mint, returns raw_token
exactly once), GET (caller's own, no raw), DELETE {id} (caller's
own; 404 deliberately covers both 'missing' and 'not yours').
Mint validation rejects bound keys carrying instance:* scopes (422).
* AdminsState gains the api keys repo; PATCH set_active(false) now
expires every active key for that user alongside session wipe —
Phase 3.5 deactivation symmetry.
* picloud lib wires PostgresApiKeyRepository through AuthDeps into
AdminsState + ApiKeysState; api_keys_router merges into the
guarded_admin layer.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* auth_middleware: split into resolve_principal → verify_session OR
verify_api_key (selected by the pic_ prefix). Both paths converge on
Principal as the request extension; require_admin keeps working as
a #[deprecated] alias for require_authenticated. AuthState gains an
api_keys repo; the cookie path is unchanged.
* api-key path takes the first 8 chars after pic_ as the indexed
lookup key, Argon2-verifies each candidate, soft-rejects deactivated
users, and updates last_used_at inline.
* auth_api: /auth/me now consumes Extension<Principal> and re-fetches
the user row so username updates surface immediately.
* picloud: AuthDeps + AuthState wired with PostgresApiKeyRepository;
the layer call switches to require_authenticated.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* admin_user_repo: surface instance_role + email on AdminUserRow /
Credentials; create() now takes instance_role; add
update_instance_role, list_active_owners, count_other_active_owners.
* admin_users_api: DTO + create/patch accept instance_role (defaults
to Admin on create — only env-var bootstrap defaults to Owner).
PATCH and DELETE enforce the last-owner guard alongside the
existing last-active-admin guard.
* app_members_repo: new — implements AuthzRepo::membership via the
app_members table plus upsert/remove/list_for_user/list_for_app.
* api_key_repo: new — create / find_active_by_prefix / touch_last_used
/ list_for_user / get / delete_by_id_and_user / expire_all_for_user.
Separates ApiKeyRow (no hash) from ApiKeyVerification (hash, for
the middleware verifier) so handlers can't leak the hash.
* auth_bootstrap + picloud tests: pass Owner on the bootstrap seed
and on the test admin seed respectively; in-memory test repo
implements the new trait methods.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Implements the three-layer capability check from blueprint §11.6:
role grant (instance role + app_members) ∩ scope intersection (for
API keys) ∩ app binding (for bound keys). Capabilities are finer than
scopes (AppWriteScript vs AppWriteRoute, AppManageDomains vs
AppAdmin) so a script:write-only key cannot mutate routes; scopes
stay at the seven values the blueprint locks down.
In-memory AuthzRepo fixture in the test module covers the full
matrix: owner / admin / member behavior, scope intersection, bound
key isolation, and instance:* denial on bound keys.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Apps become the isolation boundary for scripts, routes, domains, and
later data. Doing this now — while the surface is small — avoids
several migrations on populated tables once v1.1 data-plane services
ship.
Schema (migration 0005_apps.sql):
- New tables: apps, app_domains (with shape_key UNIQUE for collision
detection), app_slug_history (for permanent slug-rename redirects).
- app_id added to scripts, routes, execution_logs (non-null, cascading
rules per row).
- Script-name uniqueness becomes per-app; the route unique index is
swapped for an app-scoped version.
- The "default" app is seeded unconditionally with a localhost claim;
existing scripts/routes backfill into it. Fresh installs additionally
get the Hello World seed via seed_hello_world_if_fresh after
migrations run (idempotent — only fires when the default app has no
scripts).
Orchestrator dispatch is two-phase: AppDomainTable resolves Host →
app_id (most-specific match wins, exact beats wildcard), then the
existing route matcher runs against that app's partitioned slice via
RouteTable. Unknown hosts return 404 at the app layer with a clear
message; /api/v1/execute/{id} still works as the implicit
__internal__ claim, decoupled from any public domain.
Manager API: full CRUD for /api/v1/admin/apps/* and
/api/v1/admin/apps/{id_or_slug}/domains/*, with slug:check + force
takeover semantics implementing the rename-history flow (two-step
check → confirm, never a single endpoint). Script create requires
app_id; list accepts ?app= filter. Route create validates host
against the parent app's claims; conflict detection stays strictly
intra-app.
Dashboard: /admin/apps and /admin/apps/{slug} (overview + scripts +
domains + settings tabs, with slug-history-aware redirects). Root
path redirects to the apps list. Script detail page gains an app
breadcrumb and threads app_id into the route preview.
Deferred per design: per-app admin roles. The require_admin middleware
remains the seam where role checks will slot in later.
Blueprint §11.5 and roadmap updated to reflect what shipped; docs/
versioning.md notes the schema 3 → 5 bump.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the regression risk of the admin API and dashboard being open
to anyone reaching the bound port. Required foundation before v1.1
data-plane services land.
Per-user accounts (admin_users), Argon2id passwords, env-var bootstrap
of the first admin that becomes inert once any admin exists, opaque
32-byte session token doubling as bearer credential, 24h sliding TTL
configurable via PICLOUD_SESSION_TTL_HOURS. is_active column lets
admins be deactivated without losing audit history; last-active-admin
guard on DELETE and on PATCH that flips is_active to false (sessions
also wiped on deactivation).
require_admin middleware fronts every /api/v1/admin/* route. The data
plane (/api/v1/execute/{id}), /healthz, /version, and user routes
stay open. picloud admin reset-password <username> subcommand handles
recovery without going through HTTP.
Dashboard gains /admin/login and /admin/admins surfaces, a top-bar
user menu, and a token store with a localStorage echo so refreshes
don't sign you out. Cookie-based auth works in parallel for non-SPA
clients.
Forward compatibility: future RBAC tables (admin_roles,
admin_user_roles) join on admin_users.id; the auth middleware is the
seam where role checks slot in. Email, 2FA, passkeys, and personal
API tokens are all additive without touching admin_users.
Blueprint §11.4 updated to reflect what actually shipped.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Scripts can now answer at user-chosen paths (e.g. /greet, /greet/:name,
/webhooks/*), on user-chosen hosts (strict or *.example.com wildcards),
on user-chosen methods. The internal /api/v1/execute/{id} endpoint
stays as the always-available ID-based bypass.
Routing rules (decided in design with the user; see chat history):
Path kinds:
exact /greet literal
prefix /greet/* strict-subtree; stored as "/greet/";
does NOT match bare /greet (add an
exact route for that case)
param /users/:id :name captures one whole segment;
mid-segment colons are rejected;
{name} is reserved for a future SDK
Host kinds:
any no Host header constraint
strict sub.example.com literal match (case-insensitive)
wildcard *.example.com suffix match; multi-level subdomains OK
Within-kind uniqueness:
two routes of the same kind that could match the same request
conflict at config time. Algorithm (orchestrator_core::routing::
conflict):
exact: literal equality
prefix: literal equality (longer-prefix coexists; longer wins
at request time)
param: same segment count + same literals at every
literal-vs-literal position (the user's example:
:id vs :userId at same shape is a conflict)
Request-time precedence:
exact > param > prefix
among non-exact: more leading-literal segments wins
tie: param > prefix (more constrained)
within prefix: longest matching prefix wins
host bucket: strict > wildcard (longer suffix) > any; fall through
to less specific buckets when path doesn't match
Reserved path prefixes: /api/, /admin/, /healthz, /version
Routes that look invalid at config time return 422 with the precise
parse error; conflicting routes return 409 with the conflicting route
in the body (so the dashboard can render the conflict inline).
What landed:
* 0003_routes.sql — routes table (host_kind, host, host_param_name,
path_kind, path, method, script_id) with UNIQUE index on the
literal binding tuple. Schema 2 → 3.
* shared::Route / HostKind / PathKind — flat storage shape that
crosses wire boundaries cleanly.
* orchestrator_core::routing — four sub-modules, all unit-tested:
pattern.rs (16 tests) parse + validate + display
conflict.rs (12 tests) within-kind overlap predicate
matcher.rs (12 tests) runtime dispatch (specificity-aware)
table.rs Arc<RwLock<Vec<CompiledRoute>>>
shared by manager (writes) and
orchestrator (reads); atomic replace
after each admin write
* manager-core::route_admin — five new admin endpoints under
/api/v1/admin:
POST /scripts/{id}/routes create
GET /scripts/{id}/routes list per script
DELETE /routes/{route_id} delete (refreshes table)
POST /routes:check pre-flight conflict check
(powers the dashboard's
live conflict warning)
POST /routes:match synthetic URL → matched
route + extracted params
(powers the dashboard's
match-preview tool)
Stored path strings stay raw (user-typed); normalization
happens only in the in-memory CompiledRoute so re-parses are
idempotent.
* orchestrator_core::api::user_routes_router — fallback handler
mounted in picloud after the system routes. Reads Host /
method / path / query from the request, dispatches via the
table, builds an ExecRequest with params/query/rest filled,
calls the executor, writes to the log sink. 10 MiB body cap.
* executor-core::ctx (SDK 1.0 → 1.1) — adds
ctx.request.params (map of named-param captures)
ctx.request.query (parsed query string)
ctx.request.rest (suffix for prefix routes; "" otherwise)
All three are always present (empty when not applicable) so
scripts can read them unconditionally.
* picloud::build_app — now async; loads routes at startup,
populates the shared table, mounts route_admin_router under
/api/v1/admin alongside the script CRUD, and the user-routes
fallback at the app root.
* caddy/Caddyfile + Caddyfile.prod widened: anything not
/healthz, /version, /api/v1/admin/*, /api/v1/execute/*,
/api/* (404 sunset), or /admin/* (dashboard) → picloud.
* Dashboard moves to /admin/* via SvelteKit paths.base. Its
internal Caddy strips the prefix and serves with SPA fallback.
All in-app links use $app/paths. The dashboard URL is now
http://localhost:8000/admin/ — one-time break for the new
URL freedom users gained.
* PICLOUD_PUBLIC_BASE_URL env var, exposed via /version so the
dashboard renders full URLs for routes regardless of the
operator's external port / TLS setup.
* memory_limit_mb stays in the schema, still v1.3+ advisory.
Verified live through Caddy:
/version → schema 3, sdk 1.1, public_base_url
GET /admin/ → 200, dashboard HTML containing "PiCloud"
POST /api/v1/admin/scripts → 201
POST .../scripts/{id}/routes (path=/greet/:name) → 201
GET /greet/alice?lang=en → 200 {"name":"alice","q":"en"}
POST conflicting route → 409 with conflicting_route body
POST /admin/foo route → 422 "reserved"
POST /api/v1/admin/routes:match → matched + params extracted
GET /unbound-path → 404 JSON
Tests:
* 40 routing unit tests (pattern + conflict + matcher tables)
* 14 executor-core unit tests (one new for ctx.request.params/
query/rest exposure)
* 32 integration tests (10 new for routing CRUD + dispatch +
conflict + reserved + specificity tie-break + match preview +
delete invalidation + /version returns public_base_url)
* default cargo test --workspace stays green; opt-in via
DATABASE_URL + --include-ignored for the integration suite
Bumps: schema 2 → 3; SDK 1.0 → 1.1; product 0.3.0 → 0.4.0.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds optional per-script overrides for the six Rhai sandbox knobs
(max_operations, max_string_size, max_array_size, max_map_size,
max_call_levels, max_expr_depth). The executor merges its defaults
with each script's overrides on every call; the manager validates
overrides against an admin-set ceiling at write time, so the
executor trusts whatever is stored.
Storage chose JSONB on the existing scripts table over six new
columns: lets future knobs land as code-only changes, keeps the
sparse common case (most scripts override nothing) cheap to store
and serialize, and matches how the manager + executor pass the
config across the wire.
* 0002_sandbox.sql — ALTER TABLE scripts ADD COLUMN sandbox
JSONB NOT NULL DEFAULT '{}'
* shared::ScriptSandbox — six Option<u64> fields with
deny_unknown_fields so typos surface as 422
* Script.sandbox + ExecRequest.sandbox_overrides — typed end
to end; cluster mode just serializes the same struct
* executor-core::Limits::with_overrides — field-by-field
replacement; tests cover the override actually tightening
the live engine
* manager-core::SandboxCeiling — built-in conservative
defaults (10M ops, 1 MiB strings, 100k array/map, 128
call/expr depth); env vars override per knob, invalid
values warn-and-skip rather than blocking boot
* manager-core admin API — POST/PUT accept `sandbox`; values
above the ceiling return 422 with the specific field +
requested + ceiling; absent or `{}` keeps platform defaults
* picloud all-in-one — wires SandboxCeiling::from_env() into
AdminState
* memory_limit_mb stays in the schema, marked v1.3+ advisory
(no enforcement until OS-level isolation lands with
cluster-mode executors)
Verified live through Caddy:
* /version reports schema 2, product 0.3.0
* Script with max_operations: 500 → 507 on a 10k-iteration loop
* Same script after PUT raising to 1M → succeeds, returns 10000
* POST with max_operations: 1_000_000_000 → 422 (exceeds ceiling)
Tests:
* 13 executor-core unit tests (added 2 for override semantics)
* 20 integration tests (added 6 for sandbox CRUD + ceiling +
unknown-field rejection + executor honoring overrides)
* default cargo test --workspace stays green (integration tests
remain #[ignore]'d until DATABASE_URL is set)
Bumps:
* schema 1 → 2
* product 0.2.0 → 0.3.0
* SDK unchanged (scripts see nothing new)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three threads landing together because they share a public surface
(the new execution_log shape) and verifying any one in isolation
would mean re-doing the work later.
== (A) execution log persistence ==
* shared::ExecutionLog + ExecutionStatus carry the audit-trail
shape that flows from the orchestrator through the sink and
back out via the manager's logs endpoint.
* shared::ExecutionLogSink trait — abstraction the orchestrator
writes through. In single-process MVP mode the manager's
Postgres-backed impl is plugged in directly; in cluster mode
(v1.3+) the orchestrator's impl will post over HTTP to the
manager. Trait lives in `shared` so neither *-core crate has
to know about the other.
* manager-core::PostgresExecutionLogSink writes to the
execution_logs table (already in the initial migration);
PostgresExecutionLogRepository reads them back, paginated.
AdminState now carries both a script repo and a log repo, so
`admin_router` exposes `GET /scripts/{id}/logs?limit=&offset=`
capped at 200 rows per page to keep the dashboard responsive.
* orchestrator-core::DataPlaneState gains `log_sink`. The
execute handler builds an ExecutionLog on every outcome —
success, error, timeout, budget-exceeded — and awaits the
sink. Sink failures are logged at warn and DO NOT mask the
user-facing result, since "we couldn't write the audit row"
is a separate concern from "the script ran".
* picloud binary refactored into a lib (`build_app(pool)` is
the seam) + thin bin shell. Same Postgres pool backs the
script repo, the log repo, and the sink — no double pool.
== (B) dashboard ==
* Typed API client extended with `scripts.logs(id, opts)`,
`scripts.update/remove`, and `execute(id, body, headers)`.
Plain `fetch` wrapper now surfaces server-side error
messages via a typed ApiError so the UI can render them.
* `/` — create-script form now actually creates; on success
the list reloads. List entries link to detail.
* `/scripts/[id]` — new detail route: source editor with save
(calls update, version bumps); Test invoke panel that sends
arbitrary JSON body + headers to /api/execute and shows the
response; Recent executions panel reading from /logs with
expandable per-row request/response/script-log views.
Delete button with confirm. SPA-routed; Caddy serves
`build/` with the same index.html fallback.
== (C) integration tests ==
* crates/picloud/tests/api.rs — 14 sqlx::test cases driving
`build_app` through an axum_test::TestServer against a fresh
Postgres DB per test. Covers: health, full script CRUD,
duplicate-name conflict, invalid-source rejection on both
create and update, execute echoing the body, status+header
passthrough, 404 on missing scripts, error-path executions
landing in the audit log with the right status.
* Tests are `#[ignore]` by default so plain `cargo test
--workspace` stays green without infrastructure. Opt-in via:
`docker compose up -d postgres && \
DATABASE_URL=postgres://picloud:picloud@127.0.0.1:15432/picloud \
cargo test -p picloud --test api -- --include-ignored`
Verified live through Caddy on :8000: three logged invocations
land in the logs endpoint with the right structured `data` on
each `log::info`/`log::warn`, error-path executions are still
captured with status=error, dashboard list + SPA detail route
both reachable.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Brings the MVP feature set online: upload a Rhai script, get an HTTP
endpoint that runs it sandboxed in-process, list/update/delete it, and
have invalid sources rejected at upload time. Verified live through
Caddy with a full lifecycle (`create → list → get → execute → update
→ delete`) plus error paths (syntax error, duplicate name, deleted).
Layout — every concern lands behind the trait seam its layer owns, so
cluster-mode in v1.3+ is a swap of two impls, not a rewrite:
* shared::ScriptValidator — manager calls into validation without
a hard dep on executor-core; executor-core impls the trait on
`Engine`. Pinned in shared so neither crate has to know about
the other.
* executor-core::Engine — real Rhai engine: sandbox limits (max
operations / string size / map size / call depth), disabled
`print`, blocked `import` (DummyModuleResolver), `log::trace
/info/warn/error` registered as a static module with shared
log-capture buffer (no `log::debug` because `debug` is a Rhai
reserved keyword — `log::trace` covers the same need).
- `ctx` is pushed as a Scope constant exposing
execution_id, script_id, script_name, request_id,
invocation_type, request.{path,headers,body}.
- Response convention: a Map with `statusCode` is the
structured shape (`{statusCode, headers?, body}`); any
other return value is a 200 with the value as the body.
- Engine::execute is now synchronous (pure compute); the
async wrapper + wall-clock timeout live in
LocalExecutorClient, which spawns_blocking and applies a
300s hard ceiling regardless of per-script config.
- 10 unit tests cover validate, exec, structured response,
ctx exposure, log capture, op-budget enforcement, runtime
errors, blocked imports, JSON round-tripping.
* manager-core::repo — full sqlx CRUD over the `scripts` table,
with proper unique-violation handling for duplicate names.
Embedded migrations via `sqlx::migrate!` (one initial
`0001_init.sql` for pgcrypto + scripts + execution_logs).
* manager-core::api — `admin_router` mounts `/scripts` and
`/scripts/{id}`. Create + Update validate source through the
injected `ScriptValidator` before persistence. Returns proper
422/409/404 status codes via `ApiError::IntoResponse`.
* orchestrator-core::api — `data_plane_router` mounts
`/execute/{id}`: resolves the script through `ScriptResolver`,
constructs the `ExecRequest` from headers+body, awaits
`ExecutorClient::execute(..., timeout)`, translates the
`ExecResponse` to an axum `Response` with header passthrough.
Maps `ExecError` variants to 422/504/502/507.
* picloud all-in-one — opens the pool, runs migrations, builds
one engine, nests both routers under `/api/admin` and `/api`,
enables structured JSON tracing and graceful shutdown on
SIGTERM. Single `PostgresScriptRepository` Arc is shared by
the admin router (writes) and the resolver (reads).
Other changes:
* Workspace axum bump 0.7 → 0.8 for the `{id}` path syntax
matching the route definitions.
* Workspace clippy: allow `needless_pass_by_value` and
`boxed_local` to keep API ergonomics over pedantic noise.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sets up the PiCloud monorepo as a Cargo workspace organised around the
three-service architecture (manager / orchestrator / executor), each
backed by a *-core library crate so the same logic powers both the MVP
all-in-one `picloud` binary and the future split-process cluster mode.
* crates/shared, executor-core, orchestrator-core, manager-core
define the library surface and trait seams between the three
services (`ExecutorClient`, `ScriptResolver`, `ScriptRepository`).
* crates/picloud is the MVP entrypoint; serves /healthz on 8080
(override via PICLOUD_BIND).
* crates/picloud-{manager,orchestrator,executor} are skeleton
binaries that keep the crate boundaries honest until cluster
mode is built out in v1.3+.
* docs/git-workflow.md defines the trunk-based workflow:
short-lived branches, Conventional Commits, separate hotfix
flow with mandatory reproduction tests.
* CLAUDE.md captures the working rules for future Claude sessions.
Workspace passes `cargo fmt`, `cargo clippy -D warnings` (with
pedantic enabled), and `cargo test --workspace`. The all-in-one
binary responds on `/healthz` and `/`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
