Commit Graph

43 Commits

Author SHA1 Message Date
MechaCat02
dbdd398d90 feat(email): native SMTP-listener ingress (A3)
Some checks failed
CI / Rust — fmt, clippy, test (push) Failing after 33m37s
CI / Dashboard — check (push) Successful in 9m49s
An opt-in receive-only SMTP listener (PICLOUD_SMTP_BIND) so an MX can point
straight at PiCloud. It speaks minimal SMTP (HELO/EHLO/MAIL/RCPT/DATA/RSET/
NOOP/QUIT), resolves each RCPT TO mailbox to the app-owned email trigger that
claims it, and inserts an Email outbox row the dispatcher fires — the same tail
as the HMAC webhook, unchanged.

- migration 0076: email_trigger_details.inbound_address (case-insensitively
  unique among app triggers) + TriggerRepo::email_inbound_target_by_address /
  SmtpInboundTarget; the interactive create-email API accepts inbound_address.
- crates/picloud/src/smtp.rs: a small tokio accept loop + session state machine
  + a testable deliver() core + a minimal RFC-5322 header/body split. DATA is
  size-capped with dot-unstuffing; no AUTH/STARTTLS (TLS terminates upstream —
  the recipient address + per-app isolation are the boundary).
- spawned in run_server alongside axum::serve, sharing the pool, on the same
  shutdown signal.

Pinned by a picloud integration test (deliver → one Email outbox row for a
known mailbox, none for an unknown one) + smtp.rs unit tests (address parse,
header/body split). Multipart/MIME decoding is a documented follow-up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 21:01:46 +02:00
MechaCat02
80fbf2e642 test: give three no-teeth assertions their teeth
- roles: member invoke asserts the script's actual stdout, not just exit 0;
- api: app-slug script filter asserts the returned script's name;
- workflow_orchestrator: unknown-workflow asserts WorkflowError::NotFound,
  not just is_err().

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 20:13:12 +02:00
MechaCat02
612074af04 test: give the DDL and global-count suites their own database
Turning CI on surfaced `atomic_write` as intermittently failing under load —
self-inflicted. Its fault injection is `CREATE TRIGGER ... ON outbox`, which takes
an ACCESS EXCLUSIVE lock on a table every other suite is concurrently inserting
into. The header even claimed it "cannot affect any test running in parallel";
that was wrong — scoping the trigger by app_id bounds which ROWS it rejects, not
the table lock installing it takes. Shipping that alongside "run the DB tests in
CI" would have poisoned the signal.

The fix is the one already used for the e2e suites: a private database per test.
That logic (build a migrated template once, clone it per test via
`CREATE DATABASE ... TEMPLATE`) was duplicated in `picloud/tests/common`, and this
would have been a third copy — so it moves into a shared `picloud-test-support`
dev-dependency crate, and `picloud/tests/common` now re-exports it. The invariant
it encodes: manager-core's claim loops are global by design, so a test must not
share a database with anything that runs them OR with anything that does DDL.

Moved onto it:
- `atomic_write` — the DDL fault injection above; its per-test cleanup is now
  deleted (the database is thrown away).
- `outbox_reclaim` — asserts `reclaim_stale_claims` returns exactly 0, a count
  over the WHOLE outbox table. On the shared DB any stale row from any other suite
  (or a killed picloud that died holding a claim) would fail it permanently, until
  someone cleared the table by hand.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 07:27:35 +02:00
MechaCat02
b5ce4cec01 ci: run the DB-backed tests that CI was silently skipping
CI ran `cargo test --workspace` with no `--include-ignored`, so it executed 927
tests and skipped 237 — every DB-backed integration test is
`#[ignore = "needs DATABASE_URL..."]`, which covers ALL of api.rs, ALL of
authz.rs, and the entire CLI journey suite. The isolation and RBAC tests existed
but never ran (AUDIT.md F-Q-014, logged and never remediated). CI provides
Postgres, so it can run them.

Three things had to be right for that to go green:

- **`--all-targets`, not a bare workspace run.** `-- --include-ignored` un-ignores
  not just `#[ignore]` tests but also ` ```ignore ` DOCTESTS, which are
  illustrative pseudocode that does not compile. `--all-targets` runs lib/bins/
  integration tests but excludes doctests (the same reason clippy uses it); a
  separate `--doc` step runs the doctests without the flag. Structural, so a
  future pseudocode doctest can't silently break CI either.

- **The CLI journeys are their own step.** They spawn a real picloud whose
  dispatcher/orchestrator claim loops are global by design; run concurrently with
  the manager-core suites on the shared database they would claim those suites'
  outbox and workflow rows. Sequential steps keep the live server off the DB
  while the other suites use it. The step also rebuilds `-p picloud` first (the
  harness execs the prebuilt binary) and sets the dev-mode env the server needs.

- **A higher `max_connections`.** `#[sqlx::test]` pools are lazy, but mass-parallel
  test startup briefly opens many at once (each test creates its own throwaway
  database); on a many-core box that transient spike exceeded the default 100 and
  Postgres answered "sorry, too many clients already". Steady-state peak is only
  ~26; 500 absorbs the spike with room to spare. Serving the app needs nothing
  like this many. (This is the local compose ceiling; a small CI runner's default
  100 has ample headroom for its lower parallelism.)

Also fixes the test the CI gap had let rot: api.rs asserted `v["schema"] == 66`
with a hand-bump comment, and since nothing ran it, it sat broken from migration
0066 to 0073. It now asserts `/version` surfaces the live constants
(`migrations::latest_version()`, `SDK_VERSION`) — the wiring — while value drift
stays caught by schema_snapshot + check-versioning. A constant hand-synced to
another constant is a chore, not a test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 07:27:21 +02:00
MechaCat02
251d7fe3dd test(e2e): give each dispatcher test its own database
`dispatcher_e2e` failed about one run in three, on a different test each time,
and never under `--test-threads=1`. Root cause, not timing:

Each test calls `build_app`, which spawns a REAL dispatcher, and they all shared
one database — so they shared one `outbox`. `OutboxRepo::claim_due` is
deliberately not app-scoped (in production one dispatcher serves the whole
instance, so claiming any due row is correct). Test A's dispatcher would
therefore claim test B's row, test A would finish, its server would be dropped
mid-dispatch, and the claim was stranded. Test B polled for its handler's side
effect until it timed out.

The harness was modelling something production never does: N independent
instances sharing one database. So the fix belongs in the harness. Weakening the
dispatcher (app-scoping `claim_due`) would be wrong, and shortening
`PICLOUD_OUTBOX_CLAIM_TIMEOUT_SEC` to fit a 10s test window would make production
abandon the live claims of scripts that may legitimately run 300s.

`tests/common` now hands each test its own database. Replaying 73 migrations per
test is too slow, so the first caller builds a migrated TEMPLATE database once
and every test clones it (`CREATE DATABASE ... TEMPLATE` is a file copy), guarded
by a cluster-wide advisory lock because Postgres will not clone a template that
has an open connection. Names are derived from (suite, test), so a test reclaims
its own database on entry: a crashed run leaves nothing to clean up.

Only the five local `pool_or_skip` wrappers change — all 33 call sites are
untouched, and the skip-when-`DATABASE_URL`-is-unset contract is preserved.
(`#[sqlx::test]` already does this and is what `api.rs`/`authz.rs` use, but it
requires `DATABASE_URL` and would force `#[ignore]`, silently dropping these
tests from the default `cargo test --workspace` gate.)

This also stops the e2e tests leaving stranded claims behind in the shared dev
database, which is the likely source of the occasional lone journey failure.

Before: dispatcher_e2e red ~1 run in 3. After: workspace green twice over,
journeys 157/157.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 21:40:27 +02:00
MechaCat02
80a0d31cd2 fix(files): make file writes transactional and close a quota bypass
Audit #6 and #8 for the last of the three stores, plus a bypass found on
the way.

**#6.** create/update/delete wrote the metadata row, then emitted
best-effort, so an outbox failure left a committed file whose trigger never
fired. `atomic_write::FilesWriter` commits the metadata row and the fan-out
together.

Files are the one store where the ordering is subtle, because the BYTES live
on disk and cannot join a transaction:

  * create/update — blob first, then commit metadata + fan-out. A rollback
    unlinks the blob. (A crash at that exact point still orphans it; that
    hazard predates this change — the repo already wrote the blob and then
    inserted the row in a separate, failable statement — and the orphan is
    inert, referenced by nothing.)
  * delete — commit the metadata removal + fan-out FIRST, then unlink. The
    reverse order would destroy the bytes of a row that a rollback keeps,
    leaving a file that can never be read.

**#8.** `GroupFilesService::create` read `total_bytes` on one connection and
wrote on another, so concurrent uploads each saw the same pre-write total and
together overshot the ceiling. This is the worst instance of the race in the
codebase: the ceiling is DISK (10 GiB by default) and one file may be 100 MB,
so a racing fleet overshoots by gigabytes. `PostgresGroupFilesWriter` takes
the per-group advisory lock (on its own `files` key) across the check and the
write.

**The bypass.** `GroupFilesService::update` checked NO quota at all — so a
1-byte file could be updated to a 100 MB one without the ceiling ever being
consulted, repeatedly, for unbounded disk. It now checks the projected total
(the replaced file's bytes subtracted in SQL, so a same-size-or-smaller
update near the cap still goes through).

Also drive-by: `queue_e2e` asserted the ack the instant the marker appeared,
but the marker is written DURING the handler and the ack happens after it
returns — a zero-tolerance race. It polls now. (This does not fix the
suite's flakiness, which reproduces on the pre-pass commit too.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 20:09:03 +02:00
MechaCat02
ba97c35aaf feat(projects): projects table + owner_project FK; ProjectId/Project types
First commit of §7 multi-repo ownership. A 'project' is a repo-root that
declaratively manages a slice of the shared group tree; each group node is
owned by exactly one project (single-owner-per-node). This lands the registry
+ shared types and wires the inert 0047 `owner_project` seam — no behavior
change yet (claim logic is the next commit).

- migration 0066: `projects` table (UUID pk, unique slug, name, created_by →
  admin_users ON DELETE SET NULL); `groups.owner_project` gains its FK →
  projects(id) ON DELETE SET NULL (un-claim, never cascade-destroy a tree) +
  an index.
- shared: `ProjectId` (id_type! macro) + `Project` struct; `Group` gains
  `owner_project: Option<ProjectId>`.
- group_repo: GROUP_COLS + the ancestors recursive-CTE term now carry
  `owner_project`, so `ancestors()` surfaces ownership for the nearest-claimed
  fold; GroupRow + From updated.
- schema snapshot re-blessed; /version schema assertion 65 → 66.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 20:01:57 +02:00
MechaCat02
ae98063f87 test(api): pin /version schema assertion to migration 65 (D2/D3)
Some checks failed
CI / Rust — fmt, clippy, test (push) Failing after 18m47s
CI / Dashboard — check (push) Successful in 9m47s
D2/D3 added 0064 + 0065; re-pin the #[ignore]-gated version test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 22:40:20 +02:00
MechaCat02
6b1831aea2 test(api): pin /version schema assertion to migration 63
The #[ignore]-gated version_includes_public_base_url pinned schema=42 but
migrations accreted to 63 across the v1.2 template track (…0060–0063), so
it failed under `--include-ignored`. Re-pin to current reality per the
test's own intent.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 20:38:05 +02:00
MechaCat02
e676eb9ba7 fix(audit-2026-06-11/F-SE-H-04): scrub runtime error detail from public data-plane responses
ExecError::Runtime strings (filesystem paths, "blocked by SSRF policy:
link-local" cloud-metadata reconnaissance, pool-exhaustion timing,
panic fragments, and attacker-thrown strings) were returned verbatim in
the public /api/v1/execute + user-route 502 bodies. This handler only
serves anonymous data-plane callers (the authenticated script-test path
is in manager-core and keeps verbose errors), so log the full text
server-side under a correlation id and return a stable generic message
plus the id for operator grep.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-12 18:23:58 +02:00
MechaCat02
31ecfae86e fix(audit-2026-06-11/revocation-lag): evict PrincipalCache on credential change
The PrincipalCache (token-hash -> resolved Principal, 60s TTL) had no
eviction hook, so deactivation / password change / API-key revocation
flipped the DB rows but the cache kept serving the stale principal for
up to the TTL window. Closes the audit's PrincipalCache revocation-lag
Medium and greens authz::deactivating_user_revokes_their_api_keys.

- PrincipalCache::evict_user(user_id) + evict_token(hash).
- One shared cache Arc threaded into AuthState / AdminsState /
  ApiKeysState (a per-state cache would let the middleware's own copy
  keep authenticating a revoked principal).
- Evict on: deactivation, password change (both evict_user), logout
  (evict_token, precise), single API-key delete (evict_user).
- H-E1 note: DB-write invalidation stays best-effort by design (a blip
  must not undo the rotation); the cache evict is what makes it take
  effect on the next request.
- Renamed bearer_and_cookie_produce_same_principal ->
  session_token_and_api_key_produce_same_principal (cookie auth was
  removed in C-1; the test never tested cookies).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-12 18:12:54 +02:00
MechaCat02
19f3647f0e test(audit-2026-06-11): fix status-code + rotted /version assertions
Postgres-backed integration tests surfaced two assertion fixes while
verifying the audit branch:

* email_inbound::create_without_inbound_secret_is_rejected (added in the
  H-B2 commit) asserted 400; TriggersApiError::Invalid maps to 422
  (UNPROCESSABLE_ENTITY). Corrected.

* api::version_includes_public_base_url had two rotted assertions —
  `schema` pinned at 6 (it's migrations::latest_version(), which had
  climbed to 41 and is now 42 after 0042_secrets_envelope_version.sql)
  and `sdk` pinned at "1.1" (the crate is at "1.10"). The test is
  #[ignore]-gated so the rot went unnoticed in normal runs. Both pinned
  to current reality. These were pre-existing failures, not caused by
  the security changes.

Verified against a throwaway Postgres: schema_snapshot, api (53),
authz (28 of 29 — see below), dispatcher_e2e, email_inbound, invoke_e2e,
queue_e2e all green.

Known pre-existing failure (NOT fixed here, out of Tier 1+2 scope):
authz::deactivating_user_revokes_their_api_keys fails identically on the
pre-branch merge-base. It's the PrincipalCache revocation-lag Medium the
audit flagged as unfixed (resolve_principal caches by token-hash with no
eviction on deactivation; lag bounded by the 60s TTL). Deferred to a
Tier 3/4 pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-12 17:27:37 +02:00
MechaCat02
8b5a02b3ef fix(audit-2026-06-11/H-B2): inbound_secret mandatory + bad-signature lockout on email triggers
Two gaps:

1. /api/v1/admin/apps/{id}/triggers/email accepted `inbound_secret: null`
   and treated it as "this trigger is unsigned". The runtime then
   skipped HMAC verification entirely. URL discovery (logs, ops
   dashboards, bruteforce on the 122-bit TriggerId space) then fan-out-
   amplified into the outbox for free — anonymous DoS via the
   dispatcher.

2. Even signed triggers had no per-trigger rate limit on signature-
   verify failures, so an attacker who knew the URL could pump unsigned
   POSTs to force Argon2-equivalent work (HMAC verify + nonce-dedup +
   stored-secret decrypt) at line rate.

Fix:

* triggers_api::create_email_trigger now requires a non-empty
  inbound_secret. 400 on missing / empty / whitespace-only.
* email_inbound_api::receive_inbound_email returns 401 immediately when
  the resolved target has no inbound_secret_encrypted column (pre-fix
  rows). The previous `if let Some(ct), Some(nonce)` branch is gone.
* New `BadSignatureLimiter`: per-`(app_id, trigger_id)` sliding-window
  bucket (10 fails / 60 s). On lockout the receiver returns 429 with
  Retry-After instead of 401. record_failure runs on both the
  "no-secret" 401 path and any verify_signature failure.

Test fallout:
* email_inbound integration tests that relied on None secret: removed
  the now-impossible `unsigned_trigger_accepts_without_signature` test,
  replaced with `create_without_inbound_secret_is_rejected` covering
  null / "" / "   "; updated `malformed_body_is_422` and
  `cross_app_path_is_404` to pass + sign with a secret.
* Two new unit tests pin the limiter behavior (burst lockout + per-
  trigger independence).

Audit refs: security_audit/08_dos_resource.md#h-3,
security_audit/09_external_integrations.md#f-ext-m-001.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-11 20:49:31 +02:00
MechaCat02
05ed9b00bb fix(stage-6): dashboard hardening + audit Lows cherry-pick
Closes 4 dashboard hardening findings and 5 of the Lows from the audit.

Dashboard hardening:
- Subtabs no longer re-fetch the app via api.apps.get on every page
  load. users/files/dead-letters drop the fetch outright (the variable
  was set but never read); queues + queues/[name] now consume the
  layout's AppContext via getContext for the page title. The layout's
  reloadApp() owns the historical-slug redirect — subtab-local redirect
  blocks are removed so there's no race.
- The global :global(details > summary::before) chevron is now scoped
  to details.chevron. The script editor's "Advanced sandbox" details
  and the inbound-email-shape help-text both opt in; the script
  exec-list logs no longer inherit a spurious chevron.
- deriveTab now matches the path segment by anchored ===, so a future
  /apps/<slug>/queues-archived route wouldn't activate the queues tab.

Lows cherry-pick:
- ExecError gains Serialize/Deserialize derives + a snake_case tag so
  RemoteExecutorClient (cluster mode v1.3+) can round-trip the variant.
- triggers_api rejects queue triggers whose visibility_timeout_secs is
  below the dispatcher's per-message executor budget; with no minimum
  the reclaim task races the handler and the queue silently
  double-delivers. Existing test using 5s updated to 30s.
- New migration 0040: execution_logs.script_id cascade switched from
  ON DELETE CASCADE to ON DELETE SET NULL so deleting a script no
  longer wipes the forensic history that motivated the delete.
- New migration 0041: dead_letters composite index on
  (app_id, created_at DESC) so the "list all" dashboard view stops
  falling back to seqscan + sort when unresolved=false.
- Schema snapshot re-blessed.

Deferred to v1.2: the ExecRequest principal serde(skip) marker
(documented in-place; the cluster-mode PR will introduce the wire-safe
snapshot at that point) and the `pic --help` mention of
`picloud admin reset-password` (one-line follow-up).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-10 21:50:58 +02:00
MechaCat02
649246213e fix(stage-3): queue DL fan-out + emit-failure visibility
Closes the queue DL fan-out gap (audit Medium) and surfaces the
KV/docs/files non-transactional emit gap to operators.

- handle_queue_failure previously called queue.dead_letter to write the
  DL row but never invoked fan_out_dead_letter. The comment claimed
  "the outbox arm fires registered dead_letter handlers off the new row"
  — but queue DL rows are written via a separate path that bypasses the
  outbox entirely, so handlers filtered on source="queue" sat idle
  forever. Refactor fan_out_dead_letter to take a DeadLetterFanOutCtx
  struct so both the outbox arm and the queue arm can call it; the
  queue arm constructs a TriggerEvent::Queue from the claimed message
  and passes it through.

- New integration test queue_dead_letter_fans_out_to_dead_letter_handler
  registers a dead_letter trigger filtered on "queue", forces queue
  exhaustion, asserts the handler fires with the correctly-shaped event.

- KV/docs/files services committed the data write then ran events.emit
  as a separate operation, logging-and-swallowing on failure. Bump the
  six call sites from tracing::warn to tracing::error with an
  event_emit_failure=true marker so operators can grep them. The full
  single-tx repo refactor (extending ServiceEventEmitter with
  emit_in_tx + tx-aware *_repo methods) is documented in kv_service.rs
  as a v1.2 follow-up — it's a meaningful redesign that deserves its
  own pass (pubsub_service::fan_out_publish is the reference shape).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-10 21:22:15 +02:00
MechaCat02
3c9816daf3 fix(stage-1): audit High-severity backend correctness + CI unblocker
Closes 4 High-severity audit findings:

- describe_event broken for HTTP-async payloads: was reading source/op
  JSON fields that HttpDispatchPayload doesn't carry, producing empty
  source on DL rows. from_wire("") then fell back to Kv on replay, the
  dispatcher tried to resolve a non-existent trigger, and replay no-op'd
  silently. Now branches on OutboxSourceKind: HTTP rows emit
  "<method> <path>" + source="http"; Invoke rows emit "invoke_async" +
  source="invoke"; TriggerEvent payloads keep top-level field extraction.

- HTTP outbound Authorization leaked across cross-origin redirects.
  Manual redirect loop (Policy::none) reused header_map on every hop
  and only scrubbed Content-Type for POST->GET. Now compares
  url::Url::origin() across hops and strips Authorization,
  Proxy-Authorization, Cookie when origin changes — matching reqwest's
  default policy.

- F-S-010 inbound email HMAC had no replay protection. Signature was
  computed over body only with no timestamp or nonce, so captured POSTs
  were replayable indefinitely. Adds X-Picloud-Timestamp header bound
  into the HMAC input (signed string is ts || "." || body), 300s
  tolerance window, and a process-local LRU keyed on
  (ts, sha256(body)) with 600s TTL to catch within-window replays.
  Breaking change: webhook senders must include the timestamp header.

- expected_schema.txt re-blessed for migrations 0036-0039 (the previous
  snapshot stopped at 0035 so CI would have gone red on first run).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-10 21:03:10 +02:00
MechaCat02
c3baa87415 chore(v1.1.9): clippy clean — workspace -D warnings
Surfaced + fixed during the F3 attestation:

- executor-core/sdk/queue.rs: drop redundant .map(|()| ()) call sites;
  enqueue_blocking discards QueueMessageId via .map(|_id| ()) (intentional)
- executor-core/sdk/retry.rs: hoist use std::collections::hash_map::DefaultHasher
  + use std::hash::Hasher to the top of the file (clippy::items_after_statements);
  replace `as i64` casts with i64::try_from + clear comment about jitter
  bound (clippy::cast_possible_wrap)
- executor-core/sdk/invoke.rs: move _LIMITS_IS_COPY const before #[cfg(test)]
  mod tests (clippy::items_after_test_module)
- manager-core/dispatcher.rs: dispatch_one_queue gains #[allow(too_many_lines)]
  (it's the queue tick's whole logic — split makes it less readable than
  the lint); .map(...).unwrap_or(default) → .map_or(default, ...)
- picloud/lib.rs: Limits { trigger_depth_max, ..Limits::default() } via
  struct-update instead of let-mut-assign (clippy::field_reassign_with_default)
- tests: r#"..."# → r"..." where there are no `"` inside (clippy::needless_raw_string_hashes);
  combine InvokeTarget::Name | InvokeTarget::Path arms with identical bodies
  in sdk_invoke.rs (clippy::match_same_arms)

cargo fmt --all -- --check 2>&1 | tail -3: no output (exit 0)
cargo clippy --workspace --all-targets --all-features -- -D warnings:
  Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.41s
Workspace lib smoke: 432 passing (306 manager-core + 74 executor-core + 34 shared + 18 orchestrator-core).
DB-gated E2E (against docker compose postgres on localhost:15432):
  queue_e2e: 4 ok / invoke_e2e: 4 ok / retry_e2e: 3 ok / migration_queue_messages: 4 ok / schema_snapshot: 1 ok

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 11:02:22 +02:00
MechaCat02
c38c46b8bc test(v1.1.9): fix invoke_e2e + retry_e2e — admin bypass + rhai shape
Running the E2E suites against real Postgres surfaced three shape bugs
in the test scripts that caused false failures:

invoke_e2e:
- invoke_cross_app_rejects used two TestServer instances (one per app),
  but the second server's Owner admin isn't a member of the first
  server's app. Replaced with a single server that creates both apps
  via the same Owner admin (which has implicit access to every app).
- invoke_depth_limit_exceeds_cleanly: the recurser script had its own
  try/catch, so when the depth limit fired inside the deepest call the
  caught error became the BODY of a 200 response (which invoke()
  returns to the caller). The outer caller's try/catch never saw a
  throw → assertion failed. Rewrote so the recurser propagates throws
  (no inner try-catch); the outer caller's try-catch surfaces the
  depth error all the way up.

retry_e2e:
- All three tests used HTTP routes which need a domain claim the test
  apps don't have (`no app claims host ""` 404s). Switched to the
  admin bypass POST /api/v1/execute/{id} — same pattern dispatcher_e2e
  uses. Sidesteps the per-app domain matcher entirely.
- retry_run_surfaces_last_error_after_max_attempts: try-catch is a
  statement in Rhai, not an expression, so the block didn't evaluate
  to the catch arm's map. Refactored to bind to `let out` inside the
  catch arm, then return `#{ statusCode: 200, body: out }` as the
  final expression.

All 11 v1.1.9 E2E tests now pass against Postgres:
  queue_e2e: 4 passed (33s — exercises retry + dead-letter)
  invoke_e2e: 4 passed (2s)
  retry_e2e: 3 passed (2.5s)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 10:54:52 +02:00
MechaCat02
328d7d55e6 test(v1.1.9): queue_e2e + invoke_e2e + retry_e2e + migration test
Four new test binaries, DB-gated via the established pool_or_skip()
pattern (returns Some(pool) when DATABASE_URL is set; prints a skip
notice and returns None otherwise — so plain `cargo test` stays green
locally while CI runs them).

crates/picloud/tests/queue_e2e.rs (4 tests):
  - queue_receive_acks_on_success: enqueue → consumer fires → KV
    marker observable → queue row deleted (ack worked)
  - queue_receive_dead_letters_after_max_attempts: throwing handler
    retries 3× via the dispatcher's compute_backoff path → dead_letters
    row written, queue row deleted
  - queue_one_consumer_per_queue_rejected: second trigger create for the
    same (app_id, queue_name) returns 4xx with the documented "already
    has a consumer trigger" message (advisory-lock guard works)
  - queue_visibility_timeout_reclaim: insert message with stale claim
    (claimed_at = NOW() - 60s, visibility_timeout = 5s); either the
    dispatcher re-claims after reclaim runs, or the claim clears

crates/picloud/tests/invoke_e2e.rs (4 tests):
  - invoke_by_name_same_app_returns_value: callee returns body.x+1;
    caller invokes by name and writes the result to KV (42 round-trips)
  - invoke_cross_app_rejects: callee in app_B, caller in app_A; error
    string contains "different app" or "CrossApp"
  - invoke_depth_limit_exceeds_cleanly: recursive script throws with
    "depth" in the message at the bound (Limits::trigger_depth_max=8)
  - invoke_async_enqueues_outbox_row: invoke_async returns execution_id,
    dispatcher fires the OutboxSourceKind::Invoke arm, callee marker
    appears

crates/picloud/tests/retry_e2e.rs (3 tests):
  - retry_run_eventually_succeeds_inside_http_handler: counter mutates
    across 3 retries through a real HTTP route, returns 3
  - retry_run_surfaces_last_error_after_max_attempts: max_attempts=2,
    always throws — caller's try/catch surfaces "boom"
  - retry_on_codes_filters_unmatched_errors: counter == 1 after a
    throw NOT in the codes list (immediate surface, no retry)

crates/manager-core/tests/migration_queue_messages.rs (4 tests):
  - queue_messages_table_exists_with_expected_columns: shape +
    nullability of every column
  - queue_trigger_details_table_exists: shape
  - queue_widens_trigger_kind_and_outbox_source_kind: triggers.kind
    CHECK admits 'queue'; outbox.source_kind admits 'invoke'
  - queue_messages_dispatch_index_is_partial: idx_queue_messages_dispatch
    has WHERE claim_token IS NULL (guards against future migrations
    accidentally dropping the partial)

All 15 tests pass clean skip when DATABASE_URL is unset. Workspace unit
suite remains at 432 passing (306 manager-core + 74 executor-core + 34
shared + 18 orchestrator-core).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 10:25:22 +02:00
MechaCat02
02335a8132 fix(v1.1.7-dead-letter): wire dispatcher → list_matching_dead_letter
dead_letter triggers have been registerable since v1.1.1 but their
handlers never fired: dispatcher::handle_failure wrote the dead_letters
row and stopped — list_matching_dead_letter had no production caller.
Any deploy v1.1.1–v1.1.6 with dead_letter triggers had silently
non-functional handlers.

The fix: after the dead-letter row is inserted on retry exhaustion, fan
out to matching dead_letter triggers (filtered by source / originating
trigger_id / script_id) and enqueue one outbox row per match carrying a
real-shape TriggerEvent::DeadLetter (the §6 brief field names were stale
— used the actual variant: dead_letter_id, original: Box<TriggerEvent>,
attempts, last_error, trigger_id, script_id, first/last_attempt_at).
The recursion-stop (a handler's own failure isn't re-dead-lettered)
is upheld by the existing is_dead_letter_handler short-circuit.

Tests (DB-gated): handler actually fires with the nested original event;
existing row-create test now also asserts handler-fire; source_filter
excludes non-matching; failing handler does not recurse.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 22:30:25 +02:00
MechaCat02
1f78937dd2 feat(v1.1.7-email-inbound): webhook receiver + email:receive trigger
Inbound email: a provider POSTs a normalized JSON message to
POST /api/v1/email-inbound/{app_id}/{trigger_id}; the public receiver
verifies the optional HMAC signature, builds a TriggerEvent::Email, and
enqueues an outbox row the dispatcher delivers like any async trigger.
Handlers see ctx.event.email = #{from,to,cc,subject,text,html,
received_at,message_id}.

- migration 0024: widen triggers.kind + outbox.source_kind CHECKs to
  'email'; new email_trigger_details table.
- TriggerKind::Email, TriggerDetails::Email{has_inbound_secret},
  OutboxSourceKind::Email, TriggerEvent::Email; dispatcher routes the
  email row via the generic resolve_trigger path.
- Admin POST /apps/{id}/triggers/email (validate_trigger_target; module
  + cross-app rejection). inbound_secret is stored ENCRYPTED via the
  master key (deviation from the brief's plaintext default; decrypted
  per inbound request — see HANDBACK §7).
- Dashboard: email trigger form on the Triggers tab + webhook URL +
  expected-payload help.
- 8 DB-gated e2e tests (202/401/404/422/cross-app/handler-fire) +
  receiver unit tests (HMAC verify, secret round-trip, payload parse).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 22:24:35 +02:00
MechaCat02
2d11090d1a feat(v1.1.7-secrets): secrets SDK + table + admin API + dashboard
Encrypted per-app secrets, reachable from scripts as
secrets::{get,set,delete,list}(name) and managed from the dashboard
Secrets tab. Values are AES-256-GCM-sealed with the process master key
(picloud_shared::crypto) before they touch Postgres; the repo only ever
sees ciphertext + nonce. JSON round-trip preserves Rhai types.

- migration 0023_secrets.sql (PRIMARY KEY (app_id, name)).
- SecretsService trait (picloud-shared) + SecretsServiceImpl + repo
  (manager-core), wired into the Services bundle and Rhai engine.
- Capability::AppSecretsRead/Write (→ script:read / script:write); no
  new Scope variants (seven-scope commitment).
- Admin API GET/POST/DELETE /apps/{id}/secrets (list returns names +
  updated_at, never values).
- build_app now takes a MasterKey, sourced from PICLOUD_SECRET_KEY in
  main.rs; test callers pass a fixed test key.
- 64 KB value cap (PICLOUD_SECRET_MAX_VALUE_BYTES); no ServiceEvent
  emission (secret writes don't fire triggers, by design).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 21:37:17 +02:00
MechaCat02
fcbcc576a2 feat(v1.1.6): realtime channels + v1.1.5 follow-ups + version bumps
Server-side realtime SSE on per-app pub/sub topics, plus the three
v1.1.5 follow-ups and the version bumps.

Realtime:
- topics registry (0021) + admin endpoints + Capability::AppTopicManage
  (-> app:admin; no new scope).
- GET /realtime/topics/{topic} SSE endpoint (orchestrator-core data
  plane): Host -> app, RealtimeAuthority gate (404 missing/internal,
  401 bad/absent token), broadcast::Receiver stream + heartbeat.
- RealtimeBroadcaster / RealtimeEvent / RealtimeAuthority traits
  (picloud-shared); InProcessBroadcaster + GC (orchestrator-core);
  DB-backed RealtimeAuthorityImpl (manager-core). Publish path fans out
  to in-process subscribers after the durable outbox commit (best-effort,
  panic-isolated).
- HMAC subscriber tokens (subscriber_token.rs) + app_secrets table (0022)
  + pubsub::subscriber_token SDK (schema 1.6 -> 1.7). TTL clamp + env
  overrides.
- Dashboard Topics tab (register/list/edit/delete, prominent external
  badge, flip confirmation).

v1.1.5 follow-ups:
- Empty blobs accepted (NewFile/FileUpdate::validate) + round-trip test.
- Orphan *.tmp.* sweeper (spawn_files_orphan_sweep).
- Dispatcher e2e tests, one per trigger kind (DATABASE_URL-gated).

Versions: workspace 1.1.6, SDK 1.7, dashboard 0.12.0. Schema-snapshot
golden re-blessed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 20:18:50 +02:00
MechaCat02
3dbead426f test(v1.1.3-modules): resolver, cache, validator, kind-rejection coverage
Adds ~46 new tests across the v1.1.3 surface:

executor-core/tests/modules.rs (NEW, 23 tests):
- resolver_loads_simple_module / endpoint_can_import_module /
  module_can_import_module — end-to-end through Engine::execute.
- resolver_cross_app_blocked / resolver_cross_app_module_not_found /
  module_cache_keyed_by_app — same-name modules in different apps
  resolve independently; cross-app lookup returns ModuleNotFound.
- resolver_self_import_detected / resolver_circular_detected —
  cycle detector reports the chain.
- resolver_depth_limit_enforced / resolver_depth_limit_just_under_succeeds.
- resolver_module_not_found / resolver_backend_error_surfaces.
- resolver_runtime_validation_rejects_top_level_expr — defense-in-
  depth: a module with a top-level expression that bypassed the
  admin gate is rejected at resolve time.
- module_cache_hit_reuses_compiled_module /
  module_cache_stale_invalidated_on_updated_at_change /
  module_cache_lru_evicts_when_capacity_exceeded.
- validate_module_{accepts_fn_const_import_only,
  rejects_top_level_let, rejects_top_level_expr,
  rejects_top_level_while}.
- validate_endpoint_{extracts_literal_imports,
  top_level_expr_still_allowed,
  skips_dynamic_imports_in_imports_list}.

orchestrator-core/src/client.rs cache_tests (6 tests):
- cache_hit_when_identity_matches / cache_invalidated_when_updated_at_changes
  / distinct_script_ids_cache_independently / lru_eviction_caps_cache_size
  / script_identity_is_copy / compile_error_does_not_poison_cache.

shared/src/script.rs kind_tests (3 tests):
- default_is_endpoint / round_trips_through_serde_lowercase
  / parse_str_round_trip.

manager-core/src/triggers_api.rs v1.1.3 tests (6 tests):
- kv_trigger_rejects_module_target / docs_trigger_rejects_module_target
  / dl_trigger_rejects_module_target — modules cannot be trigger
  targets.
- kv_trigger_rejects_missing_script / kv_trigger_rejects_cross_app_script
  — closes the latent v1.1.1/v1.1.2 isolation gap.
- kv_trigger_accepts_endpoint_target — happy path through the
  validate_trigger_target check.

picloud/tests/api.rs (8 #[ignore]'d Postgres-gated integration tests):
- create_script_default_kind_is_endpoint / create_module_kind_persists.
- create_module_with_top_level_expr_rejected /
  create_module_with_reserved_name_rejected.
- route_bind_rejects_module.
- endpoint_imports_module_end_to_end /
  module_edit_visible_on_next_invocation / cross_app_import_blocked.

Lint cleanup along the way:
- `ScriptKind::from_str` renamed to `parse_str` to dodge the
  `should_implement_trait` lint (FromStr's `Result<…,Err>` shape
  doesn't fit a 0-info lookup).
- `derive(Default)` on `ScriptKind` (Endpoint marked `#[default]`).
- Match-arm collapse in `check_module_shape` for Import + Noop.
- `#[allow(clippy::too_many_lines)]` on `resolve()` (the bridge
  logic is genuinely cohesive and would lose clarity if split).
- Elided `'r` lifetime on `StackGuard`.

Three gates clean on this commit's HEAD:
- cargo fmt --all -- --check: clean
- cargo clippy --all-targets --all-features -- -D warnings: clean
- cargo test --workspace: 358 passed, 140 ignored (Postgres-gated)
- npm run check: 0 errors, 0 warnings

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-03 07:18:18 +02:00
MechaCat02
b42e273479 fix(test): admin_is_implicit_app_admin uses force=true on app delete
The test creates a script in the default app earlier in the body, so a
plain DELETE /apps/default hits the soft no-cascade guard and 409s
before the capability check runs. The intent is to validate that admin
holds AppAdmin everywhere, not to exercise the cascade contract — pass
?force=true so we reach the gate we're trying to test.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 20:21:38 +02:00
MechaCat02
4644ea4919 feat(manager-core): admin is implicit app_admin; delete-script needs AppAdmin
Aligns the canonical capability rules with how the dashboard now shadows
its UI. Instance admins become implicit app_admin on every app (only
InstanceManageSettings stays owner-only), and the script-delete handler
moves from AppWriteScript to AppAdmin so editors can save but not delete.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 19:27:32 +02:00
MechaCat02
e6fc6e6a0e test(picloud): close two app_members test gaps
- `membership_makes_app_appear_in_members_app_list` previously seeded
  the membership via the repo helper; switch to the public POST
  endpoint so the test actually exercises the full HTTP round-trip
  the dashboard depends on.
- Add `add_member_with_missing_user_id_is_rejected` to pin the
  Axum-JsonRejection 4xx contract on malformed POST bodies.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-27 22:00:28 +02:00
MechaCat02
b7175cc581 chore: rustfmt fixups for app_members files
Trailing-comma format! cleanup from `cargo fmt --all`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-27 21:39:25 +02:00
MechaCat02
248571dcde test(picloud): authz coverage for app members CRUD
Adds 16 integration tests against a real Postgres covering the new
/api/v1/admin/apps/{id_or_slug}/members surface:

- list / add / patch / remove against an explicit member row
- 409 on duplicate, 422 on inactive target, 422 on owner/admin target
- 404 on PATCH without an existing row; 204 idempotent DELETE
- viewer-as-bob receives 403 on every mutating verb
- both slug and UUID paths resolve to the same body
- bob-with-app_admin can manage the member list, including removing
  himself (load-bearing for the no-last-app-admin-guard decision)
- granting a `member` user a viewer membership makes the app appear
  in their `GET /admin/apps` list (was empty before)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-27 21:33:59 +02:00
MechaCat02
33697a2766 feat(api): expose caller's effective app role via my_role
GET /api/v1/admin/apps/{id_or_slug} now returns an `AppRole`-typed
`my_role` alongside the existing app fields, computed server-side from
the Principal: `Owner → app_admin` and `Admin → editor` (both
implicit per blueprint §11.6), `Member → app_members.role` (looked up
via the existing `AuthzRepo::membership` already in `AppsState`).

The dashboard uses this single field to decide whether to render
admin-only surfaces (Members tab, etc.) instead of duplicating the
implicit-grant rules on the client side — keeps API and UI gate logic
identical with one round-trip.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-27 21:25:23 +02:00
MechaCat02
0c9f11558a feat(manager-core,picloud): accept email on admin create + patch
The /admins create/patch endpoints now plumb email through to the
repo so the dashboard's invite + edit forms aren't silently dropping
it on the floor. Discovered during smoke testing — the database
column existed and was exposed in the response DTO, but neither
the request DTO nor the repo's create() accepted it.

CreateAdminRequest gains optional email; PatchAdminRequest gains
email with JSON Merge Patch semantics:
  absent     → don't change
  null       → clear (write NULL)
  "<string>" → set to that value

The tri-state needs Option<Option<String>> with a tiny custom
deserializer; serde collapses absent and null otherwise.

normalize_email() trims, treats blanks as None, and rejects
obviously bogus values (no '@', >254 chars) with a 422. Real
email verification is a future concern.

Repo trait gains an email parameter on create() and a new
update_email() method. The unique-violation branch in create now
inspects constraint() to distinguish duplicate username from
duplicate email.

Integration test exercises create-with-email, PATCH null clears,
PATCH value sets, PATCH without email key no-ops on email.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-27 19:27:52 +02:00
MechaCat02
39a6df2bfe fix(picloud): use is_some_and in /auth/me test (clippy)
clippy::map_unwrap_or — drop the map().unwrap_or(false) for the
flatter is_some_and(Value::is_null).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-27 08:08:09 +02:00
MechaCat02
3688c26cb4 feat(manager-core,picloud): expose instance_role + email on /auth/me
Login and /auth/me now return the same shape — id, username,
instance_role, email — so the dashboard can gate UI on role from
either the login response or the layout's me() refetch without an
extra round-trip.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-27 07:39:06 +02:00
MechaCat02
2aab92af31 style: cargo fmt across Phase 3.5 changes
Pure formatting pass — no behavior changes. Catches the line-wrapping
drift across the new authz / api_keys / middleware / handler edits
that piled up during the implementation.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 22:21:37 +02:00
MechaCat02
063595be31 test(picloud): integration tests for Phase 3.5 authz (11 cases)
Covers the matrix laid out in the plan:
* bootstrap admin lands as Owner
* owner / admin / member access matrices on the default app
* bearer pic_ key and cookie session resolve to the same Principal
* read-only key cannot write (scope intersection)
* bound key cannot escape its app
* member listing isolation at SQL for /admin/apps + /admin/scripts
* deactivating a user expires every API key for them
* mint rejects bound key carrying instance:* scopes (422)
* list_active_owners returns the right set for the startup warning

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 22:19:24 +02:00
MechaCat02
30a1584667 chore: bump product to 0.6.0; multi-owner startup warning
Phase 3.5 ships → product minor bump under pre-1.0 rules (any surface
bump triggers minor). Schema is now 6 (0006_users_authz.sql); API
remains 1 (additive endpoints + new credential type, no breaking
shape changes). docs/versioning.md updated.

main.rs gets warn_on_multi_owner_install() which fires once after
bootstrap when more than one active owner exists — points the
operator at PATCH /admin/admins/{id} for cleanup. Soft-fail on DB
error (does not block startup).

The api-test schema assertion was updated to expect 6.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 22:15:45 +02:00
MechaCat02
44db8d107a feat(manager-core): repos + admin patch for Phase 3.5 schema
* admin_user_repo: surface instance_role + email on AdminUserRow /
  Credentials; create() now takes instance_role; add
  update_instance_role, list_active_owners, count_other_active_owners.
* admin_users_api: DTO + create/patch accept instance_role (defaults
  to Admin on create — only env-var bootstrap defaults to Owner).
  PATCH and DELETE enforce the last-owner guard alongside the
  existing last-active-admin guard.
* app_members_repo: new — implements AuthzRepo::membership via the
  app_members table plus upsert/remove/list_for_user/list_for_app.
* api_key_repo: new — create / find_active_by_prefix / touch_last_used
  / list_for_user / get / delete_by_id_and_user / expire_all_for_user.
  Separates ApiKeyRow (no hash) from ApiKeyVerification (hash, for
  the middleware verifier) so handlers can't leak the hash.
* auth_bootstrap + picloud tests: pass Owner on the bootstrap seed
  and on the test admin seed respectively; in-memory test repo
  implements the new trait methods.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 21:49:54 +02:00
MechaCat02
4c41374db4 feat(manager-core,orchestrator-core): multi-app scoping (Phase 3b)
Apps become the isolation boundary for scripts, routes, domains, and
later data. Doing this now — while the surface is small — avoids
several migrations on populated tables once v1.1 data-plane services
ship.

Schema (migration 0005_apps.sql):
- New tables: apps, app_domains (with shape_key UNIQUE for collision
  detection), app_slug_history (for permanent slug-rename redirects).
- app_id added to scripts, routes, execution_logs (non-null, cascading
  rules per row).
- Script-name uniqueness becomes per-app; the route unique index is
  swapped for an app-scoped version.
- The "default" app is seeded unconditionally with a localhost claim;
  existing scripts/routes backfill into it. Fresh installs additionally
  get the Hello World seed via seed_hello_world_if_fresh after
  migrations run (idempotent — only fires when the default app has no
  scripts).

Orchestrator dispatch is two-phase: AppDomainTable resolves Host →
app_id (most-specific match wins, exact beats wildcard), then the
existing route matcher runs against that app's partitioned slice via
RouteTable. Unknown hosts return 404 at the app layer with a clear
message; /api/v1/execute/{id} still works as the implicit
__internal__ claim, decoupled from any public domain.

Manager API: full CRUD for /api/v1/admin/apps/* and
/api/v1/admin/apps/{id_or_slug}/domains/*, with slug:check + force
takeover semantics implementing the rename-history flow (two-step
check → confirm, never a single endpoint). Script create requires
app_id; list accepts ?app= filter. Route create validates host
against the parent app's claims; conflict detection stays strictly
intra-app.

Dashboard: /admin/apps and /admin/apps/{slug} (overview + scripts +
domains + settings tabs, with slug-history-aware redirects). Root
path redirects to the apps list. Script detail page gains an app
breadcrumb and threads app_id into the route preview.

Deferred per design: per-app admin roles. The require_admin middleware
remains the seam where role checks will slot in later.

Blueprint §11.5 and roadmap updated to reflect what shipped; docs/
versioning.md notes the schema 3 → 5 bump.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 21:03:05 +02:00
MechaCat02
6891496589 feat(manager-core): admin auth gate (Phase 3a)
Closes the regression risk of the admin API and dashboard being open
to anyone reaching the bound port. Required foundation before v1.1
data-plane services land.

Per-user accounts (admin_users), Argon2id passwords, env-var bootstrap
of the first admin that becomes inert once any admin exists, opaque
32-byte session token doubling as bearer credential, 24h sliding TTL
configurable via PICLOUD_SESSION_TTL_HOURS. is_active column lets
admins be deactivated without losing audit history; last-active-admin
guard on DELETE and on PATCH that flips is_active to false (sessions
also wiped on deactivation).

require_admin middleware fronts every /api/v1/admin/* route. The data
plane (/api/v1/execute/{id}), /healthz, /version, and user routes
stay open. picloud admin reset-password <username> subcommand handles
recovery without going through HTTP.

Dashboard gains /admin/login and /admin/admins surfaces, a top-bar
user menu, and a token store with a localStorage echo so refreshes
don't sign you out. Cookie-based auth works in parallel for non-SPA
clients.

Forward compatibility: future RBAC tables (admin_roles,
admin_user_roles) join on admin_users.id; the auth middleware is the
seam where role checks slot in. Email, 2FA, passkeys, and personal
API tokens are all additive without touching admin_users.

Blueprint §11.4 updated to reflect what actually shipped.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 19:30:25 +02:00
MechaCat02
07e2a62d98 feat: custom routing — bind scripts to your own URLs
Scripts can now answer at user-chosen paths (e.g. /greet, /greet/:name,
/webhooks/*), on user-chosen hosts (strict or *.example.com wildcards),
on user-chosen methods. The internal /api/v1/execute/{id} endpoint
stays as the always-available ID-based bypass.

Routing rules (decided in design with the user; see chat history):

  Path kinds:
    exact   /greet              literal
    prefix  /greet/*            strict-subtree; stored as "/greet/";
                                does NOT match bare /greet (add an
                                exact route for that case)
    param   /users/:id          :name captures one whole segment;
                                mid-segment colons are rejected;
                                {name} is reserved for a future SDK

  Host kinds:
    any                         no Host header constraint
    strict  sub.example.com     literal match (case-insensitive)
    wildcard *.example.com      suffix match; multi-level subdomains OK

  Within-kind uniqueness:
    two routes of the same kind that could match the same request
    conflict at config time. Algorithm (orchestrator_core::routing::
    conflict):
      exact:  literal equality
      prefix: literal equality (longer-prefix coexists; longer wins
              at request time)
      param:  same segment count + same literals at every
              literal-vs-literal position (the user's example:
              :id vs :userId at same shape is a conflict)

  Request-time precedence:
    exact > param > prefix
    among non-exact: more leading-literal segments wins
    tie: param > prefix (more constrained)
    within prefix: longest matching prefix wins
    host bucket: strict > wildcard (longer suffix) > any; fall through
    to less specific buckets when path doesn't match

  Reserved path prefixes: /api/, /admin/, /healthz, /version

  Routes that look invalid at config time return 422 with the precise
  parse error; conflicting routes return 409 with the conflicting route
  in the body (so the dashboard can render the conflict inline).

What landed:

  * 0003_routes.sql — routes table (host_kind, host, host_param_name,
    path_kind, path, method, script_id) with UNIQUE index on the
    literal binding tuple. Schema 2 → 3.

  * shared::Route / HostKind / PathKind — flat storage shape that
    crosses wire boundaries cleanly.

  * orchestrator_core::routing — four sub-modules, all unit-tested:
      pattern.rs (16 tests)  parse + validate + display
      conflict.rs (12 tests) within-kind overlap predicate
      matcher.rs (12 tests)  runtime dispatch (specificity-aware)
      table.rs               Arc<RwLock<Vec<CompiledRoute>>>
                             shared by manager (writes) and
                             orchestrator (reads); atomic replace
                             after each admin write

  * manager-core::route_admin — five new admin endpoints under
    /api/v1/admin:
      POST   /scripts/{id}/routes      create
      GET    /scripts/{id}/routes      list per script
      DELETE /routes/{route_id}        delete (refreshes table)
      POST   /routes:check             pre-flight conflict check
                                       (powers the dashboard's
                                       live conflict warning)
      POST   /routes:match             synthetic URL → matched
                                       route + extracted params
                                       (powers the dashboard's
                                       match-preview tool)
    Stored path strings stay raw (user-typed); normalization
    happens only in the in-memory CompiledRoute so re-parses are
    idempotent.

  * orchestrator_core::api::user_routes_router — fallback handler
    mounted in picloud after the system routes. Reads Host /
    method / path / query from the request, dispatches via the
    table, builds an ExecRequest with params/query/rest filled,
    calls the executor, writes to the log sink. 10 MiB body cap.

  * executor-core::ctx (SDK 1.0 → 1.1) — adds
      ctx.request.params  (map of named-param captures)
      ctx.request.query   (parsed query string)
      ctx.request.rest    (suffix for prefix routes; "" otherwise)
    All three are always present (empty when not applicable) so
    scripts can read them unconditionally.

  * picloud::build_app — now async; loads routes at startup,
    populates the shared table, mounts route_admin_router under
    /api/v1/admin alongside the script CRUD, and the user-routes
    fallback at the app root.

  * caddy/Caddyfile + Caddyfile.prod widened: anything not
    /healthz, /version, /api/v1/admin/*, /api/v1/execute/*,
    /api/* (404 sunset), or /admin/* (dashboard) → picloud.

  * Dashboard moves to /admin/* via SvelteKit paths.base. Its
    internal Caddy strips the prefix and serves with SPA fallback.
    All in-app links use $app/paths. The dashboard URL is now
    http://localhost:8000/admin/ — one-time break for the new
    URL freedom users gained.

  * PICLOUD_PUBLIC_BASE_URL env var, exposed via /version so the
    dashboard renders full URLs for routes regardless of the
    operator's external port / TLS setup.

  * memory_limit_mb stays in the schema, still v1.3+ advisory.

Verified live through Caddy:
  /version              → schema 3, sdk 1.1, public_base_url
  GET /admin/           → 200, dashboard HTML containing "PiCloud"
  POST /api/v1/admin/scripts → 201
  POST .../scripts/{id}/routes (path=/greet/:name) → 201
  GET /greet/alice?lang=en → 200 {"name":"alice","q":"en"}
  POST conflicting route → 409 with conflicting_route body
  POST /admin/foo route → 422 "reserved"
  POST /api/v1/admin/routes:match → matched + params extracted
  GET /unbound-path → 404 JSON

Tests:
  * 40 routing unit tests (pattern + conflict + matcher tables)
  * 14 executor-core unit tests (one new for ctx.request.params/
    query/rest exposure)
  * 32 integration tests (10 new for routing CRUD + dispatch +
    conflict + reserved + specificity tie-break + match preview +
    delete invalidation + /version returns public_base_url)
  * default cargo test --workspace stays green; opt-in via
    DATABASE_URL + --include-ignored for the integration suite

Bumps: schema 2 → 3; SDK 1.0 → 1.1; product 0.3.0 → 0.4.0.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 18:18:16 +02:00
MechaCat02
f51924fdbc feat: per-script Rhai sandbox overrides with admin ceiling
Adds optional per-script overrides for the six Rhai sandbox knobs
(max_operations, max_string_size, max_array_size, max_map_size,
max_call_levels, max_expr_depth). The executor merges its defaults
with each script's overrides on every call; the manager validates
overrides against an admin-set ceiling at write time, so the
executor trusts whatever is stored.

Storage chose JSONB on the existing scripts table over six new
columns: lets future knobs land as code-only changes, keeps the
sparse common case (most scripts override nothing) cheap to store
and serialize, and matches how the manager + executor pass the
config across the wire.

  * 0002_sandbox.sql — ALTER TABLE scripts ADD COLUMN sandbox
    JSONB NOT NULL DEFAULT '{}'
  * shared::ScriptSandbox — six Option<u64> fields with
    deny_unknown_fields so typos surface as 422
  * Script.sandbox + ExecRequest.sandbox_overrides — typed end
    to end; cluster mode just serializes the same struct
  * executor-core::Limits::with_overrides — field-by-field
    replacement; tests cover the override actually tightening
    the live engine
  * manager-core::SandboxCeiling — built-in conservative
    defaults (10M ops, 1 MiB strings, 100k array/map, 128
    call/expr depth); env vars override per knob, invalid
    values warn-and-skip rather than blocking boot
  * manager-core admin API — POST/PUT accept `sandbox`; values
    above the ceiling return 422 with the specific field +
    requested + ceiling; absent or `{}` keeps platform defaults
  * picloud all-in-one — wires SandboxCeiling::from_env() into
    AdminState
  * memory_limit_mb stays in the schema, marked v1.3+ advisory
    (no enforcement until OS-level isolation lands with
    cluster-mode executors)

Verified live through Caddy:
  * /version reports schema 2, product 0.3.0
  * Script with max_operations: 500 → 507 on a 10k-iteration loop
  * Same script after PUT raising to 1M → succeeds, returns 10000
  * POST with max_operations: 1_000_000_000 → 422 (exceeds ceiling)

Tests:
  * 13 executor-core unit tests (added 2 for override semantics)
  * 20 integration tests (added 6 for sandbox CRUD + ceiling +
    unknown-field rejection + executor honoring overrides)
  * default cargo test --workspace stays green (integration tests
    remain #[ignore]'d until DATABASE_URL is set)

Bumps:
  * schema 1 → 2
  * product 0.2.0 → 0.3.0
  * SDK unchanged (scripts see nothing new)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 16:26:12 +02:00
MechaCat02
0473d295af feat: versioning scheme — lockstep crates + four independent surfaces
Establish how versions are assigned, bumped, and checked across the
five things that actually change for users: the product itself, the
Rhai SDK, the HTTP API, the database schema, and the inter-service
wire (reserved for cluster mode). Crates ship in lockstep — drift
between picloud-shared and picloud-manager-core is fiction since
they always release together — but surfaces are versioned and
checked at their natural boundaries.

  * docs/versioning.md is the authoritative reference: what gets a
    version, the per-surface compatibility rules, how each surface
    bump cascades to the product version (loose pre-1.0, strict
    post-1.0), and the five enforcement mechanisms (lockstep at
    compile time, /version at runtime, golden SDK contract tests,
    migration replay, CI guardrail).

  * shared::version exposes four constants — PRODUCT_VERSION (from
    CARGO_PKG_VERSION), SDK_VERSION ("1.0"), API_VERSION (1),
    WIRE_VERSION (1). Scripts read SDK_VERSION as ctx.sdk_version
    and can feature-detect against it.

  * Workspace inheritance: `[workspace.package] version = "0.2.0"`
    is the single point of truth; every crate uses
    `version.workspace = true`. dashboard/package.json mirrors.

  * Routes move to /api/v1/* — both control plane
    (/api/v1/admin/*) and data plane (/api/v1/execute/{id}).
    Picloud composes them via a single `/api/v{API_VERSION}` nest,
    so the next major is a copy-paste-and-bump. Caddyfile (dev and
    prod) routes /api/v1/* to picloud and 404s any other /api/*
    so old clients fail loudly instead of getting the SPA shell.
    Dashboard client + integration tests updated.

  * /healthz remains a plain "ok" string (k8s probes); /version is
    the new JSON endpoint returning every surface version in one
    place — product, sdk, api, schema (from
    manager-core::migrations::latest_version), wire.

  * Reasonable bump rationale: API path changes are breaking by
    definition, so 0.1.0 → 0.2.0 (pre-1.0 license to bump minor on
    any breaking change). SDK starts at 1.0 because scripts depend
    on it more strictly than the product depends on its internals;
    we'd rather promise SDK stability early than pull the rug.

Verified live:
  * /healthz → "ok" (plain text)
  * /version → {product:"0.2.0",sdk:"1.0",api:1,schema:1,wire:1}
  * /api/v1/admin/scripts → 200
  * /api/admin/scripts → 404 with error JSON (sunset major)
  * Script can read ctx.sdk_version → "1.0"
  * All 14 integration tests pass against new paths
  * 11 executor-core unit tests pass (added one for sdk_version
    exposure with the major.minor format invariant)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 00:31:08 +02:00
MechaCat02
777f4af628 feat: persist execution logs + dashboard detail view + integration tests
Three threads landing together because they share a public surface
(the new execution_log shape) and verifying any one in isolation
would mean re-doing the work later.

== (A) execution log persistence ==

  * shared::ExecutionLog + ExecutionStatus carry the audit-trail
    shape that flows from the orchestrator through the sink and
    back out via the manager's logs endpoint.

  * shared::ExecutionLogSink trait — abstraction the orchestrator
    writes through. In single-process MVP mode the manager's
    Postgres-backed impl is plugged in directly; in cluster mode
    (v1.3+) the orchestrator's impl will post over HTTP to the
    manager. Trait lives in `shared` so neither *-core crate has
    to know about the other.

  * manager-core::PostgresExecutionLogSink writes to the
    execution_logs table (already in the initial migration);
    PostgresExecutionLogRepository reads them back, paginated.
    AdminState now carries both a script repo and a log repo, so
    `admin_router` exposes `GET /scripts/{id}/logs?limit=&offset=`
    capped at 200 rows per page to keep the dashboard responsive.

  * orchestrator-core::DataPlaneState gains `log_sink`. The
    execute handler builds an ExecutionLog on every outcome —
    success, error, timeout, budget-exceeded — and awaits the
    sink. Sink failures are logged at warn and DO NOT mask the
    user-facing result, since "we couldn't write the audit row"
    is a separate concern from "the script ran".

  * picloud binary refactored into a lib (`build_app(pool)` is
    the seam) + thin bin shell. Same Postgres pool backs the
    script repo, the log repo, and the sink — no double pool.

== (B) dashboard ==

  * Typed API client extended with `scripts.logs(id, opts)`,
    `scripts.update/remove`, and `execute(id, body, headers)`.
    Plain `fetch` wrapper now surfaces server-side error
    messages via a typed ApiError so the UI can render them.

  * `/` — create-script form now actually creates; on success
    the list reloads. List entries link to detail.

  * `/scripts/[id]` — new detail route: source editor with save
    (calls update, version bumps); Test invoke panel that sends
    arbitrary JSON body + headers to /api/execute and shows the
    response; Recent executions panel reading from /logs with
    expandable per-row request/response/script-log views.
    Delete button with confirm. SPA-routed; Caddy serves
    `build/` with the same index.html fallback.

== (C) integration tests ==

  * crates/picloud/tests/api.rs — 14 sqlx::test cases driving
    `build_app` through an axum_test::TestServer against a fresh
    Postgres DB per test. Covers: health, full script CRUD,
    duplicate-name conflict, invalid-source rejection on both
    create and update, execute echoing the body, status+header
    passthrough, 404 on missing scripts, error-path executions
    landing in the audit log with the right status.

  * Tests are `#[ignore]` by default so plain `cargo test
    --workspace` stays green without infrastructure. Opt-in via:
    `docker compose up -d postgres && \
       DATABASE_URL=postgres://picloud:picloud@127.0.0.1:15432/picloud \
       cargo test -p picloud --test api -- --include-ignored`

Verified live through Caddy on :8000: three logged invocations
land in the logs endpoint with the right structured `data` on
each `log::info`/`log::warn`, error-path executions are still
captured with status=error, dashboard list + SPA detail route
both reachable.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 00:16:32 +02:00