Compare commits

...

79 Commits

Author SHA1 Message Date
MechaCat02
813bc46640 docs(examples): add the CMS proof-of-concept dogfooding artifact
Some checks failed
CI / Rust — fmt, clippy, test (push) Failing after 39m53s
CI / Dashboard — check (push) Successful in 10m15s
A WordPress-inspired CMS whose entire backend is PiCloud Rhai scripts
deployed with pic apply, plus a SvelteKit frontend. Built to dogfood the
project tool, CLI, and SDK; exercises docs/kv/files/users, docs+queue+cron
+pubsub triggers, the transactional-outbox notification chain, a docs
before-interceptor, set_if CAS, SSE, per-app CORS, env overlays, and a
durable Workflow (validate -> enrich || seo -> publish -> finalize) started
with workflow::start and polled with workflow::run_status, visualized live
with Svelte Flow.

FINDINGS.md / SECURITY.md capture every gap, surprise, and security issue
found (each tagged [PiCloud] vs [CMS]); the platform fixes for the
actionable ones ship in the preceding commit.

Also folds in the read-only `pic` allowlist entries added to
.claude/settings.json during the session (fewer-permission-prompts).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 20:15:29 +02:00
MechaCat02
5682be366c fix(cms-poc): remediate findings surfaced by the CMS proof-of-concept
Additive fixes surfaced by building a full CMS on PiCloud
(examples/cms-poc/). One migration (0077_apps_cors.sql); no destructive
changes.

Security:
- Close the 502 info-leak on user routes: an uncaught script error (incl.
  a throw) leaked the app UUID + script fn names + source line/col. The
  user-route (inbox) path now shares scrub_runtime_detail with the
  execute-by-id path — raw detail is logged under a correlation id and the
  client sees only "script execution error (ref: <uuid>)".

Added:
- docs::find $contains operator (array membership via JSONB @>), per-app
  and group-shared — the inverse of $in.
- App-user role management from API/CLI: pic users add-role / rm-role,
  backed by the apps/{id}/users/{user_id}/roles endpoints (AppUsersAdmin).
- Per-app CORS: pic apps cors set, apps.cors_allowed_origins; the
  orchestrator echoes an allowed Origin and answers OPTIONS preflight.
- Non-JSON request bodies: form-urlencoded -> object, text/* -> string,
  other -> base64; ctx.request.content_type added. Malformed JSON still 400s.
- Binary responses via #{ body_base64 } with a script-set Content-Type.
- workflow::run_status(run_id) SDK (F-038): the starting script can poll a
  run — #{ status, output, error, steps } or () when no such run belongs to
  the app (app_id is the isolation boundary); same AppInvoke gate as start.

Fixed:
- pic apply "app not found" is now actionable (points at pic apps create).
- Interceptor- (F-021) and workflow-step-bound (F-037) scripts no longer
  warn "no route or trigger" at plan time — both count as reachability.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 20:15:17 +02:00
MechaCat02
dbdd398d90 feat(email): native SMTP-listener ingress (A3)
Some checks failed
CI / Rust — fmt, clippy, test (push) Failing after 33m37s
CI / Dashboard — check (push) Successful in 9m49s
An opt-in receive-only SMTP listener (PICLOUD_SMTP_BIND) so an MX can point
straight at PiCloud. It speaks minimal SMTP (HELO/EHLO/MAIL/RCPT/DATA/RSET/
NOOP/QUIT), resolves each RCPT TO mailbox to the app-owned email trigger that
claims it, and inserts an Email outbox row the dispatcher fires — the same tail
as the HMAC webhook, unchanged.

- migration 0076: email_trigger_details.inbound_address (case-insensitively
  unique among app triggers) + TriggerRepo::email_inbound_target_by_address /
  SmtpInboundTarget; the interactive create-email API accepts inbound_address.
- crates/picloud/src/smtp.rs: a small tokio accept loop + session state machine
  + a testable deliver() core + a minimal RFC-5322 header/body split. DATA is
  size-capped with dot-unstuffing; no AUTH/STARTTLS (TLS terminates upstream —
  the recipient address + per-app isolation are the boundary).
- spawned in run_server alongside axum::serve, sharing the pool, on the same
  shutdown signal.

Pinned by a picloud integration test (deliver → one Email outbox row for a
known mailbox, none for an unknown one) + smtp.rs unit tests (address parse,
header/body split). Multipart/MIME decoding is a documented follow-up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 21:01:46 +02:00
MechaCat02
0dd03af0c6 feat(interceptors): hook set_if (CAS) + pic interceptors ls (§9.4 M12)
Closes the §9.4 track:
- set_if (compare-and-swap), per-app and shared, now runs the same (kv, set)
  before/after interceptor as set — otherwise CAS was a silent bypass of a set
  policy. The before-hook can transform the new value; the after-hook sees the
  swapped bool.
- read-only pic interceptors ls --app|--group: app shows the RESOLVED chain
  (every marker guarding its writes, nearest-owner-wins), group its own markers.
  New apply_service::interceptor_report + InterceptorInfo, /apps|groups/{id}/
  interceptors routes (AppRead/GroupScriptsRead), client + cmd mirroring
  extension-points.

CLAUDE.md updated: §9.4 service interceptors are now COMPLETE (M1-M12).
Pinned by a journey: set_if of a guarded key is denied while a free key swaps,
and interceptors ls lists the kv/set guard (12 interceptor journeys green).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 20:44:42 +02:00
MechaCat02
ef16d48a8c feat(interceptors): extend before/after hooks to docs/files/queue/pubsub/http (§9.4 M7-M11)
Every non-KV mutating op now runs the same before/after interceptor machinery:
- docs create/update/delete (+ group) — create/update honor the M4 data transform;
- files create/update/delete (+ group) — allow/deny only (the blob is never
  surfaced to the hook, value = None);
- queue enqueue (+ shared) — transform-capable; after reports the message id;
- pubsub publish_durable (+ shared topic) — transform-capable;
- http request — all verbs funnel through two svc.request sites; collection =
  method, key = url, body not surfaced.
ictx threaded into the free-fn/handle paths that lacked it (http was _ictx;
queue/pubsub per-app publish).

validate_bundle_for generalized to a per-service allowed-ops map (kv set/delete;
docs/files create/update/delete; queue enqueue; pubsub publish; http request) —
unknown service/op still rejected. INVARIANT enforced + verified: every allowed
(service, op) has a matching runtime hook, so no validated-but-unhooked
fail-open. Pinned by a new docs-create deny journey (11 interceptor journeys green).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 20:37:18 +02:00
MechaCat02
faf04174c4 perf(interceptors): per-execution resolve cache (§9.4 M6)
Memoizes the interceptor chain resolution per execution tree, keyed by
(app_id, service, op). The FIRST hooked-eligible op pays the one chain query;
every later op reuses it — so N kv::sets in a script with no interceptors now
issue ONE resolve, not N (the dominant, zero-marker case caches an empty chain).
InterceptorCacheScope is an RAII scope entered in execute_ast next to the
emission budget, same re-entrancy model: a synchronous invoke/interceptor
re-entry shares the cache, and it is cleared at the outermost boundary so a
pooled thread never serves a foreign app's cache. Pinned by an executor-core
test: 25 sets → 1 resolve, and a fresh execution → a new resolve.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 20:22:11 +02:00
MechaCat02
8b70d52d43 feat(interceptors): per-interceptor wall-clock timeout (§9.4 M5)
A [[interceptors]] marker can set timeout_ms (migration 0075, CHECK > 0); a
runaway guard (loop {}) is interrupted and the op DENIED (fail closed) within
budget rather than hanging the write path. The effective deadline is
min(caller-remaining, now + timeout) — a hook can only tighten, never extend,
its caller's deadline. NULL uses PICLOUD_INTERCEPTOR_TIMEOUT_MS (default 5s).

Wiring: run_resolved_blocking splits into a core + a _with_timeout variant that
computes the effective deadline from engine::ambient_deadline() and runs via
execute_ast_with_deadline; run_one_hook passes the marker's timeout_ms (or the
env default). timeout_ms threaded end to end — manifest, plan, BundleInterceptor,
the reconcile diff (part of the mutable body: a timeout change is an Update),
insert_interceptor_tx, resolve_chain/list_for_owner/list_on_app_chain +
SealedInterceptor/InterceptorMarker, and interceptor_service → ResolvedInterceptor.
Schema snapshot re-blessed. Pinned by a journey: a loop{} guard with
timeout_ms=100 is denied and its write does not persist.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 20:18:03 +02:00
MechaCat02
5592f1c97e feat(interceptors): KV after-hooks + before-hook data transform (§9.4 M3+M4)
M4: a before-interceptor returning #{ allowed: true, data: ... } rewrites the
value actually written (threaded through the chain; each hook sees the prior
transform), size-capped at MAX_JSON_MATERIALIZE_BYTES. Delete never transforms.

M3: an 'after' phase interceptor runs once the write has committed, with the
write's result in its payload. After-hooks observe/deny but CANNOT roll back —
a deny surfaces as an operation error while the write persists (documented at
the call site).

Adds phase authoring end to end: a [[interceptors]] entry gains phase =
before|after (default before), threaded through the plan wire, BundleInterceptor,
validate (phase in {before,after}), the reconcile diff/insert/prune key
(service/op/phase), and the repo (insert/delete/list_for_owner/list_on_app_chain,
+ the marker's phase). run_before now returns the transformed value; run_after is
new; both share one fail-closed per-entry runner. Pinned by two journeys: a
before-hook transforms the stored value, and an after-delete hook sees
result==true, denies, yet the key stays deleted.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 19:55:44 +02:00
MechaCat02
be0c618672 feat(interceptors): ordered before-chains + cycle guard + ctx plumbing (§9.4 M1+M2)
M1: introduces a clone-cheap InterceptorCtx { interceptors, self_engine,
limits } threaded into every SDK register fn (kv/docs/files/queue/pubsub/http)
so the non-KV services carry the hook seam (unused until M7-M11); KvHandle
collapses its three fields into one ictx.

M2: replaces the nearest-only resolver with ordered before/after chains. The
trait becomes resolve(cx, service, op) -> InterceptorChain { before, after }
(migration 0074 adds a phase column + phase-aware unique indexes). The before
-chain runs ancestor->app (depth DESC) so a group compliance guard can't be
bypassed by a descendant; single-marker behavior is byte-identical to before.
An identity cycle guard (thread-local visited-set keyed by script_id) denies a
detected cycle, alongside the existing binary re-entrancy break. Fail-closed
verdict preserved (allow only on #{ allowed: true }; a Dangling entry or a
missing engine back-ref denies); app_id still derives from cx.app_id only.

after-chains resolve but stay unused until M3. Pinned by two new interceptor
journeys (ancestor->app chain ordering; self-referential no-recurse).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 22:54:51 +02:00
MechaCat02
eae2ee08f1 feat(triggers): shared dead-letter trigger fan-out (B2)
A group declares a declaratively-authored [[triggers.dead_letter]] shared=true
handler; when a message in the group's SHARED queue is exhausted, the
dispatcher's q_terminal group branch (after persisting to group_dead_letters)
fans out to it via list_matching_shared_dead_letter(owning_group, "queue", …),
each outbox row stamped the WRITER app_id (the consuming app — the M2 shared
-write model), so the handler runs under the consumer. The per-app
list_matching_dead_letter gained AND t.shared = FALSE (the shared flag is the
namespace boundary); the owning-group filter is the isolation boundary.

Adds BundleTrigger::DeadLetter + a DeadLetterTriggerSpec manifest kind (group
+shared only — validate_bundle_for rejects app-owned or non-shared, and exempts
it from the shared-requires-a-collection rule); insert_trigger_tx now accepts
dead_letter and writes dead_letter_trigger_details; current_trigger_identity
matches a group-shared dead_letter so re-apply is a NoOp (app-owned ones stay
diff-invisible). Pinned by tests/shared_dead_letter.rs (owning group matches,
per-app query does not, foreign group does not).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 22:20:24 +02:00
MechaCat02
6b2dcd41a6 feat(metrics): per-app observability dashboard over execution_logs (A2)
Aggregates the existing execution_logs table (no hot-path instrumentation):
ExecutionLogRepository::summarize_for_app returns counts, error rate, latency
avg/p50/p95 (percentile_cont), by-status/by-source breakdowns, and an hourly
series over a trailing window (clamped 1h..90d). New metrics_api router at
GET /api/v1/admin/apps/{id}/metrics?window= (AppLogRead), and a dashboard
Metrics tab (summary cards + an inline-SVG per-hour bar chart, no JS dep).

Pinned by a manager-core test asserting the exact rollup (percentiles included)
and the empty-app zero case.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 21:42:18 +02:00
MechaCat02
35345e62de feat(cli): pic pull --group round-trips a group node
Exports a group as a [group] manifest: its scripts, [[routes]]/[[triggers.*]]
TEMPLATES (with the group-only sealed/shared flags), a shared collections
entry, [vars]/[secrets], extension_points, and [suppress]. Groups own no
workflows.

The group trigger/route reports were display-only (no ops, dispatch, retry,
host_kind, cron tz, queue timeout) so a pulled manifest could not re-plan
clean. Enriched TriggerTemplateInfo with the full internally-tagged
TriggerDetails JSON + dispatch_mode + retry_max_attempts, and RouteTemplateInfo
with raw manifest-shaped fields (method/host_kind/host/host_param_name/
path_kind/dispatch_mode); pic routes ls --group now applies the ANY/host munge
client-side, consistent with the app pic routes ls. pull.rs factors the
script-writing + trigger-decoding it shares with the app path.

Pinned by a pull journey: a group with a sealed+shared kv trigger, a sealed
route, a shared collection, and a var round-trips and the exported manifest
re-applies clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 21:19:59 +02:00
MechaCat02
e4a58dc140 feat(cli): per-app docs admin read (pic docs ls/get --app)
Fills the one per-app admin-read gap kv/files already covered. New
docs_api.rs mirrors kv_api (GET /apps/{id}/docs[/{collection}/{doc_id}],
AppDocsRead capability, DocsRepo::list/get) with the group_blobs_api docs
response shape so the CLI shares one deserialize. DocsCmd gains optional
--app/--group (mutually exclusive, like KvCmd) dispatched via
require_one_owner; new client docs_list/docs_get. Read-only by design,
matching kv/files. Pinned by a collections journey: a script writes the
app's own docs collection, the operator lists + fetches it with no script.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 20:53:11 +02:00
MechaCat02
8fc78cd07f docs: correct the stale 'per-app email secret deferred' note
Per-app (non-shared) email inbound secrets already work and are the
original path: an app-owned email trigger resolves its inbound_secret_ref
against the app's own secret store and seals it AAD-bound to the app
(SecretOwner::App, AAD email:{app_id}); M5.5 generalized that app path to
groups, not the reverse, and the inbound path recovers SecretOwner::App
when materialized_from is NULL. Only the interactive API's inline-vs-named
-ref nuance remains, and it is deliberate. Fixes the CLAUDE.md Track-A
tail and the design-doc M5.5 tail that called it deferred.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 20:31:10 +02:00
MechaCat02
80fbf2e642 test: give three no-teeth assertions their teeth
- roles: member invoke asserts the script's actual stdout, not just exit 0;
- api: app-slug script filter asserts the returned script's name;
- workflow_orchestrator: unknown-workflow asserts WorkflowError::NotFound,
  not just is_err().

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 20:13:12 +02:00
MechaCat02
8e68f3a1cb test(crypto): pin the dev-mode acknowledgement gate
Extracts the dev-mode acknowledgement decision into a pure
check_dev_acknowledgement(secret, dev_mode, dev_ack) called by from_env
before key resolution, and pins that dev mode without a secret and
without PICLOUD_DEV_INSECURE_KEY is refused. Behavior-preserving.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 20:13:12 +02:00
MechaCat02
4bf518c86f test(executor): pin the emission budget through the real pubsub SDK
Drives PICLOUD_MAX_EMISSIONS_PER_EXECUTION through the live
pubsub::publish_durable surface (not just the emit_budget counter): a
1001-publish loop errors naming the budget and the service sees at most
1000. Mutation-verified (removing charge_emission from publish fails it).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 20:13:00 +02:00
MechaCat02
a09c346e83 test(dispatcher): pin trigger_depth ceiling in the async outbox path
An outbox row whose trigger_depth exceeds max_trigger_depth is dropped
without executing the handler. Mirrors the disabled-http drop harness;
mutation-verified (weakening the depth gate fails it).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 20:13:00 +02:00
MechaCat02
789325b6db test(authz): pin the editor role-floor on group pubsub + queue write caps
Mirrors group_kv_caps_resolve_by_role_up_the_chain for the two other
group write capabilities: a Viewer is denied GroupPubsubPublish and
GroupQueueEnqueue, an Editor is allowed, and an outsider is denied.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 20:13:00 +02:00
MechaCat02
977abc25d0 test(quota): pin docs ceiling, group docs byte race, and files disk-ordering rollback
Extends the atomic_write suite with three write-path gaps:
- an app cannot exceed PICLOUD_APP_DOCS_MAX_ROWS (create-only, update
  net-zero, delete frees a slot) via a real PostgresDocsWriter;
- concurrent writers cannot push a group past its docs total-bytes quota
  (16 racers against a 450-byte ceiling under pg_advisory_xact_lock);
- a files create with a broken outbox leaves no row AND no blob on disk,
  while a files delete with a broken outbox keeps both the row and the
  readable blob (the disk-ordering invariant).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 20:12:45 +02:00
MechaCat02
345db4a076 test: close the DB-suite hermeticity + vacuous-skip gaps
Three related fixes from the test audit.

**The CLI journey fixture gets its own database.** It spawned a real picloud —
whose dispatcher/orchestrator claim loops are global by design (one instance owns
one database) — against the shared dev DB, so it could claim the manager-core
suites' outbox/workflow rows (the same class of bug already fixed for the e2e
suites, one binary over). It now clones one dedicated database per journey run
from the migrated template. test-support gains `named_test_db_url` (explicit
stable name) + a blocking wrapper for the sync `LazyLock` fixture. The one journey
that talks to Postgres directly (dead-letter injection) now uses the fixture's DB
URL, not the base DATABASE_URL, so it hits the database the server reads.

**workflow_orchestrator moves to per-test databases.** Its `claim_ready_step` is
global, so the old harness serialized every test behind a process-wide CLAIM_LOCK
AND ran `DELETE FROM workflow_runs` (unscoped — it wiped every app's runs) before
each one. A private database per test makes the global claim see only that test's
rows, so both the lock and the unscoped DELETE are deleted.

**DB-backed suites fail loud instead of skipping green.** ~15 manager-core suites
`return None` when DATABASE_URL is unset and report PASS — so in any environment
that lost its database the entire integration surface reports green while running
nothing (why the CI gap went unnoticed for so long). New
`picloud_test_support::abort_if_db_required` panics when `PICLOUD_REQUIRE_DB` is
set (CI now sets it) but DATABASE_URL is not, injected into each suite's skip
path. Local runs without the var still skip cleanly.

Mutation-verified: with PICLOUD_REQUIRE_DB=1 and DATABASE_URL unset, a suite
panics; without the var it skips.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 19:39:13 +02:00
MechaCat02
2445074c87 test(cli): pin the group-create RBAC denial as HTTP 403
`creating_a_group_requires_group_admin_on_the_parent` asserted only that the
member's apply exited non-zero — so it would pass even with the GroupAdmin check
removed, because the apply would still fail for an unrelated reason (a 404 on the
attach-point lookup, a credential error, a 500). It now asserts the stderr carries
`HTTP 403` — specifically the authz denial — matching the rest of the RBAC suite.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 19:38:51 +02:00
MechaCat02
77397e4cb3 test(auth): pin the admin-session absolute-TTL cap
`PICLOUD_SESSION_ABSOLUTE_TTL_HOURS` (audit C1, migration 0070) is the whole
mitigation for a stolen-but-warm admin token, and nothing tested it — grepping
`absolute_expires_at` across all tests returned zero hits.

New DB-backed suite pins the authoritative enforcement: the lookup filter
`absolute_expires_at > NOW()`. A session with a future sliding `expires_at` but a
past absolute cap must NOT resolve (the stolen-but-warm case), and `touch` cannot
resurrect it even when it pushes `expires_at` a year out — the filter is the
backstop regardless of the middleware clamp.

Mutation-verified: dropping the `absolute_expires_at > NOW()` predicate makes both
tests fail. Runs on a private database via picloud-test-support.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 19:38:51 +02:00
MechaCat02
27bda66be6 test(services): pin that the per-value byte caps reject before authz
CLAUDE.md calls the KV/docs/pubsub/queue value-size caps an anti-DoS rail:
oversized payloads are rejected "before authz so anonymous public scripts can't
DoS Postgres." Only queue had a test, and it used an allow-all authz + anon cx —
which catches a DROPPED cap but not a REORDERED one, because an anon cx passes
script_gate regardless.

Each service now has an ordering-proof test: a DENYING authz repo + an
AUTHENTICATED member cx, so a size-check-first service returns *TooLarge while an
authz-first one would return Forbidden. Each pairs it with an under-cap control
through the same denied cx (returns Forbidden) to prove the cx really is denied,
so the *TooLarge case genuinely bypassed authz. Queue's pre-existing test is
upgraded to the same shape (+ a shared member_cx helper).

Mutation-verified: moving the KV size check after authz flips its result to
Forbidden and the test fails.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 19:38:51 +02:00
MechaCat02
a7fb7ea23c test(sdk): give the isolation and retry-negative tests teeth
Three tests that would pass against a broken implementation:

- `kv`/`docs` cross-app isolation asserted only that app B sees nothing — an
  execution_id-keyed (or script_id-keyed) bridge would pass that AND every
  single-execution round-trip, with app-scoped persistence entirely gone. Added
  the positive control: app A re-reads its own value in a LATER execution (a fresh
  execution_id/script_id via `baseline_request`), which only holds if storage is
  keyed by app_id.

- `retry_policy_rejects_bogus_backoff` asserted a bare `is_err()`, which passes if
  `retry::policy` is unregistered (ErrorFunctionNotFound), on a script typo, or if
  every policy is rejected. Now it pins the message (`backoff`) and adds a
  valid-backoff control that must succeed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 19:38:31 +02:00
MechaCat02
95d6b95272 feat(executor): expose ctx.trigger_depth and pin the invoke depth bump
`invoke_callee_sees_incremented_depth` asserted NOTHING — it ended `let _ = resp`
with a comment that `ctx.trigger_depth` wasn't exposed, so it would have passed
even if `invoke` forwarded the depth unchanged, i.e. never incrementing the
chain-depth counter that bounds runaway trigger loops.

`build_ctx_map` now surfaces `ctx.trigger_depth` (read-only: 0 for direct ingress,
+1 per synchronous `invoke` re-entry or dispatched handler) — the value the
author intended to read and a legitimate diagnostic for a script. The test now
pins the increment: the caller is depth 0, the invoke callee must see 1.

Mutation-verified: forwarding the depth unchanged makes the callee see 0 and the
test fails.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 19:38:31 +02:00
MechaCat02
36272c8dac test(groups): pin the anon + isolation gates on the queue and pubsub services
`GroupQueueServiceImpl` shipped with ZERO tests; `GroupPubsubServiceImpl` had four
that exercised neither boundary — its resolver always resolved and its principal
was always an instance Owner (who bypasses the capability check). So on the two
newest shared services, both the isolation boundary and the anonymous fail-closed
gate were unpinned — and they matter most here: a shared-queue consumer is
materialized into EVERY descendant app, so an unauthorized enqueue is
attacker-controlled input fanned out as script execution across a whole subtree.

Both now get the pair their three older siblings have:
`unrelated_app_gets_collection_not_shared` (with a positive control, so a
reject-everything impl can't pass it) and an anon/non-editor fail-closed test that
also confirms reads (depth) stay open.

Mutation-verified: swapping `script_gate_require_principal` for `script_gate` —
the one-token edit that would let an unauthenticated public route enqueue into any
ancestor group's shared queue — makes the anonymous enqueue succeed, and the test
catches it exactly there.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 07:27:53 +02:00
MechaCat02
73c634ae0d test(groups): pin the shared-collection isolation boundary against real SQL
`resolve_owning_group`'s ancestor-chain CTE is the isolation boundary for EVERY
group shared collection — kv, docs, files, topics, queues. Its own doc says so.
Yet no test exercised the actual SQL: the tests that claimed to
(`unrelated_app_gets_collection_not_shared` in the three older group services,
`realtime_authority`'s SSE 404) inject a `FakeResolver` HashMap the test itself
populates — the foreign app resolves to `None` only because the test never
inserted it. They prove the service propagates a `None`; they cannot see the
query that produces it. Dropping `JOIN chain c ON gc.group_id = c.group_owner`
would let ANY app resolve ANY group's shared collection — a cross-tenant breach —
and the whole unit suite would stay green.

This drives the real query over a real tree: a descendant resolves, a
sibling-subtree app gets nothing, the kind filter is part of the boundary (a kv
and a docs collection of one name are distinct stores), nearest-declaration
shadows (security-relevant — a deep app must reach the nearer group's store), an
app-owned marker shares with nobody, and lookup is case-insensitive.

Mutation-verified: with the chain join dropped, the boundary assertion fires — and
under that breach `app_mid` resolves a FOREIGN group's `catalog` left in the DB by
another suite, i.e. the leak reproduces live.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 07:27:53 +02:00
MechaCat02
612074af04 test: give the DDL and global-count suites their own database
Turning CI on surfaced `atomic_write` as intermittently failing under load —
self-inflicted. Its fault injection is `CREATE TRIGGER ... ON outbox`, which takes
an ACCESS EXCLUSIVE lock on a table every other suite is concurrently inserting
into. The header even claimed it "cannot affect any test running in parallel";
that was wrong — scoping the trigger by app_id bounds which ROWS it rejects, not
the table lock installing it takes. Shipping that alongside "run the DB tests in
CI" would have poisoned the signal.

The fix is the one already used for the e2e suites: a private database per test.
That logic (build a migrated template once, clone it per test via
`CREATE DATABASE ... TEMPLATE`) was duplicated in `picloud/tests/common`, and this
would have been a third copy — so it moves into a shared `picloud-test-support`
dev-dependency crate, and `picloud/tests/common` now re-exports it. The invariant
it encodes: manager-core's claim loops are global by design, so a test must not
share a database with anything that runs them OR with anything that does DDL.

Moved onto it:
- `atomic_write` — the DDL fault injection above; its per-test cleanup is now
  deleted (the database is thrown away).
- `outbox_reclaim` — asserts `reclaim_stale_claims` returns exactly 0, a count
  over the WHOLE outbox table. On the shared DB any stale row from any other suite
  (or a killed picloud that died holding a claim) would fail it permanently, until
  someone cleared the table by hand.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 07:27:35 +02:00
MechaCat02
b5ce4cec01 ci: run the DB-backed tests that CI was silently skipping
CI ran `cargo test --workspace` with no `--include-ignored`, so it executed 927
tests and skipped 237 — every DB-backed integration test is
`#[ignore = "needs DATABASE_URL..."]`, which covers ALL of api.rs, ALL of
authz.rs, and the entire CLI journey suite. The isolation and RBAC tests existed
but never ran (AUDIT.md F-Q-014, logged and never remediated). CI provides
Postgres, so it can run them.

Three things had to be right for that to go green:

- **`--all-targets`, not a bare workspace run.** `-- --include-ignored` un-ignores
  not just `#[ignore]` tests but also ` ```ignore ` DOCTESTS, which are
  illustrative pseudocode that does not compile. `--all-targets` runs lib/bins/
  integration tests but excludes doctests (the same reason clippy uses it); a
  separate `--doc` step runs the doctests without the flag. Structural, so a
  future pseudocode doctest can't silently break CI either.

- **The CLI journeys are their own step.** They spawn a real picloud whose
  dispatcher/orchestrator claim loops are global by design; run concurrently with
  the manager-core suites on the shared database they would claim those suites'
  outbox and workflow rows. Sequential steps keep the live server off the DB
  while the other suites use it. The step also rebuilds `-p picloud` first (the
  harness execs the prebuilt binary) and sets the dev-mode env the server needs.

- **A higher `max_connections`.** `#[sqlx::test]` pools are lazy, but mass-parallel
  test startup briefly opens many at once (each test creates its own throwaway
  database); on a many-core box that transient spike exceeded the default 100 and
  Postgres answered "sorry, too many clients already". Steady-state peak is only
  ~26; 500 absorbs the spike with room to spare. Serving the app needs nothing
  like this many. (This is the local compose ceiling; a small CI runner's default
  100 has ample headroom for its lower parallelism.)

Also fixes the test the CI gap had let rot: api.rs asserted `v["schema"] == 66`
with a hand-bump comment, and since nothing ran it, it sat broken from migration
0066 to 0073. It now asserts `/version` surfaces the live constants
(`migrations::latest_version()`, `SDK_VERSION`) — the wiring — while value drift
stays caught by schema_snapshot + check-versioning. A constant hand-synced to
another constant is a chore, not a test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 07:27:21 +02:00
MechaCat02
251d7fe3dd test(e2e): give each dispatcher test its own database
`dispatcher_e2e` failed about one run in three, on a different test each time,
and never under `--test-threads=1`. Root cause, not timing:

Each test calls `build_app`, which spawns a REAL dispatcher, and they all shared
one database — so they shared one `outbox`. `OutboxRepo::claim_due` is
deliberately not app-scoped (in production one dispatcher serves the whole
instance, so claiming any due row is correct). Test A's dispatcher would
therefore claim test B's row, test A would finish, its server would be dropped
mid-dispatch, and the claim was stranded. Test B polled for its handler's side
effect until it timed out.

The harness was modelling something production never does: N independent
instances sharing one database. So the fix belongs in the harness. Weakening the
dispatcher (app-scoping `claim_due`) would be wrong, and shortening
`PICLOUD_OUTBOX_CLAIM_TIMEOUT_SEC` to fit a 10s test window would make production
abandon the live claims of scripts that may legitimately run 300s.

`tests/common` now hands each test its own database. Replaying 73 migrations per
test is too slow, so the first caller builds a migrated TEMPLATE database once
and every test clones it (`CREATE DATABASE ... TEMPLATE` is a file copy), guarded
by a cluster-wide advisory lock because Postgres will not clone a template that
has an open connection. Names are derived from (suite, test), so a test reclaims
its own database on entry: a crashed run leaves nothing to clean up.

Only the five local `pool_or_skip` wrappers change — all 33 call sites are
untouched, and the skip-when-`DATABASE_URL`-is-unset contract is preserved.
(`#[sqlx::test]` already does this and is what `api.rs`/`authz.rs` use, but it
requires `DATABASE_URL` and would force `#[ignore]`, silently dropping these
tests from the default `cargo test --workspace` gate.)

This also stops the e2e tests leaving stranded claims behind in the shared dev
database, which is the likely source of the occasional lone journey failure.

Before: dispatcher_e2e red ~1 run in 3. After: workspace green twice over,
journeys 157/157.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 21:40:27 +02:00
MechaCat02
8f35050499 fix(groups): don't let with_events silently downgrade a transactional writer
`with_events()` and `with_atomic_writes()` both install a writer, last call wins.
Appending them in the wrong order in the host would swap the transactional writer
back out for the best-effort one — silently reverting the write-path invariant AND
dropping the per-group quota enforcement that lives in it, with nothing in the type
system to catch it.

The services now carry an `atomic` flag: once `with_atomic_writes` has run,
`with_events` warns and no-ops instead of clobbering it. Also enforce the group
byte ceiling in `BestEffortGroupFilesWriter::update`, so the non-transactional
writer can't be used as an unmetered bypass and a unit test can catch its removal.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 21:12:01 +02:00
MechaCat02
1fc4080800 fix(outbox): never resurrect a synchronous row on reclaim
The stale-claim reclaimer added earlier in this branch un-claimed every timed-out
row, including SYNCHRONOUS ones. `reply_to.is_some()` is the schema's "never
retry" signal (0009_outbox.sql) and the dispatcher honours it: those rows are an
HTTP request someone is blocked on, and re-running one means re-running its side
effects — a charge, an email, an external POST — for a caller that is long gone.

So a dispatcher dying mid-sync-dispatch would have its row resurrected 10 minutes
later and executed a second time, for nobody. Before the reclaimer those rows were
merely stranded; the reclaimer turned inert into harmful.

The reclaim now gates on `reply_to IS NULL`. Stale sync rows are DELETED instead:
they can never be dispatched again and nobody is waiting on them, so leaving them
would just accumulate garbage.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 21:12:01 +02:00
MechaCat02
fa79b4ee45 fix(files): check the quota BEFORE the destructive blob write
Self-review of this branch caught a data-loss bug I introduced with the
per-app / group files ceilings: the quota check ran AFTER the blob write.

`write_atomic_at` renames the new bytes over the FINAL path, so by the time
the ceiling refused an update the previous bytes were already gone. The
per-app path then unlinked the blob (row survives, file destroyed — every
read 404s); the group path left the new bytes in place under the old row's
checksum (every read fails `Corrupted`). Either way a user permanently lost
a file merely by exceeding a quota — a strictly worse outcome than the
unchecked-update bypass the ceiling was added to close.

The check now precedes the write on create AND update, per-app and group. As
a bonus this stops an over-quota caller driving unbounded write+unlink disk
churn: the ceiling now bounds I/O, not just stored bytes. The blob still goes
down inside the transaction, under the advisory lock, so a rollback unlinks
it and nothing can reference it in between.

The original test passed against the bug: it asserted the refused update did
not change the stored byte TOTAL — true, while the blob was already
destroyed. The regression test now reads the file back through the
checksum-verifying `FsFilesRepo::get`, and was confirmed to fail with
`Corrupted` against the old ordering.

Also in the KV writer (same file): drop the redundant pre-read on the hottest
write path. `set`/`set_if` did a SELECT purely to learn whether the write
would add a row, when the upsert already returns the previous value. Check
after the insert instead (`>` not `>=`) and let the transaction roll back on
refusal — identical outcome, one round-trip fewer, and an update pays nothing
at all. `kv_repo::get_on` and `FsFilesRepo::final_path` are now unused and
deleted; `FilesRepo::delete` delegates to the `delete_meta_on` + `unlink_blob`
helpers rather than re-implementing them.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 21:11:49 +02:00
MechaCat02
735b2daedf docs(users): warn against wiring a users trigger to the non-transactional emit
`users_service` still write-then-emits at nine call sites. That is currently
INERT — `triggers.kind` has no `users` value, so the outbox emitter has no arm
for the source and drops the event — but it is the exact shape audit #6
removed from every other stateful service, and it would be reintroduced by
whoever adds a users trigger kind and finds the plumbing already there.

Note that this one does not even log the failure (`let _ = ...`).

Documented at the emit site, pointing at `atomic_write` and the write-path
invariant in CLAUDE.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 20:47:58 +02:00
MechaCat02
c859c9aab2 fix(outbox): reclaim stale claims — a crash mid-dispatch stranded events forever
Chasing the e2e flakiness turned up a production durability bug, not a test
bug.

The dispatcher claims an outbox row, executes it, then either deletes it
(success) or reschedules it (failure) — both of which clear the claim. If
the PROCESS DIES in between, neither runs. And `claim_due` only ever selects
`claimed_at IS NULL`. Nothing else in the codebase clears `outbox.claimed_at`
— grep it: there are exactly three writers, and those are two of them.

So a crash or restart mid-dispatch stranded every in-flight row PERMANENTLY.
Its trigger never fired and no retry could notice. The outbox is the
universal trigger path, so the loss covered kv / docs / files / cron /
pubsub / email / invoke_async / dead-letter alike. This is the same
durability class as the audit's #6 (lost trigger event), which was just
fixed at the WRITE end — this is the same hole at the READ end.

Every other claim-based store already had the safety net: `queue_messages`
and `group_queue_messages` have `reclaim_visibility_timeouts`,
`workflow_steps` has its own reclaim. The outbox was the one that didn't.

`OutboxRepo::reclaim_stale_claims(timeout)` returns rows whose claim is older
than `PICLOUD_OUTBOX_CLAIM_TIMEOUT_SEC` (default 600s), run from the
dispatcher's existing reclaim ticker. The default is deliberately generous —
a script may run for up to 300s (the `scripts.timeout_seconds` CHECK), so a
claim held past twice that is abandoned rather than slow; reclaiming a row a
LIVE dispatcher is still working on would double-execute it (survivable —
dispatch is at-least-once — but not worth courting).

A reclaim does NOT bump `attempt_count`: the handler never ran, so it must
not consume the row's retry budget, or repeated restarts would dead-letter an
event that executed zero times. (Same reasoning as the transient queue
`release` fixed earlier in this branch.)

This is also the root cause of the flaky e2e suites: each test drops its
dispatcher, and `claim_due` is not scoped per app or per dispatcher, so a
test's dispatcher could claim another test's row and strand it on teardown.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 20:38:29 +02:00
MechaCat02
966b209d2e feat(quota): add per-app storage ceilings
An app's OWN store had no ceiling of any kind — only the group-SHARED
collections did, which is the inversion of what you'd expect. And
`authz::script_gate` returns `Ok(())` when there is no principal, so a
PUBLIC UNAUTHENTICATED route could reach `files::collection(..).create(..)`
and write blobs in a loop until the disk filled. `PICLOUD_FILES_MAX_FILE_SIZE_BYTES`
caps ONE file at 100 MB; nothing capped the count. Same for KV keys and docs
against Postgres.

Ceilings: `PICLOUD_APP_KV_MAX_ROWS` / `PICLOUD_APP_DOCS_MAX_ROWS` (100k) and
`PICLOUD_APP_FILES_MAX_TOTAL_BYTES` (10 GiB).

They are enforced DIFFERENTLY from the group ones, on purpose — porting the
group design as-is would have been a serious regression:

  * KV/docs check a ROW count, on INSERT only, with NO lock. `kv::set` is the
    hottest write path in the system; taking a per-app advisory lock on every
    set would serialize an app's entire data plane, and a `SUM(...)` scan would
    make write cost grow with the app's stored size. What the lock buys is
    small — unlocked overshoot is bounded by write concurrency (32) against a
    ceiling of 100k, ~0.03%. These are anti-DoS rails, not billing. An update
    adds no row and is bounded by the per-value cap, so it pays nothing at all.
  * No per-app BYTE ceiling for KV/docs: every value is already capped at
    `PICLOUD_KV_MAX_VALUE_BYTES`, so `max_rows x max_value_bytes` is ALREADY a
    finite bound. A second scan would buy nothing.
  * FILES are the exception and DO lock + sum: one blob may be 100 MB, so 32
    racing uploads could overshoot by gigabytes of real disk, and a file write
    is heavy enough that the lock and the SUM are lost in the noise. Checked on
    create AND update — an update that skipped the ceiling would be a free
    bypass (the same bug just fixed for group files: grow a 1-byte file to
    100 MB, repeat).

`group_quota` is renamed `quota`: it now serves both owners, and the module
doc is where the two enforcement strategies are contrasted.

Pinned by tests/atomic_write.rs: the key ceiling refuses a new key while
still allowing an update at the ceiling (rejecting updates would brick an app
the moment it filled up — worse than the DoS the rail exists to stop), and 10
concurrent uploads cannot exceed the disk ceiling, nor can an update grow past it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 20:32:32 +02:00
MechaCat02
3de75fe4a8 docs: record the transactional-outbox write-path invariant
The three rules a future change to the data-plane write path must not
break, and the two that are counter-intuitive enough to be re-broken:

  * everything inside a tx runs on that tx's connection (reaching for a
    second pooled connection while holding one can deadlock the pool), and
    `shared` must stay sqlx-free — the transactional path is manager-core
    internal, the `ServiceEventEmitter` trait stays connection-free;
  * a per-group quota needs a LOCK, not just a transaction — under READ
    COMMITTED each tx's COUNT/SUM sees a snapshot without the others'
    uncommitted rows, so concurrent writers all pass the check anyway
    (measured: 19 rows stored against a ceiling of 5);
  * files order around the disk write, which cannot join a transaction.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 20:14:45 +02:00
MechaCat02
80a0d31cd2 fix(files): make file writes transactional and close a quota bypass
Audit #6 and #8 for the last of the three stores, plus a bypass found on
the way.

**#6.** create/update/delete wrote the metadata row, then emitted
best-effort, so an outbox failure left a committed file whose trigger never
fired. `atomic_write::FilesWriter` commits the metadata row and the fan-out
together.

Files are the one store where the ordering is subtle, because the BYTES live
on disk and cannot join a transaction:

  * create/update — blob first, then commit metadata + fan-out. A rollback
    unlinks the blob. (A crash at that exact point still orphans it; that
    hazard predates this change — the repo already wrote the blob and then
    inserted the row in a separate, failable statement — and the orphan is
    inert, referenced by nothing.)
  * delete — commit the metadata removal + fan-out FIRST, then unlink. The
    reverse order would destroy the bytes of a row that a rollback keeps,
    leaving a file that can never be read.

**#8.** `GroupFilesService::create` read `total_bytes` on one connection and
wrote on another, so concurrent uploads each saw the same pre-write total and
together overshot the ceiling. This is the worst instance of the race in the
codebase: the ceiling is DISK (10 GiB by default) and one file may be 100 MB,
so a racing fleet overshoots by gigabytes. `PostgresGroupFilesWriter` takes
the per-group advisory lock (on its own `files` key) across the check and the
write.

**The bypass.** `GroupFilesService::update` checked NO quota at all — so a
1-byte file could be updated to a 100 MB one without the ceiling ever being
consulted, repeatedly, for unbounded disk. It now checks the projected total
(the replaced file's bytes subtracted in SQL, so a same-size-or-smaller
update near the cap still goes through).

Also drive-by: `queue_e2e` asserted the ack the instant the marker appeared,
but the marker is written DURING the handler and the ack happens after it
returns — a zero-tolerance race. It polls now. (This does not fix the
suite's flakiness, which reproduces on the pre-pass commit too.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 20:09:03 +02:00
MechaCat02
58bf0ab3ec fix(docs): make docs writes transactional and the group quota a bound
Audit #6 and #8, applied to the docs store — the same two bugs KV had, in
the same two places.

Per-app docs (#6): create/update/delete wrote the row, then emitted
best-effort. An outbox failure left a committed doc whose trigger never
fired. They now go through `atomic_write::DocsWriter`, whose Postgres impl
writes and fans out on one connection in one transaction.

Group docs (#8): the service read `count_rows`/`projected_total_bytes` on
pooled connections and wrote on another, so concurrent creators each saw the
same pre-write count and together overshot the ceiling.
`PostgresGroupDocsWriter` takes a per-group advisory lock — on its OWN
`docs`-namespaced key, so it serializes against other docs writers to that
group but not against KV writers, which draw on a separate ceiling.

The bespoke `check_total_bytes` (with its own copy of the upper-bound fast
path) is gone; group docs now shares `group_quota::check_group_write` with
group KV, so the row ceiling, the projected-bytes ceiling, and the fast path
that skips the O(n) SUM scan have exactly one implementation between them.

tests/atomic_write.rs gains the docs row-quota race (16 concurrent creators
against a ceiling of 4).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 19:45:26 +02:00
MechaCat02
b086713e3d fix(quota): make the per-group ceiling a bound, not a hint
Audit #8. `GroupKvServiceImpl` read the group's usage (`count_rows` /
`projected_total_bytes`) on one pooled connection and wrote on another. N
concurrent writers therefore each observed the same pre-write total, each
concluded it had room, and together sailed past the ceiling. The code
called this out as "best-effort ... quotas are safety rails, not exact
accounting" — but the overshoot is not small. With a ceiling of 5 and 20
concurrent writers, 19 rows land.

Route group-KV mutations through `atomic_write::GroupKvWriter`, whose
Postgres impl runs the quota reads, the row write, and the shared-trigger
fan-out on ONE connection in ONE transaction — and, crucially, takes a
per-group `pg_advisory_xact_lock` first.

The lock is the fix, not the transaction. Putting the check inside the
writing tx is necessary but NOT sufficient: under READ COMMITTED each
transaction's `COUNT(*)`/`SUM(...)` still sees a snapshot without the other
writers' uncommitted rows, so they all still pass. Serializing the
check-then-write per group is what makes a writer see its predecessor's row.
The lock key is namespaced per kind (kv/docs/…) so writes drawing on
different ceilings don't contend. A delete takes no lock — it only frees
space.

The quota POLICY (row ceiling, projected-bytes ceiling, and the
upper-bound fast path that skips the O(n) SUM scan for a group nowhere near
its cap) moves to one place, `group_quota::check_group_write`, over a small
`GroupUsage` trait. Both backends — the transactional one reading through
`&mut *tx`, and the in-memory one the unit tests use — share it, so there is
exactly one copy of the decision.

`set_if` gains a correctness nicety on the way: the row ceiling now keys on
whether the write ADDS a row, so a compare-and-swap against an absent key
with a `Some` precondition (which cannot swap, and so consumes nothing) is
no longer charged for one.

tests/atomic_write.rs pins both ceilings against 20/16 concurrent writers.
Both tests fail without the lock (19 rows stored against a ceiling of 5).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 19:37:11 +02:00
MechaCat02
01e4e5a8f5 fix(kv): commit a write and its trigger fan-out in one transaction
Audit #6. `KvServiceImpl` wrote the row, waited for it to commit, then
asked the emitter to resolve matching triggers and insert outbox rows —
a second transaction on a second connection. If that second step failed,
the row was permanently in the store with its trigger having never
fired: invisible to the caller (the write "succeeded"), unrecoverable by
any retry, and only ever logged. The code said as much:

    // Audit finding (Medium): this is non-transactional with the data
    // write — the row above has already committed by the time emit()
    // runs, so an emit failure means triggers silently don't fire.

Introduce `atomic_write::KvWriter` — the mutating half of the service (the
write AND the fan-out it produces), so the two can share a transaction:

  * `PostgresKvWriter` opens a tx, writes via `kv_repo::*_on(&mut *tx, …)`,
    runs the fan-out on the SAME connection via `emit_on(&mut *tx, …)`, and
    commits. An emit failure now rolls the write back and surfaces to the
    script, which is the honest outcome — the caller learns the write did
    not happen instead of silently getting a store that disagrees with its
    triggers.
  * `BestEffortKvWriter` keeps the old write-then-log-on-emit-failure
    semantics for the in-memory unit tests (no Postgres).

Both sit behind one trait, so the service body has a single code path and
the choice is a constructor detail (`with_atomic_writes(pool)` in the host).

Pool-deadlock rule, documented on the module: everything inside the tx runs
on the tx's connection. A writer that held a tx and then reached for a
second pooled connection could starve — the pool is sized to the execution
concurrency cap, so N executions each wanting 2 connections deadlock. The
fan-out takes `&mut *tx` and never touches a repo.

`tests/atomic_write.rs` pins it by injecting an outbox failure (a Postgres
BEFORE-INSERT trigger scoped to the test's own app_id, so parallel tests are
unaffected): the set errors and the key is NOT in the store; likewise a
delete rolls back rather than dropping a key nothing downstream hears about.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 19:28:17 +02:00
MechaCat02
a2360a9464 refactor(outbox): make the trigger fan-out connection-scoped
`OutboxEventEmitter` resolved matching triggers and inserted outbox rows
through `Arc<dyn TriggerRepo>` / `Arc<dyn OutboxRepo>`, i.e. against the
pool — each query on whatever connection it happened to get. That makes a
transactional outbox impossible: the data write and the outbox rows can
never share a transaction, so a crash (or an outbox error) between them
silently loses the trigger event.

Take the emitter down to a connection instead of a pool:

  * `outbox_repo::insert_on(exec, row)` and `trigger_repo::list_matching_on`
    / `list_matching_shared_on(exec, ...)` are generic over `PgExecutor`, so
    the same SQL serves a pooled connection or a `&mut *tx`. The repo trait
    methods delegate to them — the SQL keeps exactly one home.
  * `emit_on` / `emit_shared_on` take a `&mut PgConnection` and run the whole
    fan-out on it. The `ServiceEventEmitter` impl acquires one pooled
    connection and calls them, so behaviour is unchanged today; a caller
    holding a transaction can now pass `&mut *tx` and have the outbox rows
    commit with the write.
  * `OutboxEventEmitter::new` takes the `PgPool` directly (it was only ever
    constructed once, in the host wiring).

Also collapses the three copy-pasted per-app match queries (kv/docs/files
differ only in the `kind` discriminator and detail table) and the three
`emit_*` bodies into one `plan()` + one match fn, so the suppression
anti-join, the chain walk, and the empty-ops-means-any-op semantic each
exist once rather than three times.

No behaviour change — pure refactor. It is the seam the transactional
write lands on next.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 19:20:41 +02:00
MechaCat02
155c5471b7 fix(quota): measure the per-group byte projection with one consistent metric
The §11.6 M4 byte quota mixed two incompatible byte measures: `used` came from
Postgres as `octet_length(value::text)` (jsonb CANONICAL text — whitespace after
`:` and `,`, keys normalized), while `old_len`/`new_len` were computed in Rust
with `serde_json::to_vec` (COMPACT, no spaces). The projection
`used - old_len + new_len` therefore subtracted and added a *smaller* measure
than what is actually stored, so a group could be admitted over its ceiling —
the cap drifted permissive. The two forms are not reconcilable in Rust (jsonb
also reorders/dedups keys), so the projection has to be measured by Postgres.

Add `projected_total_bytes` to `GroupKvRepo` (keyed by collection+key) and
`GroupDocsRepo` (keyed by the replaced doc id, `None` on create). Each computes
`SUM(canonical) - existing_row(canonical) + new_value(canonical)` in ONE query,
binding the new value as compact TEXT and re-parsing it through `::jsonb::text`
so PG measures it in exactly the form it stores. All three terms now share one
metric. The services call it instead of the mixed Rust math; the near-cap
fast-path (`rows_after * max_value_bytes <= ceiling` → skip the SUM) is kept, so
the common case still does no extra work. The docs `update` path no longer needs
its old-doc fetch (the subtraction happens in SQL), removing a read.

The in-memory test repos implement the same projection with their own (compact)
metric, so the quota unit tests keep their exact byte arithmetic. No schema
change.

Note: the check-then-write TOCTOU (concurrent writers can each pass and
collectively overshoot) is NOT addressed here — it needs a transaction spanning
the check and the write, the same missing infrastructure as the transactional
outbox. Tracked separately.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 07:18:14 +02:00
MechaCat02
fc70f5e224 fix(materialize): take the per-(app,queue) lock so a consumer slot can't double-fill
The one-consumer-per-(app_id, queue_name) invariant has no DB unique index (the
parent's app_id and the detail's queue_name live in different tables), so it is
enforced by a pg_advisory_xact_lock on hashtext(app_id||queue_name) around a
SELECT-then-INSERT. But `materialize` guarded its own check-then-insert only
with the GLOBAL rematerialize lock, not that per-(app,queue) key — so a
tree-apply / app-create rematerialize could interleave with an interactive
`create_queue_trigger` for the same (app, queue): each SELECT sees no consumer
(the other's INSERT uncommitted), both INSERT, and the app ends up with TWO
enabled consumers draining the same queue, splitting messages unpredictably.

Fix: `materialize_one` now resolves the template's queue_name and takes the SAME
`advisory_lock_key(app_id, queue_name)` (exposed `pub(crate)`) before its slot
check + insert, so the two paths mutually exclude. No deadlock: the interactive
path never acquires the global rematerialize lock, so there is no lock cycle,
and the rematerialize lock already serializes reconcilers against each other.
No schema change.

stateful_templates DB + journey coverage stays green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 22:31:42 +02:00
MechaCat02
8b62b137c0 fix(executor): cap per-execution durable fan-out width
`trigger_depth` bounds how DEEP a trigger/invoke chain runs, but nothing bounded
how WIDE one execution could fan out. A single anonymous request running
`for i in 0..1_000_000 { invoke_async("w", #{}) }` costs a few Rhai ops plus one
cheap outbox INSERT per iteration, so within the op / wall-clock budget it could
write ~10^5 durable rows — each dispatched as its own execution — flooding the
outbox and dispatcher (a durable-amplification DoS).

Add a per-execution ceiling on DURABLE emissions (`invoke_async`,
`pubsub::publish_durable`, `queue::enqueue`, and the shared-topic/-queue
variants), env `PICLOUD_MAX_EMISSIONS_PER_EXECUTION` (default 1000). It's a
thread-local counter wrapped by a re-entrancy-aware `EmissionBudgetScope` around
every `execute_ast`: the OUTERMOST scope resets it, so a fresh dispatched
handler (a new pooled-thread task) gets a full budget while a synchronous
`invoke()` / interceptor re-entry nested in the same call stack SHARES it (fan-
out counted across the whole synchronous chain). The outermost Drop re-zeroes
the counter so a pooled thread never leaks a count into the next task.

Pinned by an `emit_budget` unit test (outermost resets, nested shares, ceiling
trips); the invoke/queue/pubsub/workflow journeys (small counts) stay green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 22:27:32 +02:00
MechaCat02
590b98b60f fix(suppress): don't over-decline a nearer descendant's own route/trigger
A group's template suppression is "coarse by reference" (path for routes,
handler-name for triggers), and both resolution points dropped ANY inherited
row matching that reference across the whole subtree — including one a NEARER
descendant group deliberately re-declared. So an ancestor group G that declines
a far-ancestor's `/x` (or `audit` handler) would also silently kill a child
group H's OWN `/x` / `audit`-bound trigger at the same reference, violating the
documented "an owner can only decline what it inherits, never a descendant's
own rows" invariant.

Fix: gate each decline on the chain DEPTH of the suppressor vs the target's
owner — a suppressor at depth `d_s` may only decline a row whose owner is
strictly ABOVE it (`target_depth > d_s`):
- Routes: `list_route_suppressions` now returns `(app, path, suppressor_depth)`;
  the rebuild folds it to the min depth per `(app, path)` and
  `compile_effective_routes` skips an inherited route only when
  `route.depth > suppressor_depth`. (This also subsumes the old `depth > 0`
  inherited-only gate.)
- Triggers: the dispatch anti-join gains `AND sc.depth < c.depth` (the
  suppressor's chain depth below the trigger owner's), correlating on the outer
  `chain c` that every kv/docs/files/pubsub match query already binds.

An app's own suppression is depth 0 → still declines anything it inherits; a
group's suppression declines only what that group itself inherits. `sealed`
still overrides. No schema change.

Pinned by a new `compile_effective_routes` unit test (a depth-1 descendant
route survives a depth-2 suppression that declines a depth-3 template) and a
new `group_suppression` DB test (same for triggers); all existing suppression /
sealed / template journeys stay green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 22:17:01 +02:00
MechaCat02
31e6fc964d fix(queue): don't count never-executed transient releases against max_attempts
`claim` pre-increments `attempt` before the handler runs, so a real execution
failure legitimately counts toward `max_attempts`. But the two TRANSIENT
release paths — gate saturation (server at capacity) and script-disabled-at-
fire-time — re-queue the message WITHOUT executing it, and previously used the
same `nack` that leaves the pre-increment in place. Under sustained overload a
message could therefore be dead-lettered after N gate-rejections having run
zero times (each rejection burning a retry).

Add a non-counting `release` to both `QueueRepo` and `GroupQueueRepo`
(re-queue + `attempt = GREATEST(attempt - 1, 0)`, mirroring `nack` otherwise)
and route the two transient paths through a new `Dispatcher::q_release`. A real
handler failure still uses `nack` and counts. No schema change.

Pinned by a new `queue_release` DB test (release undoes the claim increment;
nack keeps it) and the updated `disabled_queue_consumer_...` dispatcher test
(now asserts a release, not a nack).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 22:05:32 +02:00
MechaCat02
bdcd3dc7f4 fix(executor): bound Rhai→JSON materialization to prevent anonymous OOM
The per-element Rhai sandbox caps (max_string_size 64 KiB, max_array_size /
max_map_size 10 000) do NOT bound the *materialized* size: Rhai shares strings
and arrays by Rc, so a 10 000-element array of one aliased 64 KiB string is
cheap to build (~10 000 ops, far under the 1 M op budget) yet dealiases to
~640 MiB of distinct JSON. Every Dynamic→JSON conversion deep-copies the
aliases; the HTTP response body path had NO size cap at all, and the KV/docs
value caps run only AFTER full materialization — so a handful of anonymous
requests could OOM the node (32 concurrent × ~640 MiB ≈ 20 GiB) on the stated
consumer-hardware target.

Add a byte-budgeted materializer that bails the moment the budget is exceeded,
so the transient allocation is bounded regardless of aliasing:
- bridge.rs: `dynamic_to_json_capped(value, max) -> Result<Json, JsonSizeError>`
  (charges a running budget) + `MAX_JSON_MATERIALIZE_BYTES` (16 MiB hard rail,
  far above the 256 KiB business caps). `dynamic_to_json` stays infallible for
  best-effort paths (logging) but is now internally bounded — it collapses an
  over-limit value to a marker string instead of OOMing.
- Route every user-value boundary through the fail-closed capped form: KV
  set/set_if (+ the CAS `expected`), docs create/update/find, secrets set,
  http request body, workflow input, `json::stringify`, and the HTTP RESPONSE
  body (previously the one fully-uncapped exit → now a 500). `invoke` args and
  the pubsub/queue message materializers get the same byte budget in place.
- `impl From<JsonSizeError> for Box<EvalAltResult>` so SDK sites protect
  themselves with a bare `?`.

Pinned by bridge unit tests: an aliased 10 000×64 KiB array errors on the
budget rather than materializing, and the infallible wrapper yields a marker
instead of OOMing. Workspace 914 + journeys 157/157 green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 21:50:16 +02:00
MechaCat02
d08df88df5 fix(interceptors): harden the §9.4 slice — seal, fail-closed, re-entrancy, shared coverage
An end-to-end audit of the (unmerged) interceptor slice surfaced seven ways a
KV allow/deny guard could be silently defeated or misbehave. Close all of them:

1. Shared-collection bypass — `kv::shared_collection(...).set/delete` skipped
   the hook entirely, so any `(kv, set/delete)` guard was circumvented by
   choosing the shared handle. `GroupKvHandle` now runs the same before-op hook.

2. Seal to the declaring owner — the marker resolved nearest-owner-wins but its
   script name was then re-resolved on the CALLER's chain, so a descendant app
   could shadow a group's mandatory guard with a same-named local script.
   `resolve_before` now returns the script sealed to the owner that declared the
   marker (one LEFT JOIN in manager-core), so the executor never re-resolves by
   name. A descendant can only override with its OWN explicit marker.

3. Re-entrancy — an "allow + audit-log" guard that itself wrote KV re-triggered
   itself to the depth cap and then DENIED the original write (plus ~8x
   resolve/compile/execute amplification). A thread-local guard makes a write
   performed by an interceptor bypass interception.

4. Fail-closed verdict — only `#{ allowed: false }` denied; a bare bool, a
   typo'd key, a non-bool, or a bare unit all ALLOWED. Now allow ONLY on an
   explicit `#{ allowed: true }`; every other shape denies.

5/6. Fail-closed edges — a dangling (missing/disabled) interceptor script and a
   missing engine back-reference now DENY instead of allowing.

7. AppInvoke coupling — resolution no longer routes through `invoke.resolve`, so
   installing a guard no longer denies writes to principals who hold KV-write
   but not AppInvoke.

The seam (`InterceptorService::resolve_before`) now returns an
`InterceptorResolution { None | Dangling | Run(ResolvedScript) }`; the executor
runs the sealed script straight through the shared `run_resolved_blocking` core
and drops its `InvokeService` dependency. executor-core stays Postgres-free.
No migration change (0073 unchanged).

Pinned by three new journeys in tests/interceptors.rs: the seal (a same-named
app script does NOT shadow the group's guard), shared-collection coverage, and
the fail-closed verdict. Full journey suite 157/157.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 21:20:20 +02:00
MechaCat02
2fc9476f9e feat(interceptors): §9.4 service interceptors — thin KV allow/deny slice
Smallest honest vertical slice of §9.4: a `[[interceptors]]` block (app OR
group) binds a script to run BEFORE `kv::set`/`delete`; it reads the operation
context (`ctx.request.body`: service, action, collection, key, value, caller
ids) and returns `#{ allowed, reason }` — `allowed == false` denies the op (the
write never runs, the caller gets a runtime error).

Reuses two existing mechanisms rather than inventing new ones:
- Registration mirrors extension points (§5.5): a marker table
  `0073_interceptors.sql` (owner-polymorphic app_id/group_id XOR, keyed
  (service, op) → script), `interceptor_repo` (insert/delete/list + the
  nearest-owner-wins `resolve_before` chain walk), reconciled through the
  declarative apply exactly like `vars` (create/update/delete, prunable).
- Execution reuses the `invoke()` re-entry path: the new `InterceptorService`
  (shared trait + Postgres-backed impl) only RESOLVES the script name (keeping
  executor-core Postgres-free); the executor's `sdk::interceptor::run_before`
  resolves that name and runs it via `run_resolved_blocking` (extracted from
  `invoke_blocking` — shared depth bound + AST cache). An un-hooked write pays
  one indexed `Ok(None)` resolve; no interceptor ⇒ zero overhead.

Nearest-owner-wins so an app overrides a group's interceptor, and a group
interceptor is inherited by every descendant app — the chain walk is the
isolation boundary (a sibling subtree never matches). `validate_bundle_for`
restricts the MVP to `service = "kv"`, `op ∈ {set, delete}`, one marker per
(service, op).

Deferred (documented in §9.4): the `data` transform return, services other
than kv, `after_*` hooks, chaining + circular-dependency guard, the timeout
policy, and a `pic interceptors ls` read surface (needs a server route).

Pinned by `tests/interceptors.rs` (deny blocks the write; allow passes;
group→app inheritance), schema snapshot re-blessed. 154/154 journeys pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 20:44:50 +02:00
MechaCat02
fcd9451ab1 docs: reconcile roadmap with shipped v1.2 Workflows track
The blueprint, CLAUDE.md, and the groups design doc still described the
Workflows track as "not-yet-started" and listed a per-app `materialized`
column as deferred — both stale. Workflows shipped M1–M6 (DAG execution +
conditional `when`, nested sub-workflows, durable orchestrator,
`workflow::start` SDK + admin API + `pic workflows`, dashboard DAG +
run-history; migrations 0071/0072), and D1's `materialized` column shipped.

Update the status lines to mark Workflows shipped (leaving §9.4 service
interceptors as the one remaining Workflows-track item and cluster mode as
next), bump the "migrations through 0070" note to 0072, and drop the stale
`materialized`-deferred line. Docs-only.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 20:03:12 +02:00
MechaCat02
3da70a66c7 feat(cli): add pic docs/dead-letters ls group read mirrors
Two read-only operator surfaces existed server-side but the CLI was app-only:

- `pic dead-letters ls --group <slug>` — mirrors the app listing onto a group's
  shared-queue dead-letters (§11.6 D3, `GET /groups/{id}/dead-letters`), via the
  existing `require_one_owner`/`OwnerRef` dispatch. List-only (show/replay/
  resolve stay app-only, matching the server's operator surface); a group DL row
  shows its `collection` and carries no trigger, so it renders its own columns.
- `pic docs ls/get --group <slug>` — a new `Docs` command (`cmds/docs.rs`)
  wrapping `GET /groups/{id}/docs[/{collection}/{id}]`, mirroring `pic kv`.
  Group-only: per-app docs have no admin read route yet (unlike kv/files), so
  there is no `--app` variant — noted for a later pass.

Read-only by design (writes go through the SDK). New client methods +
`GroupDocsListDto`/`GroupDeadLetterDto`. Pinned by a new
`operator_reads_shared_docs_and_group_dead_letters_via_cli` journey
(152/152 pass).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 20:00:37 +02:00
MechaCat02
bc2538b5b5 fix(cli): reject unknown fields in manifest entry specs (fail-loud on typos)
`ManifestScript`, `ManifestRoute`, and every trigger spec
(`KvTriggerSpec`…`QueueTriggerSpec`) lacked `#[serde(deny_unknown_fields)]`,
while their parent structs all had it. So a typo inside a `[[triggers.kv]]` /
`[[routes]]` / `[[scripts]]` entry (e.g. `collection` for `collection_glob`,
`methid` for `method`) was silently dropped instead of erroring — the manifest
applied with the mistyped directive missing. Add the attribute to the 9 entry
structs (all fields are exhaustive, so `pull`-emitted TOML still round-trips —
verified against the full journey suite, 151/151). Pinned by a new
`unknown_key_in_an_entry_spec_is_rejected` unit test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 19:50:02 +02:00
MechaCat02
486cc44a06 refactor(authz): route users_service through the shared authz-verdict mapper
`users_service::require` open-coded the `AuthzDenied → {Forbidden, Backend}`
match that `authz::script_gate` centralizes for every other stateful service —
the one straggler. Extract the mapping into `authz::require_mapped` (used now by
both `script_gate` variants and `users_service::require`), so the verdict→error
mapping lives in exactly one place. Also refresh the stale `to_app_user` comment
(it referenced an unshipped "commit 8" stub; roles are resolved via
`fetch_roles` and passed in). No behavior change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 19:46:22 +02:00
MechaCat02
5e6bb762f4 refactor(groups): dedup the group-service value-size check + shared-emit tail
Two clean, local dedups in the group shared-collection services:

- Extract `GroupKvServiceImpl::check_value_size` — `set` and `set_if` inlined
  the identical encode-and-cap block; now one method (mirrors the sibling
  `group_docs_service::check_data_size`).
- Add `group_collection_repo::best_effort_emit_shared` — the KV/docs/files
  services each copied the same emit-and-log-on-error tail for shared-collection
  triggers. Centralize the tail (logging the event's own `source`/`op`); each
  service still builds its own `ServiceEvent` (payload shapes differ).

Pure refactor, pinned by the existing group-service unit tests. (The
`owning_group` resolver was evaluated for dedup too but left as-is: the five
error enums diverge — `GroupFilesError::InvalidCollection` carries a `String`,
`GroupPubsubError` lacks the variant, and files/pubsub skip the empty-check kv/
docs/queue do — so a shared conversion would be a behavior change, not a
refactor.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 19:44:06 +02:00
MechaCat02
f1b480f2eb refactor(sdk): consolidate the SDK error/bridge helpers
`http.rs` hand-rolled its own `block_on` + `map_http_err`, and `secrets.rs` /
`email.rs` / `users.rs` each carried a byte-identical `runtime_err` — five
`#[allow(clippy::unnecessary_box_returns)]` across four files for one pattern.

Move `runtime_err` to `bridge.rs` beside `block_on` (single home, single allow)
and have the three bridges import it. `http.rs` now calls the shared
`bridge::block_on("http", …)` (same "http:"-prefixed error, so messages are
unchanged) and its `err` validation helper delegates to `bridge::runtime_err`.
Deletes 4 duplicate fns and 3 of the 5 allows; no behavior change (pinned by
the executor-core SDK contract tests).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 19:36:31 +02:00
MechaCat02
aa3f0531f0 fix(workflows): pull hard-errors on a definition-less workflow, not silent skip
Some checks failed
CI / Rust — fmt, clippy, test (push) Failing after 23m16s
CI / Dashboard — check (push) Successful in 9m54s
Re-review flagged that the previous warn-and-skip (aeebdd2) traded a loud
failure for a SILENT prune hazard: against a server too old to return a
workflow's `definition`, pull dropped it with only a stderr warning, exited 0,
and wrote a workflow-less manifest — which a later `apply --prune` uses to
DELETE the server-side workflow (and `pull --force` would clobber a
hand-authored `[[workflows]]` block first).

A live workflow always has ≥1 step (zero-step is rejected at apply), so an
empty `definition` can only mean an out-of-date server. Pull now bails BEFORE
writing anything — consistent with the existing clobber / unsafe-name fail-fast
gates — so nothing partial hits disk and the operator gets an actionable error.
wire_workflow_to_manifest reverts to infallible (the guard upholds its
precondition).

Verified: fmt + clippy -D warnings clean; 5 workflow CLI journeys pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 19:19:03 +02:00
MechaCat02
aeebdd2f3f fix(workflows): close re-review gaps in the pull round-trip + partial-failure UI
A re-review of eaf5ace surfaced three follow-ups, all in the fixes themselves:

- pull vs an old server: a returned empty `definition` can only mean a server
  predating the definition-in-list field (a real zero-step workflow is rejected
  at apply). Emitting a stepless `[[workflows]]` block would just fail the next
  apply, so `wire_workflow_to_manifest` now warns and skips it instead.
- base_ms/backoff defaults: the pull mapper keyed its default-omission off a
  hardcoded `500` literal. `default_base_ms` is now `pub` + re-exported, and the
  mapper omits whatever equals `default_base_ms()` / `WorkflowBackoff::default()`
  — no drift if a default ever changes.
- dashboard: a run that succeeds with an `on_error = continue` step failure now
  carries a partial-failure note in `run.error` (from the compute_advance
  change); the run-detail view rendered it as a red failure, contradicting the
  green "succeeded" badge. It now renders as a `.notice` (warning) for a
  succeeded run, `.error` only for a failed one.

Verified: fmt + clippy -D warnings clean; 450 lib tests; 5 workflow CLI
journeys (incl. the pull→plan round-trip); dashboard npm run check clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 19:08:21 +02:00
MechaCat02
eaf5ace30f fix(workflows): close review gaps — pull round-trip, dup-names, reclaim budget
Adversarial review of the v1.2 Workflows track surfaced two HIGH footguns and
several correctness/hardening gaps. Fixes:

HIGH
- pull round-trip: `pic pull` dropped `[[workflows]]`, so a later
  `apply --prune` silently deleted them. The list endpoint now returns the full
  `definition`; pull rebuilds the manifest block (inverse of workflow_to_wire).
- case-colliding names: two workflows differing only by case collided in the
  reconcile diff (keyed by lower(name)), silently dropping one. Rejected up
  front in validate_bundle_for.

MEDIUM
- reclaim retry budget: a crashed attempt (no outcome) consumed the retry
  budget. reclaim_stale_steps now decrements `attempt` (floored) and clears
  `next_attempt_at`, so a crash no longer counts as a failed try.
- on_error/backoff: typed the manifest fields against the shared enums so a
  bad value fails at TOML parse with a clear message, not an opaque 500.
- dedupe depends_on: a repeated dependency inflated the Kahn in-degree past the
  single decrement, reporting a spurious cycle for a valid DAG. Count distinct.
- canceled child: a canceled sub-workflow resolved the parent step with a
  message naming the cause instead of a generic "failed"; documented that a
  run-level cancel op is not yet supported.

LOW
- list_run_steps is now app-scoped at the query (JOIN workflow_runs) rather
  than relying on caller pre-verification.
- partial failure is surfaced: a run that succeeds with on_error=continue
  failures records them in the run's `error` field.
- admin-started runs log the root_execution_id against the principal.
- documented the deliberate when(missing→false) vs template(missing→fail)
  asymmetry; corrected the claim's atomicity comment.

Tests: unit (dedupe-deps), DB-gated reclaim-budget assertion, and two new CLI
journeys (duplicate-name rejection, pull→plan clean round-trip). fmt + clippy
-D warnings clean; 450 lib + 14 orchestrator DB + 5 workflow journeys pass;
schema snapshot unchanged (no migration).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 18:56:44 +02:00
MechaCat02
bd9c3fa4f0 feat(workflows): M6 — dashboard run-history + DAG view
A per-app "Workflows" tab: list definitions, browse run history, and inspect a
run's per-step progress with a static layered DAG.

- API: the run-detail endpoint now returns each step's `depends_on` (loaded
  from the definition via a new `get_workflow_by_id` reader) so the dashboard
  can draw the graph edges — the run's step rows don't carry them.
- dashboard `$lib/api`: a `workflows` namespace (list / runs / start / run) +
  the matching types.
- `apps/[slug]/workflows/+page.svelte`: workflow table → drill into runs →
  drill into a run. The run view renders a step table plus an SVG DAG (nodes =
  steps colored by status, laid out by longest-path level; edges = depends_on,
  arrowed) and a "Start run" button. Nested/parked steps show their child run.
- AppTabBar + the per-app layout gain the Workflows tab (admin-only).

Tests: `npm run check` clean (0 errors); two Playwright smoke tests
(navigation tabs — workflows loads cleanly + renders its empty state) pass
against the full stack.

With M6 the v1.2 Workflows track (M1 schema/validation → M2 durable
orchestrator → M3 when/templating → M4 nested → M5 SDK/API/CLI → M6 dashboard)
is COMPLETE.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 17:57:56 +02:00
MechaCat02
5a630e1d9f feat(workflows): M5 — SDK workflow::start + admin run API + pic CLI
The user-facing surface for the durable DAG engine. A workflow run can now be
started three ways, all resolving the workflow by name in the caller's app
(`cx.app_id` / the resolved app — never a passed-in arg, the isolation
boundary), seeding a run the orchestrator advances.

SDK seam (shared):
- `WorkflowService` trait + `NoopWorkflowService` + `WorkflowError` in
  `shared::workflow` (mirrors `InvokeService`); added to `Services` as a
  noop-defaulted `with_workflow` builder (all `Services::new` call sites
  unchanged).
- `WorkflowServiceImpl` (manager-core): resolves + seeds a run; authenticated
  callers gated on `AppInvoke` (anonymous skips, script-as-gate), the same gate
  `invoke()` uses.
- `executor-core::sdk::workflow` — the Rhai bridge `workflow::start(name, input)`
  / `workflow::start(name)`, registered in `register_all`. Returns the run id.

Admin API (manager-core `workflows_api`, mounted in the binary):
- `GET  /apps/{app}/workflows`              — definitions (AppRead)
- `POST /apps/{app}/workflows/{name}/runs`  — start a run (AppInvoke)
- `GET  /apps/{app}/workflows/{name}/runs`  — run history (AppRead)
- `GET  /apps/{app}/workflow-runs/{run_id}` — one run + its steps (AppRead)
  `app_id` scopes every query; a foreign run 404s.

CLI (`pic workflows`): `ls` · `run <name> [--input JSON]` · `runs <name>` ·
`run-status <run_id>` — all `--app`-scoped; run definitions stay declarative
(`[[workflows]]` → `pic apply`).

Also: `list_runs_for_workflow` repo reader.

Tests: `workflow_service_start_seeds_a_run` (DB-gated — the SDK/API-shared
entry) + `workflow_run_executes_end_to_end` CLI journey (apply → `pic workflows
run` → poll `run-status` → succeeded, through the real binary + orchestrator).
fmt + clippy -D warnings clean, 449 manager-core lib tests, 14 orchestrator DB
tests, 18 executor-core lib tests, 3 workflow CLI journeys green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 17:48:38 +02:00
MechaCat02
74d5874384 feat(workflows): M4 — nested sub-workflows (start_child, parent-park, depth ceiling)
A `workflow`-kind step now starts a nested sub-workflow run instead of a
function. The orchestrator's step processing is restructured into prepare →
dispatch (`Prep`): after the shared context/`when`/input resolution, a
function step runs through the executor as before, while a workflow step:

- checks the nesting depth ceiling (`PICLOUD_WORKFLOW_MAX_DEPTH`; the child
  would run at parent depth + 1) — over-deep → the step fails, not infinite;
- resolves the child workflow by name in the run's app scope;
- `start_child_and_park`: in one token-gated tx, seeds a child run (depth + 1,
  `parent_run_id`/`parent_step_id` linkage, correlated under the same
  `root_execution_id`) and PARKS the parent step (`running`, claim_token
  cleared, `child_run_id` set). A parked step is never re-claimed (only `ready`
  steps are) nor reclaimed (only *leased* running steps). The token-gated park
  runs before the child insert, so a stale claim writes nothing (no orphan).

Each tick first runs `resume_finished_children`: a parent step parked on a
now-terminal child is resolved (child output → parent step output, or child
error → parent step failed) and the parent run advanced — idempotent via a
`status='running'` gate. The child's own steps are claimed by the same global
scan, so nesting is just more runs. A failed sub-workflow honors the parent
step's `on_error`.

Shared plumbing extracted for reuse: `advance_run_tx` (out of
`complete_step_and_advance`) and `seed_run_tx` (out of `start_run`).

Apply-time soft-warn (`workflow_nesting_warnings`, wired into `plan_warnings`):
a workflow that nests into itself — directly or via a mutual cycle within the
bundle — is flagged at plan (bounded by the depth ceiling, but almost always a
bug). Not an error.

Tests: nested output-flows-to-parent + depth-ceiling-fails-the-run (DB-gated,
end-to-end), self/mutual-cycle warning (pure unit). fmt + clippy -D warnings
clean, 449 manager-core lib tests, 13 orchestrator DB tests, 26 CLI journeys
(workflows/apply/plan/prune) green, binary boots.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 17:30:16 +02:00
MechaCat02
41a21c0551 feat(workflows): M3 — conditional when + input templating at run time
Wire the two pure M1 modules into the orchestrator's execute path:

- Before running a step, build a `RunContext` from the run input + every prior
  succeeded step's output (`load_run_step_outputs`, read fresh at execution
  time so a step sees all upstream results).
- `workflow_expr::eval` the step's `when` condition (if any): false → the step
  is `skipped` — never executed — and counts as satisfied-but-empty for its
  dependents (a new `StepOutcome::Skipped`, written in `complete_step_and_advance`
  then advanced; `compute_advance` already treats skipped as satisfying deps).
- `workflow_template::resolve` the step's `input` against the context: an
  exact single `{{ ref }}` preserves the referenced JSON type, an embedded ref
  interpolates as text; a missing reference is a hard step failure (a definition
  bug surfaced, not hidden), never a silent null.

`when` and templates were already parse-validated at apply time (M1); this is
the runtime half.

Tests (DB-gated, end-to-end via a scripted fake executor):
- `when_false_skips_step` — b skipped, never executed, omitted from run output
- `step_output_flows_into_downstream_input` — a's `{n:7}` feeds b's input,
  type-preserved for a bare ref and interpolated in a larger string
- `missing_input_ref_fails_the_step` — an unresolved ref fails b → run fails

Verified: cargo fmt, clippy -D warnings clean, 448 manager-core lib tests,
11 workflow_orchestrator DB tests (3 new).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 17:18:01 +02:00
MechaCat02
a55c2f112e feat(workflows): M2 — durable orchestrator worker (claim/execute/advance)
The v1.2 Workflows durable DAG engine gains its runtime. A dedicated
background worker (`workflow_orchestrator.rs`, mirroring `cron_scheduler`,
NOT folded into the dispatcher) advances every in-flight run step-by-step,
durably, surviving restarts.

Per tick, two phases:
  A. Claim + execute — up to a small batch of `ready`, due steps are claimed
     with the same `FOR UPDATE SKIP LOCKED` competing-consumer lease the queue
     uses (`claim_ready_step`), one execution-gate permit per step acquired
     BEFORE the claim so the shared gate bounds real parallelism. Each step
     resolves its function by name in the run's app scope (never a
     script-passed arg — the isolation boundary), builds an `ExecRequest`, and
     runs through the injected `ExecutorClient`. Claimed steps run concurrently.
  B. Advance — the outcome is written and the DAG advanced in one token-gated
     transaction (`complete_step_and_advance`): pending steps whose deps are
     satisfied flip to `ready`, and the run's terminal status is recomputed.
     Fan-in falls out (a join waits until its last dep flips it); a stale worker
     matches zero rows and writes nothing.

The graph-advance decision is a pure, DB-free function (`compute_advance`) —
promotions + terminal run status, folding `on_error` fail-vs-continue and
skipped/failed dependency satisfaction — so it is unit-tested in isolation.

Retry uses the step's own policy via `compute_backoff`; a second, slower
cadence reclaims steps leased by a crashed worker (`reclaim_stale_steps`) — the
durability safety net. Steps run with no principal (like invoke_async), and log
under the new `ExecutionSource::Workflow` so `pic logs` surfaces them.

M2 executes function steps only; `when` + input templating land in M3, nested
sub-workflows in M4. Seams present (`StepTarget`, `run_input`, `workflow_depth`).

- migration 0072: widen the `execution_logs.source` CHECK with `workflow`
- shared: `ExecutionSource::Workflow`, `StepStatus::is_terminal`
- workflow_repo: run/step state — `start_run`, `claim_ready_step`,
  `complete_step_and_advance`, `reclaim_stale_steps`, `get_run`,
  `list_run_steps`, `compute_advance` (+ 7 pure unit tests)
- workflow_orchestrator: the worker + config (`PICLOUD_WORKFLOW_*` knobs),
  spawned in `picloud/src/lib.rs` beside the dispatcher/cron scheduler
- tests: 8 DB-gated integration tests (linear, parallel fan-out/fan-in, retry,
  on_error fail/continue, double-complete idempotency, stale-lease reclaim,
  end-to-end tick with a fake executor)

Verified: cargo fmt, clippy -D warnings clean, 448 manager-core lib tests,
8 workflow_orchestrator DB tests, schema snapshot reblessed, M1 workflow CLI
journeys still pass (binary boots with the orchestrator wired in).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 17:12:42 +02:00
MechaCat02
44f992cbb0 feat(workflows): M1 — schema + definition validation + declarative reconcile
First milestone of the v1.2 Workflows track (blueprint §9.1/§9.2): the durable
DAG engine's foundation — the definition model, its validation, and the
declarative `apply` reconcile path. No execution yet (the orchestrator is M2).

- Migration 0071_workflows: `workflows` (owner-polymorphic like scripts, app-
  owned in M1), `workflow_runs`, and `workflow_run_steps` (the per-step
  competing-consumer lease table the M2 orchestrator will claim). Schema golden
  reblessed.
- `shared::workflow`: the `WorkflowDefinition` / `WorkflowStepDef` DTOs (shared
  by the CLI, apply, and the future orchestrator) + run/step status enums.
- Pure, DB-free `workflow_template` (input `{{ input.x }}` / `{{ steps.a.output.y }}`
  resolution, type-preserving) and `workflow_expr` (a small safe JSON-predicate
  evaluator for `when` — keeps manager-core free of a scripting engine).
- `workflow_repo`: read trait + reconcile tx free-fns (insert/update/delete).
- apply_service: `BundleWorkflow` + `Plan.workflows` + `CurrentState.workflows`
  + `ApplyReport.workflows_*`; server-side `validate_workflow_definition`
  (unique/acyclic steps via topological sort, deps exist, function XOR workflow,
  `when`/template parse) rejecting on a group node; `diff_workflows` by
  lower(name) (Update on a definition change) + in-tx reconcile.
- CLI: `[[workflows]]` + `[[workflows.steps]]` manifest structs → wire bundle;
  plan/apply render + report counts.
- Tests: 16 manager-core lib tests (template, expr, definition validation) +
  the `workflows` CLI journey (apply → NoOp → prune; cyclic DAG rejected).

Deferred to later milestones: the orchestrator worker (M2), conditional/template
runtime wiring (M3), nested sub-workflows (M4), the SDK/API/CLI run surface
(M5), the dashboard (M6). The `group_id` column + run/step tables ship now so
those slot in without a migration churn.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 16:46:32 +02:00
MechaCat02
c8c4f012ff docs: reconcile blueprint/design-doc/CLAUDE.md to shipped v1.2 state
A five-agent roadmap-verification pass found the v1.2 Hierarchies code fully and
correctly implemented, but the three source-of-truth docs carried stale
"deferred/remaining/v1.3" notes — and one self-contradiction — for features that
have since shipped (§3 M3 server-side approval gate, §6 group create/reparent, the
Track A closeout, and the 2026-07-11 audit remediation). Reconcile all three to
reflect the true code; no behavior change.

CLAUDE.md: retract the inline "groups pre-exist" / "per-env approval deferred" /
"shared-topic SSE deferred" / "group dead-letter store deferred" / "email v0/no-AAD"
notes → mark §6, §3 M3, and Track A M1–M6 shipped; add per-group KV/docs byte
quotas + set_if; note migration 0063; refresh the "Out of MVP" section.

design doc: fix the header ("Phases 4–6 remain" → all shipped); resolve the §11.6
self-contradiction (the "Deferred" list still named shared-topic SSE / byte quotas /
set_if / operator admin API, all shipped — line 1317 already said SSE shipped); mark
the D3 dead-letter store shipped; retract the stale "groups pre-exist / §6 deferred"
forward references.

blueprint (comprehensive sweep): dashboard Alpine.js → SvelteKit + CodeMirror
(diagram, §3.3, tech table); Docker-per-execution → embedded in-process Rhai
(diagram + data flow); §12 Phase 4 "current focus" → shipped through v1.1.9; Phase 5
"in active development" + "Remaining" block → Hierarchies complete; per-app RBAC
"v1.3+" → shipped Phase 3.5; SDK reference → handle-pattern notation note, S3 tag
v1.1 → v1.3+; MVP schema + docker-compose flagged non-authoritative; §9 header noted
Hierarchies-shipped / Workflows-future; top status line refreshed.

Also fix two behavior-neutral stale in-code comments (sdk/kv.rs `kv::shared` →
`kv::shared_collection`; dispatcher.rs `q_terminal` "no group dead-letter store yet"
→ dead-letters to group_dead_letters, Track A M2).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 14:53:34 +02:00
MechaCat02
c03c0c83c5 feat(dashboard): group data-plane read surfaces + CodeEditor readOnly fix
Remediate the dashboard-coverage and editor findings from the 2026-07-11 audit.

Close the group-surface gap: a GroupTabBar plus read-only surfaces for the v1.2
group data plane the dashboard previously exposed only via the CLI — vars,
secrets (names + timestamps only, never values), shared collections (KV/docs/
files browsers), scripts, triggers, routes, suppressions, extension points,
dead-letters, and ownership. All server data renders through Svelte's escaped
interpolation (no `{@html}`); each tab hits its existing
`/api/v1/admin/groups/{id}/...` read endpoint with independent loading/empty
state.

B7 — the CodeEditor `readOnly` prop is now reactive via a CodeMirror
Compartment, fixing the race where an authorized user opening a script on a warm
SPA nav got a permanently read-only editor (role resolved after mount). The
reconfigure preserves editor content.

A viewer lacking a read cap now sees a calm "you don't have permission" note on
every tab (structured LoadError + classifyError) instead of a red error panel,
generalizing the one-off Secrets 403 handling.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 23:48:09 +02:00
MechaCat02
a3c4267f6c feat(cli): plan warnings, apply confirm + blast-radius gate, files --group, unified owner-XOR
Remediate the CLI/DX findings from the 2026-07-11 audit.

B4 — `pic plan` now previews the apply-time desired-state warnings (disabled
binding, unreachable endpoint, dangling `[suppress]`), so plan and apply agree
for CI review. Wire fields are `#[serde(default)]` (older-server tolerant).

B5 — `pic apply` no longer mutates on a first run (no recorded plan) without a
preview + confirm, and a large blast radius now triggers an extra confirmation.
Both gates check `is_terminal()` before any read — a non-TTY never hangs on
stdin and fails closed without `--yes`; `--yes`/`--force` bypass headlessly.
`--force` help now notes it also skips these prompts.

C3 — `pic files ls`/`get` gain `--group`, mirroring `pic kv --group`, so the
shipped group shared-files admin surface is reachable from the CLI.

C4 — a shared `require_one_owner(app, group)` helper (cmds/mod.rs) gives one
canonical message for the `--app`/`--group` XOR, wired into every such site
(kv, files, vars, secrets, suppress, extension-points, triggers ls, scripts
deploy). routes ls (positional script_id) and scripts ls (lists all) keep their
own shapes deliberately.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 23:47:59 +02:00
MechaCat02
5e1bda1f32 fix(core): cron double-fire, materialize warnings, group quota fast path, SSE channel cap
Remediate the MEDIUM backend correctness/perf findings from the 2026-07-11 audit.

B1 — a group cron template toggled disabled→enabled no longer re-fires every
descendant's cron. Materialization now KEEPS a disabled cron template's copies
(syncing `enabled` in place) instead of delete+recreate, preserving
`last_fired_at`; the scheduler already skips disabled copies, so keeping them is
inert. Queue/email arms still delete-on-disable (freeing the one-consumer slot).

B2 — `rematerialize_stateful_templates` warnings are now logged at the
app-create/delete and group-reparent chokepoints (were silently dropped); only
the `Err` arm was handled before.

B3 — the group KV/docs byte-quota check skips the O(rows) `SUM(octet_length(..))`
scan when a cheap upper bound (`rows_after * max_value_bytes`) is already under
the cap, so the full aggregate only runs near-cap. Every row is validated
≤ max_value_bytes, so the bound is sound (never lets an over-quota write through).

B6 — the in-process SSE broadcaster caps live channels per map
(`PICLOUD_REALTIME_MAX_CHANNELS`, default 100k); a subscribe that would open a
NEW channel past the cap is refused with 503 + `Retry-After: 1` rather than
growing the (app,topic)/(group,topic) maps unboundedly. Check+insert is atomic
under the map mutex (no TOCTOU); an existing-channel subscribe is exempt.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 23:47:47 +02:00
MechaCat02
86448beb06 fix(security): server-authoritative approval gate + auth/session/file hardening
Remediate the HIGH and security-relevant findings from the 2026-07-11 audit.

H1 — the per-env approval gate is now server-authoritative. The governing
project is resolved from the target node's nearest-claimed ancestor
(`governing_env_policy`/`_tree` + `governing_project_id` +
`ProjectRepository::get_environments_by_id`), independent of the client-supplied
`[project]`. Omitting or spoofing the project block can no longer skip a gate the
owning project established; a to-create group resolves its declared parent's
chain so a fresh subtree node inherits the gate. Fails closed on any read error.

H2 — the API-key prefix slice (`&rest[..8]`) is now the boundary-safe
`rest.get(..8)`, so an attacker-supplied multibyte bearer can't panic the request
task (unauthenticated per-request DoS). Regression test added.

C1 — admin sessions gain an absolute lifetime cap (migration 0070,
`PICLOUD_SESSION_ABSOLUTE_TTL_HOURS`, default 30d): `lookup` filters it, `touch`
clamps the sliding bump to it, so a continuously-used or stolen-but-warm token
self-expires. Mirrors the data-plane app-user cap.

C2 — `Cache-Control: no-store` on the login and API-key-mint responses (the two
that return a raw credential), so a proxy/CDN/browser cache can't retain it.

B8 — file downloads are header-safe: `sanitize_stored_filename` guarantees a
valid `HeaderValue` (no panic on a control-char name) and BOTH the per-app and
group download paths now set attachment + `X-Content-Type-Options: nosniff` +
a restrictive CSP, closing a group-path stored-XSS gap.

Also folds in the server-side plan-warning plumbing (`plan_warnings`,
`PlanResult::warnings`) and the `app_only_reject` message helper that the CLI
plan-preview change builds on, plus operator security notes (reads-open shared-
topic SSE; the `--env` label is advisory, not a boundary).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 23:47:35 +02:00
MechaCat02
b8b047368e feat(realtime): external SSE subscription for group shared topics (§11.6 D2 / Track A M6)
Shared TOPICS fanned out only to in-cluster trigger handlers; external clients
could not subscribe (per-app topics already can). Add SSE for shared topics.

- RealtimeBroadcaster gains a parallel (group_id, topic) channel map:
  subscribe_group / publish_group / drop_group_topic (default no-ops so
  NoopRealtimeBroadcaster + test doubles are untouched). InProcessBroadcaster
  implements them with a second map; GC + channel_count span both.
- Route GET /realtime/shared/topics/{topic}: Host->app dispatch (as the per-app
  route), then RealtimeAuthority::authorize_subscribe_shared resolves the OWNING
  GROUP from the app's chain (kind=topic, root segment). Reads-open model — the
  resolution IS the authorization, consistent with in-script shared reads; a
  foreign-subtree app never resolves (404, the isolation boundary). No principal
  machinery needed.
- GroupPubsubServiceImpl::with_realtime bridges a shared-topic publish to the
  owning-group channel (best-effort) after the durable trigger fan-out.
- Host wires the broadcaster into the group pubsub service + the collection
  resolver into the authority.

Auth-model note: chose reads-open (subtree app's Host is the grant) over
"authenticated principal + GroupKvRead" — it's both simpler and faithful to how
shared-collection reads already work. Pinned by realtime broadcaster group-map
tests, realtime_api shared-route tests (404 + stream), and
group_pubsub_service::publish_bridges_to_the_group_broadcaster. No migration.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 15:33:39 +02:00
MechaCat02
0f05c270d1 feat(kv): set_if compare-and-swap for KV (per-app + shared + SDK) (Track A M5)
No atomic CAS existed, so concurrent writers to the same KV key raced. Add
set_if(key, expected, new): writes only when the current value equals expected,
or (expected absent) only when the key is missing — the primitive for lock-free
counters / optimistic concurrency.

- KvService + GroupKvService traits gain set_if with a defaulted "unsupported"
  impl (Noop + other impls untouched); the real services override it.
- kv_repo/group_kv_repo: atomic set_if — a single UPDATE ... WHERE value =
  $expected (JSONB semantic equality) or INSERT ... ON CONFLICT DO NOTHING.
  Atomic without a transaction; returns whether the write happened.
- Services enforce the same value-size cap + (group) writes-authed + row/byte
  quotas as set; the event fires only on a successful swap.
- Executor Rhai SDK: kv::collection(c).set_if(key, expected, new) and the
  shared_collection variant; Rhai () for expected means "only if absent".

Pinned by kv_service/group_kv_service set_if unit tests (CAS + fail-closed) +
sdk_kv::kv_set_if_compare_and_swap (through a real script). No migration.
docs/sdk-shape.md documents the primitive.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 15:19:03 +02:00
MechaCat02
636106e9ea feat(quota): per-group total-byte quotas for shared KV + docs (Track A M4)
M3 quotas capped a group's shared KV/docs by ROW count only; a group could still
blow past storage with few-but-huge values. Add the symmetric total-bytes ceiling
(files already had one).

- group_quota: PICLOUD_GROUP_{KV,DOCS}_MAX_TOTAL_BYTES env vars (default 256 MiB)
  + accessors.
- GroupKvRepo/GroupDocsRepo: total_bytes(group) = SUM(octet_length(value::text))
  (default Ok(0) so non-Postgres impls skip the check).
- GroupKv/DocsServiceImpl: check the PROJECTED total (old value's bytes
  subtracted, new added) on every set/create/update, so a same-or-smaller update
  near the cap is still allowed (unlike a naive used+new check). New
  TotalBytesQuotaExceeded errors; +with_max_total_bytes for tests.
- KV set switched has->get to obtain the old value's size for the delta.

Pinned by group_kv_service + group_docs_service per_group_byte_quota_uses_the_
projected_total unit tests (reject over-cap, allow same/smaller update near cap).
No migration. CLAUDE.md env-var table updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 15:09:01 +02:00
MechaCat02
1a69778c0c feat(secrets): AAD-bind email-trigger inbound secrets v0->v1 (Track A M3)
email_trigger_details.inbound_secret_encrypted was sealed v0 (no AAD, bound only
to the master key) because the table had no version column — a ciphertext could
in principle be relocated between rows (audit 2026-06-11 H-D1, Medium). Bring the
AAD versioning the `secrets` table gained in 0042 to email secrets.

- Migration 0069: `email_trigger_details.inbound_secret_version SMALLINT DEFAULT 0`.
- secrets_service: `seal_email`/`open_email` seal v1 with AAD bound to the
  SEALING OWNER (`email:{app}` / `email:group:{group}`) — deliberately NOT the
  per-row trigger_id, so a group template's sealed bytes stay valid when the
  materializer copies them verbatim to each descendant (all share the group AAD).
  v0 rows keep their exact legacy no-AAD read path.
- Both email-trigger create paths (declarative apply resolve_and_seal +
  triggers_api create_email_trigger) seal v1 under the trigger's owner; the
  version threads through CreateEmailTrigger + insert_email_trigger_tx.
- materialize copies inbound_secret_version verbatim with the bytes.
- email_inbound_target recovers the sealing owner (materialized_from ->
  template.group_id, else the app) so receive_inbound_email opens a v1 secret
  under the right AAD.

Cross-app/tenant relocation now fails the GCM tag; a same-owner swap is not
distinguished (accepted low-severity residual). Pinned by
email_secret_aad::materialized_email_copy_decrypts_under_the_group_aad (the
novel materialized-copy path) + the extended secret_round_trips_through_seal_open
lib test. Schema golden reblessed; materialization + email e2e regressions green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 14:58:35 +02:00
MechaCat02
fd4336e883 feat(queue): dead-letter store for group shared queues (§11.6 D3 / Track A M2)
An exhausted SHARED durable-queue message was `drop_exhausted()`-ed with a
warning — silent data loss. Per-app queues persist to `dead_letters` (0010);
add the symmetric group store so an exhausted shared-queue message is preserved
and operator-visible.

- Migration 0068: `group_dead_letters`, keyed by (group_id, collection),
  CASCADE on the group (an app delete leaves the data — it belongs to the
  group, not the consuming app).
- `GroupQueueRepo::dead_letter` (replaces `drop_exhausted`): one tx that INSERTs
  the dead-letter + DELETEs the live message, filtered by claim_token so a lost
  lease can't dead-letter a re-claimed message (mirrors queue_repo::dead_letter).
- Dispatcher `q_terminal` shared arm now dead-letters instead of dropping. It
  returns None (not the dl id) so the per-app `fan_out_dead_letter` is SKIPPED —
  firing the consuming app's per-app handlers on a shared message (competing
  consumers → nondeterministic app) would be wrong. Fan-out to a *shared*
  dead_letter trigger is deferred (needs a new trigger kind).
- Read side: `GroupDeadLetterRepo::list_for_group` backs a new read-only
  operator endpoint GET /api/v1/admin/groups/{id}/dead-letters (GroupKvRead,
  mirrors the M4 group-blobs surface). `pic dead-letters ls --group` deferred
  (optional; the HTTP endpoint is the operator surface).

Pinned by group_queue::dead_letter_moves_an_exhausted_message_to_the_group_store
(store move + claim-token-mismatch guard + operator read). Schema golden
reblessed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 14:41:04 +02:00
MechaCat02
c06a9e801e feat(apply): make the per-env approval gate hermetic (§3 M3 / Track A M1)
The per-env approval policy was applier-supplied — a hand-crafted request
that omitted `project.environments` was ungated, and flipping a gate to
`confirm = false` in the same request un-gated it. Persist the policy
server-side and enforce against `persisted ∪ declared`.

- Migration 0067: `project_environments (project_id, env_name, confirm)`,
  CASCADE on the project. Written declaratively (delete-then-insert) inside
  `upsert_project_tx`, same tx as the project row + node claim.
- `ProjectRepository::get_environments_by_slug` (read side) +
  `ApplyService::effective_env_policy` union the persisted policy with the
  request's declared one (confirm if EITHER says so — monotonic).
- `env_gate_check` now evaluates the effective policy; the three handlers load
  it before the gate. `plan.approvals_required` is the effective gated set, so
  CI sees a persisted gate even when the manifest omits it.
- The union rule closes both bypasses: an omitted policy still trips a
  persisted gate, and an ungating apply must itself pass the gate (the
  declarative replace then takes effect next time) — TOCTOU-safe.

Pinned by env_approval::{persisted_policy_gates_a_request_that_omits_it,
flipping_a_gate_to_false_in_the_same_request_still_gates} +
projects_repo::get_environments_by_slug_returns_persisted_policy. Schema golden
reblessed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 14:31:15 +02:00
MechaCat02
4b4b3148b7 feat(apply): server-side enforcement of per-env approval gating (§3 M3 / §4.2)
The per-env approval gate (`[project.environments]`, `confirm = true`) was
client-side only — the policy was `#[serde(skip_serializing)]`, so a patched or
older CLI applying to a confirm-required env bypassed it entirely. This makes
the gate server-enforced, admin-gated, and audited (the deferred §4.2 piece).

- Wire: `ProjectDecl` gains `environments` (the CLI stops skip_serializing it and
  sends the selected `env` + the `--approve` set on the apply request). The
  server RE-DERIVES the gate from the request (`env_gate_check`): a
  confirm-required env not in `approved_envs` → 409 `ApprovalRequired`.
- Admin gate: an approved override additionally requires `AppAdmin`/`GroupAdmin`
  on EVERY declared node — a step up from the editor write caps an ordinary
  apply needs (reusing the admin caps per §4.2) — and emits a `tracing::info!`
  audit line. Enforced on all three handlers (single app, single group, tree);
  the check runs before any tx, so a denial (403) precedes any mutation.
- Token: the env policy folds into the TREE bound-plan token (a policy edit
  between plan and apply trips StateMoved). The single-node token is left
  unchanged (a bare live-state hash), so plan/apply still match without threading
  a project into it.
- CLI: single-node `apply`/`plan` now resolve the GOVERNING project (own block
  else nearest ancestor's) so a leaf apply carries the root's policy; `plan`
  surfaces `approvals_required`. Client-side `require_env_approval` fast-fail
  kept as UX.

Adapts the earlier feat/ownership-and-approval M5 (654e387) to main's DTOs; main
addresses tree group nodes by slug only, so the admin loop mirrors authz_tree
(app: resolve_app_id fail-closed; group: get_by_slug, skip to-create).

Threat-model note (honest scope, in the doc + code comments): this closes the
patched/older-CLI bypass and admin-gates + audits the override, but is NOT a
hermetic boundary against a hand-crafted request that OMITS the policy — the
policy is applier-supplied, not persisted server state. Full closure (persist a
`project_environments` source of truth + a server-determined env) is deferred.

Review (adversarial pass): fixed a MEDIUM (single-node `plan` didn't resolve the
governing project, so a leaf plan under-reported the gate `apply` enforces) and a
LOW (duplicate clippy attr); corrected overstated "can't bypass" wording.

Tests: ProjectDecl gate unit test; env_approval journeys
`server_enforces_the_gate_against_a_non_stock_client` (raw request → 409 without
approval, admin-approved → 200) and `approving_a_gated_apply_requires_admin`
(editor + --approve → 403). 417 lib tests + 33 CLI journeys + workspace clippy
-D warnings + fmt clean. No migration.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 14:01:04 +02:00
MechaCat02
bbfdb1bf47 fix(apply): bound the blast-radius subtree down-walk CTE
`group_blast_radius`'s `app_chain` up-walk is depth-capped (`ac.depth < 64`,
added by 76926de to stop a should-be-impossible `groups.parent_id` cycle from
spinning the recursive CTE forever), but the sibling `subtree` down-walk in the
same query was left unbounded. A `parent_id` cycle would loop `subtree`
indefinitely and hang the plan — the exact exposure 76926de set out to close,
left half-done on the descendant side.

Add a `depth` column + `WHERE s.depth < 64` to `subtree`, mirroring `app_chain`.
Defense-in-depth: the reparent cycle-guard already prevents `parent_id` cycles,
so this hardens against an unreachable state — but that is precisely the bar
76926de applied. Surfaced by a post-adoption audit of the §7/§6 ownership code.

Verified: 416 manager-core lib tests; 12 ownership/create/divergence/approval
journeys (incl. plan_previews_ownership_and_blast_radius, which drives this
query); clippy -D warnings + fmt clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 13:25:06 +02:00
278 changed files with 32319 additions and 2451 deletions

View File

@@ -37,6 +37,25 @@
"Bash(dig *)",
"Bash(host *)",
"Bash(./target/debug/pic plan *)",
"Bash(./target/debug/pic config *)",
"Bash(./target/debug/pic whoami)",
"Bash(./target/debug/pic pull *)",
"Bash(./target/debug/pic apps domains ls *)",
"Bash(./target/debug/pic routes ls *)",
"Bash(./target/debug/pic kv get *)",
"Bash(./target/debug/pic kv ls *)",
"Bash(./target/debug/pic dead-letters ls *)",
"Bash(/home/fabi/PiCloud/target/debug/pic plan *)",
"Bash(/home/fabi/PiCloud/target/debug/pic config *)",
"Bash(/home/fabi/PiCloud/target/debug/pic whoami)",
"Bash(/home/fabi/PiCloud/target/debug/pic pull *)",
"Bash(/home/fabi/PiCloud/target/debug/pic apps domains ls *)",
"Bash(/home/fabi/PiCloud/target/debug/pic routes ls *)",
"Bash(/home/fabi/PiCloud/target/debug/pic kv get *)",
"Bash(/home/fabi/PiCloud/target/debug/pic kv ls *)",
"Bash(/home/fabi/PiCloud/target/debug/pic dead-letters ls *)",
"Bash(ls)",
"Bash(ls *)",
"Bash(pwd)",

View File

@@ -10,6 +10,11 @@ env:
# Matches what docker-compose produces locally; the schema-snapshot
# guardrail and any other DB-backed tests run against this service.
DATABASE_URL: postgres://picloud:picloud@localhost:5432/picloud
# The DB-backed suites skip themselves when DATABASE_URL is unset. In CI it is
# always set, so this makes that skip a hard error: if the database ever goes
# missing (a broken service container, a lost env), the build goes RED instead
# of green-but-empty. Honored by picloud_test_support::abort_if_db_required.
PICLOUD_REQUIRE_DB: "1"
jobs:
rust:
@@ -47,11 +52,49 @@ jobs:
- name: Clippy
run: cargo clippy --all-targets --all-features -- -D warnings
# Runs the whole workspace, including the schema-snapshot guardrail
# (it picks up DATABASE_URL from the env above and the postgres
# service; without a DB it would skip cleanly).
- name: Test
run: cargo test --workspace
# `--include-ignored` is load-bearing. Every DB-backed integration test is
# `#[ignore = "needs DATABASE_URL..."]`, and CI omitted the flag — so CI ran
# 927 tests and silently skipped 237, among them ALL of authz.rs and api.rs
# and the entire CLI journey suite. The isolation and RBAC tests existed but
# never executed (AUDIT.md F-Q-014). CI does provide Postgres, so run them.
#
# `--all-targets` (not a bare `cargo test`) is deliberate: it runs the lib,
# bins, and integration tests but NOT doctests. `-- --include-ignored`
# un-ignores not just `#[ignore]` tests but also ` ```ignore ` DOCTESTS,
# which are illustrative pseudocode that does not compile — so a bare
# `cargo test ... -- --include-ignored` fails on them. Doctests run in their
# own step below, without the flag. (Clippy already uses `--all-targets` for
# the same doctest-excluding reason.)
#
# The CLI journeys are a SEPARATE step, and deliberately not part of the
# workspace run: they spawn a real picloud whose dispatcher/orchestrator
# claim loops are global by design (one instance owns one database). Run
# concurrently with the manager-core suites — which share this database —
# it would claim their outbox and workflow rows out from under them. Keeping
# the steps sequential keeps that live server off the shared DB while the
# other suites are using it.
- name: Test (workspace, including DB-backed tests)
run: cargo test --workspace --exclude picloud-cli --all-targets -- --include-ignored
# Doctests, run WITHOUT --include-ignored so ` ```ignore ` snippets stay
# ignored. `--all-targets` above skips these, so nothing else covers them.
- name: Doctests
run: cargo test --workspace --doc
# The journey harness execs the prebuilt target/debug/picloud and does NOT
# rebuild it, so a stale binary would silently test old server code.
- name: Build picloud (the journey harness execs this binary)
run: cargo build -p picloud
# The spawned server inherits this env; without a secret key it aborts at
# startup and every journey fails as "/healthz never returned 200".
# `--all-targets` for the same reason as above (the journeys are `#[ignore]`
# integration tests; picloud-cli's doctests, if any, run in the Doctests step).
- name: Test (CLI journeys)
env:
PICLOUD_DEV_MODE: "true"
PICLOUD_DEV_INSECURE_KEY: i-understand-this-is-insecure
run: cargo test -p picloud-cli --all-targets -- --include-ignored
dashboard:
name: Dashboard — check

View File

@@ -1,5 +1,62 @@
# PiCloud Changelog
## Unreleased — CMS proof-of-concept remediation
Fixes surfaced by building a full CMS on PiCloud (`examples/cms-poc/`). Pure
additive surface + one migration (`0077_apps_cors.sql`), no destructive changes.
### Security
- **502 info-leak closed on user routes.** An uncaught script error (incl. a
`throw`) on a user route returned HTTP 502 whose body embedded the app UUID,
script function names, and source line/col. The 2026-06-11 audit scrub had
only covered the execute-by-id path; the user-route (inbox) path still leaked.
Both paths now share `scrub_runtime_detail` — the raw detail is logged
server-side under a correlation id and the client sees only
`script execution error (ref: <uuid>)`.
### Added
- **`docs::find` `$contains` operator** — array-membership via JSONB `@>`, e.g.
`docs::find("posts", #{ tags: #{ "$contains": "rust" } })` finds every post
tagged `rust`. The inverse of `$in`. Covers per-app and group-shared docs.
- **App-user role management from the API/CLI** — `pic users add-role --app <app>
<user_id> <role>` / `rm-role`, backed by `POST`/`DELETE
/api/v1/admin/apps/{id}/users/{user_id}/roles[/{role}]` (`AppUsersAdmin`).
Role assignment was previously reachable only from a Rhai script.
- **Per-app CORS** — `pic apps cors set <app> <origins…>` (or `*`), stored on
`apps.cors_allowed_origins`. The orchestrator echoes an allowed `Origin` on
every response and answers the `OPTIONS` preflight. Empty = off (unchanged).
- **Non-JSON request bodies** — a user route no longer 400s on a non-JSON body.
`application/x-www-form-urlencoded` → `ctx.request.body` object, `text/*` →
string, other types → base64 string; `ctx.request.content_type` added as a
convenience. JSON (or unspecified) is unchanged; malformed JSON still 400s.
- **Binary responses** — a script may return `#{ statusCode, headers,
body_base64 }`; the platform decodes the base64 and returns raw bytes with the
script-set `Content-Type` (default `application/octet-stream`).
- **`workflow::run_status(run_id)` SDK** (F-038) — the script that started a run
can now poll it: returns `#{ status, output, error, steps: #{ name: status } }`
or `()` when no such run belongs to the app. Read-only, same-app scoped (the
run's `app_id` is the isolation boundary), same `AppInvoke` gate as
`workflow::start`. Closes the gap where run status was platform-admin-API only.
### Fixed
- **`pic apply` "app not found" is now actionable** — points at
`pic apps create <slug>` (apply reconciles groups but never creates an app).
- **Interceptor- and workflow-step-bound scripts no longer warned "no route or
trigger"** at plan time — an `[[interceptors]]` binding (F-021) *and* a
`[[workflows]]` step `function` (F-037) now count as reachability.
### Notes
- **Workflow `skipped` ≠ `failed` (F-039, by design).** A step whose `when`
evaluates false is `skipped` and counts as *satisfied-but-empty* for its
dependents — so a step downstream of a skipped one still runs. To gate a
dependent on an upstream actually having run, give it its own `when`
(e.g. `when = "steps.publish.output.published == true"`), or have the upstream
step return an explicit outcome the dependent checks.
## v1.1.9 — Durable Queues & Function Composition (unreleased)
Per-app durable queues + same-app function composition + caller-

File diff suppressed because one or more lines are too long

12
Cargo.lock generated
View File

@@ -1939,6 +1939,7 @@ dependencies = [
"picloud-manager-core",
"picloud-orchestrator-core",
"picloud-shared",
"picloud-test-support",
"serde",
"serde_json",
"sha2 0.10.9",
@@ -1964,6 +1965,7 @@ dependencies = [
"libc",
"percent-encoding",
"picloud-shared",
"picloud-test-support",
"postgres",
"predicates",
"reqwest",
@@ -2043,6 +2045,7 @@ dependencies = [
"picloud-executor-core",
"picloud-orchestrator-core",
"picloud-shared",
"picloud-test-support",
"rand 0.8.6",
"reqwest",
"serde",
@@ -2074,6 +2077,7 @@ version = "1.1.9"
dependencies = [
"async-trait",
"axum",
"base64",
"chrono",
"lru",
"picloud-executor-core",
@@ -2110,6 +2114,14 @@ dependencies = [
"uuid",
]
[[package]]
name = "picloud-test-support"
version = "1.1.9"
dependencies = [
"sqlx",
"tokio",
]
[[package]]
name = "pin-project-lite"
version = "0.2.17"

View File

@@ -10,6 +10,7 @@ members = [
"crates/picloud-orchestrator",
"crates/picloud-executor",
"crates/picloud-cli",
"crates/test-support",
]
[workspace.package]
@@ -63,6 +64,7 @@ futures = "0.3"
rhai = { version = "=1.24", features = ["sync", "serde"] }
# Postgres (manager-core only — others stay DB-free)
picloud-test-support = { path = "crates/test-support" }
sqlx = { version = "0.8", features = ["runtime-tokio-rustls", "postgres", "uuid", "chrono", "json", "macros", "migrate"] }
# Config

View File

@@ -44,6 +44,14 @@ fn current_deadline() -> Option<Instant> {
CURRENT_DEADLINE.with(Cell::get)
}
/// The current-thread execution deadline, if one is installed. Public so the
/// interceptor hook can compute a per-hook timeout as `min(ambient, now + t)` —
/// a hook must never be able to EXTEND its caller's deadline (§9.4 M5).
#[must_use]
pub fn ambient_deadline() -> Option<Instant> {
current_deadline()
}
use chrono::{DateTime, Utc};
use picloud_shared::{
ScriptId, ScriptValidator, SdkCallCx, Services, TriggerEvent, ValidatedScript, ValidationError,
@@ -57,7 +65,9 @@ use crate::module_resolver::{
};
use crate::sandbox::Limits;
use crate::sdk;
use crate::sdk::bridge::{dynamic_to_json, json_to_dynamic};
use crate::sdk::bridge::{
dynamic_to_json, dynamic_to_json_capped, json_to_dynamic, MAX_JSON_MATERIALIZE_BYTES,
};
use crate::types::{
ExecError, ExecRequest, ExecResponse, ExecStats, InvocationType, LogEntry, LogLevel,
};
@@ -278,6 +288,17 @@ impl Engine {
/// cache hands compiled ASTs in directly; this path skips the
/// per-call compile.
pub fn execute_ast(&self, ast: &Arc<AST>, req: ExecRequest) -> Result<ExecResponse, ExecError> {
// Audit #2: bound per-execution durable fan-out. The scope is
// re-entrancy aware — a fresh dispatched execution (a new task on a
// pooled thread) resets the budget, while a synchronous invoke() /
// interceptor re-entry nested in the SAME call stack shares it, so
// fan-out is counted across the whole synchronous chain. Held to the end
// of the call; its Drop re-zeroes the counter on the outermost exit.
let _emit_scope = crate::sdk::emit_budget::EmissionBudgetScope::enter();
// §9.4 M6: memoize interceptor chain resolution for this execution tree
// (same re-entrancy model as the emission budget); cleared at the
// outermost boundary so a pooled thread never reuses a foreign app's cache.
let _interceptor_cache = crate::sdk::interceptor::InterceptorCacheScope::enter();
let effective_limits = self.limits.with_overrides(&req.sandbox_overrides);
let logs: Arc<Mutex<Vec<LogEntry>>> = Arc::new(Mutex::new(Vec::new()));
let mut engine = build_engine(effective_limits, Some(logs.clone()));
@@ -337,12 +358,13 @@ impl Engine {
|m| m.into_inner().unwrap_or_default(),
);
let (status_code, headers, body) = parse_response(value)?;
let (status_code, headers, body, body_base64) = parse_response(value)?;
Ok(ExecResponse {
status_code,
headers,
body,
body_base64,
logs,
stats: ExecStats {
duration_ms: u64::try_from(duration.as_millis()).unwrap_or(u64::MAX),
@@ -489,6 +511,14 @@ fn build_ctx_map(req: &ExecRequest) -> Map {
"invocation_type".into(),
invocation_type_str(req.invocation_type).into(),
);
// Read-only depth of this call in a trigger/invoke chain: 0 for direct
// ingress, +1 per synchronous re-entry (`invoke`) or dispatched handler.
// Exposed so a script can observe where it sits in the chain (the same value
// the platform bounds via `trigger_depth_max`).
ctx.insert(
"trigger_depth".into(),
(i64::from(req.trigger_depth)).into(),
);
let mut request = Map::new();
request.insert("path".into(), req.path.clone().into());
@@ -502,6 +532,20 @@ fn build_ctx_map(req: &ExecRequest) -> Map {
request.insert("body".into(), json_to_dynamic(req.body.clone()));
// F-026 convenience: the request Content-Type (also present in `headers`).
// Lets a script branch on how to read `body` — parsed JSON for a JSON
// request, a raw string for `text/*`, or a base64 string for other binary
// types — without a case-insensitive header lookup. `()` when absent.
let content_type = req
.headers
.iter()
.find(|(k, _)| k.eq_ignore_ascii_case("content-type"))
.map(|(_, v)| v.clone());
request.insert(
"content_type".into(),
content_type.map_or(Dynamic::UNIT, Into::into),
);
// SDK 1.1 additions — route-captured params, query string, prefix
// tail. Empty when not applicable so scripts can always read them.
let mut params = Map::new();
@@ -739,7 +783,9 @@ fn invocation_type_str(it: InvocationType) -> &'static str {
// Response parsing
// ----------------------------------------------------------------------------
fn parse_response(value: Dynamic) -> Result<(u16, BTreeMap<String, String>, Json), ExecError> {
type ParsedResponse = (u16, BTreeMap<String, String>, Json, Option<String>);
fn parse_response(value: Dynamic) -> Result<ParsedResponse, ExecError> {
// Convention: a Map with a `statusCode` field is the structured shape.
// Anything else is treated as a 200 response with the value as body.
if value.is_map() {
@@ -749,10 +795,14 @@ fn parse_response(value: Dynamic) -> Result<(u16, BTreeMap<String, String>, Json
}
}
}
Ok((200, BTreeMap::new(), dynamic_to_json(&value)))
// The response body is otherwise an uncapped Rhai→JSON exit — bound it so a
// cheaply-aliased huge value can't OOM the node on materialization.
let body = dynamic_to_json_capped(&value, MAX_JSON_MATERIALIZE_BYTES)
.map_err(|e| ExecError::InvalidResponse(e.to_string()))?;
Ok((200, BTreeMap::new(), body, None))
}
fn parse_structured_response(map: Map) -> Result<(u16, BTreeMap<String, String>, Json), ExecError> {
fn parse_structured_response(map: Map) -> Result<ParsedResponse, ExecError> {
let status_dyn = map
.get("statusCode")
.ok_or_else(|| ExecError::InvalidResponse("missing statusCode".into()))?;
@@ -771,9 +821,29 @@ fn parse_structured_response(map: Map) -> Result<(u16, BTreeMap<String, String>,
}
}
let body = map.get("body").map_or(Json::Null, dynamic_to_json);
// F-026: a `body_base64` string field returns raw bytes with the script-set
// `Content-Type`, instead of JSON-encoding `body`. When set, `body` is
// ignored (the bytes win) so an author can't accidentally send both.
let body_base64 = match map.get("body_base64") {
Some(b) => Some(
b.clone()
.into_string()
.map_err(|_| ExecError::InvalidResponse("body_base64 must be a string".into()))?,
),
None => None,
};
Ok((status_code, headers, body))
let body = if body_base64.is_some() {
Json::Null
} else {
match map.get("body") {
Some(b) => dynamic_to_json_capped(b, MAX_JSON_MATERIALIZE_BYTES)
.map_err(|e| ExecError::InvalidResponse(e.to_string()))?,
None => Json::Null,
}
};
Ok((status_code, headers, body, body_base64))
}
// ----------------------------------------------------------------------------

View File

@@ -43,6 +43,15 @@ where
})
}
/// Build a Rhai runtime error from a message. Returns the boxed error
/// directly because every caller needs a `Box<EvalAltResult>` (Rhai's error
/// type) — the shared home for the pattern the SDK bridges (secrets, email,
/// users, …) previously each copied.
#[allow(clippy::unnecessary_box_returns)]
pub(crate) fn runtime_err(msg: &str) -> Box<EvalAltResult> {
EvalAltResult::ErrorRuntime(msg.into(), rhai::Position::NONE).into()
}
/// Convert a `serde_json::Value` into a Rhai `Dynamic` suitable for
/// pushing into a script's scope. Numbers prefer the narrowest type
/// (`i64` over `f64`); anything that can't round-trip falls back to a
@@ -76,35 +85,173 @@ pub fn json_to_dynamic(value: Json) -> Dynamic {
}
}
/// Convert a Rhai `Dynamic` back to a `serde_json::Value`. Custom Rhai
/// types (timestamps, user-registered modules) fall back to their
/// `Display` form so they appear as strings in JSON output rather than
/// failing the response build.
/// Hard ceiling on the byte size of a single Rhai→JSON materialization.
///
/// The per-element Rhai sandbox caps (`max_string_size` 64 KiB,
/// `max_array_size`/`max_map_size` 10 000) do NOT bound the *materialized*
/// total: Rhai shares strings/arrays by `Rc`, so a 10 000-element array of one
/// aliased 64 KiB string is cheap to build (~10 000 ops) yet dealiases to
/// ~640 MiB of distinct JSON. Without a ceiling a handful of anonymous requests
/// could OOM the node. `dynamic_to_json_capped` charges a running byte budget
/// and bails as soon as it is exceeded, so the transient allocation is bounded
/// to roughly this ceiling regardless of aliasing. This is far above any
/// legitimate single value (the KV/docs value caps are 256 KiB); it is a
/// last-resort anti-OOM rail, not a business limit.
pub const MAX_JSON_MATERIALIZE_BYTES: usize = 16 * 1024 * 1024;
/// A Rhai value was too large to materialize into JSON within its byte budget.
#[derive(Debug, Clone, Copy)]
pub struct JsonSizeError {
pub limit: usize,
}
impl std::fmt::Display for JsonSizeError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(
f,
"value too large to serialize to JSON (exceeds the {}-byte limit)",
self.limit
)
}
}
impl std::error::Error for JsonSizeError {}
impl From<JsonSizeError> for Box<EvalAltResult> {
fn from(e: JsonSizeError) -> Self {
runtime_err(&e.to_string())
}
}
/// Convert a Rhai `Dynamic` back to a `serde_json::Value`, **infallibly** and
/// **bounded** by [`MAX_JSON_MATERIALIZE_BYTES`]. A value that would exceed the
/// ceiling collapses to a short marker string rather than OOMing — safe for
/// best-effort paths (logging) where a hard error is undesirable. Paths where
/// silently substituting a marker would be wrong (writes, the HTTP response
/// body, `invoke` args) must use [`dynamic_to_json_capped`] and surface the
/// error instead.
///
/// Custom Rhai types (timestamps, user-registered modules) fall back to their
/// `Display` form so they appear as strings rather than failing the build.
#[must_use]
pub fn dynamic_to_json(value: &Dynamic) -> Json {
dynamic_to_json_capped(value, MAX_JSON_MATERIALIZE_BYTES)
.unwrap_or_else(|_| Json::String("<value too large>".to_string()))
}
/// Bounded `Dynamic`→JSON conversion: materializes `value`, charging a running
/// byte budget starting at `max_bytes`, and returns [`JsonSizeError`] the moment
/// the budget is exceeded — so it never builds the full oversized tree.
///
/// # Errors
/// Returns [`JsonSizeError`] if the materialized JSON would exceed `max_bytes`.
pub fn dynamic_to_json_capped(value: &Dynamic, max_bytes: usize) -> Result<Json, JsonSizeError> {
let mut remaining = max_bytes;
to_json_bounded(value, &mut remaining, max_bytes)
}
/// Deduct `n` bytes from `remaining`, or fail if the budget is exhausted.
fn charge(remaining: &mut usize, n: usize, limit: usize) -> Result<(), JsonSizeError> {
match remaining.checked_sub(n) {
Some(left) => {
*remaining = left;
Ok(())
}
None => Err(JsonSizeError { limit }),
}
}
fn to_json_bounded(
value: &Dynamic,
remaining: &mut usize,
limit: usize,
) -> Result<Json, JsonSizeError> {
if value.is_unit() {
return Json::Null;
charge(remaining, 4, limit)?;
return Ok(Json::Null);
}
if let Ok(b) = value.as_bool() {
return Json::Bool(b);
charge(remaining, 5, limit)?;
return Ok(Json::Bool(b));
}
if let Ok(i) = value.as_int() {
return Json::Number(i.into());
charge(remaining, 8, limit)?;
return Ok(Json::Number(i.into()));
}
if let Ok(f) = value.as_float() {
return serde_json::Number::from_f64(f).map_or(Json::Null, Json::Number);
charge(remaining, 8, limit)?;
return Ok(serde_json::Number::from_f64(f).map_or(Json::Null, Json::Number));
}
if value.is_string() {
return Json::String(value.clone().into_string().unwrap_or_default());
let s = value.clone().into_string().unwrap_or_default();
charge(remaining, s.len() + 2, limit)?;
return Ok(Json::String(s));
}
if let Some(arr) = value.clone().try_cast::<rhai::Array>() {
return Json::Array(arr.iter().map(dynamic_to_json).collect());
charge(remaining, 2, limit)?; // "[]"
let mut out = Vec::with_capacity(arr.len().min(1024));
for v in &arr {
charge(remaining, 1, limit)?; // ","
out.push(to_json_bounded(v, remaining, limit)?);
}
return Ok(Json::Array(out));
}
if let Some(map) = value.clone().try_cast::<Map>() {
charge(remaining, 2, limit)?; // "{}"
let mut out = serde_json::Map::new();
for (k, v) in map {
out.insert(k.to_string(), dynamic_to_json(&v));
charge(remaining, k.len() + 4, limit)?; // "key":,
out.insert(k.to_string(), to_json_bounded(&v, remaining, limit)?);
}
return Json::Object(out);
return Ok(Json::Object(out));
}
let s = value.to_string();
charge(remaining, s.len() + 2, limit)?;
Ok(Json::String(s))
}
#[cfg(test)]
mod tests {
use super::*;
use rhai::{Array, Dynamic};
#[test]
fn small_value_round_trips_under_the_cap() {
let d = Dynamic::from("hello");
assert_eq!(
dynamic_to_json_capped(&d, MAX_JSON_MATERIALIZE_BYTES).unwrap(),
Json::String("hello".into())
);
}
#[test]
fn aliased_large_array_is_rejected_not_materialized() {
// The OOM regression: one 64 KiB string aliased 10 000×. Cheap to build
// (Rc-shared), ~640 MiB if dealiased. The bounded converter must bail on
// the byte budget instead of allocating the full tree.
let big = "x".repeat(64 * 1024);
let mut arr = Array::new();
for _ in 0..10_000 {
arr.push(Dynamic::from(big.clone())); // ImmutableString: shared, cheap
}
let d = Dynamic::from(arr);
// A 1 MiB budget is far below the ~640 MiB dealiased size → must error.
let err = dynamic_to_json_capped(&d, 1024 * 1024).unwrap_err();
assert_eq!(err.limit, 1024 * 1024);
}
#[test]
fn infallible_wrapper_yields_marker_instead_of_oom() {
// ~300 × 64 KiB aliased ≈ 19 MiB dealiased, just over the 16 MiB default
// ceiling — enough to trip the infallible wrapper's marker fallback
// without paying to build a pathologically huge test array.
let big = "x".repeat(64 * 1024);
let mut arr = Array::new();
for _ in 0..300 {
arr.push(Dynamic::from(big.clone()));
}
// The infallible path must NOT OOM — it collapses to a marker string.
let out = dynamic_to_json(&Dynamic::from(arr));
assert_eq!(out, Json::String("<value too large>".into()));
}
Json::String(value.to_string())
}

View File

@@ -27,7 +27,10 @@ use picloud_shared::{DocId, DocRow, DocsService, GroupDocsService, SdkCallCx, Se
use rhai::{Array, Dynamic, Engine as RhaiEngine, EvalAltResult, Map, Module};
use uuid::Uuid;
use super::bridge::{block_on, dynamic_to_json, json_to_dynamic};
use super::bridge::{
block_on, dynamic_to_json_capped, json_to_dynamic, MAX_JSON_MATERIALIZE_BYTES,
};
use super::interceptor::InterceptorCtx;
/// Per-call handle captured by the Rhai SDK. Cheap to clone (two Arcs
/// plus an owned string).
@@ -36,6 +39,8 @@ pub struct DocsHandle {
collection: String,
service: Arc<dyn DocsService>,
cx: Arc<SdkCallCx>,
// §9.4: carried so `create`/`update`/`delete` run the before/after hook (M8).
ictx: InterceptorCtx,
}
/// §11.6 shared-collection handle, returned by `docs::shared_collection(name)`.
@@ -47,9 +52,16 @@ pub struct GroupDocsHandle {
collection: String,
service: Arc<dyn GroupDocsService>,
cx: Arc<SdkCallCx>,
// §9.4: carried so shared `create`/`update`/`delete` run the hook too (M8).
ictx: InterceptorCtx,
}
pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<SdkCallCx>) {
pub(super) fn register(
engine: &mut RhaiEngine,
services: &Services,
cx: Arc<SdkCallCx>,
ictx: InterceptorCtx,
) {
let docs_service = services.docs.clone();
let group_docs_service = services.group_docs.clone();
@@ -57,6 +69,7 @@ pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<Sdk
{
let docs_service = docs_service.clone();
let cx = cx.clone();
let ictx = ictx.clone();
module.set_native_fn(
"collection",
move |name: &str| -> Result<DocsHandle, Box<EvalAltResult>> {
@@ -67,6 +80,7 @@ pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<Sdk
collection: name.to_string(),
service: docs_service.clone(),
cx: cx.clone(),
ictx: ictx.clone(),
})
},
);
@@ -74,6 +88,7 @@ pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<Sdk
{
let group_docs_service = group_docs_service.clone();
let cx = cx.clone();
let ictx = ictx.clone();
module.set_native_fn(
"shared_collection",
move |name: &str| -> Result<GroupDocsHandle, Box<EvalAltResult>> {
@@ -84,6 +99,7 @@ pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<Sdk
collection: name.to_string(),
service: group_docs_service.clone(),
cx: cx.clone(),
ictx: ictx.clone(),
})
},
);
@@ -116,12 +132,23 @@ fn register_create(engine: &mut RhaiEngine) {
engine.register_fn(
"create",
|handle: &mut DocsHandle, data: Map| -> Result<String, Box<EvalAltResult>> {
let json = dynamic_to_json_capped(&Dynamic::from(data), MAX_JSON_MATERIALIZE_BYTES)?;
// §9.4 before-op interceptor (allow/deny + M4 data-transform).
let write_val = run_before(handle, "create", "", Some(&json))?.unwrap_or(json);
let written = write_val.clone();
let h = handle.clone();
let json = dynamic_to_json(&Dynamic::from(data));
let id = block_on("docs", async move {
h.service.create(&h.cx, &h.collection, json).await
h.service.create(&h.cx, &h.collection, write_val).await
})?;
Ok(id.to_string())
let id_str = id.to_string();
run_after(
handle,
"create",
"",
Some(&written),
serde_json::Value::String(id_str.clone()),
)?;
Ok(id_str)
},
);
}
@@ -145,7 +172,7 @@ fn register_find(engine: &mut RhaiEngine) {
"find",
|handle: &mut DocsHandle, filter: Map| -> Result<Array, Box<EvalAltResult>> {
let h = handle.clone();
let json = dynamic_to_json(&Dynamic::from(filter));
let json = dynamic_to_json_capped(&Dynamic::from(filter), MAX_JSON_MATERIALIZE_BYTES)?;
let rows = block_on("docs", async move {
h.service.find(&h.cx, &h.collection, json).await
})?;
@@ -162,7 +189,7 @@ fn register_find_one(engine: &mut RhaiEngine) {
"find_one",
|handle: &mut DocsHandle, filter: Map| -> Result<Dynamic, Box<EvalAltResult>> {
let h = handle.clone();
let json = dynamic_to_json(&Dynamic::from(filter));
let json = dynamic_to_json_capped(&Dynamic::from(filter), MAX_JSON_MATERIALIZE_BYTES)?;
let row = block_on("docs", async move {
h.service.find_one(&h.cx, &h.collection, json).await
})?;
@@ -175,14 +202,23 @@ fn register_update(engine: &mut RhaiEngine) {
engine.register_fn(
"update",
|handle: &mut DocsHandle, id: &str, data: Map| -> Result<(), Box<EvalAltResult>> {
let h = handle.clone();
let parsed_id = parse_doc_id(id)?;
let json = dynamic_to_json(&Dynamic::from(data));
let json = dynamic_to_json_capped(&Dynamic::from(data), MAX_JSON_MATERIALIZE_BYTES)?;
let write_val = run_before(handle, "update", id, Some(&json))?.unwrap_or(json);
let written = write_val.clone();
let h = handle.clone();
block_on("docs", async move {
h.service
.update(&h.cx, &h.collection, parsed_id, json)
.update(&h.cx, &h.collection, parsed_id, write_val)
.await
})
})?;
run_after(
handle,
"update",
id,
Some(&written),
serde_json::Value::Null,
)
},
);
}
@@ -191,15 +227,62 @@ fn register_delete(engine: &mut RhaiEngine) {
engine.register_fn(
"delete",
|handle: &mut DocsHandle, id: &str| -> Result<bool, Box<EvalAltResult>> {
let h = handle.clone();
let parsed_id = parse_doc_id(id)?;
block_on("docs", async move {
// Delete carries no value, so the before-hook never transforms.
run_before(handle, "delete", id, None)?;
let h = handle.clone();
let was_present = block_on("docs", async move {
h.service.delete(&h.cx, &h.collection, parsed_id).await
})
})?;
run_after(
handle,
"delete",
id,
None,
serde_json::Value::Bool(was_present),
)?;
Ok(was_present)
},
);
}
/// §9.4: run the before-op interceptor for a per-app docs write.
fn run_before(
handle: &DocsHandle,
op: &'static str,
key: &str,
value: Option<&serde_json::Value>,
) -> Result<Option<serde_json::Value>, Box<EvalAltResult>> {
super::interceptor::run_before(
&handle.ictx,
&handle.cx,
"docs",
op,
&handle.collection,
key,
value,
)
}
fn run_after(
handle: &DocsHandle,
op: &'static str,
key: &str,
value: Option<&serde_json::Value>,
result: serde_json::Value,
) -> Result<(), Box<EvalAltResult>> {
super::interceptor::run_after(
&handle.ictx,
&handle.cx,
"docs",
op,
&handle.collection,
key,
value,
result,
)
}
fn register_list(engine: &mut RhaiEngine) {
// Zero-arg form: full page from the start.
engine.register_fn(
@@ -264,12 +347,22 @@ fn register_group_create(engine: &mut RhaiEngine) {
engine.register_fn(
"create",
|handle: &mut GroupDocsHandle, data: Map| -> Result<String, Box<EvalAltResult>> {
let json = dynamic_to_json_capped(&Dynamic::from(data), MAX_JSON_MATERIALIZE_BYTES)?;
let write_val = group_run_before(handle, "create", "", Some(&json))?.unwrap_or(json);
let written = write_val.clone();
let h = handle.clone();
let json = dynamic_to_json(&Dynamic::from(data));
let id = block_on("docs", async move {
h.service.create(&h.cx, &h.collection, json).await
h.service.create(&h.cx, &h.collection, write_val).await
})?;
Ok(id.to_string())
let id_str = id.to_string();
group_run_after(
handle,
"create",
"",
Some(&written),
serde_json::Value::String(id_str.clone()),
)?;
Ok(id_str)
},
);
}
@@ -293,7 +386,7 @@ fn register_group_find(engine: &mut RhaiEngine) {
"find",
|handle: &mut GroupDocsHandle, filter: Map| -> Result<Array, Box<EvalAltResult>> {
let h = handle.clone();
let json = dynamic_to_json(&Dynamic::from(filter));
let json = dynamic_to_json_capped(&Dynamic::from(filter), MAX_JSON_MATERIALIZE_BYTES)?;
let rows = block_on("docs", async move {
h.service.find(&h.cx, &h.collection, json).await
})?;
@@ -310,7 +403,7 @@ fn register_group_find_one(engine: &mut RhaiEngine) {
"find_one",
|handle: &mut GroupDocsHandle, filter: Map| -> Result<Dynamic, Box<EvalAltResult>> {
let h = handle.clone();
let json = dynamic_to_json(&Dynamic::from(filter));
let json = dynamic_to_json_capped(&Dynamic::from(filter), MAX_JSON_MATERIALIZE_BYTES)?;
let row = block_on("docs", async move {
h.service.find_one(&h.cx, &h.collection, json).await
})?;
@@ -323,14 +416,23 @@ fn register_group_update(engine: &mut RhaiEngine) {
engine.register_fn(
"update",
|handle: &mut GroupDocsHandle, id: &str, data: Map| -> Result<(), Box<EvalAltResult>> {
let h = handle.clone();
let parsed_id = parse_doc_id(id)?;
let json = dynamic_to_json(&Dynamic::from(data));
let json = dynamic_to_json_capped(&Dynamic::from(data), MAX_JSON_MATERIALIZE_BYTES)?;
let write_val = group_run_before(handle, "update", id, Some(&json))?.unwrap_or(json);
let written = write_val.clone();
let h = handle.clone();
block_on("docs", async move {
h.service
.update(&h.cx, &h.collection, parsed_id, json)
.update(&h.cx, &h.collection, parsed_id, write_val)
.await
})
})?;
group_run_after(
handle,
"update",
id,
Some(&written),
serde_json::Value::Null,
)
},
);
}
@@ -339,15 +441,62 @@ fn register_group_delete(engine: &mut RhaiEngine) {
engine.register_fn(
"delete",
|handle: &mut GroupDocsHandle, id: &str| -> Result<bool, Box<EvalAltResult>> {
let h = handle.clone();
let parsed_id = parse_doc_id(id)?;
block_on("docs", async move {
group_run_before(handle, "delete", id, None)?;
let h = handle.clone();
let was_present = block_on("docs", async move {
h.service.delete(&h.cx, &h.collection, parsed_id).await
})
})?;
group_run_after(
handle,
"delete",
id,
None,
serde_json::Value::Bool(was_present),
)?;
Ok(was_present)
},
);
}
/// §9.4: before/after hooks for a shared (group) docs write — mirror the per-app
/// helpers but over `GroupDocsHandle`.
fn group_run_before(
handle: &GroupDocsHandle,
op: &'static str,
key: &str,
value: Option<&serde_json::Value>,
) -> Result<Option<serde_json::Value>, Box<EvalAltResult>> {
super::interceptor::run_before(
&handle.ictx,
&handle.cx,
"docs",
op,
&handle.collection,
key,
value,
)
}
fn group_run_after(
handle: &GroupDocsHandle,
op: &'static str,
key: &str,
value: Option<&serde_json::Value>,
result: serde_json::Value,
) -> Result<(), Box<EvalAltResult>> {
super::interceptor::run_after(
&handle.ictx,
&handle.cx,
"docs",
op,
&handle.collection,
key,
value,
result,
)
}
fn register_group_list(engine: &mut RhaiEngine) {
engine.register_fn(
"list",

View File

@@ -26,7 +26,7 @@
use std::sync::Arc;
use super::bridge::block_on;
use super::bridge::{block_on, runtime_err};
use picloud_shared::{OutboundEmail, SdkCallCx, Services};
use rhai::{Array, Engine as RhaiEngine, EvalAltResult, Map, Module};
@@ -124,8 +124,3 @@ fn addresses(opts: &Map, key: &str) -> Result<Vec<String>, Box<EvalAltResult>> {
}
}
}
#[allow(clippy::unnecessary_box_returns)]
fn runtime_err(msg: &str) -> Box<EvalAltResult> {
EvalAltResult::ErrorRuntime(msg.into(), rhai::Position::NONE).into()
}

View File

@@ -0,0 +1,124 @@
//! Per-execution fan-out ceiling (audit fix #2).
//!
//! `trigger_depth` bounds how DEEP a trigger/invoke chain can go, but nothing
//! bounded how WIDE a single execution could fan out: one anonymous request
//! running `for i in 0..1_000_000 { invoke_async("w", #{}) }` costs a few Rhai
//! ops per iteration plus one cheap outbox INSERT, so within the op / wall-clock
//! budget it could write ~10^5 durable rows — each dispatched as its own
//! execution — flooding the outbox/queue (a durable amplification DoS).
//!
//! This caps the number of DURABLE emissions (`invoke_async`, `publish_durable`,
//! `enqueue`, and their group-shared variants) a single execution may make.
//! The counter is a thread-local. [`EmissionBudgetScope`] is created around
//! EVERY `execute_ast` and is **re-entrancy aware**: only the OUTERMOST scope
//! on a thread resets the counter. A dispatched handler (queue/trigger/
//! `invoke_async`/cron/workflow) is a fresh task on a pooled thread, so its
//! scope is outermost → it resets and gets a full budget. A synchronous
//! `invoke()` / interceptor re-entry runs a nested `execute_ast` in the SAME
//! call stack, so its scope is inner → it does NOT reset, and the whole
//! synchronous chain shares one budget (fan-out counted across it). The
//! outermost Drop re-zeroes the counter so a pooled thread never leaks a count
//! into the next task.
use std::cell::Cell;
use std::sync::LazyLock;
use rhai::EvalAltResult;
use crate::sdk::bridge::runtime_err;
/// Default max durable emissions per execution. Generous for a legit batch
/// producer, far below the ~10^5 a runaway loop can reach within budget.
const DEFAULT_MAX_EMISSIONS: u32 = 1000;
static MAX_EMISSIONS: LazyLock<u32> = LazyLock::new(|| {
std::env::var("PICLOUD_MAX_EMISSIONS_PER_EXECUTION")
.ok()
.and_then(|v| v.parse::<u32>().ok())
.filter(|&n| n > 0)
.unwrap_or(DEFAULT_MAX_EMISSIONS)
});
thread_local! {
static EMISSIONS: Cell<u32> = const { Cell::new(0) };
/// Scope nesting depth on this thread. `0` when no execution is active.
static ACTIVE: Cell<u32> = const { Cell::new(0) };
}
/// Charge one durable emission against the current execution's budget. Returns
/// an `Err` (aborting the emit) once the ceiling is exceeded.
///
/// # Errors
/// Returns a Rhai runtime error when the per-execution emission ceiling is hit.
pub(super) fn charge_emission(service: &str) -> Result<(), Box<EvalAltResult>> {
EMISSIONS.with(|c| {
let next = c.get().saturating_add(1);
if next > *MAX_EMISSIONS {
return Err(runtime_err(&format!(
"{service}: per-execution emission limit exceeded (max {} durable \
invoke_async/publish/enqueue calls per execution)",
*MAX_EMISSIONS
)));
}
c.set(next);
Ok(())
})
}
/// RAII scope wrapping one `execute_ast`. Re-entrancy aware: the OUTERMOST scope
/// on a thread zeroes the counter on entry and on final exit; nested scopes (a
/// synchronous re-entry in the same call stack) leave it alone, so the chain
/// shares one budget.
pub(crate) struct EmissionBudgetScope;
impl EmissionBudgetScope {
pub(crate) fn enter() -> Self {
ACTIVE.with(|a| {
if a.get() == 0 {
EMISSIONS.with(|c| c.set(0));
}
a.set(a.get() + 1);
});
Self
}
}
impl Drop for EmissionBudgetScope {
fn drop(&mut self) {
ACTIVE.with(|a| {
let n = a.get().saturating_sub(1);
a.set(n);
if n == 0 {
EMISSIONS.with(|c| c.set(0));
}
});
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn outermost_scope_resets_but_nested_shares_budget() {
let outer = EmissionBudgetScope::enter();
for _ in 0..DEFAULT_MAX_EMISSIONS {
charge_emission("test").expect("under budget");
}
// A NESTED scope (synchronous re-entry) must NOT reset — the budget is
// already exhausted and stays that way.
{
let _nested = EmissionBudgetScope::enter();
assert!(
charge_emission("test").is_err(),
"a nested re-entry shares the exhausted budget"
);
}
// Still exhausted after the nested scope drops (outer still active).
assert!(charge_emission("test").is_err());
drop(outer);
// A fresh OUTERMOST scope resets.
let _fresh = EmissionBudgetScope::enter();
charge_emission("test").expect("reset by a fresh outermost scope");
}
}

View File

@@ -24,6 +24,7 @@
use std::sync::Arc;
use super::bridge::block_on;
use super::interceptor::InterceptorCtx;
use picloud_shared::{
FileMeta, FileUpdate, FilesService, GroupFilesService, NewFile, SdkCallCx, Services,
};
@@ -36,6 +37,10 @@ pub struct FilesHandle {
collection: String,
service: Arc<dyn FilesService>,
cx: Arc<SdkCallCx>,
// §9.4: carried so `create`/`update`/`delete` run the before/after hook (M9).
// Files carry bytes, so the hook sees service/op/collection/key only — the
// blob is never materialized into a `value`, and there is no data transform.
ictx: InterceptorCtx,
}
/// §11.6 shared-collection handle, returned by `files::shared_collection(name)`.
@@ -47,9 +52,16 @@ pub struct GroupFilesHandle {
collection: String,
service: Arc<dyn GroupFilesService>,
cx: Arc<SdkCallCx>,
// §9.4: carried so shared `create`/`update`/`delete` run the hook too (M9).
ictx: InterceptorCtx,
}
pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<SdkCallCx>) {
pub(super) fn register(
engine: &mut RhaiEngine,
services: &Services,
cx: Arc<SdkCallCx>,
ictx: InterceptorCtx,
) {
let files_service = services.files.clone();
let group_files_service = services.group_files.clone();
@@ -57,6 +69,7 @@ pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<Sdk
{
let files_service = files_service.clone();
let cx = cx.clone();
let ictx = ictx.clone();
module.set_native_fn(
"collection",
move |name: &str| -> Result<FilesHandle, Box<EvalAltResult>> {
@@ -67,6 +80,7 @@ pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<Sdk
collection: name.to_string(),
service: files_service.clone(),
cx: cx.clone(),
ictx: ictx.clone(),
})
},
);
@@ -74,6 +88,7 @@ pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<Sdk
{
let group_files_service = group_files_service.clone();
let cx = cx.clone();
let ictx = ictx.clone();
module.set_native_fn(
"shared_collection",
move |name: &str| -> Result<GroupFilesHandle, Box<EvalAltResult>> {
@@ -84,6 +99,7 @@ pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<Sdk
collection: name.to_string(),
service: group_files_service.clone(),
cx: cx.clone(),
ictx: ictx.clone(),
})
},
);
@@ -116,6 +132,9 @@ fn register_create(engine: &mut RhaiEngine) {
let name = require_string(&meta, "name")?;
let content_type = require_string(&meta, "content_type")?;
let data = require_blob(&meta, "data")?;
// §9.4 before-op interceptor (allow/deny only — no transform for
// files, the blob is not surfaced to the hook).
run_before(handle, "create", "")?;
let h = handle.clone();
let new = NewFile {
name,
@@ -125,7 +144,14 @@ fn register_create(engine: &mut RhaiEngine) {
let id = block_on("files", async move {
h.service.create(&h.cx, &h.collection, new).await
})?;
Ok(id.to_string())
let id_str = id.to_string();
run_after(
handle,
"create",
"",
serde_json::Value::String(id_str.clone()),
)?;
Ok(id_str)
},
);
}
@@ -165,16 +191,18 @@ fn register_update(engine: &mut RhaiEngine) {
let data = require_blob(&meta, "data")?;
let name = optional_string(&meta, "name")?;
let content_type = optional_string(&meta, "content_type")?;
run_before(handle, "update", id)?;
let h = handle.clone();
let id = id.to_string();
let id_owned = id.to_string();
let upd = FileUpdate {
data,
name,
content_type,
};
block_on("files", async move {
h.service.update(&h.cx, &h.collection, &id, upd).await
})
h.service.update(&h.cx, &h.collection, &id_owned, upd).await
})?;
run_after(handle, "update", id, serde_json::Value::Null)
},
);
}
@@ -183,15 +211,52 @@ fn register_delete(engine: &mut RhaiEngine) {
engine.register_fn(
"delete",
|handle: &mut FilesHandle, id: &str| -> Result<bool, Box<EvalAltResult>> {
run_before(handle, "delete", id)?;
let h = handle.clone();
let id = id.to_string();
block_on("files", async move {
h.service.delete(&h.cx, &h.collection, &id).await
})
let id_owned = id.to_string();
let was_present = block_on("files", async move {
h.service.delete(&h.cx, &h.collection, &id_owned).await
})?;
run_after(handle, "delete", id, serde_json::Value::Bool(was_present))?;
Ok(was_present)
},
);
}
/// §9.4: before/after hooks for a per-app files write. Files pass `value = None`
/// (the blob is never surfaced to the hook), so the before-hook only allows or
/// denies — it never transforms.
fn run_before(handle: &FilesHandle, op: &'static str, key: &str) -> Result<(), Box<EvalAltResult>> {
super::interceptor::run_before(
&handle.ictx,
&handle.cx,
"files",
op,
&handle.collection,
key,
None,
)
.map(|_| ())
}
fn run_after(
handle: &FilesHandle,
op: &'static str,
key: &str,
result: serde_json::Value,
) -> Result<(), Box<EvalAltResult>> {
super::interceptor::run_after(
&handle.ictx,
&handle.cx,
"files",
op,
&handle.collection,
key,
None,
result,
)
}
fn register_list(engine: &mut RhaiEngine) {
engine.register_fn(
"list",
@@ -271,6 +336,7 @@ fn register_group_create(engine: &mut RhaiEngine) {
let name = require_string(&meta, "name")?;
let content_type = require_string(&meta, "content_type")?;
let data = require_blob(&meta, "data")?;
group_run_before(handle, "create", "")?;
let h = handle.clone();
let new = NewFile {
name,
@@ -280,7 +346,14 @@ fn register_group_create(engine: &mut RhaiEngine) {
let id = block_on("files", async move {
h.service.create(&h.cx, &h.collection, new).await
})?;
Ok(id.to_string())
let id_str = id.to_string();
group_run_after(
handle,
"create",
"",
serde_json::Value::String(id_str.clone()),
)?;
Ok(id_str)
},
);
}
@@ -320,16 +393,18 @@ fn register_group_update(engine: &mut RhaiEngine) {
let data = require_blob(&meta, "data")?;
let name = optional_string(&meta, "name")?;
let content_type = optional_string(&meta, "content_type")?;
group_run_before(handle, "update", id)?;
let h = handle.clone();
let id = id.to_string();
let id_owned = id.to_string();
let upd = FileUpdate {
data,
name,
content_type,
};
block_on("files", async move {
h.service.update(&h.cx, &h.collection, &id, upd).await
})
h.service.update(&h.cx, &h.collection, &id_owned, upd).await
})?;
group_run_after(handle, "update", id, serde_json::Value::Null)
},
);
}
@@ -338,15 +413,55 @@ fn register_group_delete(engine: &mut RhaiEngine) {
engine.register_fn(
"delete",
|handle: &mut GroupFilesHandle, id: &str| -> Result<bool, Box<EvalAltResult>> {
group_run_before(handle, "delete", id)?;
let h = handle.clone();
let id = id.to_string();
block_on("files", async move {
h.service.delete(&h.cx, &h.collection, &id).await
})
let id_owned = id.to_string();
let was_present = block_on("files", async move {
h.service.delete(&h.cx, &h.collection, &id_owned).await
})?;
group_run_after(handle, "delete", id, serde_json::Value::Bool(was_present))?;
Ok(was_present)
},
);
}
/// §9.4: before/after hooks for a shared (group) files write — mirror the
/// per-app helpers but over `GroupFilesHandle`.
fn group_run_before(
handle: &GroupFilesHandle,
op: &'static str,
key: &str,
) -> Result<(), Box<EvalAltResult>> {
super::interceptor::run_before(
&handle.ictx,
&handle.cx,
"files",
op,
&handle.collection,
key,
None,
)
.map(|_| ())
}
fn group_run_after(
handle: &GroupFilesHandle,
op: &'static str,
key: &str,
result: serde_json::Value,
) -> Result<(), Box<EvalAltResult>> {
super::interceptor::run_after(
&handle.ictx,
&handle.cx,
"files",
op,
&handle.collection,
key,
None,
result,
)
}
fn register_group_list(engine: &mut RhaiEngine) {
engine.register_fn(
"list",

View File

@@ -32,11 +32,13 @@
use std::collections::BTreeMap;
use std::sync::Arc;
use picloud_shared::{HttpError, HttpRequest, HttpResponse, HttpService, SdkCallCx, Services};
use picloud_shared::{HttpRequest, HttpResponse, HttpService, SdkCallCx, Services};
use rhai::{Dynamic, Engine as RhaiEngine, EvalAltResult, Map, Module};
use tokio::runtime::Handle as TokioHandle;
use super::bridge::{dynamic_to_json, json_to_dynamic};
use super::bridge::{
block_on, dynamic_to_json_capped, json_to_dynamic, runtime_err, MAX_JSON_MATERIALIZE_BYTES,
};
use super::interceptor::InterceptorCtx;
/// Bridge-side defaults (the service clamps server-side too). The
/// `MAX_*` ceilings stay `i64` because they're compared against the
@@ -49,20 +51,31 @@ const MAX_REDIRECTS: i64 = 10;
const ALLOWED_OPT_KEYS: [&str; 4] = ["headers", "timeout_ms", "follow_redirects", "max_redirects"];
pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<SdkCallCx>) {
// `http` registers its verbs as module native fns (no handle struct), so the
// interceptor ctx is captured into each request closure directly. Every verb
// funnels through `invoke` / `invoke_form`, which run the §9.4 before/after hook
// around `svc.request` (M11). The hook sees `service = "http"`, `op =
// "request"`, `collection = <METHOD>`, `key = <url>`; the body is never
// surfaced, so there is no data transform.
pub(super) fn register(
engine: &mut RhaiEngine,
services: &Services,
cx: Arc<SdkCallCx>,
ictx: InterceptorCtx,
) {
let svc = services.http.clone();
let mut module = Module::new();
// Bodyless verbs: (url) / (url, opts).
for verb in ["get", "head"] {
register_bodyless(&mut module, verb, &svc, &cx);
register_bodyless(&mut module, verb, &svc, &cx, &ictx);
}
// Body verbs: (url) / (url, body) / (url, body, opts).
for verb in ["post", "put", "patch", "delete"] {
register_body(&mut module, verb, &svc, &cx);
register_body(&mut module, verb, &svc, &cx, &ictx);
}
register_post_form(&mut module, &svc, &cx);
register_request(&mut module, &svc, &cx);
register_post_form(&mut module, &svc, &cx, &ictx);
register_request(&mut module, &svc, &cx, &ictx);
engine.register_static_module("http", module.into());
}
@@ -72,17 +85,18 @@ fn register_bodyless(
verb: &'static str,
svc: &Arc<dyn HttpService>,
cx: &Arc<SdkCallCx>,
ictx: &InterceptorCtx,
) {
{
let (svc, cx) = (svc.clone(), cx.clone());
let (svc, cx, ictx) = (svc.clone(), cx.clone(), ictx.clone());
module.set_native_fn(verb, move |url: &str| {
invoke(&svc, &cx, verb, url, None, None)
invoke(&ictx, &svc, &cx, verb, url, None, None)
});
}
{
let (svc, cx) = (svc.clone(), cx.clone());
let (svc, cx, ictx) = (svc.clone(), cx.clone(), ictx.clone());
module.set_native_fn(verb, move |url: &str, opts: Map| {
invoke(&svc, &cx, verb, url, None, Some(&opts))
invoke(&ictx, &svc, &cx, verb, url, None, Some(&opts))
});
}
}
@@ -92,61 +106,72 @@ fn register_body(
verb: &'static str,
svc: &Arc<dyn HttpService>,
cx: &Arc<SdkCallCx>,
ictx: &InterceptorCtx,
) {
{
let (svc, cx) = (svc.clone(), cx.clone());
let (svc, cx, ictx) = (svc.clone(), cx.clone(), ictx.clone());
module.set_native_fn(verb, move |url: &str| {
invoke(&svc, &cx, verb, url, None, None)
invoke(&ictx, &svc, &cx, verb, url, None, None)
});
}
{
let (svc, cx) = (svc.clone(), cx.clone());
let (svc, cx, ictx) = (svc.clone(), cx.clone(), ictx.clone());
module.set_native_fn(verb, move |url: &str, body: Dynamic| {
invoke(&svc, &cx, verb, url, Some(body), None)
invoke(&ictx, &svc, &cx, verb, url, Some(body), None)
});
}
{
let (svc, cx) = (svc.clone(), cx.clone());
let (svc, cx, ictx) = (svc.clone(), cx.clone(), ictx.clone());
module.set_native_fn(verb, move |url: &str, body: Dynamic, opts: Map| {
invoke(&svc, &cx, verb, url, Some(body), Some(&opts))
invoke(&ictx, &svc, &cx, verb, url, Some(body), Some(&opts))
});
}
}
fn register_post_form(module: &mut Module, svc: &Arc<dyn HttpService>, cx: &Arc<SdkCallCx>) {
fn register_post_form(
module: &mut Module,
svc: &Arc<dyn HttpService>,
cx: &Arc<SdkCallCx>,
ictx: &InterceptorCtx,
) {
{
let (svc, cx) = (svc.clone(), cx.clone());
let (svc, cx, ictx) = (svc.clone(), cx.clone(), ictx.clone());
module.set_native_fn("post_form", move |url: &str, form: Map| {
invoke_form(&svc, &cx, url, &form, None)
invoke_form(&ictx, &svc, &cx, url, &form, None)
});
}
{
let (svc, cx) = (svc.clone(), cx.clone());
let (svc, cx, ictx) = (svc.clone(), cx.clone(), ictx.clone());
module.set_native_fn("post_form", move |url: &str, form: Map, opts: Map| {
invoke_form(&svc, &cx, url, &form, Some(&opts))
invoke_form(&ictx, &svc, &cx, url, &form, Some(&opts))
});
}
}
fn register_request(module: &mut Module, svc: &Arc<dyn HttpService>, cx: &Arc<SdkCallCx>) {
fn register_request(
module: &mut Module,
svc: &Arc<dyn HttpService>,
cx: &Arc<SdkCallCx>,
ictx: &InterceptorCtx,
) {
{
let (svc, cx) = (svc.clone(), cx.clone());
let (svc, cx, ictx) = (svc.clone(), cx.clone(), ictx.clone());
module.set_native_fn("request", move |method: &str, url: &str| {
invoke(&svc, &cx, method, url, None, None)
invoke(&ictx, &svc, &cx, method, url, None, None)
});
}
{
let (svc, cx) = (svc.clone(), cx.clone());
let (svc, cx, ictx) = (svc.clone(), cx.clone(), ictx.clone());
module.set_native_fn("request", move |method: &str, url: &str, body: Dynamic| {
invoke(&svc, &cx, method, url, Some(body), None)
invoke(&ictx, &svc, &cx, method, url, Some(body), None)
});
}
{
let (svc, cx) = (svc.clone(), cx.clone());
let (svc, cx, ictx) = (svc.clone(), cx.clone(), ictx.clone());
module.set_native_fn(
"request",
move |method: &str, url: &str, body: Dynamic, opts: Map| {
invoke(&svc, &cx, method, url, Some(body), Some(&opts))
invoke(&ictx, &svc, &cx, method, url, Some(body), Some(&opts))
},
);
}
@@ -237,20 +262,21 @@ fn dispatch_body(body: Dynamic) -> Result<EncodedBody, Box<EvalAltResult>> {
return Ok((Some(s.into_bytes()), Some("text/plain".to_string())));
}
if body.is_map() || body.is_array() {
let json = dynamic_to_json(&body);
let json = dynamic_to_json_capped(&body, MAX_JSON_MATERIALIZE_BYTES)?;
let bytes = serde_json::to_vec(&json)
.map_err(|e| err(format!("could not encode JSON body: {e}")))?;
return Ok((Some(bytes), Some("application/json".to_string())));
}
// Scalars (int/float/bool) → JSON-encode for consistency.
let json = dynamic_to_json(&body);
let json = dynamic_to_json_capped(&body, MAX_JSON_MATERIALIZE_BYTES)?;
let bytes =
serde_json::to_vec(&json).map_err(|e| err(format!("could not encode body: {e}")))?;
Ok((Some(bytes), Some("application/json".to_string())))
}
#[allow(clippy::needless_pass_by_value)]
#[allow(clippy::needless_pass_by_value, clippy::too_many_arguments)]
fn invoke(
ictx: &InterceptorCtx,
svc: &Arc<dyn HttpService>,
cx: &Arc<SdkCallCx>,
method: &str,
@@ -260,6 +286,9 @@ fn invoke(
) -> Result<Dynamic, Box<EvalAltResult>> {
let opts = parse_opts(opts)?;
let method_uc = method.to_ascii_uppercase();
// §9.4 before-op interceptor (allow/deny only; the body is not surfaced to
// the hook). `collection` = the HTTP method, `key` = the URL.
super::interceptor::run_before(ictx, cx, "http", "request", &method_uc, url, None)?;
let bodyless = matches!(method_uc.as_str(), "GET" | "HEAD");
let (encoded, content_type) = if bodyless {
(None, None)
@@ -270,7 +299,7 @@ fn invoke(
};
let req = HttpRequest {
method: method_uc,
method: method_uc.clone(),
url: url.to_string(),
headers: opts.headers,
body: encoded,
@@ -280,12 +309,27 @@ fn invoke(
max_redirects: opts.max_redirects,
script_id: Some(cx.script_id.to_string()),
};
let resp = block_on(svc, cx, req)?;
let resp = {
let svc = svc.clone();
let cx = cx.clone();
block_on("http", async move { svc.request(&cx, req).await })?
};
super::interceptor::run_after(
ictx,
cx,
"http",
"request",
&method_uc,
url,
None,
serde_json::Value::Null,
)?;
Ok(response_to_dynamic(&resp))
}
#[allow(clippy::needless_pass_by_value)]
fn invoke_form(
ictx: &InterceptorCtx,
svc: &Arc<dyn HttpService>,
cx: &Arc<SdkCallCx>,
url: &str,
@@ -293,6 +337,8 @@ fn invoke_form(
opts: Option<&Map>,
) -> Result<Dynamic, Box<EvalAltResult>> {
let opts = parse_opts(opts)?;
// §9.4 before-op interceptor. `post_form` is always a POST.
super::interceptor::run_before(ictx, cx, "http", "request", "POST", url, None)?;
let mut serializer = url::form_urlencoded::Serializer::new(String::new());
for (k, v) in form {
serializer.append_pair(k.as_str(), &dyn_to_string(v));
@@ -310,7 +356,21 @@ fn invoke_form(
max_redirects: opts.max_redirects,
script_id: Some(cx.script_id.to_string()),
};
let resp = block_on(svc, cx, req)?;
let resp = {
let svc = svc.clone();
let cx = cx.clone();
block_on("http", async move { svc.request(&cx, req).await })?
};
super::interceptor::run_after(
ictx,
cx,
"http",
"request",
"POST",
url,
None,
serde_json::Value::Null,
)?;
Ok(response_to_dynamic(&resp))
}
@@ -356,36 +416,10 @@ fn dyn_to_string(v: &Dynamic) -> String {
}
}
// Rhai's native-fn error channel is `Box<EvalAltResult>`, so these
// helpers return the boxed form the call sites need.
// A validation error, prefixed like the service's runtime errors (the shared
// `bridge::block_on("http", …)` prefixes the same way, so messages are uniform).
// Delegates to `bridge::runtime_err`; the boxed return is Rhai's error channel.
#[allow(clippy::unnecessary_box_returns)]
fn err(msg: String) -> Box<EvalAltResult> {
EvalAltResult::ErrorRuntime(format!("http: {msg}").into(), rhai::Position::NONE).into()
}
/// Run the async service call from the synchronous Rhai context. Same
/// pattern as `kv`/`docs`: the script runs under `spawn_blocking`, so a
/// runtime handle is reachable and blocking on it is correct.
fn block_on(
svc: &Arc<dyn HttpService>,
cx: &Arc<SdkCallCx>,
req: HttpRequest,
) -> Result<HttpResponse, Box<EvalAltResult>> {
let handle = TokioHandle::try_current().map_err(|e| -> Box<EvalAltResult> {
EvalAltResult::ErrorRuntime(
format!("http: no tokio runtime available: {e}").into(),
rhai::Position::NONE,
)
.into()
})?;
let svc = svc.clone();
let cx = cx.clone();
handle
.block_on(async move { svc.request(&cx, req).await })
.map_err(map_http_err)
}
#[allow(clippy::unnecessary_box_returns)]
fn map_http_err(e: HttpError) -> Box<EvalAltResult> {
EvalAltResult::ErrorRuntime(format!("http: {e}").into(), rhai::Position::NONE).into()
runtime_err(&format!("http: {msg}"))
}

View File

@@ -0,0 +1,417 @@
//! §9.4 Service Interceptors — the executor-side before-op hook.
//!
//! MVP: `kv::set` / `kv::delete` (private AND group-shared collections) run an
//! allow/deny interceptor first. The hook resolves the nearest interceptor for
//! `(service, op)` on the calling app's chain via the injected
//! `InterceptorService` (a cheap indexed query; `None` = un-hooked → allow, no
//! further work). The resolver returns a script already **sealed to the owner
//! that declared the marker** and fully materialized, so we run it straight
//! through the SAME `invoke()` re-entry core (`run_resolved_blocking`) — no
//! second dispatch mechanism, no re-resolution by name (which would let a
//! descendant app shadow a group's guard).
//!
//! **Fail closed.** Everything but an explicit allow denies:
//! - a registered marker whose script is missing/disabled → deny;
//! - a registered marker with no runnable engine back-reference → deny;
//! - a return value that is not `#{ allowed: true }` → deny.
//!
//! Only a total absence of any marker allows without running anything.
//!
//! **Re-entrancy.** An interceptor that itself performs the guarded op (e.g.
//! an "allow + audit-log" hook that writes KV) must not re-trigger itself. A
//! thread-local guard, set for the duration of the interceptor's synchronous
//! execution, makes any nested op the interceptor performs bypass interception.
use std::cell::{Cell, RefCell};
use std::collections::HashMap;
use std::sync::{Arc, LazyLock};
use std::time::Duration;
use picloud_shared::{
AppId, InterceptorChain, InterceptorEntry, InterceptorService, ScriptId, SdkCallCx,
};
use rhai::EvalAltResult;
use serde_json::{json, Value as Json};
use tokio::runtime::Handle as TokioHandle;
use crate::engine::Engine;
use crate::sandbox::Limits;
use crate::sdk::bridge::runtime_err;
use crate::sdk::invoke::run_resolved_blocking_with_timeout;
/// Default per-interceptor wall-clock timeout when a marker sets no `timeout_ms`
/// (§9.4 M5). Read once from `PICLOUD_INTERCEPTOR_TIMEOUT_MS`; a runaway guard
/// (`loop {}`) is denied once this elapses rather than hanging the write path.
static DEFAULT_TIMEOUT_MS: LazyLock<u32> = LazyLock::new(|| {
std::env::var("PICLOUD_INTERCEPTOR_TIMEOUT_MS")
.ok()
.and_then(|s| s.parse::<u32>().ok())
.filter(|&v| v > 0)
.unwrap_or(5000)
});
/// Clone-cheap bundle of the §9.4 interceptor dependencies threaded into every
/// SDK service `register` fn (M1). Carries the resolver, the `invoke()` re-entry
/// engine back-reference, and the sandbox limits — everything `run_before`
/// (and, from M3, `run_after`) needs. A few `Arc`s plus a small `Limits` copy.
#[derive(Clone)]
pub(crate) struct InterceptorCtx {
pub interceptors: Arc<dyn InterceptorService>,
pub self_engine: Option<Arc<Engine>>,
pub limits: Limits,
}
thread_local! {
/// Re-entrancy depth: `> 0` while an interceptor script (and anything it
/// synchronously invokes) is executing on this thread. The whole re-entry
/// runs synchronously in the caller's `spawn_blocking` thread (same model
/// as `invoke()`), so a thread-local is sufficient and correct.
static IN_INTERCEPTOR: Cell<u32> = const { Cell::new(0) };
/// Identity cycle guard (M2): the `script_id`s of the interceptors currently
/// on the synchronous run stack. A chain that would run an interceptor whose
/// script is already executing is a cycle — we DENY rather than recurse.
/// Precise (keyed by script id) and independent of the binary
/// `IN_INTERCEPTOR` counter, which suppresses interception of an
/// interceptor's OWN nested writes but does not track identity.
static VISITED: RefCell<Vec<ScriptId>> = const { RefCell::new(Vec::new()) };
/// §9.4 M6: per-execution resolve cache. Keyed by `(app_id, service, op)` —
/// the chain a `(service, op)` resolves to is constant for one app across an
/// execution tree, so the FIRST hooked-eligible op of that kind pays the one
/// chain query and every later op reuses it. The dominant win is the
/// un-hooked case: an empty chain is cached once, so N `kv::set`s in a script
/// with no interceptors issue ONE resolve, not N. Cleared at the outermost
/// execution boundary so a pooled thread never serves a stale (or foreign)
/// app's cache.
static RESOLVE_CACHE: RefCell<HashMap<(AppId, String, String), InterceptorChain>> =
RefCell::new(HashMap::new());
/// Scope nesting depth for the resolve cache (mirrors the emission budget):
/// `0` when no execution is active on this thread.
static CACHE_ACTIVE: Cell<u32> = const { Cell::new(0) };
}
/// RAII scope wrapping one `execute_ast` for the §9.4 M6 resolve cache.
/// Re-entrancy aware: the OUTERMOST scope on a thread clears the cache on entry
/// and on final exit; a nested synchronous re-entry (invoke / interceptor)
/// shares the same cache, so a whole execution tree resolves each `(service,
/// op)` at most once.
pub(crate) struct InterceptorCacheScope;
impl InterceptorCacheScope {
pub(crate) fn enter() -> Self {
CACHE_ACTIVE.with(|a| {
if a.get() == 0 {
RESOLVE_CACHE.with(|c| c.borrow_mut().clear());
}
a.set(a.get() + 1);
});
Self
}
}
impl Drop for InterceptorCacheScope {
fn drop(&mut self) {
CACHE_ACTIVE.with(|a| {
let n = a.get().saturating_sub(1);
a.set(n);
if n == 0 {
RESOLVE_CACHE.with(|c| c.borrow_mut().clear());
}
});
}
}
fn currently_in_interceptor() -> bool {
IN_INTERCEPTOR.with(|c| c.get() > 0)
}
/// RAII: increment the re-entrancy counter while an interceptor runs, decrement
/// on drop (including on the `?`/panic-unwind paths).
struct ReentryGuard;
impl ReentryGuard {
fn enter() -> Self {
IN_INTERCEPTOR.with(|c| c.set(c.get().saturating_add(1)));
Self
}
}
impl Drop for ReentryGuard {
fn drop(&mut self) {
IN_INTERCEPTOR.with(|c| c.set(c.get().saturating_sub(1)));
}
}
/// RAII for the identity cycle guard: push a `script_id` on enter, pop on drop
/// (including the `?`/panic-unwind paths), so the visited set exactly tracks the
/// interceptors on the current synchronous run stack.
struct CycleGuard;
impl CycleGuard {
fn enter(id: ScriptId) -> Self {
VISITED.with(|v| v.borrow_mut().push(id));
Self
}
fn contains(id: ScriptId) -> bool {
VISITED.with(|v| v.borrow().contains(&id))
}
}
impl Drop for CycleGuard {
fn drop(&mut self) {
VISITED.with(|v| {
v.borrow_mut().pop();
});
}
}
/// The parsed outcome of ONE allowing hook: the operation passes, optionally
/// carrying a `data` transform (M4) the caller writes instead of the original.
struct HookAllow {
/// `Some` iff the hook returned a `data` key alongside `allowed: true`.
data: Option<Json>,
}
/// Run the before-op interceptor CHAIN for `(service, op)` if any are
/// registered. Returns `Ok(Some(value))` when a before-hook rewrote the payload
/// via a `data` transform (M4) — the caller writes THAT instead of the original
/// — or `Ok(None)` to write the original unchanged (un-hooked, or every hook
/// allowed without transforming). An `Err` denies the operation (the caller
/// must NOT perform the write); the FIRST deny short-circuits. `value` is the
/// original payload (`None` for delete, which never transforms).
#[allow(clippy::too_many_arguments)]
pub(super) fn run_before(
ictx: &InterceptorCtx,
cx: &Arc<SdkCallCx>,
service: &'static str,
op: &'static str,
collection: &str,
key: &str,
value: Option<&Json>,
) -> Result<Option<Json>, Box<EvalAltResult>> {
// Re-entrancy break: writes performed BY an interceptor (or anything it
// invokes) are not themselves intercepted — otherwise an "allow + write"
// hook would recurse to the depth cap and deny the original op.
if currently_in_interceptor() {
return Ok(None);
}
let chain = resolve_chain(ictx, cx, service, op)?;
if chain.before.is_empty() {
return Ok(None);
}
// Thread the (possibly rewritten) value through the chain: each before-hook
// sees the prior hook's transform in `value`, and the write uses the final.
let mut current: Option<Json> = value.cloned();
let mut transformed = false;
for entry in &chain.before {
let payload = op_payload(service, op, collection, key, current.as_ref(), cx, None);
let outcome = run_one_hook(ictx, cx, entry, service, op, &payload)?;
// M4 data-transform: honored ONLY on an allowing hook (the verdict parse
// already enforced fail-closed). Delete has no value to rewrite.
if let Some(data) = outcome.data {
if value.is_some() {
enforce_transform_size(service, op, &data)?;
current = Some(data);
transformed = true;
}
}
}
Ok(transformed.then(|| current.unwrap_or(Json::Null)))
}
/// Run the after-op interceptor CHAIN (M3) once the write has committed.
/// `result` is the write's outcome (serialized), surfaced to the hook so it can
/// observe/audit. After-hooks CANNOT roll back — the write already happened; a
/// deny here surfaces as an operation error to the caller but the write
/// persists. Returns `Ok(())` when every after-hook allowed (or none exist), an
/// `Err` on the first deny.
#[allow(clippy::too_many_arguments)]
pub(super) fn run_after(
ictx: &InterceptorCtx,
cx: &Arc<SdkCallCx>,
service: &'static str,
op: &'static str,
collection: &str,
key: &str,
value: Option<&Json>,
result: Json,
) -> Result<(), Box<EvalAltResult>> {
if currently_in_interceptor() {
return Ok(());
}
let chain = resolve_chain(ictx, cx, service, op)?;
if chain.after.is_empty() {
return Ok(());
}
for entry in &chain.after {
let payload = op_payload(service, op, collection, key, value, cx, Some(&result));
// After-hooks observe/deny; a returned `data` is ignored (the write is
// already durable, so there is nothing to rewrite for the kv slice).
run_one_hook(ictx, cx, entry, service, op, &payload)?;
}
Ok(())
}
/// Resolve the before+after chain for `(service, op)` — every marker visible on
/// the calling app's chain, ordered per phase. Memoized per execution tree
/// (§9.4 M6): the FIRST call for a `(app_id, service, op)` pays the chain query;
/// later calls reuse the cached result. An `Err` (no tokio runtime, or a backend
/// failure) makes the caller fail closed rather than silently allow.
fn resolve_chain(
ictx: &InterceptorCtx,
cx: &Arc<SdkCallCx>,
service: &'static str,
op: &'static str,
) -> Result<InterceptorChain, Box<EvalAltResult>> {
let cache_key = (cx.app_id, service.to_string(), op.to_string());
if let Some(hit) = RESOLVE_CACHE.with(|c| c.borrow().get(&cache_key).cloned()) {
return Ok(hit);
}
let handle = TokioHandle::try_current()
.map_err(|e| runtime_err(&format!("{service} interceptor: no tokio runtime: {e}")))?;
let interceptors = ictx.interceptors.clone();
let resolve_cx = cx.clone();
let chain = handle
.block_on(async move { interceptors.resolve(&resolve_cx, service, op).await })
.map_err(|e| runtime_err(&format!("{service}::{op} interceptor resolve: {e}")))?;
RESOLVE_CACHE.with(|c| c.borrow_mut().insert(cache_key, chain.clone()));
Ok(chain)
}
/// Build the operation-context payload handed to a hook. `result` is `Some` only
/// for an after-hook (the write's outcome).
fn op_payload(
service: &str,
op: &str,
collection: &str,
key: &str,
value: Option<&Json>,
cx: &SdkCallCx,
result: Option<&Json>,
) -> Json {
let mut m = json!({
"service": service,
"action": op,
"collection": collection,
"key": key,
"value": value,
"caller_script_id": cx.script_id.to_string(),
"caller_execution_id": cx.execution_id.to_string(),
});
if let (Json::Object(obj), Some(r)) = (&mut m, result) {
obj.insert("result".to_string(), r.clone());
}
m
}
/// Run ONE chain entry: fail-closed on a dangling marker, an identity cycle, a
/// missing engine back-reference, or a depth-limit breach; otherwise run the
/// sealed script and parse its verdict. Returns the allow outcome (with any
/// `data` transform) or an `Err` denying the operation.
fn run_one_hook(
ictx: &InterceptorCtx,
cx: &Arc<SdkCallCx>,
entry: &InterceptorEntry,
service: &str,
op: &str,
payload: &Json,
) -> Result<HookAllow, Box<EvalAltResult>> {
let resolved = match entry {
// A guard is declared but its script is missing/disabled — fail closed.
InterceptorEntry::Dangling(name) => {
return Err(runtime_err(&format!(
"{service}::{op} denied: interceptor script `{name}` is missing or disabled"
)));
}
InterceptorEntry::Run(r) => r,
};
let script = &resolved.script;
// Identity cycle guard: if this interceptor's script is already on the run
// stack, running it again would recurse forever — DENY.
if CycleGuard::contains(script.script_id) {
return Err(runtime_err(&format!(
"{service}::{op} denied: interceptor cycle detected: `{}`",
script.name
)));
}
// A guard is registered but the engine back-reference isn't installed (a bare
// test engine without `set_self_weak`): we cannot run it, so DENY — a
// resolvable guard we can't execute must not fail open. Production always
// installs the back-ref.
let Some(self_engine) = ictx.self_engine.as_ref() else {
return Err(runtime_err(&format!(
"{service}::{op} denied: interceptor `{}` cannot run (engine back-reference not installed)",
script.name
)));
};
// Depth bound (shared with invoke / trigger fan-out).
if cx.trigger_depth + 1 > ictx.limits.trigger_depth_max {
return Err(runtime_err(&format!(
"{service}::{op} interceptor `{}`: depth limit exceeded (max {})",
script.name, ictx.limits.trigger_depth_max
)));
}
// Run the sealed script with the operation context as its request body, under
// the re-entrancy guard (its own writes bypass interception), the identity
// cycle guard (its own script id is on the stack), and a per-hook timeout
// (§9.4 M5 — the marker's `timeout_ms` or the env default, tightened to at
// most the caller's remaining deadline). A runaway guard is denied, not hung.
let timeout = Duration::from_millis(u64::from(
resolved.timeout_ms.unwrap_or(*DEFAULT_TIMEOUT_MS),
));
let ret = {
let _reentry = ReentryGuard::enter();
let _cycle = CycleGuard::enter(script.script_id);
run_resolved_blocking_with_timeout(
self_engine,
cx,
script,
payload.clone(),
&format!("{service}::{op} interceptor `{}`", script.name),
Some(timeout),
)?
};
// Fail-closed verdict: allow ONLY on an explicit `#{ allowed: true }`. A bare
// bool, a typo'd key, a non-bool `allowed`, a bare unit, or any other shape
// DENIES — a control whose job is to deny must not allow by omission.
match ret {
Json::Object(mut m) if m.get("allowed") == Some(&Json::Bool(true)) => Ok(HookAllow {
data: m.remove("data"),
}),
Json::Object(m) => {
let reason = m
.get("reason")
.and_then(Json::as_str)
.unwrap_or("denied by interceptor");
Err(runtime_err(&format!(
"{service}::{op} denied by interceptor `{}`: {reason}",
script.name
)))
}
_ => Err(runtime_err(&format!(
"{service}::{op} denied by interceptor `{}`: no allow verdict returned",
script.name
))),
}
}
/// Cap a `data` transform at the same byte ceiling the SDK uses for a Rhai→JSON
/// materialization, so an interceptor can't rewrite a small write into an
/// unbounded blob before it reaches the service's own value cap.
fn enforce_transform_size(service: &str, op: &str, data: &Json) -> Result<(), Box<EvalAltResult>> {
let len = serde_json::to_vec(data)
.map(|v| v.len())
.unwrap_or(usize::MAX);
if len > crate::sdk::bridge::MAX_JSON_MATERIALIZE_BYTES {
return Err(runtime_err(&format!(
"{service}::{op} interceptor transform too large ({len} bytes exceeds the {}-byte limit)",
crate::sdk::bridge::MAX_JSON_MATERIALIZE_BYTES
)));
}
Ok(())
}

View File

@@ -42,7 +42,7 @@ use tokio::runtime::Handle as TokioHandle;
use crate::engine::Engine;
use crate::sandbox::Limits;
use crate::sdk::bridge::json_to_dynamic;
use crate::sdk::bridge::{json_to_dynamic, MAX_JSON_MATERIALIZE_BYTES};
use crate::types::{ExecRequest, InvocationType};
pub(super) fn register(
@@ -85,6 +85,9 @@ pub(super) fn register(
module.set_native_fn(
"invoke_async",
move |target: Dynamic, args: Dynamic| -> Result<String, Box<EvalAltResult>> {
// Audit #2: count this durable emission against the per-execution
// fan-out ceiling before doing any work.
crate::sdk::emit_budget::charge_emission("invoke_async")?;
let target = parse_target(target)?;
let args_json = args_to_json(&args)?;
let svc = svc.clone();
@@ -163,9 +166,69 @@ fn invoke_blocking(
.into()
})?;
let execution_id = ExecutionId::new();
// The callee's return is the response `body` JSON. Convert back to
// Dynamic for the caller. Status code + headers are dropped; the
// function-call mental model is "return value", not HTTP response.
let body = run_resolved_blocking(self_engine, cx, &resolved, args_json, &target_label)?;
Ok(json_to_dynamic(body))
}
/// Synchronous same-engine re-entry: build the callee `ExecRequest` (inheriting
/// the caller's app/principal/root/depth+1 and the callee's lexical owner),
/// compile through the per-Engine AST cache (F-P-004), execute, and return the
/// response `body` JSON. Shared by `invoke()` and the §9.4 interceptor hook —
/// the caller is responsible for the depth check (both do it before resolving).
/// `label` prefixes any compile/execute error.
/// Run a resolved script through the `invoke()` re-entry core, inheriting the
/// caller's ambient execution deadline.
pub(super) fn run_resolved_blocking(
self_engine: &Arc<Engine>,
cx: &Arc<SdkCallCx>,
resolved: &picloud_shared::ResolvedScript,
body_json: Json,
label: &str,
) -> Result<Json, Box<EvalAltResult>> {
run_resolved_core(
self_engine,
cx,
resolved,
body_json,
label,
crate::engine::ambient_deadline(),
)
}
/// Like [`run_resolved_blocking`] but bounds the run by a per-hook `timeout`
/// (§9.4 M5). The effective deadline is `min(ambient, now + timeout)` — a hook
/// can only ever TIGHTEN, never extend, its caller's deadline.
pub(super) fn run_resolved_blocking_with_timeout(
self_engine: &Arc<Engine>,
cx: &Arc<SdkCallCx>,
resolved: &picloud_shared::ResolvedScript,
body_json: Json,
label: &str,
timeout: Option<std::time::Duration>,
) -> Result<Json, Box<EvalAltResult>> {
let ambient = crate::engine::ambient_deadline();
let own = timeout.map(|d| std::time::Instant::now() + d);
let effective = match (ambient, own) {
(Some(a), Some(o)) => Some(a.min(o)),
(a, None) => a,
(None, o) => o,
};
run_resolved_core(self_engine, cx, resolved, body_json, label, effective)
}
fn run_resolved_core(
self_engine: &Arc<Engine>,
cx: &Arc<SdkCallCx>,
resolved: &picloud_shared::ResolvedScript,
body_json: Json,
label: &str,
deadline: Option<std::time::Instant>,
) -> Result<Json, Box<EvalAltResult>> {
let req = ExecRequest {
execution_id,
execution_id: ExecutionId::new(),
request_id: cx.request_id,
script_id: resolved.script_id,
script_name: resolved.name.clone(),
@@ -173,7 +236,7 @@ fn invoke_blocking(
path: "/invoke".into(),
method: String::new(),
headers: BTreeMap::new(),
body: args_json,
body: body_json,
params: BTreeMap::new(),
query: BTreeMap::new(),
rest: String::new(),
@@ -183,42 +246,24 @@ fn invoke_blocking(
// script's imports resolve from the group even when invoked by an
// app. `None` falls back to `App(cx.app_id)` in the engine.
script_owner: resolved.owner,
// Same-app invoke is a function call, not a re-auth boundary —
// inherit the caller's principal.
// Same-app re-entry is not a re-auth boundary — inherit the principal.
principal: cx.principal.clone(),
trigger_depth: cx.trigger_depth + 1,
root_execution_id: cx.root_execution_id,
is_dead_letter_handler: cx.is_dead_letter_handler,
event: None,
};
// F-P-004: synchronous re-entry — route through the per-Engine
// AST cache so each callee parses once per (script_id, updated_at),
// not once per invoke. Composed workflows multiply parse cost by
// depth; the cache cuts that to constant compile + N executions.
let ast = self_engine
.compile_for_identity(resolved.script_id, resolved.updated_at, &resolved.source)
.map_err(|e| -> Box<EvalAltResult> {
EvalAltResult::ErrorRuntime(
format!("invoke({target_label}): {e}").into(),
rhai::Position::NONE,
)
.into()
EvalAltResult::ErrorRuntime(format!("{label}: {e}").into(), rhai::Position::NONE).into()
})?;
let resp = self_engine
.execute_ast(&ast, req)
.execute_ast_with_deadline(&ast, req, deadline)
.map_err(|e| -> Box<EvalAltResult> {
EvalAltResult::ErrorRuntime(
format!("invoke({target_label}): {e}").into(),
rhai::Position::NONE,
)
.into()
EvalAltResult::ErrorRuntime(format!("{label}: {e}").into(), rhai::Position::NONE).into()
})?;
// The callee's return is the response `body` JSON. Convert back to
// Dynamic for the caller. Status code + headers are dropped; the
// function-call mental model is "return value", not HTTP response.
Ok(json_to_dynamic(resp.body))
Ok(resp.body)
}
/// Accept a string (route path OR script name) or a Rhai script-id
@@ -247,8 +292,35 @@ fn parse_target(target: Dynamic) -> Result<InvokeTarget, Box<EvalAltResult>> {
}
/// Convert Rhai Dynamic → JSON. Rejects FnPtr at any depth (closures
/// don't survive invoke boundaries).
/// don't survive invoke boundaries) and is bounded by
/// [`MAX_JSON_MATERIALIZE_BYTES`] so a cheaply-aliased huge args value can't OOM
/// the node on materialization (same anti-OOM rail as `dynamic_to_json_capped`).
fn args_to_json(value: &Dynamic) -> Result<Json, Box<EvalAltResult>> {
let mut remaining = MAX_JSON_MATERIALIZE_BYTES;
args_to_json_bounded(value, &mut remaining)
}
fn args_charge(remaining: &mut usize, n: usize) -> Result<(), Box<EvalAltResult>> {
match remaining.checked_sub(n) {
Some(left) => {
*remaining = left;
Ok(())
}
None => Err(EvalAltResult::ErrorRuntime(
format!(
"invoke: args too large to serialize (exceeds the {MAX_JSON_MATERIALIZE_BYTES}-byte limit)"
)
.into(),
rhai::Position::NONE,
)
.into()),
}
}
fn args_to_json_bounded(
value: &Dynamic,
remaining: &mut usize,
) -> Result<Json, Box<EvalAltResult>> {
if value.is::<rhai::FnPtr>() {
return Err(EvalAltResult::ErrorRuntime(
"invoke: args must not contain FnPtr / closures".into(),
@@ -258,40 +330,52 @@ fn args_to_json(value: &Dynamic) -> Result<Json, Box<EvalAltResult>> {
}
if value.is_blob() {
let blob = value.clone().into_blob().unwrap_or_default();
return Ok(Json::String(STANDARD.encode(&blob)));
let encoded = STANDARD.encode(&blob);
args_charge(remaining, encoded.len() + 2)?;
return Ok(Json::String(encoded));
}
if value.is_unit() {
args_charge(remaining, 4)?;
return Ok(Json::Null);
}
if let Ok(b) = value.as_bool() {
args_charge(remaining, 5)?;
return Ok(Json::Bool(b));
}
if let Ok(i) = value.as_int() {
args_charge(remaining, 8)?;
return Ok(Json::Number(i.into()));
}
if let Ok(f) = value.as_float() {
args_charge(remaining, 8)?;
return Ok(serde_json::Number::from_f64(f).map_or(Json::Null, Json::Number));
}
if value.is_string() {
return Ok(Json::String(
value.clone().into_string().unwrap_or_default(),
));
let s = value.clone().into_string().unwrap_or_default();
args_charge(remaining, s.len() + 2)?;
return Ok(Json::String(s));
}
if let Some(arr) = value.clone().try_cast::<Array>() {
let mut out = Vec::with_capacity(arr.len());
args_charge(remaining, 2)?;
let mut out = Vec::with_capacity(arr.len().min(1024));
for v in &arr {
out.push(args_to_json(v)?);
args_charge(remaining, 1)?;
out.push(args_to_json_bounded(v, remaining)?);
}
return Ok(Json::Array(out));
}
if let Some(map) = value.clone().try_cast::<Map>() {
args_charge(remaining, 2)?;
let mut out = serde_json::Map::new();
for (k, v) in map {
out.insert(k.to_string(), args_to_json(&v)?);
args_charge(remaining, k.len() + 4)?;
out.insert(k.to_string(), args_to_json_bounded(&v, remaining)?);
}
return Ok(Json::Object(out));
}
Ok(Json::String(value.to_string()))
let s = value.to_string();
args_charge(remaining, s.len() + 2)?;
Ok(Json::String(s))
}
/// `Limits` is `Copy`; passed by value into `register` so closures take

View File

@@ -5,6 +5,8 @@
//! widgets.set("k", #{ n: 1 });
//! let v = widgets.get("k"); // value or () if absent
//! if widgets.has("k") { ... }
//! widgets.set_if("k", (), #{ n: 0 }); // insert only if ABSENT -> bool
//! widgets.set_if("k", old, new); // swap only if current == old -> bool
//! widgets.delete("k"); // bool (was-present)
//! let page = widgets.list(); // returns #{ keys: [...], next_cursor: () }
//! ```
@@ -31,40 +33,54 @@ use std::sync::Arc;
use picloud_shared::{GroupKvService, KvService, SdkCallCx, Services};
use rhai::{Array, Dynamic, Engine as RhaiEngine, EvalAltResult, Map, Module};
use super::bridge::{block_on, dynamic_to_json, json_to_dynamic};
use super::bridge::{
block_on, dynamic_to_json_capped, json_to_dynamic, JsonSizeError, MAX_JSON_MATERIALIZE_BYTES,
};
use super::interceptor::InterceptorCtx;
/// Per-call handle captured by the Rhai SDK. Cheap to clone (two Arcs
/// plus an owned string).
/// Per-call handle captured by the Rhai SDK. Cheap to clone (a few Arcs plus an
/// owned string). Carries the §9.4 interceptor ctx so `set`/`delete` can run a
/// before-op allow/deny hook (the resolver + the `invoke()` re-entry engine).
#[derive(Clone)]
pub struct KvHandle {
collection: String,
service: Arc<dyn KvService>,
cx: Arc<SdkCallCx>,
ictx: InterceptorCtx,
}
/// §11.6 shared-collection handle, returned by `kv::shared_collection(name)`. A distinct
/// Rhai type from `KvHandle` so the method set can diverge (e.g. shared
/// collections never grow triggers) and a script's choice of private-vs-shared
/// scope is explicit. Routes through the `GroupKvService`, which resolves the
/// owning group from `cx.app_id` (the script never names a group).
/// owning group from `cx.app_id` (the script never names a group). Carries the
/// §9.4 interceptor ctx too so a `(kv, set/delete)` guard covers shared-
/// collection writes — otherwise the shared handle would be a silent bypass.
#[derive(Clone)]
pub struct GroupKvHandle {
collection: String,
service: Arc<dyn GroupKvService>,
cx: Arc<SdkCallCx>,
ictx: InterceptorCtx,
}
pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<SdkCallCx>) {
pub(super) fn register(
engine: &mut RhaiEngine,
services: &Services,
cx: Arc<SdkCallCx>,
ictx: InterceptorCtx,
) {
let kv_service = services.kv.clone();
let group_kv_service = services.group_kv.clone();
// `kv::collection(name)` / `kv::shared_collection(name)` — both constructors live in
// the `kv` static module so the script-visible calls are `kv::collection`
// and `kv::shared`.
// and `kv::shared_collection` (`shared` alone is a Rhai reserved word).
let mut module = Module::new();
{
let kv_service = kv_service.clone();
let cx = cx.clone();
let ictx = ictx.clone();
module.set_native_fn(
"collection",
move |name: &str| -> Result<KvHandle, Box<EvalAltResult>> {
@@ -75,6 +91,7 @@ pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<Sdk
collection: name.to_string(),
service: kv_service.clone(),
cx: cx.clone(),
ictx: ictx.clone(),
})
},
);
@@ -82,6 +99,7 @@ pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<Sdk
{
let group_kv_service = group_kv_service.clone();
let cx = cx.clone();
let ictx = ictx.clone();
module.set_native_fn(
"shared_collection",
move |name: &str| -> Result<GroupKvHandle, Box<EvalAltResult>> {
@@ -92,6 +110,7 @@ pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<Sdk
collection: name.to_string(),
service: group_kv_service.clone(),
cx: cx.clone(),
ictx: ictx.clone(),
})
},
);
@@ -105,6 +124,7 @@ pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<Sdk
register_get(engine);
register_set(engine);
register_set_if(engine);
register_has(engine);
register_delete(engine);
register_list(engine);
@@ -114,11 +134,26 @@ pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<Sdk
register_group_get(engine);
register_group_set(engine);
register_group_set_if(engine);
register_group_has(engine);
register_group_delete(engine);
register_group_list(engine);
}
/// Map a Rhai `expected` argument to the CAS precondition: Rhai unit `()` means
/// "expected ABSENT" (insert-if-absent); any other value is the expected current
/// value.
fn expected_from_dynamic(expected: &Dynamic) -> Result<Option<serde_json::Value>, JsonSizeError> {
if expected.is_unit() {
Ok(None)
} else {
Ok(Some(dynamic_to_json_capped(
expected,
MAX_JSON_MATERIALIZE_BYTES,
)?))
}
}
fn register_get(engine: &mut RhaiEngine) {
engine.register_fn(
"get",
@@ -136,11 +171,82 @@ fn register_set(engine: &mut RhaiEngine) {
engine.register_fn(
"set",
|handle: &mut KvHandle, key: &str, value: Dynamic| -> Result<(), Box<EvalAltResult>> {
let json = dynamic_to_json_capped(&value, MAX_JSON_MATERIALIZE_BYTES)?;
// §9.4 before-op interceptor (allow/deny + M4 data-transform). A
// denial errors here; a returned `data` rewrites the written value.
let write_val = super::interceptor::run_before(
&handle.ictx,
&handle.cx,
"kv",
"set",
&handle.collection,
key,
Some(&json),
)?
.unwrap_or(json);
let written = write_val.clone();
let h = handle.clone();
let json = dynamic_to_json(&value);
block_on("kv", async move {
h.service.set(&h.cx, &h.collection, key, json).await
})
h.service.set(&h.cx, &h.collection, key, write_val).await
})?;
// §9.4 M3 after-hook (observe/audit; the write already committed).
super::interceptor::run_after(
&handle.ictx,
&handle.cx,
"kv",
"set",
&handle.collection,
key,
Some(&written),
serde_json::Value::Null,
)
},
);
}
fn register_set_if(engine: &mut RhaiEngine) {
// `handle.set_if(key, expected, new)` — CAS. `expected = ()` means "only if
// absent". Returns `true` if the swap happened, `false` if the precondition
// failed.
engine.register_fn(
"set_if",
|handle: &mut KvHandle,
key: &str,
expected: Dynamic,
new: Dynamic|
-> Result<bool, Box<EvalAltResult>> {
let exp = expected_from_dynamic(&expected)?;
let json = dynamic_to_json_capped(&new, MAX_JSON_MATERIALIZE_BYTES)?;
// §9.4 M12: CAS is a WRITE, so it runs the `(kv, set)` guard too —
// otherwise `set_if` would be a silent bypass of a `set` policy.
let write_val = super::interceptor::run_before(
&handle.ictx,
&handle.cx,
"kv",
"set",
&handle.collection,
key,
Some(&json),
)?
.unwrap_or(json);
let written = write_val.clone();
let h = handle.clone();
let swapped = block_on("kv", async move {
h.service
.set_if(&h.cx, &h.collection, key, exp, write_val)
.await
})?;
super::interceptor::run_after(
&handle.ictx,
&handle.cx,
"kv",
"set",
&handle.collection,
key,
Some(&written),
serde_json::Value::Bool(swapped),
)?;
Ok(swapped)
},
);
}
@@ -161,14 +267,73 @@ fn register_delete(engine: &mut RhaiEngine) {
engine.register_fn(
"delete",
|handle: &mut KvHandle, key: &str| -> Result<bool, Box<EvalAltResult>> {
// Delete carries no value, so the before-hook never transforms.
super::interceptor::run_before(
&handle.ictx,
&handle.cx,
"kv",
"delete",
&handle.collection,
key,
None,
)?;
let h = handle.clone();
block_on("kv", async move {
let was_present = block_on("kv", async move {
h.service.delete(&h.cx, &h.collection, key).await
})
})?;
super::interceptor::run_after(
&handle.ictx,
&handle.cx,
"kv",
"delete",
&handle.collection,
key,
None,
serde_json::Value::Bool(was_present),
)?;
Ok(was_present)
},
);
}
/// §9.4: run the before-op interceptor for a shared-collection write, so a
/// `(kv, set/delete)` guard covers the shared handle too (not just private KV).
fn group_run_before(
handle: &GroupKvHandle,
op: &'static str,
key: &str,
value: Option<&serde_json::Value>,
) -> Result<Option<serde_json::Value>, Box<EvalAltResult>> {
super::interceptor::run_before(
&handle.ictx,
&handle.cx,
"kv",
op,
&handle.collection,
key,
value,
)
}
fn group_run_after(
handle: &GroupKvHandle,
op: &'static str,
key: &str,
value: Option<&serde_json::Value>,
result: serde_json::Value,
) -> Result<(), Box<EvalAltResult>> {
super::interceptor::run_after(
&handle.ictx,
&handle.cx,
"kv",
op,
&handle.collection,
key,
value,
result,
)
}
fn register_list(engine: &mut RhaiEngine) {
// Zero-arg form — full page, no cursor.
engine.register_fn(
@@ -234,11 +399,47 @@ fn register_group_set(engine: &mut RhaiEngine) {
engine.register_fn(
"set",
|handle: &mut GroupKvHandle, key: &str, value: Dynamic| -> Result<(), Box<EvalAltResult>> {
let json = dynamic_to_json_capped(&value, MAX_JSON_MATERIALIZE_BYTES)?;
// §9.4 before-op interceptor also guards shared-collection writes
// (allow/deny + M4 data-transform).
let write_val = group_run_before(handle, "set", key, Some(&json))?.unwrap_or(json);
let written = write_val.clone();
let h = handle.clone();
let json = dynamic_to_json(&value);
block_on("kv", async move {
h.service.set(&h.cx, &h.collection, key, json).await
})
h.service.set(&h.cx, &h.collection, key, write_val).await
})?;
group_run_after(handle, "set", key, Some(&written), serde_json::Value::Null)
},
);
}
fn register_group_set_if(engine: &mut RhaiEngine) {
engine.register_fn(
"set_if",
|handle: &mut GroupKvHandle,
key: &str,
expected: Dynamic,
new: Dynamic|
-> Result<bool, Box<EvalAltResult>> {
let exp = expected_from_dynamic(&expected)?;
let json = dynamic_to_json_capped(&new, MAX_JSON_MATERIALIZE_BYTES)?;
// §9.4 M12: CAS on a shared collection runs the `(kv, set)` guard too.
let write_val = group_run_before(handle, "set", key, Some(&json))?.unwrap_or(json);
let written = write_val.clone();
let h = handle.clone();
let swapped = block_on("kv", async move {
h.service
.set_if(&h.cx, &h.collection, key, exp, write_val)
.await
})?;
group_run_after(
handle,
"set",
key,
Some(&written),
serde_json::Value::Bool(swapped),
)?;
Ok(swapped)
},
);
}
@@ -259,10 +460,19 @@ fn register_group_delete(engine: &mut RhaiEngine) {
engine.register_fn(
"delete",
|handle: &mut GroupKvHandle, key: &str| -> Result<bool, Box<EvalAltResult>> {
group_run_before(handle, "delete", key, None)?;
let h = handle.clone();
block_on("kv", async move {
let was_present = block_on("kv", async move {
h.service.delete(&h.cx, &h.collection, key).await
})
})?;
group_run_after(
handle,
"delete",
key,
None,
serde_json::Value::Bool(was_present),
)?;
Ok(was_present)
},
);
}

View File

@@ -16,8 +16,10 @@ pub mod cx;
pub mod dead_letters;
pub mod docs;
pub mod email;
pub mod emit_budget;
pub mod files;
pub mod http;
pub mod interceptor;
pub mod invoke;
pub mod kv;
pub mod pubsub;
@@ -27,8 +29,12 @@ pub mod secrets;
pub mod stdlib;
pub mod users;
pub mod vars;
pub mod workflow;
pub use bridge::{dynamic_to_json, json_to_dynamic};
pub use bridge::{
dynamic_to_json, dynamic_to_json_capped, json_to_dynamic, JsonSizeError,
MAX_JSON_MATERIALIZE_BYTES,
};
pub use cx::SdkCallCx;
use std::sync::Arc;
@@ -54,17 +60,26 @@ pub fn register_all(
limits: Limits,
self_engine: Option<Arc<Engine>>,
) {
kv::register(engine, services, cx.clone());
docs::register(engine, services, cx.clone());
// §9.4 M1: build the interceptor ctx once and thread it into every SDK
// service that has (or will gain) a before/after hook. Only `kv` runs a
// hook today; docs/files/queue/pubsub/http carry it for M7M11.
let ictx = interceptor::InterceptorCtx {
interceptors: services.interceptors.clone(),
self_engine: self_engine.clone(),
limits,
};
kv::register(engine, services, cx.clone(), ictx.clone());
docs::register(engine, services, cx.clone(), ictx.clone());
dead_letters::register(engine, services, cx.clone());
http::register(engine, services, cx.clone());
files::register(engine, services, cx.clone());
pubsub::register(engine, services, cx.clone());
queue::register(engine, services, cx.clone());
http::register(engine, services, cx.clone(), ictx.clone());
files::register(engine, services, cx.clone(), ictx.clone());
pubsub::register(engine, services, cx.clone(), ictx.clone());
queue::register(engine, services, cx.clone(), ictx);
retry::register(engine, services, cx.clone());
secrets::register(engine, services, cx.clone());
vars::register(engine, services, cx.clone());
email::register(engine, services, cx.clone());
users::register(engine, services, cx.clone());
workflow::register(engine, services, cx.clone());
invoke::register(engine, services, cx, limits, self_engine);
}

View File

@@ -24,23 +24,53 @@ use rhai::{Array, Dynamic, Engine as RhaiEngine, EvalAltResult, Map, Module};
use serde_json::Value as Json;
use tokio::runtime::Handle as TokioHandle;
use super::bridge::block_on;
use super::bridge::{block_on, MAX_JSON_MATERIALIZE_BYTES};
use super::interceptor::InterceptorCtx;
pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<SdkCallCx>) {
pub(super) fn register(
engine: &mut RhaiEngine,
services: &Services,
cx: Arc<SdkCallCx>,
ictx: InterceptorCtx,
) {
let svc = services.pubsub.clone();
let mut module = Module::new();
{
let svc = svc.clone();
let cx = cx.clone();
let ictx = ictx.clone();
module.set_native_fn(
"publish_durable",
move |topic: &str, message: Dynamic| -> Result<(), Box<EvalAltResult>> {
let json = message_to_json(&message);
let svc = svc.clone();
let cx = cx.clone();
crate::sdk::emit_budget::charge_emission("pubsub::publish_durable")?;
let json = message_to_json(&message)?;
// §9.4 before-op interceptor (allow/deny + M4 data-transform).
let write_val = super::interceptor::run_before(
&ictx,
&cx,
"pubsub",
"publish",
topic,
"",
Some(&json),
)?
.unwrap_or(json);
let written = write_val.clone();
let svc2 = svc.clone();
let cx2 = cx.clone();
block_on("pubsub", async move {
svc.publish_durable(&cx, topic, json).await
})
svc2.publish_durable(&cx2, topic, write_val).await
})?;
super::interceptor::run_after(
&ictx,
&cx,
"pubsub",
"publish",
topic,
"",
Some(&written),
Json::Null,
)
},
);
}
@@ -76,6 +106,7 @@ pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<Sdk
{
let group_svc = services.group_pubsub.clone();
let cx = cx.clone();
let ictx = ictx.clone();
module.set_native_fn(
"shared_topic",
move |name: &str| -> Result<GroupTopicHandle, Box<EvalAltResult>> {
@@ -86,6 +117,7 @@ pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<Sdk
namespace: name.to_string(),
service: group_svc.clone(),
cx: cx.clone(),
ictx: ictx.clone(),
})
},
);
@@ -103,6 +135,8 @@ pub struct GroupTopicHandle {
namespace: String,
service: Arc<dyn picloud_shared::GroupPubsubService>,
cx: Arc<SdkCallCx>,
// §9.4: carried so shared `publish` runs the before/after hook too (M11).
ictx: InterceptorCtx,
}
fn shared_publish(
@@ -110,17 +144,43 @@ fn shared_publish(
subtopic: &str,
message: Dynamic,
) -> Result<(), Box<EvalAltResult>> {
let json = message_to_json(&message);
crate::sdk::emit_budget::charge_emission("pubsub::shared_topic")?;
let json = message_to_json(&message)?;
// §9.4 before-op interceptor also guards shared-topic publishes. The
// collection is the namespace; the subtopic is the key.
let write_val = super::interceptor::run_before(
&h.ictx,
&h.cx,
"pubsub",
"publish",
&h.namespace,
subtopic,
Some(&json),
)?
.unwrap_or(json);
let written = write_val.clone();
let service = h.service.clone();
let cx = h.cx.clone();
let namespace = h.namespace.clone();
let subtopic = subtopic.to_string();
let subtopic_owned = subtopic.to_string();
block_on("pubsub", async move {
service.publish(&cx, &namespace, &subtopic, json).await
service
.publish(&cx, &namespace, &subtopic_owned, write_val)
.await
})
// The fan-out count isn't surfaced to scripts (fire-and-forget, like the
// per-app publish).
.map(|_n: u32| ())
.map(|_n: u32| ())?;
super::interceptor::run_after(
&h.ictx,
&h.cx,
"pubsub",
"publish",
&h.namespace,
subtopic,
Some(&written),
Json::Null,
)
}
fn register_shared_publish(engine: &mut RhaiEngine) {
@@ -189,38 +249,81 @@ fn mint_token(
/// Convert a Rhai `Dynamic` message into JSON, base64-encoding any
/// `Blob` (at any nesting depth). Mirrors `bridge::dynamic_to_json` but
/// adds the blob arm the pub/sub wire contract requires.
fn message_to_json(value: &Dynamic) -> Json {
/// adds the blob arm the pub/sub wire contract requires. Bounded by
/// [`MAX_JSON_MATERIALIZE_BYTES`] so a cheaply-aliased huge message can't OOM
/// the node on materialization (the per-message wire cap is enforced later, on
/// the already-materialized value).
fn message_to_json(value: &Dynamic) -> Result<Json, Box<EvalAltResult>> {
let mut remaining = MAX_JSON_MATERIALIZE_BYTES;
message_to_json_bounded(value, &mut remaining)
}
fn msg_charge(remaining: &mut usize, n: usize) -> Result<(), Box<EvalAltResult>> {
match remaining.checked_sub(n) {
Some(left) => {
*remaining = left;
Ok(())
}
None => Err(EvalAltResult::ErrorRuntime(
format!("pubsub: message too large to serialize (exceeds the {MAX_JSON_MATERIALIZE_BYTES}-byte limit)").into(),
rhai::Position::NONE,
)
.into()),
}
}
fn message_to_json_bounded(
value: &Dynamic,
remaining: &mut usize,
) -> Result<Json, Box<EvalAltResult>> {
// Blob must be checked before the generic array path (a Blob is a
// `Vec<u8>`, distinct from a Rhai `Array`).
if value.is_blob() {
let blob = value.clone().into_blob().unwrap_or_default();
return Json::String(STANDARD.encode(&blob));
let encoded = STANDARD.encode(&blob);
msg_charge(remaining, encoded.len() + 2)?;
return Ok(Json::String(encoded));
}
if value.is_unit() {
return Json::Null;
msg_charge(remaining, 4)?;
return Ok(Json::Null);
}
if let Ok(b) = value.as_bool() {
return Json::Bool(b);
msg_charge(remaining, 5)?;
return Ok(Json::Bool(b));
}
if let Ok(i) = value.as_int() {
return Json::Number(i.into());
msg_charge(remaining, 8)?;
return Ok(Json::Number(i.into()));
}
if let Ok(f) = value.as_float() {
return serde_json::Number::from_f64(f).map_or(Json::Null, Json::Number);
msg_charge(remaining, 8)?;
return Ok(serde_json::Number::from_f64(f).map_or(Json::Null, Json::Number));
}
if value.is_string() {
return Json::String(value.clone().into_string().unwrap_or_default());
let s = value.clone().into_string().unwrap_or_default();
msg_charge(remaining, s.len() + 2)?;
return Ok(Json::String(s));
}
if let Some(arr) = value.clone().try_cast::<Array>() {
return Json::Array(arr.iter().map(message_to_json).collect());
msg_charge(remaining, 2)?;
let mut out = Vec::with_capacity(arr.len().min(1024));
for v in &arr {
msg_charge(remaining, 1)?;
out.push(message_to_json_bounded(v, remaining)?);
}
return Ok(Json::Array(out));
}
if let Some(map) = value.clone().try_cast::<Map>() {
msg_charge(remaining, 2)?;
let mut out = serde_json::Map::new();
for (k, v) in map {
out.insert(k.to_string(), message_to_json(&v));
msg_charge(remaining, k.len() + 4)?;
out.insert(k.to_string(), message_to_json_bounded(&v, remaining)?);
}
return Json::Object(out);
return Ok(Json::Object(out));
}
Json::String(value.to_string())
let s = value.to_string();
msg_charge(remaining, s.len() + 2)?;
Ok(Json::String(s))
}

View File

@@ -27,7 +27,15 @@ use rhai::{Array, Dynamic, Engine as RhaiEngine, EvalAltResult, Map, Module};
use serde_json::Value as Json;
use tokio::runtime::Handle as TokioHandle;
pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<SdkCallCx>) {
use super::bridge::MAX_JSON_MATERIALIZE_BYTES;
use super::interceptor::InterceptorCtx;
pub(super) fn register(
engine: &mut RhaiEngine,
services: &Services,
cx: Arc<SdkCallCx>,
ictx: InterceptorCtx,
) {
let svc = services.queue.clone();
let mut module = Module::new();
@@ -35,11 +43,11 @@ pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<Sdk
{
let svc = svc.clone();
let cx = cx.clone();
let ictx = ictx.clone();
module.set_native_fn(
"enqueue",
move |name: &str, message: Dynamic| -> Result<(), Box<EvalAltResult>> {
let json = message_to_json(&message)?;
enqueue_blocking(&svc, &cx, name, json, EnqueueOpts::default())
app_enqueue(&ictx, &svc, &cx, name, &message, EnqueueOpts::default())
},
);
}
@@ -49,12 +57,12 @@ pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<Sdk
{
let svc = svc.clone();
let cx = cx.clone();
let ictx = ictx.clone();
module.set_native_fn(
"enqueue",
move |name: &str, message: Dynamic, opts: Map| -> Result<(), Box<EvalAltResult>> {
let json = message_to_json(&message)?;
let opts = parse_opts(&opts)?;
enqueue_blocking(&svc, &cx, name, json, opts)
app_enqueue(&ictx, &svc, &cx, name, &message, opts)
},
);
}
@@ -96,6 +104,7 @@ pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<Sdk
{
let group_svc = services.group_queue.clone();
let cx = cx.clone();
let ictx = ictx.clone();
module.set_native_fn(
"shared_collection",
move |name: &str| -> Result<GroupQueueHandle, Box<EvalAltResult>> {
@@ -106,6 +115,7 @@ pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<Sdk
collection: name.to_string(),
service: group_svc.clone(),
cx: cx.clone(),
ictx: ictx.clone(),
})
},
);
@@ -122,6 +132,40 @@ pub struct GroupQueueHandle {
collection: String,
service: Arc<dyn GroupQueueService>,
cx: Arc<SdkCallCx>,
// §9.4: carried so shared `enqueue` runs the before/after hook too (M10).
ictx: InterceptorCtx,
}
/// Per-app enqueue with the §9.4 interceptor around it: charge the emission
/// budget, run the before-hook (allow/deny + M4 data-transform), enqueue the
/// (possibly rewritten) message, then run the after-hook with the new message
/// id as the result. `key` is `""` (a queue has no key). `collection` = the
/// queue name.
fn app_enqueue(
ictx: &InterceptorCtx,
svc: &Arc<dyn QueueService>,
cx: &Arc<SdkCallCx>,
name: &str,
message: &Dynamic,
opts: EnqueueOpts,
) -> Result<(), Box<EvalAltResult>> {
crate::sdk::emit_budget::charge_emission("queue::enqueue")?;
let json = message_to_json(message)?;
let write_val =
super::interceptor::run_before(ictx, cx, "queue", "enqueue", name, "", Some(&json))?
.unwrap_or(json);
let written = write_val.clone();
let id = enqueue_blocking(svc, cx, name, write_val, opts)?;
super::interceptor::run_after(
ictx,
cx,
"queue",
"enqueue",
name,
"",
Some(&written),
Json::String(id.0.to_string()),
)
}
fn group_enqueue(
@@ -129,7 +173,20 @@ fn group_enqueue(
message: Dynamic,
opts: EnqueueOpts,
) -> Result<(), Box<EvalAltResult>> {
crate::sdk::emit_budget::charge_emission("queue::shared_collection")?;
let json = message_to_json(&message)?;
// §9.4 before-op interceptor also guards shared-queue enqueues.
let write_val = super::interceptor::run_before(
&h.ictx,
&h.cx,
"queue",
"enqueue",
&h.collection,
"",
Some(&json),
)?
.unwrap_or(json);
let written = write_val.clone();
let handle = TokioHandle::try_current().map_err(|e| -> Box<EvalAltResult> {
EvalAltResult::ErrorRuntime(
format!("queue: no tokio runtime available: {e}").into(),
@@ -140,12 +197,21 @@ fn group_enqueue(
let service = h.service.clone();
let cx = h.cx.clone();
let collection = h.collection.clone();
handle
.block_on(async move { service.enqueue(&cx, &collection, json, opts).await })
.map(|_id| ())
let id = handle
.block_on(async move { service.enqueue(&cx, &collection, write_val, opts).await })
.map_err(|err| -> Box<EvalAltResult> {
EvalAltResult::ErrorRuntime(format!("queue: {err}").into(), rhai::Position::NONE).into()
})
})?;
super::interceptor::run_after(
&h.ictx,
&h.cx,
"queue",
"enqueue",
&h.collection,
"",
Some(&written),
Json::String(id.0.to_string()),
)
}
fn group_depth<F, Fut>(h: &mut GroupQueueHandle, f: F) -> Result<i64, Box<EvalAltResult>>
@@ -192,14 +258,15 @@ fn register_shared_queue_methods(engine: &mut RhaiEngine) {
}
/// Run the async enqueue inside the synchronous Rhai context. Mirrors
/// `pubsub::block_on` but for the queue's `Result<_, QueueError>`.
/// `pubsub::block_on` but for the queue's `Result<_, QueueError>`. Returns the
/// new message id (surfaced to the §9.4 after-hook, not to the script).
fn enqueue_blocking(
svc: &Arc<dyn QueueService>,
cx: &Arc<SdkCallCx>,
name: &str,
payload: Json,
opts: EnqueueOpts,
) -> Result<(), Box<EvalAltResult>> {
) -> Result<picloud_shared::QueueMessageId, Box<EvalAltResult>> {
let handle = TokioHandle::try_current().map_err(|e| -> Box<EvalAltResult> {
EvalAltResult::ErrorRuntime(
format!("queue: no tokio runtime available: {e}").into(),
@@ -212,7 +279,6 @@ fn enqueue_blocking(
let name = name.to_string();
handle
.block_on(async move { svc.enqueue(&cx, &name, payload, opts).await })
.map(|_id| ())
.map_err(|err| -> Box<EvalAltResult> {
EvalAltResult::ErrorRuntime(format!("queue: {err}").into(), rhai::Position::NONE).into()
})
@@ -262,8 +328,33 @@ fn parse_opts(opts: &Map) -> Result<EnqueueOpts, Box<EvalAltResult>> {
/// Convert a Rhai `Dynamic` into JSON. Mirrors `pubsub::message_to_json`
/// — blobs become base64, FnPtr / closures are rejected so a script
/// can't accidentally enqueue something that won't survive a trip
/// through Postgres + back through the bridge.
/// through Postgres + back through the bridge. Bounded by
/// [`MAX_JSON_MATERIALIZE_BYTES`] so a cheaply-aliased huge message can't OOM
/// the node on materialization (the per-message payload cap is enforced later,
/// on the already-materialized value).
fn message_to_json(value: &Dynamic) -> Result<Json, Box<EvalAltResult>> {
let mut remaining = MAX_JSON_MATERIALIZE_BYTES;
message_to_json_bounded(value, &mut remaining)
}
fn msg_charge(remaining: &mut usize, n: usize) -> Result<(), Box<EvalAltResult>> {
match remaining.checked_sub(n) {
Some(left) => {
*remaining = left;
Ok(())
}
None => Err(EvalAltResult::ErrorRuntime(
format!("queue::enqueue: message too large to serialize (exceeds the {MAX_JSON_MATERIALIZE_BYTES}-byte limit)").into(),
rhai::Position::NONE,
)
.into()),
}
}
fn message_to_json_bounded(
value: &Dynamic,
remaining: &mut usize,
) -> Result<Json, Box<EvalAltResult>> {
if value.is::<rhai::FnPtr>() {
return Err(EvalAltResult::ErrorRuntime(
"queue::enqueue: messages must not contain FnPtr / closures".into(),
@@ -273,40 +364,52 @@ fn message_to_json(value: &Dynamic) -> Result<Json, Box<EvalAltResult>> {
}
if value.is_blob() {
let blob = value.clone().into_blob().unwrap_or_default();
return Ok(Json::String(STANDARD.encode(&blob)));
let encoded = STANDARD.encode(&blob);
msg_charge(remaining, encoded.len() + 2)?;
return Ok(Json::String(encoded));
}
if value.is_unit() {
msg_charge(remaining, 4)?;
return Ok(Json::Null);
}
if let Ok(b) = value.as_bool() {
msg_charge(remaining, 5)?;
return Ok(Json::Bool(b));
}
if let Ok(i) = value.as_int() {
msg_charge(remaining, 8)?;
return Ok(Json::Number(i.into()));
}
if let Ok(f) = value.as_float() {
msg_charge(remaining, 8)?;
return Ok(serde_json::Number::from_f64(f).map_or(Json::Null, Json::Number));
}
if value.is_string() {
return Ok(Json::String(
value.clone().into_string().unwrap_or_default(),
));
let s = value.clone().into_string().unwrap_or_default();
msg_charge(remaining, s.len() + 2)?;
return Ok(Json::String(s));
}
if let Some(arr) = value.clone().try_cast::<Array>() {
let mut out = Vec::with_capacity(arr.len());
msg_charge(remaining, 2)?;
let mut out = Vec::with_capacity(arr.len().min(1024));
for v in &arr {
out.push(message_to_json(v)?);
msg_charge(remaining, 1)?;
out.push(message_to_json_bounded(v, remaining)?);
}
return Ok(Json::Array(out));
}
if let Some(map) = value.clone().try_cast::<Map>() {
msg_charge(remaining, 2)?;
let mut out = serde_json::Map::new();
for (k, v) in map {
out.insert(k.to_string(), message_to_json(&v)?);
msg_charge(remaining, k.len() + 4)?;
out.insert(k.to_string(), message_to_json_bounded(&v, remaining)?);
}
return Ok(Json::Object(out));
}
Ok(Json::String(value.to_string()))
let s = value.to_string();
msg_charge(remaining, s.len() + 2)?;
Ok(Json::String(s))
}
#[cfg(test)]

View File

@@ -21,7 +21,9 @@ use std::sync::Arc;
use picloud_shared::{SdkCallCx, SecretsListPage, Services};
use rhai::{Array, Dynamic, Engine as RhaiEngine, EvalAltResult, Map, Module};
use super::bridge::{block_on, dynamic_to_json, json_to_dynamic};
use super::bridge::{
block_on, dynamic_to_json_capped, json_to_dynamic, runtime_err, MAX_JSON_MATERIALIZE_BYTES,
};
pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<SdkCallCx>) {
let svc = services.secrets.clone();
@@ -34,7 +36,7 @@ pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<Sdk
module.set_native_fn(
"set",
move |name: &str, value: Dynamic| -> Result<(), Box<EvalAltResult>> {
let json = dynamic_to_json(&value);
let json = dynamic_to_json_capped(&value, MAX_JSON_MATERIALIZE_BYTES)?;
let svc = svc.clone();
let cx = cx.clone();
block_on("secrets", async move { svc.set(&cx, name, json).await })
@@ -124,10 +126,3 @@ fn list_page_to_map(page: SecretsListPage) -> Map {
);
m
}
// Returns the boxed error directly because every caller needs a
// `Box<EvalAltResult>` (Rhai's error type), matching the other bridges.
#[allow(clippy::unnecessary_box_returns)]
fn runtime_err(msg: &str) -> Box<EvalAltResult> {
EvalAltResult::ErrorRuntime(msg.into(), rhai::Position::NONE).into()
}

View File

@@ -4,7 +4,7 @@
use rhai::{Dynamic, Engine as RhaiEngine, EvalAltResult, Module};
use crate::sdk::bridge::{dynamic_to_json, json_to_dynamic};
use crate::sdk::bridge::{dynamic_to_json_capped, json_to_dynamic, MAX_JSON_MATERIALIZE_BYTES};
pub fn register(engine: &mut RhaiEngine) {
let mut module = Module::new();
@@ -26,7 +26,7 @@ fn register_stringify(module: &mut Module) {
module.set_native_fn(
"stringify",
|v: Dynamic| -> Result<String, Box<EvalAltResult>> {
serde_json::to_string(&dynamic_to_json(&v))
serde_json::to_string(&dynamic_to_json_capped(&v, MAX_JSON_MATERIALIZE_BYTES)?)
.map_err(|e| format!("json::stringify: {e}").into())
},
);
@@ -36,7 +36,7 @@ fn register_stringify_pretty(module: &mut Module) {
module.set_native_fn(
"stringify_pretty",
|v: Dynamic| -> Result<String, Box<EvalAltResult>> {
serde_json::to_string_pretty(&dynamic_to_json(&v))
serde_json::to_string_pretty(&dynamic_to_json_capped(&v, MAX_JSON_MATERIALIZE_BYTES)?)
.map_err(|e| format!("json::stringify_pretty: {e}").into())
},
);

View File

@@ -50,7 +50,7 @@
use std::sync::Arc;
use super::bridge::block_on;
use super::bridge::{block_on, runtime_err};
use picloud_shared::{
AppUser, AppUserId, CreateUserInput, EmailTemplateOpts, GeneratedAccept, GeneratedSession,
InviteOpts, SdkCallCx, Services, UpdateUserInput, UsersListOpts, UsersListPage, UsersService,
@@ -615,8 +615,3 @@ fn optional_string(opts: &Map, key: &str) -> Option<String> {
Some(d) => Some(d.to_string()),
}
}
#[allow(clippy::unnecessary_box_returns)]
fn runtime_err(msg: &str) -> Box<EvalAltResult> {
EvalAltResult::ErrorRuntime(msg.into(), rhai::Position::NONE).into()
}

View File

@@ -0,0 +1,156 @@
//! `workflow::` Rhai bridge — start a durable workflow run + read its status.
//!
//! ```rhai
//! let run_id = workflow::start("nightly_report", #{ date: "2026-07-12" });
//! let run_id = workflow::start("cleanup"); // no input
//! let st = workflow::run_status(run_id); // #{ status, output, error, steps } | ()
//! ```
//!
//! Returns the run id as a string. The owning workflow resolves from
//! `cx.app_id` inside the service — never a script arg (the isolation
//! boundary). The durable orchestrator advances the run asynchronously; the
//! script does not block on completion. `run_status` (F-038) lets the starter
//! poll progress without the platform-admin API — `()` when no such run
//! belongs to this app.
use std::sync::Arc;
use picloud_shared::{SdkCallCx, Services, WorkflowRunId, WorkflowRunView, WorkflowService};
use rhai::{Dynamic, Engine as RhaiEngine, EvalAltResult, Map, Module};
use tokio::runtime::Handle as TokioHandle;
use crate::sdk::bridge::{dynamic_to_json_capped, json_to_dynamic, MAX_JSON_MATERIALIZE_BYTES};
pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<SdkCallCx>) {
let svc = services.workflow.clone();
let mut module = Module::new();
// `workflow::start(name, input)`
{
let svc = svc.clone();
let cx = cx.clone();
module.set_native_fn(
"start",
move |name: &str, input: Dynamic| -> Result<String, Box<EvalAltResult>> {
start_blocking(&svc, &cx, name, &input)
},
);
}
// `workflow::start(name)` — no input (passes JSON null).
{
let svc = svc.clone();
let cx = cx.clone();
module.set_native_fn(
"start",
move |name: &str| -> Result<String, Box<EvalAltResult>> {
start_blocking(&svc, &cx, name, &Dynamic::UNIT)
},
);
}
// `workflow::run_status(run_id)` — read-only poll (F-038). `()` if absent.
{
let svc = svc.clone();
let cx = cx.clone();
module.set_native_fn(
"run_status",
move |run_id: &str| -> Result<Dynamic, Box<EvalAltResult>> {
run_status_blocking(&svc, &cx, run_id)
},
);
}
engine.register_static_module("workflow", module.into());
}
fn start_blocking(
svc: &Arc<dyn WorkflowService>,
cx: &Arc<SdkCallCx>,
name: &str,
input: &Dynamic,
) -> Result<String, Box<EvalAltResult>> {
let input_json = if input.is_unit() {
serde_json::Value::Null
} else {
dynamic_to_json_capped(input, MAX_JSON_MATERIALIZE_BYTES)?
};
let handle = TokioHandle::try_current().map_err(|e| -> Box<EvalAltResult> {
EvalAltResult::ErrorRuntime(
format!("workflow::start: no tokio runtime available: {e}").into(),
rhai::Position::NONE,
)
.into()
})?;
let svc = svc.clone();
let cx = cx.clone();
let name = name.to_string();
let run_id = handle
.block_on(async move { svc.start(&cx, &name, input_json).await })
.map_err(|e| -> Box<EvalAltResult> {
EvalAltResult::ErrorRuntime(
format!("workflow::start: {e}").into(),
rhai::Position::NONE,
)
.into()
})?;
Ok(run_id.to_string())
}
fn run_status_blocking(
svc: &Arc<dyn WorkflowService>,
cx: &Arc<SdkCallCx>,
run_id: &str,
) -> Result<Dynamic, Box<EvalAltResult>> {
let run_id: WorkflowRunId = run_id
.parse::<uuid::Uuid>()
.map(WorkflowRunId::from)
.map_err(|_| -> Box<EvalAltResult> {
EvalAltResult::ErrorRuntime(
format!("workflow::run_status: invalid run id {run_id:?}").into(),
rhai::Position::NONE,
)
.into()
})?;
let handle = TokioHandle::try_current().map_err(|e| -> Box<EvalAltResult> {
EvalAltResult::ErrorRuntime(
format!("workflow::run_status: no tokio runtime available: {e}").into(),
rhai::Position::NONE,
)
.into()
})?;
let svc = svc.clone();
let cx = cx.clone();
let view = handle
.block_on(async move { svc.run_status(&cx, run_id).await })
.map_err(|e| -> Box<EvalAltResult> {
EvalAltResult::ErrorRuntime(
format!("workflow::run_status: {e}").into(),
rhai::Position::NONE,
)
.into()
})?;
// Absent run → `()` so a script can test `if status == ()`.
Ok(view.map_or(Dynamic::UNIT, |v| view_to_dynamic(&v)))
}
/// Convert a run view into the Rhai shape
/// `#{ status, output, error, steps: #{ name: status } }`.
fn view_to_dynamic(v: &WorkflowRunView) -> Dynamic {
let mut m = Map::new();
m.insert("status".into(), v.status.as_str().into());
m.insert(
"output".into(),
v.output.clone().map_or(Dynamic::UNIT, json_to_dynamic),
);
m.insert(
"error".into(),
v.error.clone().map_or(Dynamic::UNIT, Into::into),
);
let mut steps = Map::new();
for s in &v.steps {
steps.insert(s.name.clone().into(), s.status.as_str().into());
}
m.insert("steps".into(), steps.into());
Dynamic::from_map(m)
}

View File

@@ -123,6 +123,13 @@ pub struct ExecResponse {
pub status_code: u16,
pub headers: BTreeMap<String, String>,
pub body: serde_json::Value,
/// F-026 binary response. When a script returns an envelope with a
/// `body_base64` field, the HTTP layer decodes it and returns the raw
/// bytes with the script-set `Content-Type` (instead of JSON-encoding
/// `body`). `None` ⇒ the normal JSON `body` path. `#[serde(default)]`
/// keeps the cluster-mode wire format backward-compatible.
#[serde(default)]
pub body_base64: Option<String>,
pub logs: Vec<LogEntry>,
pub stats: ExecStats,
}

View File

@@ -93,6 +93,26 @@ fn returns_structured_response_when_status_code_present() {
assert_eq!(resp.body, json!({ "ok": true, "msg": "created" }));
}
#[test]
fn body_base64_envelope_sets_binary_response() {
// F-026: a `body_base64` field carries raw bytes; `body` is ignored.
let src = r#"
#{ statusCode: 200,
headers: #{ "content-type": "image/png" },
body: #{ ignored: true },
body_base64: "aGVsbG8=" }
"#;
let resp = engine().execute(src, req(json!(null))).unwrap();
assert_eq!(resp.status_code, 200);
assert_eq!(resp.body_base64.as_deref(), Some("aGVsbG8="));
// `body` is nulled out when body_base64 is present.
assert_eq!(resp.body, json!(null));
assert_eq!(
resp.headers.get("content-type").map(String::as_str),
Some("image/png")
);
}
#[test]
fn ctx_exposes_request_data() {
let src = r"

View File

@@ -501,8 +501,27 @@ async fn docs_bridge_preserves_cross_app_isolation() {
v == ()
"#
);
let body = run_script(engine, &reader_src, baseline_request(app_b)).await;
assert_eq!(body, json!(true));
let body = run_script(engine.clone(), &reader_src, baseline_request(app_b)).await;
assert_eq!(body, json!(true), "app B must not see app A's document");
// Positive control — without it this test has no teeth (an execution_id-keyed
// bridge would pass the negative above). `baseline_request` mints a fresh
// execution_id/script_id per call, so this re-read under app_a is a different
// execution than the writer; it must still find A's own doc.
let reader_a = format!(
r#"
let c = docs::collection("shared");
let v = c.get("{id_a_str}");
if v == () {{ "MISSING" }} else {{ v.data.from }}
"#
);
let body = run_script(engine, &reader_a, baseline_request(app_a)).await;
assert_eq!(
body,
json!("a"),
"app A must read back its own document in a later execution — storage is \
keyed by app_id, not execution_id/script_id"
);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]

View File

@@ -194,16 +194,23 @@ async fn invoke_callee_sees_incremented_depth() {
#{ statusCode: 200, body: d }
"#
.to_string();
// The caller is direct ingress (depth 0); the callee, re-entered via
// `invoke`, must see depth 1. Previously this test asserted NOTHING (`let _ =
// resp`) because `ctx.trigger_depth` wasn't exposed — so it would have passed
// even if `invoke` forwarded the depth unchanged (never incrementing the
// chain-depth counter that bounds runaway trigger loops). `ctx.trigger_depth`
// is now surfaced in build_ctx_map, so we can pin the increment.
let req = baseline_request(app);
assert_eq!(req.trigger_depth, 0, "the caller is direct ingress");
let resp = tokio::task::spawn_blocking(move || engine.execute(&src, req))
.await
.unwrap()
.unwrap();
// ctx exposes trigger_depth? — not yet (it's not in build_ctx_map).
// Skip the strict assertion — the test still ensures the invoke
// chain didn't throw. (See HANDBACK §11 for cx.trigger_depth surface
// exposure as a v1.2 follow-up.)
let _ = resp;
assert_eq!(
resp.body,
json!(1),
"the invoke callee must see trigger_depth = caller + 1"
);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]

View File

@@ -52,6 +52,27 @@ impl KvService for InMemoryKv {
Ok(())
}
async fn set_if(
&self,
cx: &SdkCallCx,
collection: &str,
key: &str,
expected: Option<Value>,
new: Value,
) -> Result<bool, KvError> {
let mut data = self.data.lock().await;
let k = (cx.app_id, collection.to_string(), key.to_string());
let matches = match (&expected, data.get(&k)) {
(None, None) => true,
(Some(exp), Some(cur)) => exp == cur,
_ => false,
};
if matches {
data.insert(k, new);
}
Ok(matches)
}
async fn delete(&self, cx: &SdkCallCx, collection: &str, key: &str) -> Result<bool, KvError> {
Ok(self
.data
@@ -167,6 +188,27 @@ async fn kv_set_then_get_round_trip() {
assert_eq!(body, json!({ "n": 1 }));
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn kv_set_if_compare_and_swap() {
let engine = make_engine();
let app = AppId::new();
// `set_if(key, (), new)` inserts only when absent; `set_if(key, expected,
// new)` swaps only on match. Returns a bool each time.
let src = r#"
let c = kv::collection("counters");
let first = c.set_if("n", (), 1); // absent -> inserts, true
let dup = c.set_if("n", (), 2); // present -> false
let bad = c.set_if("n", 99, 3); // mismatch -> false
let good = c.set_if("n", 1, 3); // match -> swaps, true
#{ first: first, dup: dup, bad: bad, good: good, final: c.get("n") }
"#;
let body = run_script(engine, src, baseline_request(app)).await;
assert_eq!(
body,
json!({ "first": true, "dup": false, "bad": false, "good": true, "final": 3 })
);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn kv_get_missing_returns_unit() {
let engine = make_engine();
@@ -269,6 +311,99 @@ async fn kv_bridge_preserves_cross_app_isolation() {
let c = kv::collection("shared");
c.get("k")
"#;
let body = run_script(engine, reader, baseline_request(app_b)).await;
assert_eq!(body, Value::Null);
let body = run_script(engine.clone(), reader, baseline_request(app_b)).await;
assert_eq!(body, Value::Null, "app B must not see app A's value");
// Positive control — WITHOUT it this test has no teeth. `baseline_request`
// mints a fresh execution_id AND script_id each call, so this second run under
// app_a is a DIFFERENT execution/script than the writer. A bridge that keyed
// storage by execution_id or script_id (instead of app_id) would pass the
// negative above AND every single-execution round-trip, while app-scoped
// persistence was completely gone. This catches that: app A must still read
// back its own value across executions.
let body = run_script(engine, reader, baseline_request(app_a)).await;
assert_eq!(
body,
Value::from("from-a"),
"app A must read back its own value in a later execution — storage is keyed \
by app_id, not execution_id/script_id"
);
}
// --- §9.4 M6: per-execution interceptor resolve cache ---------------------
/// An `InterceptorService` that resolves to an EMPTY chain (nothing hooked) but
/// counts every `resolve` call, so a test can prove the per-execution cache
/// collapses N same-`(service, op)` resolves into one.
#[derive(Default)]
struct CountingInterceptors {
calls: std::sync::atomic::AtomicUsize,
}
#[async_trait]
impl picloud_shared::InterceptorService for CountingInterceptors {
async fn resolve(
&self,
_cx: &SdkCallCx,
_service: &str,
_op: &str,
) -> Result<picloud_shared::InterceptorChain, String> {
self.calls.fetch_add(1, std::sync::atomic::Ordering::SeqCst);
Ok(picloud_shared::InterceptorChain::default())
}
}
fn make_engine_with_interceptors(ic: Arc<CountingInterceptors>) -> Arc<Engine> {
let services = Services::new(
Arc::new(InMemoryKv::default()),
Arc::new(NoopDocsService),
Arc::new(NoopDeadLetterService),
Arc::new(NoopEventEmitter),
Arc::new(NoopModuleSource),
Arc::new(NoopHttpService),
Arc::new(picloud_shared::NoopFilesService),
Arc::new(picloud_shared::NoopPubsubService),
Arc::new(picloud_shared::NoopSecretsService),
Arc::new(picloud_shared::NoopEmailService),
Arc::new(picloud_shared::NoopUsersService),
Arc::new(picloud_shared::NoopQueueService),
Arc::new(picloud_shared::NoopInvokeService),
Arc::new(picloud_shared::NoopVarsService),
)
.with_interceptors(ic);
Arc::new(Engine::new(Limits::default(), services))
}
/// M6: within ONE execution, N `kv::set`s of the same `(service, op)` resolve
/// the interceptor chain at most once (the empty chain is cached), and a SECOND
/// execution starts fresh (the cache is cleared at the outermost boundary).
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn resolve_is_cached_per_execution() {
let ic = Arc::new(CountingInterceptors::default());
let engine = make_engine_with_interceptors(ic.clone());
let app = AppId::new();
// 25 sets in one execution — all the same (kv, set).
let src = r#"
let c = kv::collection("c");
let n = 0;
while n < 25 { c.set("k" + n, n); n += 1; }
"done"
"#;
let body = run_script(engine.clone(), src, baseline_request(app)).await;
assert_eq!(body, Value::from("done"));
assert_eq!(
ic.calls.load(std::sync::atomic::Ordering::SeqCst),
1,
"25 kv::sets in one execution must resolve the (kv, set) chain exactly once"
);
// A second execution resolves again (the cache cleared at the outermost exit).
let body = run_script(engine, src, baseline_request(app)).await;
assert_eq!(body, Value::from("done"));
assert_eq!(
ic.calls.load(std::sync::atomic::Ordering::SeqCst),
2,
"a fresh execution must not reuse the previous execution's resolve cache"
);
}

View File

@@ -19,6 +19,7 @@ use serde_json::{json, Value};
#[derive(Default)]
struct RecordingPubsub {
last: Mutex<Option<(String, Value)>>,
count: std::sync::atomic::AtomicUsize,
}
#[async_trait]
@@ -32,6 +33,7 @@ impl PubsubService for RecordingPubsub {
if topic.trim().is_empty() {
return Err(PubsubError::EmptyTopic);
}
self.count.fetch_add(1, std::sync::atomic::Ordering::SeqCst);
*self.last.lock().unwrap() = Some((topic.to_string(), message));
Ok(())
}
@@ -163,3 +165,40 @@ async fn publish_empty_topic_throws() {
.expect("spawn_blocking should not panic");
assert!(res.is_err(), "empty topic should throw");
}
/// `PICLOUD_MAX_EMISSIONS_PER_EXECUTION` bounds a one-request outbox-amplification
/// DoS. Only the counter itself was unit-tested (`emit_budget`); this drives it
/// through the REAL `pubsub::publish_durable` SDK surface — i.e. it pins that the
/// charge_emission call is actually wired into publish, not just that the counter
/// works. Uses the DEFAULT budget (1000) so it needs no env mutation: a loop of
/// 1001 publishes must error, and no more than the budget may have reached the
/// service.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn publish_beyond_the_emission_budget_is_refused() {
let svc = Arc::new(RecordingPubsub::default());
let engine = make_engine(svc.clone());
// 1001 durable publishes in one execution — one past the default ceiling.
let src = r#"
let n = 0;
while n < 1001 { pubsub::publish_durable("t", #{ i: n }); n += 1; }
"#
.to_string();
let req = baseline_request(AppId::new());
let res = tokio::task::spawn_blocking(move || engine.execute(&src, req))
.await
.expect("spawn_blocking should not panic");
let err = res.expect_err("exceeding the per-execution emission budget must error");
assert!(
format!("{err:?}").to_lowercase().contains("emission"),
"the error must name the emission budget, got: {err:?}"
);
// The rail actually stopped the amplification: the service saw at most the
// budget, not all 1001.
let seen = svc.count.load(std::sync::atomic::Ordering::SeqCst);
assert!(
seen <= 1000,
"the service must not have received more than the 1000-emission budget, saw {seen}"
);
assert!(seen > 0, "some publishes should have landed before the cap");
}

View File

@@ -140,12 +140,40 @@ async fn retry_policy_clamps_visible_at_script_layer() {
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn retry_policy_rejects_bogus_backoff() {
let engine = build_engine();
// Pin the REASON. A bare `is_err()` would pass if `retry::policy` were not
// registered at all (ErrorFunctionNotFound), or on a typo in the script
// source, or if EVERY policy were rejected — none of which is "an invalid
// backoff is rejected". The message must name `backoff`.
let src = r#"retry::policy(#{ backoff: "bogus" });"#.to_string();
let req = baseline_request(AppId::new());
let res = tokio::task::spawn_blocking(move || engine.execute(&src, req))
let err = tokio::task::spawn_blocking({
let engine = engine.clone();
move || engine.execute(&src, req)
})
.await
.unwrap()
.unwrap_err()
.to_string();
assert!(
err.to_lowercase().contains("backoff"),
"the rejection must be about the backoff value specifically, got: {err}"
);
// Control: the SAME call with a VALID backoff succeeds — proving `retry::policy`
// is registered and does not reject everything, so the failure above is
// specific to the bad value.
let ok_src = r#"
let p = retry::policy(#{ backoff: "constant", max_attempts: 2, base_ms: 1 });
retry::run(p, || 7)
"#
.to_string();
let req = baseline_request(AppId::new());
let resp = tokio::task::spawn_blocking(move || engine.execute(&ok_src, req))
.await
.unwrap();
assert!(res.is_err());
.unwrap()
.expect("a valid backoff must be accepted");
assert_eq!(resp.body, serde_json::json!(7));
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]

View File

@@ -41,4 +41,5 @@ data-encoding.workspace = true
lettre.workspace = true
[dev-dependencies]
picloud-test-support.workspace = true
tokio.workspace = true

View File

@@ -0,0 +1,25 @@
-- §3 M3 (hermetic gate): persist the per-env approval policy server-side.
--
-- Before this table the policy lived ONLY in the apply request
-- (`ProjectDecl.environments`), so a hand-crafted request that omitted it was
-- ungated — the server had nothing authoritative to check against. This makes
-- the gate hermetic: the confirm-required set is stored per project and the
-- server enforces against `persisted declared` (an env is gated if EITHER the
-- stored row OR the request says so), so an omitted or flipped-to-false policy
-- can no longer bypass a gate a prior apply established.
--
-- One owner (the project), a single boolean policy per environment name — no
-- polymorphic owner and no environment_scope (unlike `vars`/`secrets`, which
-- are data the apply targets; this is metadata ABOUT the apply). CASCADE on the
-- project mirrors 0066's philosophy: un-claiming/deleting a project drops its
-- policy, it does not linger to gate a tree the project no longer owns.
CREATE TABLE project_environments (
project_id UUID NOT NULL REFERENCES projects(id) ON DELETE CASCADE,
-- The environment name as declared in `[project.environments]` (e.g.
-- "production"); matched against the apply's `--env`.
env_name TEXT NOT NULL,
-- TRUE => applying to this env is gated (needs `--approve <env>` + admin).
confirm BOOLEAN NOT NULL,
PRIMARY KEY (project_id, env_name)
);

View File

@@ -0,0 +1,43 @@
-- §11.6 D3 (dead-letter store for group SHARED durable queues).
--
-- Previously an exhausted shared-queue message was `drop_exhausted()`-ed with a
-- warning — silent data loss. Per-app queues persist to `dead_letters` (0010);
-- this is the symmetric group store, keyed by `group_id` (the shared-queue owner)
-- + `collection` (a group has many shared queues) instead of `app_id`.
--
-- CASCADE on the group mirrors `group_queue_messages` (0065): deleting the group
-- drops its dead-letters; an app delete leaves them (the data belongs to the
-- group, not the consuming app). No `app_id` — a shared-queue message is not
-- owned by whichever descendant happened to consume it.
--
-- Fan-out to a *shared* dead-letter TRIGGER is deferred (would need a new shared
-- dead-letter trigger kind); M2 stops the data loss + makes exhausted messages
-- operator-visible (GET /api/v1/admin/groups/{id}/dead-letters).
CREATE TABLE group_dead_letters (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
group_id UUID NOT NULL REFERENCES groups(id) ON DELETE CASCADE,
-- The shared-queue collection name the exhausted message came from.
collection TEXT NOT NULL,
-- The group_queue_messages.id row that exhausted retries (already deleted).
original_event_id UUID NOT NULL,
source TEXT NOT NULL,
op TEXT NOT NULL,
-- The materialized consumer copy's trigger/script (nullable, mirrors 0010).
trigger_id UUID,
script_id UUID,
payload JSONB NOT NULL,
attempt_count INT NOT NULL,
first_attempt_at TIMESTAMPTZ NOT NULL,
last_attempt_at TIMESTAMPTZ NOT NULL,
last_error TEXT NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
resolved_at TIMESTAMPTZ,
resolution TEXT
CHECK (resolution IN
('replayed', 'ignored', 'handled_by_script', 'handler_failed'))
);
-- Operator listing: newest-first per group (GET .../groups/{id}/dead-letters).
CREATE INDEX idx_group_dead_letters_group
ON group_dead_letters (group_id, created_at DESC);

View File

@@ -0,0 +1,16 @@
-- §M5.5 / audit 2026-06-11 H-D1: AAD-bind the email-trigger inbound secret.
--
-- `email_trigger_details.inbound_secret_encrypted` was sealed v0 (no AAD, bound
-- only to the master key) because this table had no version column — so a
-- ciphertext could in principle be relocated between rows. The per-app `secrets`
-- table gained AAD versioning in 0042; this brings the same to email secrets.
--
-- The AAD binds to the SEALING OWNER (the group for a template, the app for a
-- standalone trigger) — deliberately NOT the per-row trigger_id, so a group
-- template's sealed bytes stay valid when the materializer copies them verbatim
-- to each descendant app (all share the group AAD). At open time the sealing
-- owner is recovered from `materialized_from -> template.group_id` (else the
-- app). v0 rows keep their exact pre-audit no-AAD read path (default 0).
ALTER TABLE email_trigger_details
ADD COLUMN inbound_secret_version SMALLINT NOT NULL DEFAULT 0;

View File

@@ -0,0 +1,19 @@
-- Audit 2026-07-11 (LOW C1): admin sessions had only a SLIDING expiry, so a
-- continuously-used (or stolen-but-warm) admin token never self-expired. The
-- data-plane app-user sessions (0027) already carry an absolute hard cap; give
-- the higher-privilege admin session the same. `touch` clamps the sliding bump
-- at this value, and `lookup` filters it, so an admin session dies at the later
-- of its idle window or this absolute ceiling — whichever comes first.
ALTER TABLE admin_sessions ADD COLUMN absolute_expires_at TIMESTAMPTZ;
-- Backfill live rows: cap them 30 days from creation (the new default absolute
-- TTL). A row already older than that will be swept on its next lookup.
UPDATE admin_sessions
SET absolute_expires_at = created_at + INTERVAL '30 days'
WHERE absolute_expires_at IS NULL;
ALTER TABLE admin_sessions ALTER COLUMN absolute_expires_at SET NOT NULL;
-- Backs the absolute-expiry filter on lookup / prune.
CREATE INDEX admin_sessions_absolute_expiry_idx
ON admin_sessions (absolute_expires_at);

View File

@@ -0,0 +1,120 @@
-- v1.2 Workflows (blueprint §9.1/§9.2): a durable DAG orchestration engine.
--
-- A `workflow` is a declarative graph of steps; each step invokes a function
-- (a script, by name, resolved in the app's scope) or a nested sub-workflow,
-- with `depends_on` edges, a per-step `when` condition, input mapping, and a
-- per-step retry + `on_error` policy. A `workflow::start(name, input)` (or the
-- admin run API) creates a RUN; a dedicated background orchestrator advances it
-- step-by-step, durably, surviving restarts.
--
-- Three tables:
-- * workflows — the definition (owner-polymorphic like scripts/0050:
-- app XOR group; M1 authors app-owned only, the group_id
-- column ships unused so group-owned templates stay a
-- door-open follow-up).
-- * workflow_runs — one row per start(); the durable run record.
-- * workflow_run_steps — one row per step per run; carries the competing-
-- consumer lease (claim_token/claimed_at) exactly like
-- queue_messages (0034), so parallel steps complete
-- lock-free and a crashed worker's step is reclaimable.
-- ---------------------------------------------------------------------------
-- workflows: the definition (owner-polymorphic, mirrors scripts/0050).
-- ---------------------------------------------------------------------------
CREATE TABLE workflows (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
-- exactly one of app_id / group_id (the exactly-one CHECK below). Code is
-- not data, so RESTRICT (a group can't be deleted out from under a workflow
-- it owns), mirroring scripts.
app_id UUID NULL REFERENCES apps(id) ON DELETE CASCADE,
group_id UUID NULL REFERENCES groups(id) ON DELETE RESTRICT,
name TEXT NOT NULL,
-- the parsed DAG: { "steps": [ { name, function|workflow, input, depends_on,
-- when, retry, on_error } ] }. Validated server-side (acyclic, unique step
-- names, deps exist) before it is ever written.
definition JSONB NOT NULL,
enabled BOOLEAN NOT NULL DEFAULT TRUE,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
CONSTRAINT workflows_owner_exactly_one CHECK ((group_id IS NULL) <> (app_id IS NULL))
);
-- Per-owner, case-insensitive name uniqueness (partial, one per owner column).
CREATE UNIQUE INDEX workflows_app_name_uidx
ON workflows (app_id, LOWER(name)) WHERE app_id IS NOT NULL;
CREATE UNIQUE INDEX workflows_group_name_uidx
ON workflows (group_id, LOWER(name)) WHERE group_id IS NOT NULL;
CREATE INDEX workflows_group_id_idx ON workflows (group_id) WHERE group_id IS NOT NULL;
-- ---------------------------------------------------------------------------
-- workflow_runs: one row per start(). Carries app_id NOT NULL (the data-plane
-- invariant — every run-listing query filters by app without a join) even
-- though it is derivable from the workflow.
-- ---------------------------------------------------------------------------
CREATE TABLE workflow_runs (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
workflow_id UUID NOT NULL REFERENCES workflows(id) ON DELETE RESTRICT,
app_id UUID NOT NULL REFERENCES apps(id) ON DELETE CASCADE,
status TEXT NOT NULL DEFAULT 'pending'
CHECK (status IN ('pending', 'running', 'succeeded', 'failed', 'canceled')),
input JSONB NOT NULL DEFAULT '{}'::jsonb,
output JSONB NULL,
error TEXT NULL,
-- correlates every step execution in execution_logs under one id.
root_execution_id UUID NOT NULL,
-- nesting: a sub-workflow run links back to the parent step that spawned it.
workflow_depth INT NOT NULL DEFAULT 0,
parent_run_id UUID NULL REFERENCES workflow_runs(id) ON DELETE SET NULL,
parent_step_id UUID NULL,
started_at TIMESTAMPTZ NULL,
finished_at TIMESTAMPTZ NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE INDEX workflow_runs_app_status_idx ON workflow_runs (app_id, status);
CREATE INDEX workflow_runs_workflow_idx ON workflow_runs (workflow_id, created_at DESC);
CREATE INDEX workflow_runs_parent_idx ON workflow_runs (parent_run_id) WHERE parent_run_id IS NOT NULL;
-- ---------------------------------------------------------------------------
-- workflow_run_steps: one row per step per run. The claim_token/claimed_at
-- lease is the queue_messages (0034) competing-consumer pattern — the
-- orchestrator claims a `ready` step FOR UPDATE SKIP LOCKED, and the reclaim
-- task frees a stale lease so a crashed worker's step is retried.
-- ---------------------------------------------------------------------------
CREATE TABLE workflow_run_steps (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
run_id UUID NOT NULL REFERENCES workflow_runs(id) ON DELETE CASCADE,
step_name TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'pending'
CHECK (status IN ('pending', 'ready', 'running', 'succeeded', 'failed', 'skipped')),
attempt INT NOT NULL DEFAULT 0,
max_attempts INT NOT NULL DEFAULT 1,
output JSONB NULL,
error TEXT NULL,
-- competing-consumer lease (mirrors queue_messages).
claim_token UUID NULL,
claimed_at TIMESTAMPTZ NULL,
-- retry backoff / initial dispatch gate (NULL = due immediately once ready).
next_attempt_at TIMESTAMPTZ NULL,
-- set when this step is a nested sub-workflow: the child run it is waiting on.
child_run_id UUID NULL REFERENCES workflow_runs(id) ON DELETE SET NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
UNIQUE (run_id, step_name)
);
-- The orchestrator's claim scan: ready steps whose backoff gate has elapsed.
-- (deliver-gate NOW() comparison is applied as a filter on the small partial
-- set, as in queue_messages.)
CREATE INDEX workflow_run_steps_claimable_idx
ON workflow_run_steps (next_attempt_at)
WHERE status = 'ready';
-- Advance re-eval + reclaim scans by run.
CREATE INDEX workflow_run_steps_run_idx ON workflow_run_steps (run_id);
-- Reclaim-task scan: leased steps whose claim is older than the visibility
-- timeout (bounded by in-flight step count).
CREATE INDEX workflow_run_steps_claimed_idx
ON workflow_run_steps (claimed_at)
WHERE claim_token IS NOT NULL;

View File

@@ -0,0 +1,18 @@
-- v1.2 Workflows M2: the orchestrator executes each workflow step as a normal
-- function invocation and logs it, so `pic logs` can surface (and filter by)
-- workflow-driven runs — exactly as it does for cron/queue/invoke triggers.
--
-- Add 'workflow' to the execution_logs.source CHECK (extends 0043). The column
-- default stays 'http'; this only widens the allowed set. Keep this list in
-- sync with `shared::ExecutionSource` (which gains the matching `Workflow`
-- variant) and `manager-core::OutboxSourceKind`.
--
-- Postgres has no ALTER … ALTER CONSTRAINT for a CHECK, so drop + re-add.
ALTER TABLE execution_logs DROP CONSTRAINT execution_logs_source_check;
ALTER TABLE execution_logs
ADD CONSTRAINT execution_logs_source_check
CHECK (source IN (
'http', 'kv', 'docs', 'dead_letter', 'cron',
'files', 'pubsub', 'email', 'invoke', 'queue', 'workflow'
));

View File

@@ -0,0 +1,42 @@
-- §9.4 Service Interceptors (v1.2) — before-op allow/deny hooks.
--
-- A marker `(owner, service, op) -> interceptor_script` declares that a script
-- runs BEFORE a data-plane operation and may deny it. MVP scope: `service='kv'`,
-- `op IN ('set','delete')`, allow/deny only (no data transform, no chaining,
-- no after-hooks). The interceptor is itself a script owned by the same node
-- (or an ancestor group); it is resolved + run through the existing `invoke()`
-- re-entry path, so this table holds only the MARKER — pure declaration,
-- structurally like an `extension_points` row (0051): config, not code, so
-- ON DELETE CASCADE.
--
-- Ownership is polymorphic (mirrors extension_points/vars/secrets/scripts):
-- exactly one of (app_id, group_id) is set. Resolution walks the calling app's
-- chain (app, then nearest ancestor group) and picks the nearest declaration
-- for a (service, op) — nearest-owner-wins, so an app overrides a group's
-- interceptor, the deliberate inverse of a sealed import.
CREATE TABLE interceptors (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
group_id UUID REFERENCES groups(id) ON DELETE CASCADE,
app_id UUID REFERENCES apps(id) ON DELETE CASCADE,
CONSTRAINT interceptors_owner_exactly_one
CHECK ((group_id IS NULL) <> (app_id IS NULL)),
-- The intercepted operation. MVP: service='kv', op IN ('set','delete').
service TEXT NOT NULL,
op TEXT NOT NULL,
-- Name of the interceptor script (resolved on the owner's chain at run time).
interceptor_script TEXT NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
-- One marker per (owner, service, op). Partial because the owner is split
-- across two nullable columns.
CREATE UNIQUE INDEX interceptors_group_uidx
ON interceptors (group_id, service, op) WHERE group_id IS NOT NULL;
CREATE UNIQUE INDEX interceptors_app_uidx
ON interceptors (app_id, service, op) WHERE app_id IS NOT NULL;
-- Lookup indexes for the resolver's chain join + list-by-owner.
CREATE INDEX interceptors_group_id_idx ON interceptors (group_id) WHERE group_id IS NOT NULL;
CREATE INDEX interceptors_app_id_idx ON interceptors (app_id) WHERE app_id IS NOT NULL;

View File

@@ -0,0 +1,20 @@
-- §9.4 Service Interceptors M2 — before + after phases.
--
-- The 0073 marker guarded only the BEFORE-op allow/deny hook. M2 introduces an
-- explicit `phase` so an owner can declare a `before` AND an `after` interceptor
-- for one `(service, op)`. Existing markers default to `phase='before'` — the
-- 0073 behaviour is unchanged. The partial-unique indexes gain `phase` so the
-- two phases are distinct rows per owner (one before + one after each).
ALTER TABLE interceptors
ADD COLUMN phase TEXT NOT NULL DEFAULT 'before'
CHECK (phase IN ('before', 'after'));
-- Recreate the per-owner uniqueness to be per (owner, service, op, PHASE).
DROP INDEX interceptors_group_uidx;
DROP INDEX interceptors_app_uidx;
CREATE UNIQUE INDEX interceptors_group_uidx
ON interceptors (group_id, service, op, phase) WHERE group_id IS NOT NULL;
CREATE UNIQUE INDEX interceptors_app_uidx
ON interceptors (app_id, service, op, phase) WHERE app_id IS NOT NULL;

View File

@@ -0,0 +1,10 @@
-- §9.4 Service Interceptors M5 — per-interceptor wall-clock timeout.
--
-- An optional per-marker `timeout_ms` bounds how long ONE interceptor hook may
-- run before it is denied (a runaway guard, e.g. `loop {}`, must not hang the
-- guarded write path). NULL means "use the instance default"
-- (`PICLOUD_INTERCEPTOR_TIMEOUT_MS`). The effective deadline is always tightened
-- to at most the caller's remaining budget — a hook can never EXTEND it.
ALTER TABLE interceptors
ADD COLUMN timeout_ms INTEGER
CHECK (timeout_ms IS NULL OR timeout_ms > 0);

View File

@@ -0,0 +1,13 @@
-- A3 raw SMTP ingress — an inbound email ADDRESS an email trigger listens on.
--
-- The HMAC-webhook path (v1.1.7) addresses a trigger by its UUID in the URL. A
-- native SMTP listener instead receives `RCPT TO:<address>` and must resolve
-- that address to the bound trigger. `inbound_address` is the mailbox the
-- trigger claims; NULL for a webhook-only trigger. Case-insensitively unique
-- among app-owned triggers so an address resolves to exactly one script.
ALTER TABLE email_trigger_details
ADD COLUMN inbound_address TEXT;
CREATE UNIQUE INDEX email_trigger_details_inbound_address_uidx
ON email_trigger_details (lower(inbound_address))
WHERE inbound_address IS NOT NULL;

View File

@@ -0,0 +1,11 @@
-- F-030 per-app CORS. A decoupled browser SPA on a different origin cannot
-- call an app's user routes today: the orchestrator emits no
-- `Access-Control-Allow-Origin` header and never answers an OPTIONS preflight.
--
-- This adds a per-app allow-list of origins. The orchestrator (which has no DB)
-- reads it into its in-memory cache on each domain-cache rebuild, echoes the
-- request `Origin` when it matches, and short-circuits preflights. Empty array
-- (the default) = CORS off, preserving today's behaviour. `["*"]` allows any
-- origin.
ALTER TABLE apps
ADD COLUMN cors_allowed_origins JSONB NOT NULL DEFAULT '[]'::jsonb;

View File

@@ -17,13 +17,15 @@ pub enum AdminSessionRepositoryError {
Db(#[from] sqlx::Error),
}
/// Result of a session lookup. Includes the user id (for auth context)
/// and the existing `expires_at` so the middleware can decide whether
/// the sliding window bump is worth a write.
/// Result of a session lookup. Includes the user id (for auth context),
/// the existing `expires_at` (so the middleware can decide whether the
/// sliding-window bump is worth a write), and the `absolute_expires_at`
/// hard cap the bump must clamp at (C1).
#[derive(Debug, Clone)]
pub struct AdminSessionLookup {
pub user_id: AdminUserId,
pub expires_at: DateTime<Utc>,
pub absolute_expires_at: DateTime<Utc>,
}
#[async_trait]
@@ -33,9 +35,11 @@ pub trait AdminSessionRepository: Send + Sync {
user_id: AdminUserId,
token_hash: &str,
expires_at: DateTime<Utc>,
absolute_expires_at: DateTime<Utc>,
) -> Result<(), AdminSessionRepositoryError>;
/// Look up a session by token hash. Returns `None` for missing or
/// already-expired rows (the query filters them).
/// already-expired rows — either the sliding `expires_at` OR the
/// absolute cap having passed (the query filters both).
async fn lookup(
&self,
token_hash: &str,
@@ -78,14 +82,16 @@ impl AdminSessionRepository for PostgresAdminSessionRepository {
user_id: AdminUserId,
token_hash: &str,
expires_at: DateTime<Utc>,
absolute_expires_at: DateTime<Utc>,
) -> Result<(), AdminSessionRepositoryError> {
sqlx::query(
"INSERT INTO admin_sessions (token_hash, user_id, expires_at) \
VALUES ($1, $2, $3)",
"INSERT INTO admin_sessions (token_hash, user_id, expires_at, absolute_expires_at) \
VALUES ($1, $2, $3, $4)",
)
.bind(token_hash)
.bind(user_id.into_inner())
.bind(expires_at)
.bind(absolute_expires_at)
.execute(&self.pool)
.await?;
Ok(())
@@ -95,16 +101,17 @@ impl AdminSessionRepository for PostgresAdminSessionRepository {
&self,
token_hash: &str,
) -> Result<Option<AdminSessionLookup>, AdminSessionRepositoryError> {
let row: Option<(uuid::Uuid, DateTime<Utc>)> = sqlx::query_as(
"SELECT user_id, expires_at FROM admin_sessions \
WHERE token_hash = $1 AND expires_at > NOW()",
let row: Option<(uuid::Uuid, DateTime<Utc>, DateTime<Utc>)> = sqlx::query_as(
"SELECT user_id, expires_at, absolute_expires_at FROM admin_sessions \
WHERE token_hash = $1 AND expires_at > NOW() AND absolute_expires_at > NOW()",
)
.bind(token_hash)
.fetch_optional(&self.pool)
.await?;
Ok(row.map(|(uid, exp)| AdminSessionLookup {
Ok(row.map(|(uid, exp, abs)| AdminSessionLookup {
user_id: uid.into(),
expires_at: exp,
absolute_expires_at: abs,
}))
}
@@ -144,9 +151,12 @@ impl AdminSessionRepository for PostgresAdminSessionRepository {
}
async fn prune_expired(&self) -> Result<u64, AdminSessionRepositoryError> {
let res = sqlx::query("DELETE FROM admin_sessions WHERE expires_at <= NOW()")
.execute(&self.pool)
.await?;
let res = sqlx::query(
"DELETE FROM admin_sessions \
WHERE expires_at <= NOW() OR absolute_expires_at <= NOW()",
)
.execute(&self.pool)
.await?;
Ok(res.rows_affected())
}
}

View File

@@ -18,7 +18,7 @@
use std::sync::Arc;
use axum::extract::{Path, State};
use axum::http::StatusCode;
use axum::http::{HeaderMap, StatusCode};
use axum::response::{IntoResponse, Json, Response};
use axum::routing::{delete, get};
use axum::{Extension, Router};
@@ -116,7 +116,7 @@ async fn mint_key(
State(state): State<ApiKeysState>,
Extension(principal): Extension<Principal>,
Json(input): Json<MintApiKeyRequest>,
) -> Result<(StatusCode, Json<MintApiKeyResponse>), ApiKeysError> {
) -> Result<(StatusCode, HeaderMap, Json<MintApiKeyResponse>), ApiKeysError> {
validate_name(&input.name)?;
validate_scopes(&input.scopes, input.app_id)?;
@@ -135,6 +135,8 @@ async fn mint_key(
.await?;
Ok((
StatusCode::CREATED,
// C2: the response carries the raw key exactly once — never cache it.
crate::auth_api::no_store_headers(),
Json(MintApiKeyResponse {
key: row.into(),
raw_token: minted.raw,

View File

@@ -103,6 +103,25 @@ pub trait AppRepository: Send + Sync {
/// single transaction so a partial delete cannot be observed.
async fn delete_cascade(&self, id: AppId) -> Result<(), ScriptRepositoryError>;
async fn count_scripts_in_app(&self, id: AppId) -> Result<i64, ScriptRepositoryError>;
/// F-030 per-app CORS. The allow-list of origins each app permits on its
/// user routes, keyed by app_id — read once per domain-cache rebuild to
/// push into the orchestrator (which has no DB). Default: none (CORS off).
async fn cors_all(&self) -> Result<Vec<(AppId, Vec<String>)>, ScriptRepositoryError> {
Ok(Vec::new())
}
/// The CORS allow-list for one app (`[]` = CORS disabled).
async fn get_cors(&self, _id: AppId) -> Result<Vec<String>, ScriptRepositoryError> {
Ok(Vec::new())
}
/// Replace an app's CORS allow-list. `["*"]` allows any origin.
async fn set_cors(
&self,
_id: AppId,
_origins: Vec<String>,
) -> Result<(), ScriptRepositoryError> {
Ok(())
}
}
pub struct PostgresAppRepository {
@@ -443,6 +462,32 @@ impl AppRepository for PostgresAppRepository {
.await?;
Ok(count.0)
}
async fn cors_all(&self) -> Result<Vec<(AppId, Vec<String>)>, ScriptRepositoryError> {
let rows: Vec<(uuid::Uuid, sqlx::types::Json<Vec<String>>)> =
sqlx::query_as("SELECT id, cors_allowed_origins FROM apps")
.fetch_all(&self.pool)
.await?;
Ok(rows.into_iter().map(|(id, j)| (id.into(), j.0)).collect())
}
async fn get_cors(&self, id: AppId) -> Result<Vec<String>, ScriptRepositoryError> {
let row: Option<(sqlx::types::Json<Vec<String>>,)> =
sqlx::query_as("SELECT cors_allowed_origins FROM apps WHERE id = $1")
.bind(id.into_inner())
.fetch_optional(&self.pool)
.await?;
Ok(row.map(|(j,)| j.0).unwrap_or_default())
}
async fn set_cors(&self, id: AppId, origins: Vec<String>) -> Result<(), ScriptRepositoryError> {
sqlx::query("UPDATE apps SET cors_allowed_origins = $2, updated_at = NOW() WHERE id = $1")
.bind(id.into_inner())
.bind(sqlx::types::Json(origins))
.execute(&self.pool)
.await?;
Ok(())
}
}
#[derive(sqlx::FromRow)]

View File

@@ -18,8 +18,9 @@ use serde_json::json;
use crate::app_repo::AppRepository;
use crate::apply_service::{
ApplyError, ApplyOwner, ApplyReport, ApplyService, Bundle, BundleTrigger, CollectionInfo,
ExtensionPointInfo, NodeKind, OwnershipClaim, PlanResult, ProjectDecl, RouteTemplateInfo,
StructureMode, SuppressionInfo, TreeBundle, TreePlanResult, TriggerTemplateInfo,
ExtensionPointInfo, InterceptorInfo, NodeKind, OwnershipClaim, PlanResult, ProjectDecl,
RouteTemplateInfo, StructureMode, SuppressionInfo, TreeBundle, TreePlanResult,
TriggerTemplateInfo,
};
use crate::authz::{require, AuthzDenied, Capability};
use crate::group_repo::GroupRepository;
@@ -46,9 +47,45 @@ pub fn apply_router(service: ApplyService) -> Router {
.route("/groups/{id}/routes", get(group_routes_handler))
.route("/apps/{id}/suppressions", get(app_suppressions_handler))
.route("/groups/{id}/suppressions", get(group_suppressions_handler))
.route("/apps/{id}/interceptors", get(app_interceptors_handler))
.route("/groups/{id}/interceptors", get(group_interceptors_handler))
.with_state(service)
}
/// Read-only §9.4 interceptor report for an app: the RESOLVED chain view (every
/// marker guarding its writes, nearest-owner-wins). Viewer-tier `AppRead`. Backs
/// `pic interceptors ls --app`.
async fn app_interceptors_handler(
State(svc): State<ApplyService>,
Extension(principal): Extension<Principal>,
Path(id_or_slug): Path<String>,
) -> Result<Json<Vec<InterceptorInfo>>, ApplyError> {
let app_id = resolve_app_id(svc.apps.as_ref(), &id_or_slug).await?;
require(svc.authz.as_ref(), &principal, Capability::AppRead(app_id))
.await
.map_err(map_authz)?;
let report = svc.interceptor_report(ApplyOwner::App(app_id)).await?;
Ok(Json(report))
}
/// Read-only §9.4 interceptor report for a group: its OWN declared markers.
async fn group_interceptors_handler(
State(svc): State<ApplyService>,
Extension(principal): Extension<Principal>,
Path(id_or_slug): Path<String>,
) -> Result<Json<Vec<InterceptorInfo>>, ApplyError> {
let group_id = resolve_group_id(svc.groups.as_ref(), &id_or_slug).await?;
require(
svc.authz.as_ref(),
&principal,
Capability::GroupScriptsRead(group_id),
)
.await
.map_err(map_authz)?;
let report = svc.interceptor_report(ApplyOwner::Group(group_id)).await?;
Ok(Json(report))
}
/// Read-only §11 tail suppression report for an app: the inherited templates it
/// declines (`target_kind`, `reference`). Viewer-tier `AppRead`. Backs
/// `pic suppress ls --app`.
@@ -205,12 +242,22 @@ pub struct ApplyRequest {
/// refuses (409) if the app's live state has changed since.
#[serde(default)]
pub expected_token: Option<String>,
/// §7 ownership: the declaring repo's `[project]` (absent = no claim).
/// §7 ownership: the declaring repo's `[project]` (absent = no claim). Also
/// carries the §3 M3 env approval policy (`project.environments`).
#[serde(default)]
pub project: Option<ProjectDecl>,
/// §7 ownership: reassign a node owned by another project (group-admin).
#[serde(default)]
pub takeover: bool,
/// §3 M3: the environment this apply targets (the overlay merged). Drives the
/// per-env approval gate when the project marks it `confirm = true`.
#[serde(default)]
pub env: Option<String>,
/// §3 M3: environments the actor explicitly approved (`--approve <env>`). A
/// confirm-required env is refused unless it appears here; `--yes` alone does
/// NOT add it.
#[serde(default)]
pub approved_envs: Vec<String>,
}
async fn apply_handler(
@@ -221,6 +268,21 @@ async fn apply_handler(
) -> Result<Json<ApplyReport>, ApplyError> {
let app_id = resolve_app_id(svc.apps.as_ref(), &id_or_slug).await?;
require_app_node_writes(&svc, &principal, app_id, &req.bundle, req.prune).await?;
// §3 M3: server-authoritative per-env approval gate. The governing policy is
// loaded from the project the SERVER resolves as owning this node (its
// nearest-claimed ancestor), unioned with the declared policy — so a request
// that omits or spoofs `[project]` can't bypass a gate the owning project
// established. If gated and approved, the override additionally requires
// ADMIN on the node (a step up from the editor write caps above) + is audited.
let policy = svc
.governing_env_policy(ApplyOwner::App(app_id), req.project.as_ref())
.await?;
if env_gate_check(&policy, req.env.as_deref(), &req.approved_envs)? {
require(svc.authz.as_ref(), &principal, Capability::AppAdmin(app_id))
.await
.map_err(map_authz)?;
audit_gated_apply(&principal, req.env.as_deref(), 1);
}
let claim = OwnershipClaim {
project: req.project,
takeover: req.takeover,
@@ -298,6 +360,21 @@ async fn group_apply_handler(
let group_id = resolve_group_id(svc.groups.as_ref(), &id_or_slug).await?;
svc.require_group_node_writes(&principal, group_id, &req.bundle, req.prune)
.await?;
// §3 M3: per-env approval gate (governing declared, server-resolved owner)
// — an approved gated apply requires GroupAdmin.
let policy = svc
.governing_env_policy(ApplyOwner::Group(group_id), req.project.as_ref())
.await?;
if env_gate_check(&policy, req.env.as_deref(), &req.approved_envs)? {
require(
svc.authz.as_ref(),
&principal,
Capability::GroupAdmin(group_id),
)
.await
.map_err(map_authz)?;
audit_gated_apply(&principal, req.env.as_deref(), 1);
}
let claim = OwnershipClaim {
project: req.project,
takeover: req.takeover,
@@ -328,6 +405,7 @@ struct TreeApplyRequest {
#[serde(default)]
expected_token: Option<String>,
/// §7 ownership: one `[project]` for the whole tree (the root manifest's).
/// Also carries the §3 M3 env approval policy (`project.environments`).
#[serde(default)]
project: Option<ProjectDecl>,
#[serde(default)]
@@ -336,6 +414,12 @@ struct TreeApplyRequest {
/// manifest (default `Refuse`; a pre-M2 CLI omits it).
#[serde(default)]
structure_mode: StructureMode,
/// §3 M3: the environment this apply targets (drives the per-env approval gate).
#[serde(default)]
env: Option<String>,
/// §3 M3: environments the actor explicitly approved (`--approve <env>`).
#[serde(default)]
approved_envs: Vec<String>,
}
#[derive(Deserialize)]
@@ -364,6 +448,18 @@ async fn tree_apply_handler(
Json(req): Json<TreeApplyRequest>,
) -> Result<Json<ApplyReport>, ApplyError> {
authz_tree(&svc, &principal, &req.bundle, Some(req.prune)).await?;
// §3 M3: server-authoritative per-env approval gate (governing declared).
// The governing side is the union of every declared node's owning-project
// policy, resolved server-side — omitting `[project]` can't dodge it. On an
// approved gated apply, require ADMIN on EVERY declared node (a step up from
// the editor write caps authz_tree checks) + audit.
let policy = svc
.governing_env_policy_tree(&req.bundle, req.project.as_ref())
.await?;
if env_gate_check(&policy, req.env.as_deref(), &req.approved_envs)? {
require_admin_on_every_node(&svc, &principal, &req.bundle).await?;
audit_gated_apply(&principal, req.env.as_deref(), req.bundle.nodes.len());
}
let claim = OwnershipClaim {
project: req.project,
takeover: req.takeover,
@@ -381,6 +477,82 @@ async fn tree_apply_handler(
Ok(Json(report))
}
/// §3 M3 (hermetic gate): the per-env approval gate, evaluated against the
/// EFFECTIVE policy (`governing declared`, built by
/// `ApplyService::governing_env_policy`) — the governing side is loaded from the
/// project the SERVER resolves as owning the node, so a request that omits or
/// weakens `[project]` can't bypass a gate the owning project established.
/// Returns `Ok(true)` when the target env is confirm-required
/// AND approved — the caller must then require admin + audit. `Ok(false)` when
/// not gated (no env, or `confirm = false` everywhere). `Err(ApprovalRequired)`
/// when gated and NOT in `approved_envs` (a blanket `--yes` never lands here).
fn env_gate_check(
policy: &std::collections::BTreeMap<String, bool>,
env: Option<&str>,
approved_envs: &[String],
) -> Result<bool, ApplyError> {
let Some(env) = env else {
return Ok(false);
};
if !policy.get(env).copied().unwrap_or(false) {
return Ok(false);
}
if approved_envs.iter().any(|e| e == env) {
Ok(true)
} else {
Err(ApplyError::ApprovalRequired(env.to_string()))
}
}
/// §3 M3: require `AppAdmin`/`GroupAdmin` on every declared node of a tree apply
/// (the "override a gate" authority, above editor write caps). Mirrors
/// `authz_tree`'s resolution: an app resolves UUID-or-slug (fail-closed); a group
/// node whose slug names no existing group is to-CREATE (its create is separately
/// admin-gated in the service), so it is skipped here rather than 404'd.
async fn require_admin_on_every_node(
svc: &ApplyService,
principal: &Principal,
bundle: &TreeBundle,
) -> Result<(), ApplyError> {
for node in &bundle.nodes {
match node.kind {
NodeKind::App => {
let app_id = resolve_app_id(svc.apps.as_ref(), &node.slug).await?;
require(svc.authz.as_ref(), principal, Capability::AppAdmin(app_id))
.await
.map_err(map_authz)?;
}
NodeKind::Group => {
if let Some(group) = svc
.groups
.get_by_slug(&node.slug)
.await
.map_err(|e| ApplyError::Backend(e.to_string()))?
{
require(
svc.authz.as_ref(),
principal,
Capability::GroupAdmin(group.id),
)
.await
.map_err(map_authz)?;
}
}
}
}
Ok(())
}
/// §3 M3: audit an approved gated-environment apply (actor + env + node count).
fn audit_gated_apply(principal: &Principal, env: Option<&str>, nodes: usize) {
tracing::info!(
user_id = ?principal.user_id,
env = ?env,
nodes,
"apply: approved gated-environment apply (audit)"
);
}
/// Per-node capability check for a tree plan/apply. `prune` widens an apply's
/// write requirement to every kind. A plan (`prune` ignored, no writes) needs
/// only the per-node read cap.
@@ -533,7 +705,14 @@ async fn resolve_app_id(apps: &dyn AppRepository, ident: &str) -> Result<AppId,
.await
.map_err(|e| ApplyError::Backend(e.to_string()))?
.map(|l| l.app.id)
.ok_or_else(|| ApplyError::AppNotFound(ident.to_string()))
// `pic apply` reconciles the group tree but never creates an app — make
// the miss actionable instead of a bare "app not found: cms" (F-006).
.ok_or_else(|| {
ApplyError::AppNotFound(format!(
"{ident} — apply does not create apps; create it first with \
`pic apps create {ident}`, then re-run apply"
))
})
}
fn map_authz(denied: AuthzDenied) -> ApplyError {
@@ -551,9 +730,10 @@ impl IntoResponse for ApplyError {
StatusCode::UNPROCESSABLE_ENTITY,
json!({ "error": self.to_string() }),
),
Self::StateMoved => (StatusCode::CONFLICT, json!({ "error": self.to_string() })),
// §7 ownership conflict — actionable (declare [project], --takeover).
Self::OwnershipConflict(_) => {
// 409 CONFLICT, all actionable: a stale bound plan (StateMoved), a
// §3 M3 gated env needing `--approve` (ApprovalRequired), or a §7
// ownership conflict (declare `[project]`, `--takeover`).
Self::StateMoved | Self::ApprovalRequired(_) | Self::OwnershipConflict(_) => {
(StatusCode::CONFLICT, json!({ "error": self.to_string() }))
}
Self::Forbidden => (StatusCode::FORBIDDEN, json!({ "error": self.to_string() })),

File diff suppressed because it is too large Load Diff

View File

@@ -74,6 +74,10 @@ pub fn apps_router(state: AppsState) -> Router {
"/apps/{id_or_slug}/domains/{domain_id}",
delete(delete_domain),
)
.route(
"/apps/{id_or_slug}/cors",
get(get_cors_handler).put(set_cors_handler),
)
.with_state(state)
}
@@ -134,6 +138,13 @@ pub struct SlugCheckResponse {
pub reason: Option<String>,
}
/// F-030 per-app CORS config: the set of browser origins allowed to call this
/// app's user routes. `["*"]` = any origin. Empty = CORS off.
#[derive(Debug, Serialize, Deserialize)]
pub struct CorsConfig {
pub allowed_origins: Vec<String>,
}
#[derive(Debug, Deserialize)]
pub struct CreateDomainRequest {
pub pattern: String,
@@ -513,6 +524,44 @@ async fn delete_domain(
Ok(StatusCode::NO_CONTENT)
}
/// F-030: read an app's CORS allow-list. `AppRead` — same gate as viewing
/// domain claims (CORS is host-adjacent config).
async fn get_cors_handler(
State(s): State<AppsState>,
Extension(principal): Extension<Principal>,
Path(id_or_slug): Path<String>,
) -> Result<Json<CorsConfig>, AppsApiError> {
let app = resolve_app(&*s.apps, &id_or_slug).await?.app;
require(s.authz.as_ref(), &principal, Capability::AppRead(app.id)).await?;
let allowed_origins = s.apps.get_cors(app.id).await?;
Ok(Json(CorsConfig { allowed_origins }))
}
/// F-030: replace an app's CORS allow-list, then refresh the domain cache so
/// the orchestrator serves the new policy immediately. `AppManageDomains` —
/// the same gate as domain-claim CRUD.
async fn set_cors_handler(
State(s): State<AppsState>,
Extension(principal): Extension<Principal>,
Path(id_or_slug): Path<String>,
Json(input): Json<CorsConfig>,
) -> Result<Json<CorsConfig>, AppsApiError> {
let app = resolve_app(&*s.apps, &id_or_slug).await?.app;
require(
s.authz.as_ref(),
&principal,
Capability::AppManageDomains(app.id),
)
.await?;
s.apps
.set_cors(app.id, input.allowed_origins.clone())
.await?;
refresh_domain_cache(&s).await?;
Ok(Json(CorsConfig {
allowed_origins: input.allowed_origins,
}))
}
// ----------------------------------------------------------------------------
// Helpers
// ----------------------------------------------------------------------------
@@ -571,8 +620,15 @@ async fn refresh_route_cache(state: &AppsState) {
}
// §4.5 M5: reconcile materialized stateful-template copies for the changed
// app set (best-effort; self-heals on the next mutation).
if let Err(e) = crate::materialize::rematerialize_stateful_templates(&state.pool).await {
tracing::warn!(error = %e, "apps: stateful-template materialization failed; it will self-heal");
match crate::materialize::rematerialize_stateful_templates(&state.pool).await {
Ok(warnings) => {
for w in warnings {
tracing::warn!(warning = %w, "apps: stateful-template materialization warning");
}
}
Err(e) => {
tracing::warn!(error = %e, "apps: stateful-template materialization failed; it will self-heal");
}
}
}
@@ -595,6 +651,10 @@ pub async fn refresh_domain_cache(state: &AppsState) -> Result<(), AppsApiError>
})
.collect();
state.domain_table.replace(compiled);
// F-030: push the per-app CORS allow-lists into the same in-memory cache so
// the orchestrator can answer preflights + tag responses without a DB hit.
let cors: std::collections::HashMap<_, _> = state.apps.cors_all().await?.into_iter().collect();
state.domain_table.replace_cors(cors);
Ok(())
}

File diff suppressed because it is too large Load Diff

View File

@@ -156,12 +156,19 @@ async fn login(
};
let token = generate_session_token();
let expires_at = Utc::now()
+ ChronoDuration::from_std(state.ttl).unwrap_or_else(|_| ChronoDuration::hours(24));
let now = Utc::now();
// C1: an admin session dies at the EARLIER of its sliding window and its
// absolute cap. At create the sliding window is shorter, so `expires_at`
// starts as `now + ttl`; the absolute cap `now + absolute_ttl` is stored so
// the sliding `touch` can clamp to it as the session ages.
let expires_at =
now + ChronoDuration::from_std(state.ttl).unwrap_or_else(|_| ChronoDuration::hours(24));
let absolute_expires_at = now
+ ChronoDuration::from_std(state.absolute_ttl).unwrap_or_else(|_| ChronoDuration::days(30));
if let Err(err) = state
.sessions
.create(user_id, &token.hash, expires_at)
.create(user_id, &token.hash, expires_at, absolute_expires_at)
.await
{
tracing::error!(?err, "admin_sessions insert failed");
@@ -174,6 +181,7 @@ async fn login(
(
StatusCode::OK,
no_store_headers(),
Json(LoginResponse {
user: AdminUserDto {
id: user_row.id,
@@ -188,6 +196,15 @@ async fn login(
.into_response()
}
/// `Cache-Control: no-store` for a response carrying a raw credential (a session
/// token, an API key). Prevents a browser / proxy / CDN cache from retaining the
/// secret where a later client could read it back. (C2)
pub(crate) fn no_store_headers() -> HeaderMap {
let mut headers = HeaderMap::new();
headers.insert(header::CACHE_CONTROL, HeaderValue::from_static("no-store"));
headers
}
async fn logout(State(state): State<AuthState>, req: Request<Body>) -> Response {
// Pull token without requiring a valid session (logout is idempotent).
let token = extract_token_for_logout(&req);

View File

@@ -137,6 +137,11 @@ pub struct AuthState {
pub sessions: Arc<dyn AdminSessionRepository>,
pub keys: Arc<dyn ApiKeyRepository>,
pub ttl: Duration,
/// C1: absolute hard cap on an admin session's lifetime. The sliding
/// `touch` bump is clamped at `session_start + absolute_ttl`, so even a
/// continuously-used token self-expires. Set at login via the session's
/// stored `absolute_expires_at`; this value seeds that at create time.
pub absolute_ttl: Duration,
/// F-P-009 — shared cache of resolved Principals. Constructed once
/// at startup and cloned (same `Arc`) into every router state that
/// resolves or revokes credentials, so a revocation-side eviction is
@@ -279,8 +284,10 @@ async fn verify_session(
};
// Sliding-window bump — inline so a DB blip surfaces as 500 rather
// than silent stale sessions. Same shape as Phase 3a.
let new_expires_at = Utc::now() + chrono::Duration::from_std(state.ttl).unwrap_or_default();
// than silent stale sessions. C1: clamp the bump at the session's absolute
// cap so a continuously-used token can't slide forever.
let sliding = Utc::now() + chrono::Duration::from_std(state.ttl).unwrap_or_default();
let new_expires_at = sliding.min(lookup.absolute_expires_at);
if let Err(err) = state.sessions.touch(&token_hash, new_expires_at).await {
tracing::error!(?err, "admin_sessions touch failed");
return Err(InternalError);
@@ -304,7 +311,15 @@ async fn verify_api_key(state: &AuthState, rest: &str) -> Result<Option<Principa
if rest.len() <= API_KEY_PREFIX_LEN {
return Ok(None);
}
let prefix = &rest[..API_KEY_PREFIX_LEN];
// `rest` is the raw, attacker-controlled bearer value after `pic_` — it is
// NOT yet validated as the base32 body a real key has, so a byte-index slice
// (`&rest[..8]`) could split a multibyte UTF-8 codepoint straddling byte 8
// and panic, aborting the request task (an unauthenticated DoS). `get(..)`
// returns `None` at a non-char-boundary, so a malformed bearer falls through
// to "no match" instead of panicking.
let Some(prefix) = rest.get(..API_KEY_PREFIX_LEN) else {
return Ok(None);
};
let mut candidates = match state.keys.find_active_by_prefix(prefix).await {
Ok(v) => v,
@@ -508,6 +523,20 @@ mod tests {
}
}
#[test]
fn api_key_prefix_slice_is_char_boundary_safe() {
// H2 regression (audit 2026-07-11): `rest` is the raw bearer body after
// `pic_`, sliced BEFORE any base32 validation. A byte-index slice
// `&rest[..API_KEY_PREFIX_LEN]` panics when a multibyte codepoint
// straddles byte 8. `€` is 3 bytes, so `aaaaaa€…` puts byte 8 mid-
// codepoint — the boundary-safe `get(..)` must return None, not panic.
let malformed = "aaaaaa\u{20ac}bbbb";
assert!(malformed.len() > API_KEY_PREFIX_LEN);
assert!(malformed.get(..API_KEY_PREFIX_LEN).is_none());
// A clean ASCII body slices to exactly the indexed prefix.
assert_eq!("abcdefghXYZ".get(..API_KEY_PREFIX_LEN), Some("abcdefgh"));
}
#[test]
fn evict_user_drops_only_that_users_entries() {
// Audit 2026-06-11 (PrincipalCache revocation-lag).

View File

@@ -458,6 +458,30 @@ pub enum AuthzDenied {
Repo(#[from] AuthzError),
}
/// Run [`require`] for a known principal and map the `AuthzDenied` verdict onto
/// the caller's service-specific error via `forbidden`/`backend`. The single
/// home for the `Denied → forbidden() / Repo(e) → backend(e)` mapping that
/// [`script_gate`], [`script_gate_require_principal`], and the stateful services
/// share.
///
/// # Errors
///
/// `forbidden()` on `AuthzDenied::Denied`; `backend(repo_err.to_string())` on
/// `AuthzDenied::Repo`.
pub async fn require_mapped<E>(
repo: &dyn AuthzRepo,
principal: &Principal,
cap: Capability,
forbidden: impl FnOnce() -> E,
backend: impl FnOnce(String) -> E,
) -> Result<(), E> {
match require(repo, principal, cap).await {
Ok(()) => Ok(()),
Err(AuthzDenied::Denied) => Err(forbidden()),
Err(AuthzDenied::Repo(e)) => Err(backend(e.to_string())),
}
}
/// Script-as-gate authz: anonymous public-HTTP scripts skip the check
/// (`cx.principal` is `None`); authenticated callers must hold `cap`.
///
@@ -482,11 +506,7 @@ pub async fn script_gate<E>(
let Some(principal) = cx.principal.as_ref() else {
return Ok(());
};
match require(repo, principal, cap).await {
Ok(()) => Ok(()),
Err(AuthzDenied::Denied) => Err(forbidden()),
Err(AuthzDenied::Repo(e)) => Err(backend(e.to_string())),
}
require_mapped(repo, principal, cap, forbidden, backend).await
}
/// Like [`script_gate`], but **fails closed on an anonymous principal**: a
@@ -510,11 +530,7 @@ pub async fn script_gate_require_principal<E>(
let Some(principal) = cx.principal.as_ref() else {
return Err(forbidden());
};
match require(repo, principal, cap).await {
Ok(()) => Ok(()),
Err(AuthzDenied::Denied) => Err(forbidden()),
Err(AuthzDenied::Repo(e)) => Err(backend(e.to_string())),
}
require_mapped(repo, principal, cap, forbidden, backend).await
}
// ----------------------------------------------------------------------------
@@ -1366,6 +1382,64 @@ mod tests {
);
}
/// §11.6 D2/D3: shared-topic PUBLISH and shared-queue ENQUEUE are writes, so
/// they require editor+ on the owning group. The three older group data caps
/// (KV/docs/files) have this test; these two — the newest, and the ones whose
/// blast radius is an entire subtree's handlers — did not. A `role_satisfies`
/// regression that let a Viewer publish/enqueue would go uncaught otherwise.
#[tokio::test]
async fn group_pubsub_and_queue_write_caps_require_editor() {
let repo = InMemoryAuthzRepo::default();
let root = GroupId::new();
let team = GroupId::new();
repo.add_group(root, None).await;
repo.add_group(team, Some(root)).await;
// A Viewer at root is denied BOTH writes on a descendant group.
let viewer = principal(InstanceRole::Member);
repo.grant_group(viewer.user_id, root, AppRole::Viewer)
.await;
for cap in [
Capability::GroupPubsubPublish(team),
Capability::GroupQueueEnqueue(team),
] {
assert_eq!(
can(&repo, &viewer, cap).await.unwrap(),
Decision::Deny,
"a Viewer must not publish/enqueue into a shared group collection"
);
}
// An Editor at root is allowed both (resolved up the chain).
let editor = principal(InstanceRole::Member);
repo.grant_group(editor.user_id, root, AppRole::Editor)
.await;
for cap in [
Capability::GroupPubsubPublish(team),
Capability::GroupQueueEnqueue(team),
] {
assert!(
can(&repo, &editor, cap).await.unwrap().is_allow(),
"an Editor on the owning group's ancestor must be allowed"
);
}
// An unrelated member gets neither.
let outsider = principal(InstanceRole::Member);
assert_eq!(
can(&repo, &outsider, Capability::GroupPubsubPublish(team))
.await
.unwrap(),
Decision::Deny
);
assert_eq!(
can(&repo, &outsider, Capability::GroupQueueEnqueue(team))
.await
.unwrap(),
Decision::Deny
);
}
#[tokio::test]
async fn group_docs_caps_resolve_by_role_up_the_chain() {
// §11.6 docs: same trust shape as shared KV — read is viewer+, write is

View File

@@ -231,6 +231,8 @@ impl Dispatcher {
// contend with the per-100ms dispatcher tick.
let reclaim_queue = self.queue.clone();
let reclaim_group_queue = self.group_queue.clone();
let reclaim_outbox = self.outbox.clone();
let outbox_claim_timeout = self.config.outbox_claim_timeout_secs;
let reclaim_interval =
Duration::from_millis(u64::from(self.config.queue_reclaim_interval_ms));
tokio::spawn(async move {
@@ -238,6 +240,21 @@ impl Dispatcher {
ticker.tick().await;
loop {
ticker.tick().await;
// A dispatcher that died mid-dispatch left its claimed rows
// stranded — `claim_due` only takes unclaimed rows, so without
// this they would never fire again.
match reclaim_outbox
.reclaim_stale_claims(outbox_claim_timeout)
.await
{
Ok(0) => {}
Ok(n) => tracing::warn!(
reclaimed = n,
timeout_secs = outbox_claim_timeout,
"reclaimed stale outbox claims — a dispatcher died mid-dispatch"
),
Err(e) => tracing::warn!(?e, "outbox reclaim task errored"),
}
match reclaim_queue.reclaim_visibility_timeouts().await {
Ok(0) => {}
Ok(n) => tracing::info!(reclaimed = n, "queue visibility-timeout reclaim"),
@@ -394,11 +411,38 @@ impl Dispatcher {
}
}
/// TRANSIENT release — the handler never ran (gate saturated, or the script
/// was disabled at fire time). Re-queue WITHOUT counting the claim's
/// pre-increment against `max_attempts`, so overload/disable windows can't
/// dead-letter a message that executed zero times.
async fn q_release(
&self,
c: &ActiveQueueConsumer,
id: QueueMessageId,
token: uuid::Uuid,
delay: chrono::Duration,
) -> Result<bool, String> {
match c.shared_group {
Some(_) => self
.group_queue
.release(id, token, delay)
.await
.map_err(|e| e.to_string()),
None => self
.queue
.release(id, token, delay)
.await
.map_err(|e| e.to_string()),
}
}
/// Terminal disposition of a message that can't be processed (script
/// missing / cross-app / exhausted). Per-app → dead-letter (+ the caller
/// fans out `dead_letter` triggers). Group shared queue → drop the row
/// (no group dead-letter store yet — documented D3 deferral) + warn.
/// Returns the dead-letter id only for the per-app path (drives fan-out).
/// missing / cross-app / exhausted). Per-app → dead-letter into `dead_letters`
/// (+ the caller fans out `dead_letter` triggers). Group shared queue →
/// dead-letter into `group_dead_letters` (Track A M2) and return `None` so the
/// per-app fan-out is skipped (a competing-consumer message has no single app
/// to fire per-app handlers under). Returns the dead-letter id only for the
/// per-app path (drives fan-out).
async fn q_terminal(
&self,
c: &ActiveQueueConsumer,
@@ -408,19 +452,71 @@ impl Dispatcher {
reason: &str,
) -> Option<DeadLetterId> {
match c.shared_group {
Some(_) => {
if let Err(e) = self
Some(group_id) => {
// §11.6 D3: persist the exhausted message to the group dead-letter
// store instead of dropping it. We return None (not the dl id) so
// the per-app `fan_out_dead_letter` below is SKIPPED — firing the
// consuming app's *per-app* dead_letter handlers on a shared-queue
// message (competing consumers → nondeterministic app) would be
// wrong.
//
// §11.6 B2: instead, fan out to the group's *shared* dead_letter
// handlers (`shared = true` on the owning group). Each runs under
// the WRITER app (`claimed.app_id`, the consumer that exhausted
// the message) — the M2 shared-write model.
match self
.group_queue
.drop_exhausted(claimed.id, claimed.claim_token)
.dead_letter(
claimed.id,
claimed.claim_token,
group_id,
&claimed.queue_name,
trigger_id,
script_id,
claimed.attempt,
claimed.enqueued_at,
reason,
)
.await
{
tracing::warn!(?e, "shared-queue drop failed");
Ok(dl_id) => {
tracing::warn!(
reason,
queue = %claimed.queue_name,
dead_letter_id = %dl_id.into_inner(),
"shared-queue message dead-lettered"
);
let original = TriggerEvent::Queue {
queue_name: claimed.queue_name.clone(),
message: claimed.payload.clone(),
enqueued_at: claimed.enqueued_at,
attempt: claimed.attempt,
message_id: claimed.id.to_string(),
};
self.fan_out_shared_dead_letter(
group_id,
DeadLetterFanOutCtx {
// Writer app: the consumer that exhausted the msg.
app_id: claimed.app_id,
original,
source: "queue".to_string(),
dead_letter_id: dl_id,
attempts: claimed.attempt,
last_error: reason.to_string(),
trigger_id,
script_id,
first_attempt_at: claimed.enqueued_at,
last_attempt_at: Utc::now(),
// Shared-queue messages root a depth-1 chain (the
// queue is depth 0; a DL handler ticks up).
trigger_depth: 1,
root_execution_id: None,
},
)
.await;
}
Err(e) => tracing::error!(?e, "shared-queue dead-letter write failed"),
}
tracing::warn!(
reason,
queue = %claimed.queue_name,
"shared-queue message dropped (no group dead-letter store yet)"
);
None
}
None => match self
@@ -465,8 +561,11 @@ impl Dispatcher {
// immediate (which lets the next tick re-claim). Mirrors the
// outbox arm.
let Ok(permit) = self.gate.try_acquire() else {
// Transient: never executed → release without burning the retry
// budget (else sustained overload could dead-letter a message that
// ran zero times).
let _ = self
.q_nack(
.q_release(
consumer,
claimed.id,
claimed.claim_token,
@@ -557,7 +656,7 @@ impl Dispatcher {
"queue consumer script disabled at fire time; releasing claim"
);
if let Err(e) = self
.q_nack(
.q_release(
consumer,
claimed.id,
claimed.claim_token,
@@ -565,7 +664,7 @@ impl Dispatcher {
)
.await
{
tracing::warn!(?e, "queue nack on disabled consumer failed");
tracing::warn!(?e, "queue release on disabled consumer failed");
}
drop(permit);
return Ok(());
@@ -680,9 +779,9 @@ impl Dispatcher {
// same way here.
let now = Utc::now();
let last_error = err.to_string();
// Per-app → dead-letter (+ fan out below). Shared group queue → dropped
// inside q_terminal (no group dead-letter store yet), returns None so
// the fan-out is skipped.
// Per-app → dead-letter (+ fan out below). Shared group queue →
// dead-lettered into `group_dead_letters` inside q_terminal (Track A M2),
// which returns None so the per-app fan-out is skipped.
let dl_id = self
.q_terminal(
consumer,
@@ -1434,6 +1533,83 @@ impl Dispatcher {
}
}
/// §11.6 B2: the shared analogue of `fan_out_dead_letter`. When a message in
/// a group's SHARED queue is exhausted, fire the group's `shared = true`
/// `dead_letter` handlers. Each outbox row is stamped `ctx.app_id` (the
/// writer/consumer that exhausted the message) so the handler runs under
/// that app's `SdkCallCx` — the M2 shared-write model. Best-effort, mirroring
/// the per-app path (the group dead-letter row is already durably written).
async fn fan_out_shared_dead_letter(
&self,
owning_group: picloud_shared::GroupId,
ctx: DeadLetterFanOutCtx,
) {
let DeadLetterFanOutCtx {
app_id,
original,
source,
dead_letter_id,
attempts,
last_error,
trigger_id,
script_id,
first_attempt_at,
last_attempt_at,
trigger_depth,
root_execution_id,
} = ctx;
let matches = match self
.triggers
.list_matching_shared_dead_letter(owning_group, &source, trigger_id, script_id)
.await
{
Ok(m) => m,
Err(e) => {
tracing::error!(?e, "shared dead-letter trigger lookup failed");
return;
}
};
for m in matches {
let event = TriggerEvent::DeadLetter {
dead_letter_id,
original: Box::new(original.clone()),
attempts,
last_error: last_error.clone(),
trigger_id,
script_id,
first_attempt_at,
last_attempt_at,
};
let payload = match serde_json::to_value(&event) {
Ok(p) => p,
Err(e) => {
tracing::error!(?e, "failed to serialize shared dead-letter event");
continue;
}
};
if let Err(e) = self
.outbox
.insert(NewOutboxRow {
// Writer app — the consumer that exhausted the message.
app_id,
source_kind: OutboxSourceKind::DeadLetter,
trigger_id: Some(m.trigger_id),
script_id: Some(m.script_id),
reply_to: None,
payload,
origin_principal: Some(m.registered_by_principal),
trigger_depth: trigger_depth.saturating_add(1),
root_execution_id,
})
.await
{
tracing::error!(?e, "failed to enqueue shared dead-letter handler delivery");
}
}
}
async fn deliver_inbox(&self, row: &OutboxRow, inbox_id: Uuid, result: InboxResult) {
match self.inbox.deliver(inbox_id, result.clone()).await {
InboxDeliveryOutcome::Delivered => {}
@@ -1505,6 +1681,7 @@ fn summarize(resp: &ExecResponse) -> ExecResponseSummary {
status_code: resp.status_code,
headers: resp.headers.clone(),
body: resp.body.clone(),
body_base64: resp.body_base64.clone(),
}
}
@@ -1663,13 +1840,30 @@ mod tests {
) -> Result<bool, crate::group_queue_repo::GroupQueueRepoError> {
Ok(false)
}
async fn drop_exhausted(
async fn release(
&self,
_id: QueueMessageId,
_token: Uuid,
_delay: chrono::Duration,
) -> Result<bool, crate::group_queue_repo::GroupQueueRepoError> {
Ok(false)
}
#[allow(clippy::too_many_arguments)]
async fn dead_letter(
&self,
_id: QueueMessageId,
_token: Uuid,
_group_id: picloud_shared::GroupId,
_collection: &str,
_trigger_id: Option<picloud_shared::TriggerId>,
_script_id: Option<ScriptId>,
_attempt: u32,
_first_attempt_at: chrono::DateTime<chrono::Utc>,
_last_error: &str,
) -> Result<picloud_shared::DeadLetterId, crate::group_queue_repo::GroupQueueRepoError>
{
unreachable!("shared queue not exercised")
}
async fn reclaim_visibility_timeouts(
&self,
) -> Result<u64, crate::group_queue_repo::GroupQueueRepoError> {
@@ -1937,6 +2131,7 @@ mod tests {
struct ClaimNackQueue {
claimed: ClaimedMessage,
nacked: Arc<AtomicBool>,
released: Arc<AtomicBool>,
}
#[async_trait]
@@ -1971,6 +2166,15 @@ mod tests {
self.nacked.store(true, Ordering::SeqCst);
Ok(true)
}
async fn release(
&self,
_message_id: QueueMessageId,
_claim_token: Uuid,
_retry_delay: chrono::Duration,
) -> Result<bool, crate::queue_repo::QueueRepoError> {
self.released.store(true, Ordering::SeqCst);
Ok(true)
}
async fn reclaim_visibility_timeouts(
&self,
) -> Result<u64, crate::queue_repo::QueueRepoError> {
@@ -2269,6 +2473,12 @@ mod tests {
#[async_trait]
impl OutboxRepo for UnusedOutbox {
async fn reclaim_stale_claims(
&self,
_timeout_secs: u32,
) -> Result<u64, crate::outbox_repo::OutboxRepoError> {
Ok(0)
}
async fn insert(
&self,
_row: NewOutboxRow,
@@ -2388,6 +2598,7 @@ mod tests {
};
let nacked = Arc::new(AtomicBool::new(false));
let released = Arc::new(AtomicBool::new(false));
let executed = Arc::new(AtomicBool::new(false));
let dispatcher = Dispatcher {
@@ -2408,6 +2619,7 @@ mod tests {
queue: Arc::new(ClaimNackQueue {
claimed,
nacked: nacked.clone(),
released: released.clone(),
}),
group_queue: Arc::new(NoopGroupQueue),
config: TriggerConfig::from_env(),
@@ -2418,10 +2630,16 @@ mod tests {
// 1. The disabled path returns Ok(()).
assert!(result.is_ok(), "dispatch_one_queue returned {result:?}");
// 2. The claim was released via nack.
// 2. The claim was RELEASED (transient — the handler never ran), not
// nacked: a disabled-at-fire release must not burn the retry
// budget (audit fix #4).
assert!(
nacked.load(Ordering::SeqCst),
"expected nack to release the claim for a disabled consumer"
released.load(Ordering::SeqCst),
"expected a transient release for a disabled consumer"
);
assert!(
!nacked.load(Ordering::SeqCst),
"a disabled-at-fire release must NOT count as a nack (retry budget)"
);
// 3. The executor was never reached. If the `if !script.enabled`
// gate is deleted, the flow falls through to resolve →
@@ -2504,6 +2722,14 @@ mod tests {
) -> Result<bool, crate::queue_repo::QueueRepoError> {
unimplemented!("not used by this test")
}
async fn release(
&self,
_message_id: QueueMessageId,
_claim_token: Uuid,
_retry_delay: chrono::Duration,
) -> Result<bool, crate::queue_repo::QueueRepoError> {
unimplemented!("not used by this test")
}
async fn reclaim_visibility_timeouts(
&self,
) -> Result<u64, crate::queue_repo::QueueRepoError> {
@@ -2558,6 +2784,12 @@ mod tests {
#[async_trait]
impl OutboxRepo for RecordingOutbox {
async fn reclaim_stale_claims(
&self,
_timeout_secs: u32,
) -> Result<u64, crate::outbox_repo::OutboxRepoError> {
Ok(0)
}
async fn insert(
&self,
_row: NewOutboxRow,
@@ -2665,5 +2897,76 @@ mod tests {
"executor ran for a disabled HTTP outbox row; fire-time gate missing"
);
}
/// `trigger_depth` bounds chain DEPTH. The sync `invoke()` path is pinned
/// (sdk/invoke.rs), but the ASYNC dispatcher enforcement — `dispatch_one`
/// dropping a row past `max_trigger_depth` — was not. Without it, a trigger
/// whose handler writes back to the collection it watches is an infinite
/// outbox self-amplification loop (write→trigger→write→…) from one request.
#[tokio::test]
async fn outbox_row_past_max_depth_is_dropped_without_executing() {
let app_id = AppId::new();
// ENABLED script — so what stops execution is the DEPTH gate, not the
// disabled-drop path. (The depth check runs before script resolution,
// so the repo is never even consulted, but keep it honest.)
let mut script = disabled_script(app_id);
script.enabled = true;
let config = TriggerConfig::from_env();
let row_id = Uuid::new_v4();
let row = OutboxRow {
id: row_id,
app_id,
source_kind: OutboxSourceKind::Kv,
trigger_id: Some(TriggerId::new()),
script_id: Some(script.id),
reply_to: None,
payload: serde_json::json!({}),
origin_principal: None,
// One past the ceiling — must be dropped, not dispatched.
trigger_depth: config.max_trigger_depth + 1,
root_execution_id: None,
attempt_count: 0,
next_attempt_at: Utc::now(),
created_at: Utc::now(),
};
let deleted = Arc::new(Mutex::new(None));
let executed = Arc::new(AtomicBool::new(false));
let dispatcher = Dispatcher {
outbox: Arc::new(RecordingOutbox {
deleted: deleted.clone(),
}),
triggers: Arc::new(UnusedTriggers),
scripts: Arc::new(DisabledScriptRepo { script }),
dead_letters: Arc::new(UnusedDeadLetters),
abandoned: Arc::new(UnusedAbandoned),
principals: Arc::new(OkPrincipals),
executor: Arc::new(RecordingExecutor {
executed: executed.clone(),
}),
gate: Arc::new(ExecutionGate::new(1)),
log_sink: Arc::new(UnusedLogSink),
inbox: Arc::new(UnusedInbox),
queue: Arc::new(UnusedQueue),
group_queue: Arc::new(NoopGroupQueue),
config,
instance_id: "test-instance".into(),
};
let result = dispatcher.dispatch_one(row).await;
assert!(result.is_ok(), "dispatch_one returned {result:?}");
assert_eq!(
*deleted.lock().unwrap(),
Some(row_id),
"a depth-exceeded row must be deleted (not left to re-amplify)"
);
assert!(
!executed.load(Ordering::SeqCst),
"executor ran for a depth-exceeded row — the depth gate is missing, \
so a self-writing trigger would loop unbounded"
);
}
}
}

View File

@@ -0,0 +1,186 @@
//! `/api/v1/admin/apps/{id}/docs*` — read-only docs inspection.
//!
//! Mirrors `kv_api` / `files_api` so the `pic docs … --app` CLI (and a
//! future dashboard tab) can browse stored documents without a script.
//! The per-group equivalent shipped first (`group_blobs_api`); this is
//! its per-app counterpart, filling the one admin-read gap the other
//! data-plane services already covered. **Read-only by design** — docs
//! writes go through `docs::create/update/delete` in scripts, which emit
//! change events the trigger framework depends on; an admin write would
//! bypass that, so it is deliberately out of scope (matching kv/files).
//!
//! Two operations:
//! * `GET /apps/{id}/docs?collection=<c>&cursor=&limit=` — list docs in
//! a collection (cursor-paginated).
//! * `GET /apps/{id}/docs/{collection}/{doc_id}` — fetch one document.
//!
//! Capability: `AppDocsRead`, resolved against the app loaded from the
//! path (same tier the SDK read path uses).
use std::sync::Arc;
use axum::extract::{Path, Query, State};
use axum::response::{IntoResponse, Json, Response};
use axum::routing::get;
use axum::{Extension, Router};
use picloud_shared::{AppId, Principal};
use serde::{Deserialize, Serialize};
use serde_json::json;
use uuid::Uuid;
use crate::app_repo::AppRepository;
use crate::authz::{require, AuthzDenied, AuthzRepo, Capability};
use crate::docs_repo::DocsRepo;
#[derive(Clone)]
pub struct DocsAdminState {
pub docs: Arc<dyn DocsRepo>,
pub apps: Arc<dyn AppRepository>,
pub authz: Arc<dyn AuthzRepo>,
}
pub fn docs_admin_router(state: DocsAdminState) -> Router {
Router::new()
.route("/apps/{app_id}/docs", get(list_docs))
.route("/apps/{app_id}/docs/{collection}/{doc_id}", get(get_doc))
.with_state(state)
}
#[derive(Debug, Deserialize)]
pub struct ListDocsQuery {
pub collection: String,
#[serde(default)]
pub cursor: Option<String>,
#[serde(default)]
pub limit: Option<u32>,
}
/// Mirrors the `group_blobs_api` docs shape so the CLI can share one
/// deserialize across `--app` and `--group`.
#[derive(Debug, Serialize)]
struct DocEntry {
id: String,
data: serde_json::Value,
}
#[derive(Debug, Serialize)]
struct ListDocsResponse {
docs: Vec<DocEntry>,
next_cursor: Option<String>,
}
async fn list_docs(
State(s): State<DocsAdminState>,
Extension(principal): Extension<Principal>,
Path(id_or_slug): Path<String>,
Query(q): Query<ListDocsQuery>,
) -> Result<Json<ListDocsResponse>, DocsApiError> {
let app_id = resolve_app(&*s.apps, &id_or_slug).await?;
require(
s.authz.as_ref(),
&principal,
Capability::AppDocsRead(app_id),
)
.await?;
let page = s
.docs
.list(
app_id,
&q.collection,
q.cursor.as_deref(),
q.limit.unwrap_or(0),
)
.await
.map_err(|e| DocsApiError::Backend(e.to_string()))?;
Ok(Json(ListDocsResponse {
docs: page
.docs
.into_iter()
.map(|d| DocEntry {
id: d.id.to_string(),
data: d.data,
})
.collect(),
next_cursor: page.next_cursor,
}))
}
async fn get_doc(
State(s): State<DocsAdminState>,
Extension(principal): Extension<Principal>,
Path((id_or_slug, collection, doc_id)): Path<(String, String, String)>,
) -> Result<Json<serde_json::Value>, DocsApiError> {
let app_id = resolve_app(&*s.apps, &id_or_slug).await?;
require(
s.authz.as_ref(),
&principal,
Capability::AppDocsRead(app_id),
)
.await?;
let id = doc_id.parse::<Uuid>().map_err(|_| DocsApiError::NotFound)?;
let row = s
.docs
.get(app_id, &collection, id)
.await
.map_err(|e| DocsApiError::Backend(e.to_string()))?
.ok_or(DocsApiError::NotFound)?;
Ok(Json(json!({ "id": row.id.to_string(), "data": row.data })))
}
async fn resolve_app(apps: &dyn AppRepository, ident: &str) -> Result<AppId, DocsApiError> {
crate::app_repo::resolve_app(apps, ident)
.await
.map_err(|e| DocsApiError::Backend(e.to_string()))?
.map(|l| l.app.id)
.ok_or(DocsApiError::AppNotFound)
}
#[derive(Debug, thiserror::Error)]
pub enum DocsApiError {
#[error("app not found")]
AppNotFound,
#[error("document not found")]
NotFound,
#[error("forbidden")]
Forbidden,
#[error("authorization repo error: {0}")]
AuthzRepo(String),
#[error("docs backend: {0}")]
Backend(String),
}
impl From<AuthzDenied> for DocsApiError {
fn from(d: AuthzDenied) -> Self {
match d {
AuthzDenied::Denied => Self::Forbidden,
AuthzDenied::Repo(e) => Self::AuthzRepo(e.to_string()),
}
}
}
impl IntoResponse for DocsApiError {
fn into_response(self) -> Response {
use axum::http::StatusCode;
let (status, body) = match &self {
Self::AppNotFound | Self::NotFound => {
(StatusCode::NOT_FOUND, json!({ "error": self.to_string() }))
}
Self::Forbidden => (StatusCode::FORBIDDEN, json!({ "error": self.to_string() })),
Self::AuthzRepo(e) => {
tracing::error!(error = %e, "docs admin authz error");
(
StatusCode::INTERNAL_SERVER_ERROR,
json!({ "error": "internal error" }),
)
}
Self::Backend(e) => {
tracing::error!(error = %e, "docs admin backend error");
(
StatusCode::INTERNAL_SERVER_ERROR,
json!({ "error": "internal error" }),
)
}
};
(status, Json(body)).into_response()
}
}

View File

@@ -150,6 +150,12 @@ pub enum ComparisonOp {
/// `$in` — `= ANY($M::text[])` where the value list is bound as
/// a TEXT[].
In,
/// `$contains` — JSONB containment (`@>`) for array-membership:
/// matches a doc whose stored *array* field contains the given
/// scalar element (e.g. `{ tags: { "$contains": "rust" } }` finds
/// docs tagged `rust`). The `$in` operator is the inverse (field is
/// one of a list); this one is (list field contains a value).
Contains,
}
impl ComparisonOp {
@@ -169,6 +175,7 @@ impl ComparisonOp {
"$lt" => Ok(Self::Lt),
"$lte" => Ok(Self::Lte),
"$in" => Ok(Self::In),
"$contains" => Ok(Self::Contains),
other => Err(FilterParseError::UnsupportedOperator(format!(
"docs::find: operator '{other}' is not supported in v1.1.2; planned for v1.2 advanced query"
))),
@@ -321,6 +328,7 @@ const fn op_name(op: ComparisonOp) -> &'static str {
ComparisonOp::Lt => "$lt",
ComparisonOp::Lte => "$lte",
ComparisonOp::In => "$in",
ComparisonOp::Contains => "$contains",
}
}
@@ -465,6 +473,17 @@ mod tests {
// $in needs an array.
let f = parse(json!({ "tier": { "$in": ["gold", "platinum"] } })).unwrap();
assert_eq!(f.conditions[0].op, ComparisonOp::In);
// $contains takes a scalar (the element to find in an array field).
let f = parse(json!({ "tags": { "$contains": "rust" } })).unwrap();
assert_eq!(f.conditions[0].op, ComparisonOp::Contains);
}
#[test]
fn contains_with_object_value_rejected() {
// The element to find must be a scalar, not a nested object/array.
let err = parse(json!({ "tags": { "$contains": { "k": 1 } } })).unwrap_err();
assert!(err.to_string().contains("'$contains'"));
assert!(err.to_string().contains("scalar"));
}
#[test]

View File

@@ -118,24 +118,7 @@ impl DocsRepo for PostgresDocsRepo {
collection: &str,
data: Value,
) -> Result<DocRow, DocsRepoError> {
let id = Uuid::new_v4();
let row: (DateTime<Utc>, DateTime<Utc>) = sqlx::query_as(
"INSERT INTO docs (app_id, collection, id, data) \
VALUES ($1, $2, $3, $4) \
RETURNING created_at, updated_at",
)
.bind(app_id.into_inner())
.bind(collection)
.bind(id)
.bind(&data)
.fetch_one(&self.pool)
.await?;
Ok(DocRow {
id,
data,
created_at: row.0,
updated_at: row.1,
})
create_on(&self.pool, app_id, collection, data).await
}
async fn get(
@@ -179,34 +162,7 @@ impl DocsRepo for PostgresDocsRepo {
id: DocId,
data: Value,
) -> Result<Option<Value>, DocsRepoError> {
// Same CTE shape as KV's set ([kv_repo.rs:101-132]): SELECT the
// previous data before the UPDATE so the service can emit
// `prev_data` in the update ServiceEvent. Single statement, no
// explicit transaction. Inherits KV's last-writer-wins race
// under concurrent writers; documented as a known limitation
// for v1.1.2.
let row: Option<(Option<Value>,)> = sqlx::query_as(
"WITH prev AS ( \
SELECT data FROM docs \
WHERE app_id = $1 AND collection = $2 AND id = $3 \
), \
updated AS ( \
UPDATE docs SET data = $4, updated_at = NOW() \
WHERE app_id = $1 AND collection = $2 AND id = $3 \
RETURNING 1 \
) \
SELECT (SELECT data FROM prev) FROM updated",
)
.bind(app_id.into_inner())
.bind(collection)
.bind(id)
.bind(&data)
.fetch_optional(&self.pool)
.await?;
// `row` is None when the UPDATE matched no rows (missing doc);
// Some((Some(prev),)) on success. `data` is JSONB NOT NULL so
// the inner Option is always Some when prev exists.
Ok(row.and_then(|(v,)| v))
update_on(&self.pool, app_id, collection, id, data).await
}
async fn delete(
@@ -215,17 +171,7 @@ impl DocsRepo for PostgresDocsRepo {
collection: &str,
id: DocId,
) -> Result<Option<Value>, DocsRepoError> {
let row: Option<(Value,)> = sqlx::query_as(
"DELETE FROM docs \
WHERE app_id = $1 AND collection = $2 AND id = $3 \
RETURNING data",
)
.bind(app_id.into_inner())
.bind(collection)
.bind(id)
.fetch_optional(&self.pool)
.await?;
Ok(row.map(|(v,)| v))
delete_on(&self.pool, app_id, collection, id).await
}
async fn list(
@@ -365,6 +311,20 @@ fn emit_condition<'a>(
qb: &mut QueryBuilder<'a, Postgres>,
cond: &'a crate::docs_filter::FieldCondition,
) {
// `$contains` uses JSONB containment (`@>`), which needs the field's
// *JSONB* value, not the text rendering the other operators compare
// against — so it builds its own path expression and short-circuits.
// Emits `jsonb_extract_path(data, $seg…) @> $val::jsonb`, matching a doc
// whose stored array field contains the given scalar element (F-015).
if cond.op == ComparisonOp::Contains {
push_jsonb_path_value(qb, cond.path.segments());
qb.push(" @> ");
// `Value::to_string()` is the compact JSON encoding (`"rust"`, `5`,
// `true`), i.e. a valid JSONB literal — bound, never interpolated.
qb.push_bind(cond.value.to_string());
qb.push("::jsonb");
return;
}
push_jsonb_path(qb, cond.path.segments());
match cond.op {
ComparisonOp::Eq => {
@@ -412,6 +372,7 @@ fn emit_condition<'a>(
qb.push_bind(texts);
qb.push(")");
}
ComparisonOp::Contains => unreachable!("handled before the match"),
}
}
@@ -427,6 +388,18 @@ fn push_jsonb_path<'a>(qb: &mut QueryBuilder<'a, Postgres>, segments: &'a [Strin
qb.push(")");
}
/// Like [`push_jsonb_path`] but returns the *JSONB* value rather than its
/// text rendering (`jsonb_extract_path`, no `_text`). Used by `$contains`,
/// whose `@>` operator needs a JSONB left operand. Segments bound as params.
fn push_jsonb_path_value<'a>(qb: &mut QueryBuilder<'a, Postgres>, segments: &'a [String]) {
qb.push("jsonb_extract_path(data");
for seg in segments {
qb.push(", ");
qb.push_bind(seg.as_str());
}
qb.push(")");
}
/// JSON scalar → TEXT for binding. `Value::Null` is preserved as
/// `None` so the binding lands as SQL NULL (handled specially above for
/// `Eq` / `Ne`). Arrays + objects serialize to compact JSON; the user
@@ -448,6 +421,113 @@ fn value_to_text(v: &Value) -> Option<String> {
// pin the cross-app isolation invariant at the SQL level.
// ----------------------------------------------------------------------------
// ----------------------------------------------------------------------------
// Connection-scoped mutations
//
// Generic over the executor so the same SQL serves the pooled trait methods
// above and `crate::atomic_write`, which runs the write and the trigger fan-out
// it produces on ONE connection inside ONE transaction.
// ----------------------------------------------------------------------------
pub(crate) async fn create_on<'c, E>(
exec: E,
app_id: AppId,
collection: &str,
data: Value,
) -> Result<DocRow, DocsRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let id = Uuid::new_v4();
let row: (DateTime<Utc>, DateTime<Utc>) = sqlx::query_as(
"INSERT INTO docs (app_id, collection, id, data) \
VALUES ($1, $2, $3, $4) \
RETURNING created_at, updated_at",
)
.bind(app_id.into_inner())
.bind(collection)
.bind(id)
.bind(&data)
.fetch_one(exec)
.await?;
Ok(DocRow {
id,
data,
created_at: row.0,
updated_at: row.1,
})
}
/// Returns the previous data (for the update event), `None` if the doc is
/// missing. The CTE captures the prior data alongside the UPDATE so the service
/// can emit `prev_data` without a second round trip.
pub(crate) async fn update_on<'c, E>(
exec: E,
app_id: AppId,
collection: &str,
id: DocId,
data: Value,
) -> Result<Option<Value>, DocsRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let row: Option<(Option<Value>,)> = sqlx::query_as(
"WITH prev AS ( \
SELECT data FROM docs \
WHERE app_id = $1 AND collection = $2 AND id = $3 \
), \
updated AS ( \
UPDATE docs SET data = $4, updated_at = NOW() \
WHERE app_id = $1 AND collection = $2 AND id = $3 \
RETURNING 1 \
) \
SELECT (SELECT data FROM prev) FROM updated",
)
.bind(app_id.into_inner())
.bind(collection)
.bind(id)
.bind(&data)
.fetch_optional(exec)
.await?;
Ok(row.and_then(|(v,)| v))
}
/// Returns the deleted doc's data if it existed, `None` if no such doc.
pub(crate) async fn delete_on<'c, E>(
exec: E,
app_id: AppId,
collection: &str,
id: DocId,
) -> Result<Option<Value>, DocsRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let row: Option<(Value,)> = sqlx::query_as(
"DELETE FROM docs \
WHERE app_id = $1 AND collection = $2 AND id = $3 \
RETURNING data",
)
.bind(app_id.into_inner())
.bind(collection)
.bind(id)
.fetch_optional(exec)
.await?;
Ok(row.map(|(v,)| v))
}
/// Total doc count for the app, across all its collections. Backs the per-app
/// row ceiling; run only on `create`.
pub(crate) async fn count_rows_on<'c, E>(exec: E, app_id: AppId) -> Result<u64, DocsRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let (n,): (i64,) = sqlx::query_as("SELECT COUNT(*) FROM docs WHERE app_id = $1")
.bind(app_id.into_inner())
.fetch_one(exec)
.await?;
Ok(u64::try_from(n).unwrap_or(0))
}
#[cfg(test)]
mod sql_shape_tests {
use super::*;
@@ -481,6 +561,7 @@ mod sql_shape_tests {
json!({ "$sort": { "created_at": -1 }, "$limit": 5 }),
json!({ "tier": "gold", "$sort": { "created_at": 1 } }),
json!({ "deleted_at": { "$ne": null } }),
json!({ "tags": { "$contains": "rust" } }),
];
for case in cases {
let sql = sql_for(case.clone());
@@ -567,4 +648,16 @@ mod sql_shape_tests {
let sql = sql_for(json!({ "user.email": "a@b" }));
assert!(sql.contains("jsonb_extract_path_text(data"), "sql: {sql}");
}
#[test]
fn contains_emits_jsonb_containment() {
// `$contains` must use the JSONB (not text) extractor + `@>`, with the
// element bound as a `::jsonb` parameter — never interpolated (F-015).
let sql = sql_for(json!({ "tags": { "$contains": "rust" } }));
assert!(sql.contains("jsonb_extract_path(data"), "sql: {sql}");
assert!(!sql.contains("jsonb_extract_path_text(data"), "sql: {sql}");
assert!(sql.contains(" @> "), "sql: {sql}");
assert!(sql.contains("::jsonb"), "sql: {sql}");
assert!(!sql.contains("rust"), "value leaked into SQL string: {sql}");
}
}

View File

@@ -26,10 +26,10 @@ use std::sync::Arc;
use async_trait::async_trait;
use picloud_shared::{
DocId, DocRow, DocsError, DocsListPage, DocsService, SdkCallCx, ServiceEvent,
ServiceEventEmitter,
DocId, DocRow, DocsError, DocsListPage, DocsService, SdkCallCx, ServiceEventEmitter,
};
use crate::atomic_write::{BestEffortDocsWriter, DocsWriter, PostgresDocsWriter};
use crate::authz::{self, AuthzRepo, Capability};
use crate::docs_filter::{parse_filter, FilterParseError};
use crate::docs_repo::{DocsRepo, DocsRepoError};
@@ -55,9 +55,11 @@ pub fn docs_max_value_bytes_from_env() -> usize {
}
pub struct DocsServiceImpl {
/// Reads only. Mutations go through `writer`, which owns the write AND the
/// trigger fan-out so the two can share a transaction.
repo: Arc<dyn DocsRepo>,
authz: Arc<dyn AuthzRepo>,
events: Arc<dyn ServiceEventEmitter>,
writer: Arc<dyn DocsWriter>,
max_value_bytes: usize,
}
@@ -79,13 +81,23 @@ impl DocsServiceImpl {
max_value_bytes: usize,
) -> Self {
Self {
writer: Arc::new(BestEffortDocsWriter::new(repo.clone(), events)),
repo,
authz,
events,
max_value_bytes,
}
}
/// Swap the best-effort writer for the transactional one: the write and its
/// trigger fan-out then commit together, so an outbox failure rolls the
/// write back instead of silently losing the event. The host always calls
/// this; the in-memory unit tests do not.
#[must_use]
pub fn with_atomic_writes(mut self, pool: sqlx::PgPool, max_rows: u64) -> Self {
self.writer = Arc::new(PostgresDocsWriter::new(pool, max_rows));
self
}
fn check_data_size(&self, data: &serde_json::Value) -> Result<(), DocsError> {
let encoded_len = serde_json::to_vec(data)
.map(|v| v.len())
@@ -163,30 +175,7 @@ impl DocsService for DocsServiceImpl {
validate_data(&data)?;
self.check_data_size(&data)?;
self.check_write(cx).await?;
let row = self
.repo
.create(cx.app_id, collection, data.clone())
.await?;
// Best-effort emit — a failed emit logs but does not roll back
// the write (mirrors KV's pattern).
if let Err(e) = self
.events
.emit(
cx,
ServiceEvent {
source: "docs",
op: "create",
collection: Some(collection.to_string()),
key: Some(row.id.to_string()),
payload: Some(data),
old_payload: None,
},
)
.await
{
tracing::error!(error = %e, source = "docs", op = "create", event_emit_failure = true, "event emit failed");
}
Ok(row.id)
self.writer.create(cx, collection, data).await
}
async fn get(
@@ -241,60 +230,17 @@ impl DocsService for DocsServiceImpl {
validate_data(&data)?;
self.check_data_size(&data)?;
self.check_write(cx).await?;
let previous = self
.repo
.update(cx.app_id, collection, id, data.clone())
.await?;
match previous {
Some(prev) => {
if let Err(e) = self
.events
.emit(
cx,
ServiceEvent {
source: "docs",
op: "update",
collection: Some(collection.to_string()),
key: Some(id.to_string()),
payload: Some(data),
old_payload: Some(prev),
},
)
.await
{
tracing::error!(error = %e, source = "docs", op = "update", event_emit_failure = true, "event emit failed");
}
Ok(())
}
None => Err(DocsError::NotFound),
if self.writer.update(cx, collection, id, data).await? {
Ok(())
} else {
Err(DocsError::NotFound)
}
}
async fn delete(&self, cx: &SdkCallCx, collection: &str, id: DocId) -> Result<bool, DocsError> {
validate_collection(collection)?;
self.check_write(cx).await?;
let previous = self.repo.delete(cx.app_id, collection, id).await?;
let was_present = previous.is_some();
if let Some(prev) = previous {
if let Err(e) = self
.events
.emit(
cx,
ServiceEvent {
source: "docs",
op: "delete",
collection: Some(collection.to_string()),
key: Some(id.to_string()),
payload: None,
old_payload: Some(prev),
},
)
.await
{
tracing::error!(error = %e, source = "docs", op = "delete", event_emit_failure = true, "event emit failed");
}
}
Ok(was_present)
self.writer.delete(cx, collection, id).await
}
async fn list(
@@ -506,6 +452,22 @@ mod tests {
};
arr.iter().any(|v| actual == json_text(v).as_deref())
}
Contains => {
// `actual` is the text rendering of the stored field; for an
// array field that's compact JSON. Parse it back and test
// element membership, mirroring Postgres `@>` (a scalar field
// equal to the value also "contains" it).
let Some(text) = actual else {
return false;
};
match serde_json::from_str::<serde_json::Value>(text) {
Ok(serde_json::Value::Array(items)) => {
items.iter().any(|v| json_text(v).as_deref() == want_ref)
}
Ok(other) => json_text(&other).as_deref() == want_ref,
Err(_) => false,
}
}
}
}
@@ -883,6 +845,35 @@ mod tests {
assert_eq!(hits.len(), 2);
}
#[tokio::test]
async fn find_with_contains_matches_array_membership() {
// F-015: `$contains` finds docs whose ARRAY field holds the element —
// the "posts with tag X" query that `$in` cannot express.
let s = svc();
let cx = anon_cx(AppId::new());
s.create(&cx, "posts", json!({ "tags": ["intro", "rust"] }))
.await
.unwrap();
s.create(&cx, "posts", json!({ "tags": ["rust", "async"] }))
.await
.unwrap();
s.create(&cx, "posts", json!({ "tags": ["cooking"] }))
.await
.unwrap();
let hits = s
.find(&cx, "posts", json!({ "tags": { "$contains": "rust" } }))
.await
.unwrap();
assert_eq!(hits.len(), 2, "expected the two rust-tagged posts");
let none = s
.find(&cx, "posts", json!({ "tags": { "$contains": "python" } }))
.await
.unwrap();
assert!(none.is_empty());
}
#[tokio::test]
async fn find_one_explicit_limit_is_honoured() {
// The service injects limit=1 ONLY when caller didn't set
@@ -940,4 +931,64 @@ mod tests {
s.update(&cx, "users", id, json!({ "x": 2 })).await.unwrap();
let _ = s.delete(&cx, "users", id).await.unwrap();
}
/// `PICLOUD_DOCS_MAX_VALUE_BYTES` — the docs analogue of the KV anti-DoS rail.
/// Pins that an oversized document is rejected BEFORE authz, and covers create
/// AND update (both encode-and-check). The cx is an authenticated member with
/// no role (authz-denied) so the ordering is observable: size-first returns
/// `ValueTooLarge`, authz-first would return `Forbidden`. An anon cx couldn't
/// tell the two apart.
#[tokio::test]
async fn oversized_document_is_rejected_before_authz() {
let s = DocsServiceImpl::with_max_value_bytes(
Arc::new(InMemoryDocsRepo::default()),
Arc::new(DenyingAuthzRepo),
Arc::new(NoopEventEmitter),
16,
);
let cx = member_no_role_cx(AppId::new());
let err = s
.create(&cx, "users", json!({ "blob": "x".repeat(100) }))
.await
.unwrap_err();
assert!(
matches!(err, DocsError::ValueTooLarge { limit: 16, .. }),
"an oversized create must be ValueTooLarge before authz; got {err:?}"
);
// Control: an under-cap create with the same cx is Forbidden — the cx is
// genuinely authz-denied, so the case above bypassed authz. And it seeds a
// doc via an ALLOWING service so we can then test the update path.
assert!(
matches!(
s.create(&cx, "users", json!({ "x": 1 })).await.unwrap_err(),
DocsError::Forbidden
),
"the control confirms this cx is authz-denied"
);
// Update also encodes-and-checks before authz — an update that skipped the
// cap would be a free bypass. Seed with an allowing service, then update
// through the denied one.
let seeder = DocsServiceImpl::with_max_value_bytes(
Arc::new(InMemoryDocsRepo::default()),
Arc::new(AllowingAuthzRepo),
Arc::new(NoopEventEmitter),
16,
);
let owner = owner_cx(AppId::new());
let id = seeder
.create(&owner, "users", json!({ "x": 1 }))
.await
.unwrap();
let err = seeder
.update(&owner, "users", id, json!({ "blob": "x".repeat(100) }))
.await
.unwrap_err();
assert!(
matches!(err, DocsError::ValueTooLarge { limit: 16, .. }),
"an oversized update must be ValueTooLarge too; got {err:?}"
);
}
}

View File

@@ -36,7 +36,7 @@ use serde_json::json;
use sha2::{Digest, Sha256};
use crate::outbox_repo::{NewOutboxRow, OutboxRepo, OutboxSourceKind};
use crate::secrets_service::open_legacy;
use crate::secrets_service::{open_email, SecretOwner};
use crate::trigger_repo::TriggerRepo;
type HmacSha256 = Hmac<Sha256>;
@@ -244,7 +244,20 @@ async fn receive_inbound_email(
s.bad_sig_limiter.record_failure(app_id, trigger_id);
return Err(EmailInboundError::Unauthorized);
};
let secret = decrypt_secret(&s.master_key, ct, nonce)?;
// §M5.5 / audit H-D1: recover the SEALING owner for a v1 (AAD-bound) secret —
// the source-template group for a materialized copy, else this app. v0 rows
// ignore the owner (legacy no-AAD path).
let sealing_owner = match target.sealing_group {
Some(group_id) => SecretOwner::Group(group_id),
None => SecretOwner::App(app_id),
};
let secret = decrypt_secret(
&s.master_key,
sealing_owner,
ct,
nonce,
target.inbound_secret_version,
)?;
if let Err(err) = verify_signature(&headers, &body, secret.as_bytes(), &s.nonce_dedup) {
s.bad_sig_limiter.record_failure(app_id, trigger_id);
return Err(err);
@@ -286,16 +299,18 @@ async fn receive_inbound_email(
Ok(StatusCode::ACCEPTED)
}
/// Decrypt the stored inbound secret back to its raw string. It was
/// sealed as a JSON string by the admin layer (v0, no AAD — see
/// secrets_service::seal_legacy), so `open_legacy` yields a
/// Decrypt the stored inbound secret back to its raw string. Sealed as a JSON
/// string by the admin/apply layer; `open_email` dispatches on `version` (v0 =
/// legacy no-AAD, v1 = AAD bound to the sealing `owner`), yielding a
/// `Value::String`.
fn decrypt_secret(
master_key: &MasterKey,
owner: SecretOwner,
ciphertext: &[u8],
nonce: &[u8],
version: i16,
) -> Result<String, EmailInboundError> {
let value = open_legacy(master_key, ciphertext, nonce).map_err(|_| {
let value = open_email(master_key, owner, ciphertext, nonce, version).map_err(|_| {
// Corrupted secret means we can't verify — fail closed (401).
EmailInboundError::Unauthorized
})?;
@@ -421,8 +436,9 @@ mod tests {
//! Postgres in `crates/picloud/tests/email_inbound.rs`.
use super::*;
use crate::secrets_service::seal_legacy;
use crate::secrets_service::DEFAULT_SECRET_MAX_VALUE_BYTES;
use crate::secrets_service::{seal_email, seal_legacy};
use picloud_shared::{AppId, GroupId};
fn sign(secret: &[u8], ts: i64, body: &[u8]) -> String {
let mut mac = HmacSha256::new_from_slice(secret).unwrap();
@@ -520,14 +536,36 @@ mod tests {
#[test]
fn secret_round_trips_through_seal_open() {
let key = MasterKey::from_bytes([3u8; 32]);
let (ct, nonce) = seal_legacy(
// v0 (legacy no-AAD) still opens — backward compatibility for pre-M3 rows.
let (ct0, nonce0) = seal_legacy(
&key,
&serde_json::Value::String("provider-secret".into()),
DEFAULT_SECRET_MAX_VALUE_BYTES,
)
.unwrap();
let recovered = decrypt_secret(&key, &ct, &nonce).unwrap();
let owner = SecretOwner::App(AppId::from(uuid::Uuid::from_u128(1)));
let recovered = decrypt_secret(&key, owner, &ct0, &nonce0, 0).unwrap();
assert_eq!(recovered, "provider-secret");
// v1 (AAD bound to the sealing owner) round-trips under the SAME owner.
let (ct1, nonce1, ver) = seal_email(
&key,
owner,
&serde_json::Value::String("provider-secret".into()),
DEFAULT_SECRET_MAX_VALUE_BYTES,
)
.unwrap();
assert_eq!(ver, 1);
assert_eq!(
decrypt_secret(&key, owner, &ct1, &nonce1, ver).unwrap(),
"provider-secret"
);
// A DIFFERENT owner (cross-tenant relocation) fails the GCM tag → 401.
let other = SecretOwner::Group(GroupId::from(uuid::Uuid::from_u128(2)));
assert!(
decrypt_secret(&key, other, &ct1, &nonce1, ver).is_err(),
"a v1 secret must not open under a different sealing owner"
);
let body = br#"{"from":"x@y.com"}"#;
let ts = now_ts();
let sig = sign(recovered.as_bytes(), ts, body);

View File

@@ -163,9 +163,12 @@ async fn get_file(
.await?
.ok_or(FilesApiError::NotFound)?;
let safe_ct = picloud_shared::sanitize_stored_content_type(&meta.content_type);
// The stored name is upload-supplied; sanitize to header-safe ASCII so the
// response builder can never error on a control byte (which `.expect()`
// below would turn into a per-request panic).
let disposition = format!(
"attachment; filename=\"{}\"",
meta.name.replace('"', "").replace(['\r', '\n'], "")
picloud_shared::sanitize_stored_filename(&meta.name)
);
let len = bytes.len();
Ok(Response::builder()

View File

@@ -209,11 +209,6 @@ impl FsFilesRepo {
}
Ok(())
}
fn final_path(&self, app_id: AppId, collection: &str, id: Uuid) -> PathBuf {
final_path_at(&self.config.root, &app_owner_dir(app_id), collection, id)
}
fn write_atomic(
&self,
app_id: AppId,
@@ -468,42 +463,14 @@ impl FilesRepo for FsFilesRepo {
id: Uuid,
) -> Result<Option<FileMeta>, FilesRepoError> {
Self::guard_collection(collection)?;
// SELECT + DELETE in one tx; unlink afterwards (outside the tx).
let mut tx = self.pool.begin().await?;
let row: Option<FileRow> = sqlx::query_as(
"SELECT id, collection, name, content_type, size_bytes, \
checksum_sha256, created_at, updated_at \
FROM files WHERE app_id = $1 AND collection = $2 AND id = $3 \
FOR UPDATE",
)
.bind(app_id.into_inner())
.bind(collection)
.bind(id)
.fetch_optional(&mut *tx)
.await?;
let Some(row) = row else {
tx.rollback().await?;
return Ok(None);
};
sqlx::query("DELETE FROM files WHERE app_id = $1 AND collection = $2 AND id = $3")
.bind(app_id.into_inner())
.bind(collection)
.bind(id)
.execute(&mut *tx)
.await?;
tx.commit().await?;
// Row is gone; unlink the bytes. A failure here leaves an orphan
// file (reclaimed by a future sweep) — not fatal.
let path = self.final_path(app_id, collection, id);
if let Err(e) = std::fs::remove_file(&path) {
if e.kind() != std::io::ErrorKind::NotFound {
tracing::warn!(path = %path.display(), error = %e, "files: unlink after delete failed (orphan)");
}
// `DELETE ... RETURNING` is one statement, so the old SELECT-FOR-UPDATE
// + DELETE pair is unnecessary. Unlink only AFTER the row is gone: the
// reverse order would destroy the bytes of a row that a failure keeps.
let meta = delete_meta_on(&self.pool, app_id, collection, id).await?;
if meta.is_some() {
unlink_blob(&self.config.root, &app_owner_dir(app_id), collection, id);
}
Ok(Some(row.into_meta()))
Ok(meta)
}
async fn list(
@@ -599,7 +566,7 @@ pub(crate) fn decode_cursor(cursor: &str) -> Result<Uuid, FilesRepoError> {
}
#[derive(sqlx::FromRow)]
struct FileRow {
pub(crate) struct FileRow {
id: Uuid,
collection: String,
name: String,
@@ -611,7 +578,7 @@ struct FileRow {
}
impl FileRow {
fn into_meta(self) -> FileMeta {
pub(crate) fn into_meta(self) -> FileMeta {
FileMeta {
id: self.id,
collection: self.collection,
@@ -625,6 +592,194 @@ impl FileRow {
}
}
// ----------------------------------------------------------------------------
// Connection-scoped metadata operations
//
// The BYTES are written to disk outside any transaction (they are not
// transactional and cannot be). What these give `crate::atomic_write` is a way
// to commit the metadata row and the trigger fan-out it produces TOGETHER, so a
// committed file row always has its event enqueued.
//
// A rollback after the bytes are on disk leaves an orphan blob. That hazard is
// not new — the pre-existing repo already wrote the blob, then inserted the row
// in a separate statement that could fail — and the writer now explicitly
// unlinks the blob on the rollback path, so the window is strictly smaller than
// it was.
// ----------------------------------------------------------------------------
/// The four metadata columns a blob write sets. Bundled so the `*_meta_on`
/// helpers take a target (owner, collection, id) plus one payload, rather than a
/// long positional tail it would be easy to transpose.
#[derive(Clone, Copy)]
pub(crate) struct MetaFields<'a> {
pub name: &'a str,
pub content_type: &'a str,
pub size: i64,
pub checksum: &'a str,
}
/// The same, for an update: `name`/`content_type` are `None` to keep the stored
/// value (`COALESCE`), while the bytes always change.
#[derive(Clone, Copy)]
pub(crate) struct MetaPatch<'a> {
pub name: Option<&'a str>,
pub content_type: Option<&'a str>,
pub size: i64,
pub checksum: &'a str,
}
const FILE_COLS: &str = "id, collection, name, content_type, size_bytes, \
checksum_sha256, created_at, updated_at";
pub(crate) async fn head_on<'c, E>(
exec: E,
app_id: AppId,
collection: &str,
id: Uuid,
) -> Result<Option<FileMeta>, FilesRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let row: Option<FileRow> = sqlx::query_as(&format!(
"SELECT {FILE_COLS} FROM files \
WHERE app_id = $1 AND collection = $2 AND id = $3"
))
.bind(app_id.into_inner())
.bind(collection)
.bind(id)
.fetch_optional(exec)
.await?;
Ok(row.map(FileRow::into_meta))
}
pub(crate) async fn insert_meta_on<'c, E>(
exec: E,
app_id: AppId,
collection: &str,
id: Uuid,
meta: MetaFields<'_>,
) -> Result<FileMeta, FilesRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let row: FileRow = sqlx::query_as(&format!(
"INSERT INTO files \
(app_id, collection, id, name, content_type, size_bytes, checksum_sha256) \
VALUES ($1, $2, $3, $4, $5, $6, $7) \
RETURNING {FILE_COLS}"
))
.bind(app_id.into_inner())
.bind(collection)
.bind(id)
.bind(meta.name)
.bind(meta.content_type)
.bind(meta.size)
.bind(meta.checksum)
.fetch_one(exec)
.await?;
Ok(row.into_meta())
}
/// `None` when the file row is gone (the caller maps that to `NotFound`).
pub(crate) async fn update_meta_on<'c, E>(
exec: E,
app_id: AppId,
collection: &str,
id: Uuid,
meta: MetaPatch<'_>,
) -> Result<Option<FileMeta>, FilesRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let row: Option<FileRow> = sqlx::query_as(&format!(
"UPDATE files SET \
name = COALESCE($4, name), \
content_type = COALESCE($5, content_type), \
size_bytes = $6, \
checksum_sha256 = $7, \
updated_at = NOW() \
WHERE app_id = $1 AND collection = $2 AND id = $3 \
RETURNING {FILE_COLS}"
))
.bind(app_id.into_inner())
.bind(collection)
.bind(id)
.bind(meta.name)
.bind(meta.content_type)
.bind(meta.size)
.bind(meta.checksum)
.fetch_optional(exec)
.await?;
Ok(row.map(FileRow::into_meta))
}
/// `DELETE ... RETURNING` — one statement, so no `SELECT ... FOR UPDATE` dance.
/// The caller unlinks the bytes AFTER the transaction commits: unlinking first
/// would destroy the blob of a row that a rollback then keeps.
pub(crate) async fn delete_meta_on<'c, E>(
exec: E,
app_id: AppId,
collection: &str,
id: Uuid,
) -> Result<Option<FileMeta>, FilesRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let row: Option<FileRow> = sqlx::query_as(&format!(
"DELETE FROM files \
WHERE app_id = $1 AND collection = $2 AND id = $3 \
RETURNING {FILE_COLS}"
))
.bind(app_id.into_inner())
.bind(collection)
.bind(id)
.fetch_optional(exec)
.await?;
Ok(row.map(FileRow::into_meta))
}
/// Best-effort unlink of a blob whose metadata write was rolled back (or whose
/// row was just deleted). A failure leaves an orphan — logged, never fatal.
pub(crate) fn unlink_blob(root: &Path, owner_rel: &Path, collection: &str, id: Uuid) {
let path = final_path_at(root, owner_rel, collection, id);
if let Err(e) = std::fs::remove_file(&path) {
if e.kind() != std::io::ErrorKind::NotFound {
tracing::warn!(
path = %path.display(), error = %e,
"files: unlink failed (orphan blob left on disk)"
);
}
}
}
/// The PROJECTED total stored bytes for the app AFTER this write — the current
/// SUM, minus the bytes of the file being replaced (`replacing = Some(id)` on
/// update, `None` on create), plus the incoming blob's. Backs the per-app disk
/// ceiling; the subtraction is what lets an update near the cap still go through.
pub(crate) async fn projected_total_bytes_on<'c, E>(
exec: E,
app_id: AppId,
replacing: Option<Uuid>,
incoming: i64,
) -> Result<u64, FilesRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let (n,): (i64,) = sqlx::query_as(
"SELECT ( \
COALESCE((SELECT SUM(size_bytes) FROM files WHERE app_id = $1), 0) \
- COALESCE((SELECT size_bytes FROM files WHERE app_id = $1 AND id = $2), 0) \
+ $3 \
)::BIGINT",
)
.bind(app_id.into_inner())
.bind(replacing)
.bind(incoming)
.fetch_one(exec)
.await?;
Ok(u64::try_from(n).unwrap_or(0))
}
#[cfg(test)]
mod tests {
use super::*;

View File

@@ -18,17 +18,18 @@ use std::sync::Arc;
use async_trait::async_trait;
use picloud_shared::{
sanitize_stored_content_type, validate_files_collection, FileMeta, FileUpdate, FilesError,
FilesListPage, FilesService, NewFile, SdkCallCx, ServiceEvent, ServiceEventEmitter,
FilesListPage, FilesService, NewFile, SdkCallCx, ServiceEventEmitter,
};
use uuid::Uuid;
use crate::atomic_write::{BestEffortFilesWriter, FilesWriter, PostgresFilesWriter};
use crate::authz::{self, AuthzRepo, Capability};
use crate::files_repo::{FileUpdated, FilesRepo, FilesRepoError};
use crate::files_repo::{FilesRepo, FilesRepoError};
pub struct FilesServiceImpl {
repo: Arc<dyn FilesRepo>,
authz: Arc<dyn AuthzRepo>,
events: Arc<dyn ServiceEventEmitter>,
writer: Arc<dyn FilesWriter>,
max_file_size_bytes: usize,
}
@@ -41,13 +42,28 @@ impl FilesServiceImpl {
max_file_size_bytes: usize,
) -> Self {
Self {
writer: Arc::new(BestEffortFilesWriter::new(repo.clone(), events)),
repo,
authz,
events,
max_file_size_bytes,
}
}
/// Swap the best-effort writer for the transactional one: the metadata row
/// and the trigger fan-out then commit together, so an outbox failure rolls
/// the metadata back (and unlinks the blob) instead of silently losing the
/// event. The host always calls this; the in-memory unit tests do not.
#[must_use]
pub fn with_atomic_writes(
mut self,
pool: sqlx::PgPool,
root: std::path::PathBuf,
max_total_bytes: u64,
) -> Self {
self.writer = Arc::new(PostgresFilesWriter::new(pool, root, max_total_bytes));
self
}
async fn check_read(&self, cx: &SdkCallCx) -> Result<(), FilesError> {
authz::script_gate(
&*self.authz,
@@ -69,37 +85,6 @@ impl FilesServiceImpl {
)
.await
}
/// Best-effort `ServiceEvent` emission. A failed emit is logged but
/// never rolls back the (already-durable) file write.
async fn emit(
&self,
cx: &SdkCallCx,
op: &'static str,
collection: &str,
meta: &FileMeta,
old: Option<&FileMeta>,
) {
let payload = serde_json::to_value(meta).ok();
let old_payload = old.and_then(|m| serde_json::to_value(m).ok());
if let Err(e) = self
.events
.emit(
cx,
ServiceEvent {
source: "files",
op,
collection: Some(collection.to_string()),
key: Some(meta.id.to_string()),
payload,
old_payload,
},
)
.await
{
tracing::error!(error = %e, source = "files", op, event_emit_failure = true, "event emit failed");
}
}
}
/// Parse a script-supplied id. Invalid UUIDs aren't an error shape the
@@ -132,9 +117,7 @@ impl FilesService for FilesServiceImpl {
// Audit 2026-06-11 C-2 — coerce dangerous render types to
// application/octet-stream after the shape checks pass.
new.content_type = sanitize_stored_content_type(&new.content_type);
let meta = self.repo.create(cx.app_id, collection, new).await?;
self.emit(cx, "create", collection, &meta, None).await;
Ok(meta.id)
self.writer.create(cx, collection, new).await
}
async fn head(
@@ -182,12 +165,10 @@ impl FilesService for FilesServiceImpl {
let Some(uuid) = parse_id(id) else {
return Err(FilesError::NotFound);
};
match self.repo.update(cx.app_id, collection, uuid, upd).await? {
Some(FileUpdated { new, prev }) => {
self.emit(cx, "update", collection, &new, Some(&prev)).await;
Ok(())
}
None => Err(FilesError::NotFound),
if self.writer.update(cx, collection, uuid, upd).await? {
Ok(())
} else {
Err(FilesError::NotFound)
}
}
@@ -197,16 +178,7 @@ impl FilesService for FilesServiceImpl {
let Some(uuid) = parse_id(id) else {
return Ok(false);
};
match self.repo.delete(cx.app_id, collection, uuid).await? {
Some(meta) => {
// On delete, the top-level metadata AND `prev` both carry
// the deleted row (per docs/v1.1.x design + the brief).
self.emit(cx, "delete", collection, &meta, Some(&meta))
.await;
Ok(true)
}
None => Ok(false),
}
self.writer.delete(cx, collection, uuid).await
}
async fn list(
@@ -232,6 +204,7 @@ impl FilesService for FilesServiceImpl {
mod tests {
use super::*;
use crate::authz::{AuthzError, AuthzRepo};
use crate::files_repo::FileUpdated;
use async_trait::async_trait;
use chrono::Utc;
use picloud_shared::{

View File

@@ -14,7 +14,8 @@
use std::sync::Arc;
use axum::extract::{Path, Query, State};
use axum::http::header::{CONTENT_LENGTH, CONTENT_TYPE};
use axum::http::header::{CONTENT_DISPOSITION, CONTENT_LENGTH, CONTENT_TYPE};
use axum::http::StatusCode;
use axum::response::{IntoResponse, Json, Response};
use axum::routing::get;
use axum::{Extension, Router};
@@ -24,16 +25,23 @@ use serde_json::json;
use uuid::Uuid;
use crate::authz::{require, AuthzDenied, AuthzRepo, Capability};
use crate::group_dead_letter_repo::GroupDeadLetterRepo;
use crate::group_docs_repo::GroupDocsRepo;
use crate::group_files_repo::GroupFilesRepo;
use crate::group_kv_repo::GroupKvRepo;
use crate::group_repo::GroupRepository;
/// Default/max page size for the dead-letters listing (an operator view, not a
/// bulk export).
const DEAD_LETTERS_LIMIT_DEFAULT: i64 = 100;
const DEAD_LETTERS_LIMIT_MAX: i64 = 500;
#[derive(Clone)]
pub struct GroupBlobsState {
pub kv: Arc<dyn GroupKvRepo>,
pub docs: Arc<dyn GroupDocsRepo>,
pub files: Arc<dyn GroupFilesRepo>,
pub dead_letters: Arc<dyn GroupDeadLetterRepo>,
pub groups: Arc<dyn GroupRepository>,
pub authz: Arc<dyn AuthzRepo>,
}
@@ -46,6 +54,7 @@ pub fn group_blobs_router(state: GroupBlobsState) -> Router {
.route("/groups/{id}/docs/{collection}/{doc_id}", get(get_doc))
.route("/groups/{id}/files", get(list_files))
.route("/groups/{id}/files/{collection}/{file_id}", get(get_file))
.route("/groups/{id}/dead-letters", get(list_dead_letters))
.with_state(state)
}
@@ -64,6 +73,44 @@ struct ListKeysResponse {
next_cursor: Option<String>,
}
#[derive(Debug, Deserialize)]
pub struct DeadLettersQuery {
#[serde(default)]
pub unresolved: bool,
#[serde(default)]
pub limit: Option<u32>,
}
/// One group dead-letter, as returned by the operator listing.
#[derive(Debug, Serialize)]
struct DeadLetterDto {
id: Uuid,
collection: String,
source: String,
op: String,
attempt_count: u32,
last_error: String,
created_at: String,
resolved_at: Option<String>,
payload: serde_json::Value,
}
impl From<crate::group_dead_letter_repo::GroupDeadLetterRow> for DeadLetterDto {
fn from(r: crate::group_dead_letter_repo::GroupDeadLetterRow) -> Self {
Self {
id: r.id.into_inner(),
collection: r.collection,
source: r.source,
op: r.op,
attempt_count: r.attempt_count,
last_error: r.last_error,
created_at: r.created_at.to_rfc3339(),
resolved_at: r.resolved_at.map(|t| t.to_rfc3339()),
payload: r.payload,
}
}
}
async fn list_kv(
State(s): State<GroupBlobsState>,
Extension(principal): Extension<Principal>,
@@ -241,14 +288,57 @@ async fn get_file(
.await
.map_err(|e| GroupBlobsError::Backend(e.to_string()))?
.ok_or(GroupBlobsError::NotFound)?;
Ok((
[
(CONTENT_TYPE, meta.content_type),
(CONTENT_LENGTH, bytes.len().to_string()),
],
bytes,
// Same download hardening as the per-app path (`files_api::get_file`): a
// sanitized content-type, forced `attachment` disposition with a header-safe
// filename, plus nosniff/CSP so a stored HTML/SVG shared file can't render
// inline (stored-XSS) and no attacker-influenced byte can panic the builder.
let safe_ct = picloud_shared::sanitize_stored_content_type(&meta.content_type);
let disposition = format!(
"attachment; filename=\"{}\"",
picloud_shared::sanitize_stored_filename(&meta.name)
);
let len = bytes.len();
Response::builder()
.status(StatusCode::OK)
.header(CONTENT_TYPE, safe_ct)
.header(CONTENT_DISPOSITION, disposition)
.header(CONTENT_LENGTH, len)
.header("X-Content-Type-Options", "nosniff")
.header(
"Content-Security-Policy",
"default-src 'none'; sandbox; frame-ancestors 'none'",
)
.header("Referrer-Policy", "no-referrer")
.body(axum::body::Body::from(bytes))
.map_err(|e| GroupBlobsError::Backend(e.to_string()))
}
/// §11.6 D3: list a group's shared-queue dead-letters (newest-first). Guarded by
/// `GroupKvRead` (the same read tier as the shared collections above; no new
/// capability). `?unresolved=true` filters to still-open rows; `?limit=` caps
/// the page (default 100, max 500).
async fn list_dead_letters(
State(s): State<GroupBlobsState>,
Extension(principal): Extension<Principal>,
Path(ident): Path<String>,
Query(q): Query<DeadLettersQuery>,
) -> Result<Json<Vec<DeadLetterDto>>, GroupBlobsError> {
let group_id = resolve_group(&*s.groups, &ident).await?;
require(
s.authz.as_ref(),
&principal,
Capability::GroupKvRead(group_id),
)
.into_response())
.await?;
let limit = q.limit.map_or(DEAD_LETTERS_LIMIT_DEFAULT, |n| {
i64::from(n).clamp(1, DEAD_LETTERS_LIMIT_MAX)
});
let rows = s
.dead_letters
.list_for_group(group_id, q.unresolved, limit)
.await
.map_err(|e| GroupBlobsError::Backend(e.to_string()))?;
Ok(Json(rows.into_iter().map(DeadLetterDto::from).collect()))
}
async fn resolve_group(

View File

@@ -17,7 +17,6 @@ use picloud_shared::{AppId, GroupId, ScriptOwner};
use sqlx::{PgPool, Postgres, Transaction};
use crate::config_resolver::CHAIN_LEVELS_CTE;
/// List **all** shared-collection declarations at `owner` as `(name, kind)`
/// pairs, sorted. Used by the kind-aware apply diff and `collections ls`.
pub async fn list_all_for_owner(

View File

@@ -0,0 +1,136 @@
//! `GroupDeadLetterRepo` — READ side over `group_dead_letters` (§11.6 D3).
//!
//! The group counterpart to the per-app [`crate::dead_letter_repo::DeadLetterRepo`]
//! read surface, keyed by `(group_id, collection)`. The WRITE side lives in
//! [`crate::group_queue_repo::GroupQueueRepo::dead_letter`] (co-located with the
//! queue-message delete for one-transaction atomicity, exactly as the per-app
//! `queue_repo::dead_letter` inlines its own INSERT). This repo backs the
//! read-only operator surface GET /api/v1/admin/groups/{id}/dead-letters, so an
//! operator can see shared-queue messages that exhausted their retries rather
//! than having them vanish.
use async_trait::async_trait;
use chrono::{DateTime, Utc};
use picloud_shared::{DeadLetterId, GroupId, ScriptId, TriggerId};
use sqlx::PgPool;
use uuid::Uuid;
#[derive(Debug, thiserror::Error)]
pub enum GroupDeadLetterRepoError {
#[error("database error: {0}")]
Db(#[from] sqlx::Error),
}
/// One group dead-letter row (mirror of `dead_letter_repo::DeadLetterRow`,
/// group-keyed).
#[derive(Debug, Clone)]
pub struct GroupDeadLetterRow {
pub id: DeadLetterId,
pub group_id: GroupId,
pub collection: String,
pub original_event_id: Uuid,
pub source: String,
pub op: String,
pub trigger_id: Option<TriggerId>,
pub script_id: Option<ScriptId>,
pub payload: serde_json::Value,
pub attempt_count: u32,
pub first_attempt_at: DateTime<Utc>,
pub last_attempt_at: DateTime<Utc>,
pub last_error: String,
pub created_at: DateTime<Utc>,
pub resolved_at: Option<DateTime<Utc>>,
pub resolution: Option<String>,
}
#[async_trait]
pub trait GroupDeadLetterRepo: Send + Sync {
/// A group's dead-letters, newest-first, capped at `limit` (bounded by the
/// caller). `unresolved_only` filters to `resolved_at IS NULL`.
async fn list_for_group(
&self,
group_id: GroupId,
unresolved_only: bool,
limit: i64,
) -> Result<Vec<GroupDeadLetterRow>, GroupDeadLetterRepoError>;
}
pub struct PostgresGroupDeadLetterRepo {
pool: PgPool,
}
impl PostgresGroupDeadLetterRepo {
#[must_use]
pub fn new(pool: PgPool) -> Self {
Self { pool }
}
}
const COLS: &str = "id, group_id, collection, original_event_id, source, op, \
trigger_id, script_id, payload, attempt_count, first_attempt_at, \
last_attempt_at, last_error, created_at, resolved_at, resolution";
#[async_trait]
impl GroupDeadLetterRepo for PostgresGroupDeadLetterRepo {
async fn list_for_group(
&self,
group_id: GroupId,
unresolved_only: bool,
limit: i64,
) -> Result<Vec<GroupDeadLetterRow>, GroupDeadLetterRepoError> {
let rows: Vec<RawRow> = sqlx::query_as(&format!(
"SELECT {COLS} FROM group_dead_letters \
WHERE group_id = $1 AND ($2 = FALSE OR resolved_at IS NULL) \
ORDER BY created_at DESC LIMIT $3"
))
.bind(group_id.into_inner())
.bind(unresolved_only)
.bind(limit)
.fetch_all(&self.pool)
.await?;
Ok(rows.into_iter().map(Into::into).collect())
}
}
#[derive(sqlx::FromRow)]
struct RawRow {
id: Uuid,
group_id: Uuid,
collection: String,
original_event_id: Uuid,
source: String,
op: String,
trigger_id: Option<Uuid>,
script_id: Option<Uuid>,
payload: serde_json::Value,
attempt_count: i32,
first_attempt_at: DateTime<Utc>,
last_attempt_at: DateTime<Utc>,
last_error: String,
created_at: DateTime<Utc>,
resolved_at: Option<DateTime<Utc>>,
resolution: Option<String>,
}
impl From<RawRow> for GroupDeadLetterRow {
fn from(r: RawRow) -> Self {
Self {
id: DeadLetterId(r.id),
group_id: r.group_id.into(),
collection: r.collection,
original_event_id: r.original_event_id,
source: r.source,
op: r.op,
trigger_id: r.trigger_id.map(Into::into),
script_id: r.script_id.map(Into::into),
payload: r.payload,
attempt_count: u32::try_from(r.attempt_count).unwrap_or(0),
first_attempt_at: r.first_attempt_at,
last_attempt_at: r.last_attempt_at,
last_error: r.last_error,
created_at: r.created_at,
resolved_at: r.resolved_at,
resolution: r.resolution,
}
}
}

View File

@@ -88,6 +88,30 @@ pub trait GroupDocsRepo: Send + Sync {
let _ = group_id;
Ok(0)
}
/// §11.6 M4 quota: total stored JSON bytes across the group's shared-docs
/// collections (`octet_length(data::text)`). Default `Ok(0)`.
async fn total_bytes(&self, group_id: GroupId) -> Result<u64, GroupDocsRepoError> {
let _ = group_id;
Ok(0)
}
/// §11.6 M4 quota: the PROJECTED total stored bytes for the group AFTER
/// writing `new_data` — current SUM, minus the existing doc's bytes
/// (`replacing = Some(id)` on update; `None` on create), plus the new data's
/// bytes. Computed entirely in SQL so ALL three terms use the SAME canonical
/// metric (`octet_length(data::text)`); the Rust-side subtract/add previously
/// mixed the DB canonical SUM with a `serde_json` compact length, drifting the
/// ceiling permissive. Default `Ok(0)`.
async fn projected_total_bytes(
&self,
group_id: GroupId,
replacing: Option<DocId>,
new_data: &serde_json::Value,
) -> Result<u64, GroupDocsRepoError> {
let _ = (group_id, replacing, new_data);
Ok(0)
}
}
pub struct PostgresGroupDocsRepo {
@@ -112,24 +136,7 @@ impl GroupDocsRepo for PostgresGroupDocsRepo {
collection: &str,
data: Value,
) -> Result<DocRow, GroupDocsRepoError> {
let id = Uuid::new_v4();
let row: (DateTime<Utc>, DateTime<Utc>) = sqlx::query_as(
"INSERT INTO group_docs (group_id, collection, id, data) \
VALUES ($1, $2, $3, $4) \
RETURNING created_at, updated_at",
)
.bind(group_id.into_inner())
.bind(collection)
.bind(id)
.bind(&data)
.fetch_one(&self.pool)
.await?;
Ok(DocRow {
id,
data,
created_at: row.0,
updated_at: row.1,
})
create_on(&self.pool, group_id, collection, data).await
}
async fn get(
@@ -182,25 +189,7 @@ impl GroupDocsRepo for PostgresGroupDocsRepo {
id: DocId,
data: Value,
) -> Result<Option<Value>, GroupDocsRepoError> {
let row: Option<(Option<Value>,)> = sqlx::query_as(
"WITH prev AS ( \
SELECT data FROM group_docs \
WHERE group_id = $1 AND collection = $2 AND id = $3 \
), \
updated AS ( \
UPDATE group_docs SET data = $4, updated_at = NOW() \
WHERE group_id = $1 AND collection = $2 AND id = $3 \
RETURNING 1 \
) \
SELECT (SELECT data FROM prev) FROM updated",
)
.bind(group_id.into_inner())
.bind(collection)
.bind(id)
.bind(&data)
.fetch_optional(&self.pool)
.await?;
Ok(row.and_then(|(v,)| v))
update_on(&self.pool, group_id, collection, id, data).await
}
async fn delete(
@@ -209,17 +198,7 @@ impl GroupDocsRepo for PostgresGroupDocsRepo {
collection: &str,
id: DocId,
) -> Result<Option<Value>, GroupDocsRepoError> {
let row: Option<(Value,)> = sqlx::query_as(
"DELETE FROM group_docs \
WHERE group_id = $1 AND collection = $2 AND id = $3 \
RETURNING data",
)
.bind(group_id.into_inner())
.bind(collection)
.bind(id)
.fetch_optional(&self.pool)
.await?;
Ok(row.map(|(v,)| v))
delete_on(&self.pool, group_id, collection, id).await
}
async fn list(
@@ -275,12 +254,170 @@ impl GroupDocsRepo for PostgresGroupDocsRepo {
}
async fn count_rows(&self, group_id: GroupId) -> Result<u64, GroupDocsRepoError> {
let (n,): (i64,) = sqlx::query_as("SELECT COUNT(*) FROM group_docs WHERE group_id = $1")
.bind(group_id.into_inner())
.fetch_one(&self.pool)
.await?;
count_rows_on(&self.pool, group_id).await
}
async fn total_bytes(&self, group_id: GroupId) -> Result<u64, GroupDocsRepoError> {
let (n,): (i64,) = sqlx::query_as(
"SELECT COALESCE(SUM(octet_length(data::text)), 0)::BIGINT \
FROM group_docs WHERE group_id = $1",
)
.bind(group_id.into_inner())
.fetch_one(&self.pool)
.await?;
Ok(u64::try_from(n).unwrap_or(0))
}
async fn projected_total_bytes(
&self,
group_id: GroupId,
replacing: Option<DocId>,
new_data: &serde_json::Value,
) -> Result<u64, GroupDocsRepoError> {
projected_total_bytes_on(&self.pool, group_id, replacing, new_data).await
}
}
// ----------------------------------------------------------------------------
// Connection-scoped operations
//
// Generic over the executor so the same SQL serves the pooled trait methods
// above and `crate::atomic_write`, which runs the per-group advisory lock, the
// quota reads, the write, and the shared-trigger fan-out on ONE connection
// inside ONE transaction (audit #6/#8).
// ----------------------------------------------------------------------------
pub(crate) async fn create_on<'c, E>(
exec: E,
group_id: GroupId,
collection: &str,
data: Value,
) -> Result<DocRow, GroupDocsRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let id = Uuid::new_v4();
let row: (DateTime<Utc>, DateTime<Utc>) = sqlx::query_as(
"INSERT INTO group_docs (group_id, collection, id, data) \
VALUES ($1, $2, $3, $4) \
RETURNING created_at, updated_at",
)
.bind(group_id.into_inner())
.bind(collection)
.bind(id)
.bind(&data)
.fetch_one(exec)
.await?;
Ok(DocRow {
id,
data,
created_at: row.0,
updated_at: row.1,
})
}
/// Returns the previous data (for the update event), `None` if the doc is absent.
pub(crate) async fn update_on<'c, E>(
exec: E,
group_id: GroupId,
collection: &str,
id: DocId,
data: Value,
) -> Result<Option<Value>, GroupDocsRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let row: Option<(Option<Value>,)> = sqlx::query_as(
"WITH prev AS ( \
SELECT data FROM group_docs \
WHERE group_id = $1 AND collection = $2 AND id = $3 \
), \
updated AS ( \
UPDATE group_docs SET data = $4, updated_at = NOW() \
WHERE group_id = $1 AND collection = $2 AND id = $3 \
RETURNING 1 \
) \
SELECT (SELECT data FROM prev) FROM updated",
)
.bind(group_id.into_inner())
.bind(collection)
.bind(id)
.bind(&data)
.fetch_optional(exec)
.await?;
Ok(row.and_then(|(v,)| v))
}
pub(crate) async fn delete_on<'c, E>(
exec: E,
group_id: GroupId,
collection: &str,
id: DocId,
) -> Result<Option<Value>, GroupDocsRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let row: Option<(Value,)> = sqlx::query_as(
"DELETE FROM group_docs \
WHERE group_id = $1 AND collection = $2 AND id = $3 \
RETURNING data",
)
.bind(group_id.into_inner())
.bind(collection)
.bind(id)
.fetch_optional(exec)
.await?;
Ok(row.map(|(v,)| v))
}
/// Total doc count across the group's shared-docs collections (§11.6 row quota).
pub(crate) async fn count_rows_on<'c, E>(
exec: E,
group_id: GroupId,
) -> Result<u64, GroupDocsRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let (n,): (i64,) = sqlx::query_as("SELECT COUNT(*) FROM group_docs WHERE group_id = $1")
.bind(group_id.into_inner())
.fetch_one(exec)
.await?;
Ok(u64::try_from(n).unwrap_or(0))
}
/// §11.6 M4 quota: the PROJECTED total stored bytes for the group AFTER writing
/// `new_data` — current SUM, minus the replaced doc's bytes (`replacing =
/// Some(id)` on update, `None` on create), plus the new data's. Computed
/// entirely in SQL so all three terms use the SAME canonical metric
/// (`octet_length(data::text)`).
pub(crate) async fn projected_total_bytes_on<'c, E>(
exec: E,
group_id: GroupId,
replacing: Option<DocId>,
new_data: &serde_json::Value,
) -> Result<u64, GroupDocsRepoError>
where
E: sqlx::PgExecutor<'c>,
{
// The new data is bound as compact TEXT and re-parsed through `::jsonb::text`
// so PG measures it in the same canonical form it stores. `replacing = None`
// (create) → the subtrahend row never matches → 0.
let new_text = serde_json::to_string(new_data).unwrap_or_else(|_| "null".to_string());
let (n,): (i64,) = sqlx::query_as(
"SELECT ( \
COALESCE((SELECT SUM(octet_length(data::text)) \
FROM group_docs WHERE group_id = $1), 0) \
- COALESCE((SELECT octet_length(data::text) \
FROM group_docs WHERE group_id = $1 AND id = $2), 0) \
+ octet_length($3::jsonb::text) \
)::BIGINT",
)
.bind(group_id.into_inner())
.bind(replacing)
.bind(new_text)
.fetch_one(exec)
.await?;
Ok(u64::try_from(n).unwrap_or(0))
}
fn encode_cursor(last_id: &Uuid) -> String {

View File

@@ -12,14 +12,18 @@ use std::sync::Arc;
use async_trait::async_trait;
use picloud_shared::{
DocId, DocRow, DocsListPage, GroupDocsError, GroupDocsService, GroupId, NoopEventEmitter,
SdkCallCx, ServiceEvent, ServiceEventEmitter,
SdkCallCx, ServiceEventEmitter,
};
use crate::atomic_write::{
BestEffortGroupDocsWriter, GroupDocsTarget, GroupDocsWriter, PostgresGroupDocsWriter,
};
use crate::authz::{self, AuthzRepo, Capability};
use crate::docs_filter::{parse_filter, FilterParseError};
use crate::docs_service::docs_max_value_bytes_from_env;
use crate::group_collection_repo::GroupCollectionResolver;
use crate::group_docs_repo::{GroupDocsRepo, GroupDocsRepoError};
use crate::quota::GroupWriteQuota;
/// The registry `kind` this service resolves.
const KIND_DOCS: &str = "docs";
@@ -32,8 +36,20 @@ pub struct GroupDocsServiceImpl {
/// §11.6 per-group quota: max total docs across the group's shared-docs
/// collections (`PICLOUD_GROUP_DOCS_MAX_ROWS`).
max_rows: u64,
/// §11.6: fires `shared = true` docs triggers on a shared-collection write.
events: Arc<dyn ServiceEventEmitter>,
/// §11.6 M4 per-group quota: max total stored bytes across the group's
/// shared-docs collections (`PICLOUD_GROUP_DOCS_MAX_TOTAL_BYTES`). Checked on
/// the projected total so a same/smaller update near the cap is still allowed.
max_total_bytes: u64,
/// Mutations: the quota decision, the row write, and the shared-trigger
/// fan-out, which for the host all commit in one advisory-locked
/// transaction. Reads stay on `repo`.
writer: Arc<dyn GroupDocsWriter>,
/// Set once [`Self::with_atomic_writes`] has installed the transactional
/// writer. `with_events` refuses to overwrite it — silently downgrading a
/// service back to non-transactional writes (and dropping its quota
/// enforcement) because a builder call was appended in the wrong order is
/// exactly the kind of regression the type system will not catch.
atomic: bool,
}
impl GroupDocsServiceImpl {
@@ -46,10 +62,29 @@ impl GroupDocsServiceImpl {
Self::with_max_value_bytes(repo, resolver, authz, docs_max_value_bytes_from_env())
}
/// Wire the event emitter (§11.6 shared-collection triggers).
/// Wire the event emitter (§11.6 shared-collection triggers) on the
/// best-effort writer. Superseded by [`Self::with_atomic_writes`].
#[must_use]
pub fn with_events(mut self, events: Arc<dyn ServiceEventEmitter>) -> Self {
self.events = events;
if self.atomic {
tracing::warn!(
"with_events() called after with_atomic_writes() — ignored; the \
transactional writer already resolves and writes the fan-out itself"
);
return self;
}
self.writer = Arc::new(BestEffortGroupDocsWriter::new(self.repo.clone(), events));
self
}
/// Use the transactional writer: the quota reads, the row write, and the
/// shared-trigger fan-out all run on ONE connection under a per-group
/// advisory lock, so the quota is a real bound and an emit failure rolls the
/// write back. Supersedes [`Self::with_events`].
#[must_use]
pub fn with_atomic_writes(mut self, pool: sqlx::PgPool) -> Self {
self.writer = Arc::new(PostgresGroupDocsWriter::new(pool));
self.atomic = true;
self
}
@@ -61,8 +96,13 @@ impl GroupDocsServiceImpl {
max_value_bytes: usize,
) -> Self {
Self {
events: Arc::new(NoopEventEmitter),
max_rows: crate::group_quota::group_docs_max_rows_from_env(),
writer: Arc::new(BestEffortGroupDocsWriter::new(
repo.clone(),
Arc::new(NoopEventEmitter),
)),
atomic: false,
max_rows: crate::quota::group_docs_max_rows_from_env(),
max_total_bytes: crate::quota::group_docs_max_total_bytes_from_env(),
repo,
resolver,
authz,
@@ -70,42 +110,23 @@ impl GroupDocsServiceImpl {
}
}
/// §11.6: fire `shared = true` docs triggers on the owning group.
/// Best-effort; an emit failure is logged, never surfaced.
#[allow(clippy::too_many_arguments)] // op/collection/id + new + old payloads
async fn emit_shared(
&self,
cx: &SdkCallCx,
group_id: GroupId,
op: &'static str,
collection: &str,
id: &str,
payload: Option<serde_json::Value>,
old_payload: Option<serde_json::Value>,
) {
if let Err(e) = self
.events
.emit_shared(
cx,
group_id,
ServiceEvent {
source: "docs",
op,
collection: Some(collection.to_string()),
key: Some(id.to_string()),
payload,
old_payload,
},
)
.await
{
tracing::error!(error = %e, source = "docs", op, event_emit_failure = true, "shared event emit failed");
/// Override the per-group total-bytes quota (§11.6 M4). Mainly for tests.
#[must_use]
pub fn with_max_total_bytes(mut self, max_total_bytes: u64) -> Self {
self.max_total_bytes = max_total_bytes;
self
}
/// The ceilings a shared-collection write must fit under. The row/byte
/// policy itself lives in `group_quota::check_group_write`, shared with KV.
fn quota(&self) -> GroupWriteQuota {
GroupWriteQuota {
max_rows: self.max_rows,
max_total_bytes: self.max_total_bytes,
max_value_bytes: self.max_value_bytes,
}
}
/// The structural boundary: resolve the collection to its owning group
/// (kind=`docs`) on the calling app's chain. `CollectionNotShared` when no
/// ancestor group declares it.
async fn owning_group(
&self,
cx: &SdkCallCx,
@@ -193,26 +214,17 @@ impl GroupDocsService for GroupDocsServiceImpl {
validate_data(&data)?;
self.check_data_size(&data)?;
self.check_write(cx, group_id).await?;
// §11.6 quota: a new doc must fit under the group's row ceiling.
let count = self.repo.count_rows(group_id).await?;
if count >= self.max_rows {
return Err(GroupDocsError::QuotaExceeded {
limit: usize::try_from(self.max_rows).unwrap_or(usize::MAX),
actual: usize::try_from(count).unwrap_or(usize::MAX),
});
}
let created = self.repo.create(group_id, collection, data.clone()).await?;
self.emit_shared(
cx,
group_id,
"create",
collection,
&created.id.to_string(),
Some(data),
None,
)
.await;
Ok(created.id)
self.writer
.create(
cx,
GroupDocsTarget {
group_id,
collection,
},
data,
self.quota(),
)
.await
}
async fn get(
@@ -265,25 +277,23 @@ impl GroupDocsService for GroupDocsServiceImpl {
validate_data(&data)?;
self.check_data_size(&data)?;
self.check_write(cx, group_id).await?;
match self
.repo
.update(group_id, collection, id, data.clone())
if self
.writer
.update(
cx,
GroupDocsTarget {
group_id,
collection,
},
id,
data,
self.quota(),
)
.await?
{
Some(prev) => {
self.emit_shared(
cx,
group_id,
"update",
collection,
&id.to_string(),
Some(data),
Some(prev),
)
.await;
Ok(())
}
None => Err(GroupDocsError::NotFound),
Ok(())
} else {
Err(GroupDocsError::NotFound)
}
}
@@ -295,22 +305,16 @@ impl GroupDocsService for GroupDocsServiceImpl {
) -> Result<bool, GroupDocsError> {
let group_id = self.owning_group(cx, collection).await?;
self.check_write(cx, group_id).await?;
match self.repo.delete(group_id, collection, id).await? {
Some(prev) => {
self.emit_shared(
cx,
self.writer
.delete(
cx,
GroupDocsTarget {
group_id,
"delete",
collection,
&id.to_string(),
None,
Some(prev),
)
.await;
Ok(true)
}
None => Ok(false),
}
},
id,
)
.await
}
async fn list(
@@ -447,6 +451,37 @@ mod tests {
next_cursor: None,
})
}
async fn total_bytes(&self, group_id: GroupId) -> Result<u64, GroupDocsRepoError> {
let data = self.data.lock().await;
Ok(data
.iter()
.filter(|((g, _), _)| *g == group_id)
.flat_map(|(_, docs)| docs.values())
.map(|v| serde_json::to_vec(v).map_or(0, |b| b.len() as u64))
.sum())
}
/// Same projection the Postgres impl computes in SQL, but with this
/// mock's own (compact) metric: every doc in the group EXCEPT the one
/// being replaced, plus the new data — i.e. `used - old + new`.
async fn projected_total_bytes(
&self,
group_id: GroupId,
replacing: Option<DocId>,
new_data: &serde_json::Value,
) -> Result<u64, GroupDocsRepoError> {
let data = self.data.lock().await;
let others: u64 = data
.iter()
.filter(|((g, _), _)| *g == group_id)
.flat_map(|(_, docs)| docs.iter())
.filter(|(id, _)| Some(**id) != replacing)
.map(|(_, v)| serde_json::to_vec(v).map_or(0, |b| b.len() as u64))
.sum();
let new_len = serde_json::to_vec(new_data).map_or(0, |b| b.len() as u64);
Ok(others + new_len)
}
}
#[derive(Default)]
@@ -578,4 +613,51 @@ mod tests {
.unwrap_err();
assert!(matches!(err, GroupDocsError::InvalidData));
}
#[tokio::test]
async fn per_group_byte_quota_uses_the_projected_total() {
// §11.6 M4: the shared-docs byte ceiling is checked on the PROJECTED
// total — a create over the cap is rejected, and a shrinking update near
// the cap is allowed (the delta rule).
let app = AppId::new();
let group = GroupId::new();
let mut resolver = FakeResolver::default();
resolver.map.insert((app, "articles".into()), group);
let docs = GroupDocsServiceImpl::new(
Arc::new(InMemoryGroupDocsRepo::default()),
Arc::new(resolver),
Arc::new(DenyingAuthzRepo),
)
.with_max_total_bytes(40);
let cx = cx_with(app, Some(owner()));
// {"t":"xxxxx"} encodes to 13 bytes.
let id = docs
.create(&cx, "articles", serde_json::json!({"t": "xxxxx"}))
.await
.unwrap(); // used = 13
docs.create(&cx, "articles", serde_json::json!({"t": "xxxxx"}))
.await
.unwrap(); // used = 26
// A third 13-byte doc → projected 39 ≤ 40 → ok.
docs.create(&cx, "articles", serde_json::json!({"t": "xxxxx"}))
.await
.unwrap(); // used = 39
// A fourth → projected 52 > 40 → rejected.
let err = docs
.create(&cx, "articles", serde_json::json!({"t": "xxxxx"}))
.await
.unwrap_err();
assert!(
matches!(
err,
GroupDocsError::TotalBytesQuotaExceeded { limit: 40, .. }
),
"got {err:?}"
);
// Shrinking an existing doc frees budget (projected 39 - 13 + 9 = 35).
docs.update(&cx, "articles", id, serde_json::json!({"t": "y"}))
.await
.expect("a shrinking update near the cap must be allowed");
}
}

View File

@@ -389,7 +389,7 @@ impl GroupFilesRepo for FsGroupFilesRepo {
}
#[derive(sqlx::FromRow)]
struct GroupFileRow {
pub(crate) struct GroupFileRow {
id: Uuid,
collection: String,
name: String,
@@ -401,7 +401,7 @@ struct GroupFileRow {
}
impl GroupFileRow {
fn into_meta(self) -> FileMeta {
pub(crate) fn into_meta(self) -> FileMeta {
FileMeta {
id: self.id,
collection: self.collection,
@@ -414,3 +414,146 @@ impl GroupFileRow {
}
}
}
// ----------------------------------------------------------------------------
// Connection-scoped metadata operations (see `files_repo` for the rationale)
// ----------------------------------------------------------------------------
use crate::files_repo::{MetaFields, MetaPatch};
const GROUP_FILE_COLS: &str = "id, collection, name, content_type, size_bytes, \
checksum_sha256, created_at, updated_at";
pub(crate) async fn head_on<'c, E>(
exec: E,
group_id: GroupId,
collection: &str,
id: Uuid,
) -> Result<Option<FileMeta>, GroupFilesRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let row: Option<GroupFileRow> = sqlx::query_as(&format!(
"SELECT {GROUP_FILE_COLS} FROM group_files \
WHERE group_id = $1 AND collection = $2 AND id = $3"
))
.bind(group_id.into_inner())
.bind(collection)
.bind(id)
.fetch_optional(exec)
.await?;
Ok(row.map(GroupFileRow::into_meta))
}
pub(crate) async fn insert_meta_on<'c, E>(
exec: E,
group_id: GroupId,
collection: &str,
id: Uuid,
meta: MetaFields<'_>,
) -> Result<FileMeta, GroupFilesRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let row: GroupFileRow = sqlx::query_as(&format!(
"INSERT INTO group_files \
(group_id, collection, id, name, content_type, size_bytes, checksum_sha256) \
VALUES ($1, $2, $3, $4, $5, $6, $7) \
RETURNING {GROUP_FILE_COLS}"
))
.bind(group_id.into_inner())
.bind(collection)
.bind(id)
.bind(meta.name)
.bind(meta.content_type)
.bind(meta.size)
.bind(meta.checksum)
.fetch_one(exec)
.await?;
Ok(row.into_meta())
}
pub(crate) async fn update_meta_on<'c, E>(
exec: E,
group_id: GroupId,
collection: &str,
id: Uuid,
meta: MetaPatch<'_>,
) -> Result<Option<FileMeta>, GroupFilesRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let row: Option<GroupFileRow> = sqlx::query_as(&format!(
"UPDATE group_files SET \
name = COALESCE($4, name), \
content_type = COALESCE($5, content_type), \
size_bytes = $6, \
checksum_sha256 = $7, \
updated_at = NOW() \
WHERE group_id = $1 AND collection = $2 AND id = $3 \
RETURNING {GROUP_FILE_COLS}"
))
.bind(group_id.into_inner())
.bind(collection)
.bind(id)
.bind(meta.name)
.bind(meta.content_type)
.bind(meta.size)
.bind(meta.checksum)
.fetch_optional(exec)
.await?;
Ok(row.map(GroupFileRow::into_meta))
}
pub(crate) async fn delete_meta_on<'c, E>(
exec: E,
group_id: GroupId,
collection: &str,
id: Uuid,
) -> Result<Option<FileMeta>, GroupFilesRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let row: Option<GroupFileRow> = sqlx::query_as(&format!(
"DELETE FROM group_files \
WHERE group_id = $1 AND collection = $2 AND id = $3 \
RETURNING {GROUP_FILE_COLS}"
))
.bind(group_id.into_inner())
.bind(collection)
.bind(id)
.fetch_optional(exec)
.await?;
Ok(row.map(GroupFileRow::into_meta))
}
/// §11.6 quota: the PROJECTED total stored bytes for the group AFTER this write
/// — the current SUM, minus the bytes of the file being replaced (`replacing =
/// Some(id)` on update; `None` on create), plus the incoming file's bytes.
///
/// The subtraction is what `GroupFilesService::update` was missing entirely: it
/// checked no quota at all, so a 1-byte file could be updated to a 100 MB one
/// without ever consulting the ceiling.
pub(crate) async fn projected_total_bytes_on<'c, E>(
exec: E,
group_id: GroupId,
replacing: Option<Uuid>,
incoming: i64,
) -> Result<u64, GroupFilesRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let (n,): (i64,) = sqlx::query_as(
"SELECT ( \
COALESCE((SELECT SUM(size_bytes) FROM group_files WHERE group_id = $1), 0) \
- COALESCE((SELECT size_bytes FROM group_files \
WHERE group_id = $1 AND id = $2), 0) \
+ $3 \
)::BIGINT",
)
.bind(group_id.into_inner())
.bind(replacing)
.bind(incoming)
.fetch_one(exec)
.await?;
Ok(u64::try_from(n).unwrap_or(0))
}

View File

@@ -13,12 +13,14 @@ use async_trait::async_trait;
use picloud_shared::{
sanitize_stored_content_type, validate_files_collection, FileMeta, FileUpdate, FilesListPage,
GroupFilesError, GroupFilesService, GroupId, NewFile, NoopEventEmitter, SdkCallCx,
ServiceEvent, ServiceEventEmitter,
ServiceEventEmitter,
};
use uuid::Uuid;
use crate::atomic_write::{
BestEffortGroupFilesWriter, GroupFilesQuota, GroupFilesWriter, PostgresGroupFilesWriter,
};
use crate::authz::{self, AuthzRepo, Capability};
use crate::files_repo::FileUpdated;
use crate::group_collection_repo::GroupCollectionResolver;
use crate::group_files_repo::{GroupFilesRepo, GroupFilesRepoError};
@@ -34,7 +36,16 @@ pub struct GroupFilesServiceImpl {
/// shared-files collections (`PICLOUD_GROUP_FILES_MAX_TOTAL_BYTES`).
max_total_bytes: u64,
/// §11.6: fires `shared = true` files triggers on a shared-collection write.
events: Arc<dyn ServiceEventEmitter>,
/// Mutations: the byte-quota decision, the blob + metadata write, and the
/// shared-trigger fan-out — for the host, all under one advisory-locked
/// transaction. Reads stay on `repo`.
writer: Arc<dyn GroupFilesWriter>,
/// Set once [`Self::with_atomic_writes`] has installed the transactional
/// writer. `with_events` refuses to overwrite it — silently downgrading a
/// service back to non-transactional writes (and dropping its quota
/// enforcement) because a builder call was appended in the wrong order is
/// exactly the kind of regression the type system will not catch.
atomic: bool,
}
impl GroupFilesServiceImpl {
@@ -46,56 +57,53 @@ impl GroupFilesServiceImpl {
max_file_size_bytes: usize,
) -> Self {
Self {
max_total_bytes: crate::quota::group_files_max_total_bytes_from_env(),
writer: Arc::new(BestEffortGroupFilesWriter::new(
repo.clone(),
Arc::new(NoopEventEmitter),
)),
atomic: false,
repo,
resolver,
authz,
max_file_size_bytes,
max_total_bytes: crate::group_quota::group_files_max_total_bytes_from_env(),
events: Arc::new(NoopEventEmitter),
}
}
/// Wire the event emitter (§11.6 shared-collection triggers).
/// The ceiling a shared-files write must fit under.
fn quota(&self) -> GroupFilesQuota {
GroupFilesQuota {
max_total_bytes: self.max_total_bytes,
}
}
/// Wire the event emitter (§11.6 shared-collection triggers) on the
/// best-effort writer. Superseded by [`Self::with_atomic_writes`].
#[must_use]
pub fn with_events(mut self, events: Arc<dyn ServiceEventEmitter>) -> Self {
self.events = events;
if self.atomic {
tracing::warn!(
"with_events() called after with_atomic_writes() — ignored; the \
transactional writer already resolves and writes the fan-out itself"
);
return self;
}
self.writer = Arc::new(BestEffortGroupFilesWriter::new(self.repo.clone(), events));
self
}
/// §11.6: fire `shared = true` files triggers on the owning group.
/// Best-effort; an emit failure is logged, never surfaced.
async fn emit_shared(
&self,
cx: &SdkCallCx,
group_id: GroupId,
op: &'static str,
collection: &str,
meta: &FileMeta,
old: Option<&FileMeta>,
) {
if let Err(e) = self
.events
.emit_shared(
cx,
group_id,
ServiceEvent {
source: "files",
op,
collection: Some(collection.to_string()),
key: Some(meta.id.to_string()),
payload: serde_json::to_value(meta).ok(),
old_payload: old.and_then(|m| serde_json::to_value(m).ok()),
},
)
.await
{
tracing::error!(error = %e, source = "files", op, event_emit_failure = true, "shared event emit failed");
}
/// Use the transactional writer: the byte-quota check, the metadata row, and
/// the shared-trigger fan-out run on ONE connection under a per-group
/// advisory lock, so concurrent uploads can't each see the same pre-write
/// total and together overshoot the ceiling. Supersedes
/// [`Self::with_events`].
#[must_use]
pub fn with_atomic_writes(mut self, pool: sqlx::PgPool, root: std::path::PathBuf) -> Self {
self.writer = Arc::new(PostgresGroupFilesWriter::new(pool, root));
self.atomic = true;
self
}
/// The structural boundary: resolve the collection to its owning group
/// (kind=`files`) on the calling app's chain. `CollectionNotShared` when no
/// ancestor group declares it.
async fn owning_group(
&self,
cx: &SdkCallCx,
@@ -164,19 +172,9 @@ impl GroupFilesService for GroupFilesServiceImpl {
// shape checks pass (same as app files, audit 2026-06-11 C-2).
new.content_type = sanitize_stored_content_type(&new.content_type);
self.check_write(cx, group_id).await?;
// §11.6 quota: the new file's bytes must fit under the group's total.
let used = self.repo.total_bytes(group_id).await?;
let incoming = new.data.len() as u64;
if used.saturating_add(incoming) > self.max_total_bytes {
return Err(GroupFilesError::QuotaExceeded {
limit: self.max_total_bytes,
actual: used.saturating_add(incoming),
});
}
let meta = self.repo.create(group_id, collection, new).await?;
self.emit_shared(cx, group_id, "create", collection, &meta, None)
.await;
Ok(meta.id)
self.writer
.create(cx, group_id, collection, new, self.quota())
.await
}
async fn head(
@@ -226,13 +224,14 @@ impl GroupFilesService for GroupFilesServiceImpl {
let Some(uuid) = parse_id(id) else {
return Err(GroupFilesError::NotFound);
};
match self.repo.update(group_id, collection, uuid, upd).await? {
Some(FileUpdated { new, prev }) => {
self.emit_shared(cx, group_id, "update", collection, &new, Some(&prev))
.await;
Ok(())
}
None => Err(GroupFilesError::NotFound),
if self
.writer
.update(cx, group_id, collection, uuid, upd, self.quota())
.await?
{
Ok(())
} else {
Err(GroupFilesError::NotFound)
}
}
@@ -248,14 +247,7 @@ impl GroupFilesService for GroupFilesServiceImpl {
let Some(uuid) = parse_id(id) else {
return Ok(false);
};
match self.repo.delete(group_id, collection, uuid).await? {
Some(meta) => {
self.emit_shared(cx, group_id, "delete", collection, &meta, Some(&meta))
.await;
Ok(true)
}
None => Ok(false),
}
self.writer.delete(cx, group_id, collection, uuid).await
}
async fn list(
@@ -282,6 +274,7 @@ impl GroupFilesService for GroupFilesServiceImpl {
mod tests {
use super::*;
use crate::authz::{AuthzError, AuthzRepo};
use crate::files_repo::FileUpdated;
use crate::group_files_repo::GroupFilesRepoError;
use chrono::Utc;
use picloud_shared::{

View File

@@ -39,6 +39,18 @@ pub trait GroupKvRepo: Send + Sync {
value: serde_json::Value,
) -> Result<Option<serde_json::Value>, GroupKvRepoError>;
/// Compare-and-swap. Atomically writes `new` iff the current value equals
/// `expected` (`expected = None` → iff the key is ABSENT). Returns `true`
/// when the write happened. One SQL statement — atomic without a tx.
async fn set_if(
&self,
group_id: GroupId,
collection: &str,
key: &str,
expected: Option<serde_json::Value>,
new: serde_json::Value,
) -> Result<bool, GroupKvRepoError>;
/// Returns the deleted value if present, `None` if the row didn't exist.
async fn delete(
&self,
@@ -68,6 +80,33 @@ pub trait GroupKvRepo: Send + Sync {
let _ = group_id;
Ok(0)
}
/// §11.6 M4 quota: total stored JSON bytes across all of the group's shared-KV
/// collections (`octet_length(value::text)`). Default `Ok(0)` so non-Postgres
/// impls skip the byte-quota check.
async fn total_bytes(&self, group_id: GroupId) -> Result<u64, GroupKvRepoError> {
let _ = group_id;
Ok(0)
}
/// §11.6 M4 quota: the PROJECTED total stored bytes for the group AFTER
/// writing `new_value` at `(collection, key)` — i.e. current SUM minus the
/// existing row's bytes (if any) plus the new value's bytes. Computed
/// entirely in SQL so ALL three terms use the SAME canonical metric
/// (`octet_length(value::text)`); the Rust-side subtract/add previously
/// mixed the DB canonical SUM with a `serde_json` compact length, which
/// drifted the ceiling permissive. Default `Ok(0)` (non-Postgres impls skip
/// the byte quota).
async fn projected_total_bytes(
&self,
group_id: GroupId,
collection: &str,
key: &str,
new_value: &serde_json::Value,
) -> Result<u64, GroupKvRepoError> {
let _ = (group_id, collection, key, new_value);
Ok(0)
}
}
pub struct PostgresGroupKvRepo {
@@ -93,16 +132,7 @@ impl GroupKvRepo for PostgresGroupKvRepo {
collection: &str,
key: &str,
) -> Result<Option<serde_json::Value>, GroupKvRepoError> {
let row: Option<(serde_json::Value,)> = sqlx::query_as(
"SELECT value FROM group_kv_entries \
WHERE group_id = $1 AND collection = $2 AND key = $3",
)
.bind(group_id.into_inner())
.bind(collection)
.bind(key)
.fetch_optional(&self.pool)
.await?;
Ok(row.map(|(v,)| v))
get_on(&self.pool, group_id, collection, key).await
}
async fn set(
@@ -112,27 +142,18 @@ impl GroupKvRepo for PostgresGroupKvRepo {
key: &str,
value: serde_json::Value,
) -> Result<Option<serde_json::Value>, GroupKvRepoError> {
let row: Option<(Option<serde_json::Value>,)> = sqlx::query_as(
"WITH prev AS (\
SELECT value FROM group_kv_entries \
WHERE group_id = $1 AND collection = $2 AND key = $3\
), \
upserted AS (\
INSERT INTO group_kv_entries (group_id, collection, key, value) \
VALUES ($1, $2, $3, $4) \
ON CONFLICT (group_id, collection, key) DO UPDATE \
SET value = EXCLUDED.value, updated_at = NOW() \
RETURNING 1\
) \
SELECT (SELECT value FROM prev) FROM upserted",
)
.bind(group_id.into_inner())
.bind(collection)
.bind(key)
.bind(value)
.fetch_optional(&self.pool)
.await?;
Ok(row.and_then(|(v,)| v))
set_on(&self.pool, group_id, collection, key, value).await
}
async fn set_if(
&self,
group_id: GroupId,
collection: &str,
key: &str,
expected: Option<serde_json::Value>,
new: serde_json::Value,
) -> Result<bool, GroupKvRepoError> {
set_if_on(&self.pool, group_id, collection, key, expected, new).await
}
async fn delete(
@@ -141,17 +162,7 @@ impl GroupKvRepo for PostgresGroupKvRepo {
collection: &str,
key: &str,
) -> Result<Option<serde_json::Value>, GroupKvRepoError> {
let row: Option<(serde_json::Value,)> = sqlx::query_as(
"DELETE FROM group_kv_entries \
WHERE group_id = $1 AND collection = $2 AND key = $3 \
RETURNING value",
)
.bind(group_id.into_inner())
.bind(collection)
.bind(key)
.fetch_optional(&self.pool)
.await?;
Ok(row.map(|(v,)| v))
delete_on(&self.pool, group_id, collection, key).await
}
async fn has(
@@ -217,13 +228,214 @@ impl GroupKvRepo for PostgresGroupKvRepo {
}
async fn count_rows(&self, group_id: GroupId) -> Result<u64, GroupKvRepoError> {
let (n,): (i64,) =
sqlx::query_as("SELECT COUNT(*) FROM group_kv_entries WHERE group_id = $1")
.bind(group_id.into_inner())
.fetch_one(&self.pool)
.await?;
count_rows_on(&self.pool, group_id).await
}
async fn total_bytes(&self, group_id: GroupId) -> Result<u64, GroupKvRepoError> {
let (n,): (i64,) = sqlx::query_as(
"SELECT COALESCE(SUM(octet_length(value::text)), 0)::BIGINT \
FROM group_kv_entries WHERE group_id = $1",
)
.bind(group_id.into_inner())
.fetch_one(&self.pool)
.await?;
Ok(u64::try_from(n).unwrap_or(0))
}
async fn projected_total_bytes(
&self,
group_id: GroupId,
collection: &str,
key: &str,
new_value: &serde_json::Value,
) -> Result<u64, GroupKvRepoError> {
projected_total_bytes_on(&self.pool, group_id, collection, key, new_value).await
}
}
// ----------------------------------------------------------------------------
// Connection-scoped operations
//
// Generic over the executor so the same SQL serves the pooled trait methods
// above and `crate::atomic_write`, which runs the per-group advisory lock, the
// quota reads, the write, and the shared-trigger fan-out on ONE connection
// inside ONE transaction. The quota reads in particular MUST run on the writing
// transaction's connection — that is what makes check-then-write atomic rather
// than a race (audit #8).
// ----------------------------------------------------------------------------
pub(crate) async fn get_on<'c, E>(
exec: E,
group_id: GroupId,
collection: &str,
key: &str,
) -> Result<Option<serde_json::Value>, GroupKvRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let row: Option<(serde_json::Value,)> = sqlx::query_as(
"SELECT value FROM group_kv_entries \
WHERE group_id = $1 AND collection = $2 AND key = $3",
)
.bind(group_id.into_inner())
.bind(collection)
.bind(key)
.fetch_optional(exec)
.await?;
Ok(row.map(|(v,)| v))
}
/// Upsert. Returns the previous value (`None` = this was an insert).
pub(crate) async fn set_on<'c, E>(
exec: E,
group_id: GroupId,
collection: &str,
key: &str,
value: serde_json::Value,
) -> Result<Option<serde_json::Value>, GroupKvRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let row: Option<(Option<serde_json::Value>,)> = sqlx::query_as(
"WITH prev AS (\
SELECT value FROM group_kv_entries \
WHERE group_id = $1 AND collection = $2 AND key = $3\
), \
upserted AS (\
INSERT INTO group_kv_entries (group_id, collection, key, value) \
VALUES ($1, $2, $3, $4) \
ON CONFLICT (group_id, collection, key) DO UPDATE \
SET value = EXCLUDED.value, updated_at = NOW() \
RETURNING 1\
) \
SELECT (SELECT value FROM prev) FROM upserted",
)
.bind(group_id.into_inner())
.bind(collection)
.bind(key)
.bind(value)
.fetch_optional(exec)
.await?;
Ok(row.and_then(|(v,)| v))
}
/// Compare-and-swap. Writes `new` iff the current value equals `expected`
/// (`expected = None` → iff the key is ABSENT). Returns whether it wrote.
pub(crate) async fn set_if_on<'c, E>(
exec: E,
group_id: GroupId,
collection: &str,
key: &str,
expected: Option<serde_json::Value>,
new: serde_json::Value,
) -> Result<bool, GroupKvRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let affected = match expected {
Some(exp) => sqlx::query(
"UPDATE group_kv_entries SET value = $4, updated_at = NOW() \
WHERE group_id = $1 AND collection = $2 AND key = $3 AND value = $5",
)
.bind(group_id.into_inner())
.bind(collection)
.bind(key)
.bind(new)
.bind(exp)
.execute(exec)
.await?
.rows_affected(),
None => sqlx::query(
"INSERT INTO group_kv_entries (group_id, collection, key, value) \
VALUES ($1, $2, $3, $4) \
ON CONFLICT (group_id, collection, key) DO NOTHING",
)
.bind(group_id.into_inner())
.bind(collection)
.bind(key)
.bind(new)
.execute(exec)
.await?
.rows_affected(),
};
Ok(affected == 1)
}
/// Returns the deleted value if present, `None` if the row didn't exist.
pub(crate) async fn delete_on<'c, E>(
exec: E,
group_id: GroupId,
collection: &str,
key: &str,
) -> Result<Option<serde_json::Value>, GroupKvRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let row: Option<(serde_json::Value,)> = sqlx::query_as(
"DELETE FROM group_kv_entries \
WHERE group_id = $1 AND collection = $2 AND key = $3 \
RETURNING value",
)
.bind(group_id.into_inner())
.bind(collection)
.bind(key)
.fetch_optional(exec)
.await?;
Ok(row.map(|(v,)| v))
}
/// Total key count across the group's shared-KV collections (§11.6 row quota).
pub(crate) async fn count_rows_on<'c, E>(
exec: E,
group_id: GroupId,
) -> Result<u64, GroupKvRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let (n,): (i64,) = sqlx::query_as("SELECT COUNT(*) FROM group_kv_entries WHERE group_id = $1")
.bind(group_id.into_inner())
.fetch_one(exec)
.await?;
Ok(u64::try_from(n).unwrap_or(0))
}
/// §11.6 M4 quota: the PROJECTED total stored bytes for the group AFTER writing
/// `new_value` at `(collection, key)` — current SUM, minus the existing row's
/// bytes (0 if the key is new), plus the new value's bytes. Computed entirely in
/// SQL so all three terms use the SAME canonical metric
/// (`octet_length(value::text)`); mixing the DB's canonical SUM with a
/// `serde_json` compact length drifts the ceiling permissive.
pub(crate) async fn projected_total_bytes_on<'c, E>(
exec: E,
group_id: GroupId,
collection: &str,
key: &str,
new_value: &serde_json::Value,
) -> Result<u64, GroupKvRepoError>
where
E: sqlx::PgExecutor<'c>,
{
// The new value is bound as compact TEXT and re-parsed through `::jsonb::text`
// so PG measures it in the same canonical form it stores.
// A `serde_json::Value` always serializes; fall back defensively.
let new_text = serde_json::to_string(new_value).unwrap_or_else(|_| "null".to_string());
let (n,): (i64,) = sqlx::query_as(
"SELECT ( \
COALESCE((SELECT SUM(octet_length(value::text)) \
FROM group_kv_entries WHERE group_id = $1), 0) \
- COALESCE((SELECT octet_length(value::text) \
FROM group_kv_entries \
WHERE group_id = $1 AND collection = $2 AND key = $3), 0) \
+ octet_length($4::jsonb::text) \
)::BIGINT",
)
.bind(group_id.into_inner())
.bind(collection)
.bind(key)
.bind(new_text)
.fetch_one(exec)
.await?;
Ok(u64::try_from(n).unwrap_or(0))
}
fn encode_cursor(last_key: &str) -> String {

View File

@@ -22,14 +22,18 @@ use std::sync::Arc;
use async_trait::async_trait;
use picloud_shared::{
GroupId, GroupKvError, GroupKvService, KvListPage, NoopEventEmitter, SdkCallCx, ServiceEvent,
GroupId, GroupKvError, GroupKvService, KvListPage, NoopEventEmitter, SdkCallCx,
ServiceEventEmitter,
};
use crate::atomic_write::{
BestEffortGroupKvWriter, GroupKvTarget, GroupKvWriter, PostgresGroupKvWriter,
};
use crate::authz::{self, AuthzRepo, Capability};
use crate::group_collection_repo::GroupCollectionResolver;
use crate::group_kv_repo::{GroupKvRepo, GroupKvRepoError};
use crate::kv_service::kv_max_value_bytes_from_env;
use crate::quota::GroupWriteQuota;
/// The registry `kind` this service resolves.
const KIND_KV: &str = "kv";
@@ -43,10 +47,21 @@ pub struct GroupKvServiceImpl {
/// collections (`PICLOUD_GROUP_KV_MAX_ROWS`). Checked only when inserting a
/// NEW key.
max_rows: u64,
/// §11.6: fires `shared = true` triggers on a shared-collection write.
/// Defaults to the noop emitter; the host wires the outbox emitter via
/// [`Self::with_events`].
events: Arc<dyn ServiceEventEmitter>,
/// §11.6 M4 per-group quota: max total stored bytes across the group's
/// shared-KV collections (`PICLOUD_GROUP_KV_MAX_TOTAL_BYTES`). Checked on the
/// projected total (old value subtracted, new added) so a same/smaller update
/// near the cap is still allowed.
max_total_bytes: u64,
/// Mutations: the quota decision, the row write, and the shared-trigger
/// fan-out, which for the host all commit in one advisory-locked
/// transaction. Reads stay on `repo`.
writer: Arc<dyn GroupKvWriter>,
/// Set once [`Self::with_atomic_writes`] has installed the transactional
/// writer. `with_events` refuses to overwrite it — silently downgrading a
/// service back to non-transactional writes (and dropping its quota
/// enforcement) because a builder call was appended in the wrong order is
/// exactly the kind of regression the type system will not catch.
atomic: bool,
}
impl GroupKvServiceImpl {
@@ -59,11 +74,34 @@ impl GroupKvServiceImpl {
Self::with_max_value_bytes(repo, resolver, authz, kv_max_value_bytes_from_env())
}
/// Wire the event emitter (§11.6 shared-collection triggers). Without this
/// the service uses the noop emitter and shared writes fire nothing.
/// Wire the event emitter (§11.6 shared-collection triggers) on the
/// best-effort writer. Without this — or [`Self::with_atomic_writes`], which
/// supersedes it — shared writes fire nothing.
#[must_use]
pub fn with_events(mut self, events: Arc<dyn ServiceEventEmitter>) -> Self {
self.events = events;
if self.atomic {
tracing::warn!(
"with_events() called after with_atomic_writes() — ignored; the \
transactional writer already resolves and writes the fan-out itself"
);
return self;
}
self.writer = Arc::new(BestEffortGroupKvWriter::new(self.repo.clone(), events));
self
}
/// Use the transactional writer: the quota reads, the row write, and the
/// shared-trigger fan-out all run on ONE connection under a per-group
/// advisory lock. That makes the quota an actual bound (concurrent writers
/// to a group serialize, so one sees the other's row) and makes an emit
/// failure roll the write back instead of silently losing the event.
///
/// Supersedes [`Self::with_events`] — the transactional writer resolves and
/// writes the fan-out itself and needs no injected emitter.
#[must_use]
pub fn with_atomic_writes(mut self, pool: sqlx::PgPool) -> Self {
self.writer = Arc::new(PostgresGroupKvWriter::new(pool));
self.atomic = true;
self
}
@@ -83,8 +121,13 @@ impl GroupKvServiceImpl {
max_value_bytes: usize,
) -> Self {
Self {
events: Arc::new(NoopEventEmitter),
max_rows: crate::group_quota::group_kv_max_rows_from_env(),
writer: Arc::new(BestEffortGroupKvWriter::new(
repo.clone(),
Arc::new(NoopEventEmitter),
)),
atomic: false,
max_rows: crate::quota::group_kv_max_rows_from_env(),
max_total_bytes: crate::quota::group_kv_max_total_bytes_from_env(),
repo,
resolver,
authz,
@@ -92,6 +135,13 @@ impl GroupKvServiceImpl {
}
}
/// Override the per-group total-bytes quota (§11.6 M4). Mainly for tests.
#[must_use]
pub fn with_max_total_bytes(mut self, max_total_bytes: u64) -> Self {
self.max_total_bytes = max_total_bytes;
self
}
/// The structural boundary: resolve the collection to its owning group on
/// the calling app's chain. `CollectionNotShared` when no ancestor group
/// declares it — the same outcome a foreign app gets, by construction.
@@ -110,38 +160,31 @@ impl GroupKvServiceImpl {
.ok_or_else(|| GroupKvError::CollectionNotShared(collection.to_string()))
}
/// §11.6: fire `shared = true` kv triggers on the owning group. Best-effort
/// — an emit failure is logged, never surfaced (the write already
/// committed), matching the per-app `KvServiceImpl` behaviour.
async fn emit_shared(
&self,
cx: &SdkCallCx,
group_id: GroupId,
op: &'static str,
collection: &str,
key: &str,
payload: Option<serde_json::Value>,
) {
if let Err(e) = self
.events
.emit_shared(
cx,
group_id,
ServiceEvent {
source: "kv",
op,
collection: Some(collection.to_string()),
key: Some(key.to_string()),
payload,
old_payload: None,
},
)
.await
{
tracing::error!(error = %e, source = "kv", op, event_emit_failure = true, "shared event emit failed");
/// The ceilings a shared-collection write must fit under.
fn quota(&self) -> GroupWriteQuota {
GroupWriteQuota {
max_rows: self.max_rows,
max_total_bytes: self.max_total_bytes,
max_value_bytes: self.max_value_bytes,
}
}
/// Enforce the per-value byte cap. The per-GROUP byte quota is measured
/// canonically in SQL inside the writing transaction — see
/// `group_quota::check_group_write`.
fn check_value_size(&self, value: &serde_json::Value) -> Result<(), GroupKvError> {
let encoded_len = serde_json::to_vec(value)
.map(|v| v.len())
.map_err(|e| GroupKvError::Backend(format!("encode value: {e}")))?;
if encoded_len > self.max_value_bytes {
return Err(GroupKvError::ValueTooLarge {
limit: self.max_value_bytes,
actual: encoded_len,
});
}
Ok(())
}
async fn check_read(&self, cx: &SdkCallCx, group_id: GroupId) -> Result<(), GroupKvError> {
authz::script_gate(
&*self.authz,
@@ -194,44 +237,46 @@ impl GroupKvService for GroupKvServiceImpl {
value: serde_json::Value,
) -> Result<(), GroupKvError> {
let group_id = self.owning_group(cx, collection).await?;
let encoded_len = serde_json::to_vec(&value)
.map(|v| v.len())
.map_err(|e| GroupKvError::Backend(format!("encode value: {e}")))?;
if encoded_len > self.max_value_bytes {
return Err(GroupKvError::ValueTooLarge {
limit: self.max_value_bytes,
actual: encoded_len,
});
}
self.check_value_size(&value)?;
self.check_write(cx, group_id).await?;
// Determine insert vs update for the event op (best-effort, like the
// per-app path — the has+set is non-transactional but triggers are
// fire-and-forget).
let existed = self.repo.has(group_id, collection, key).await?;
// §11.6 quota: a NEW key must fit under the group's row ceiling; an
// update of an existing key is net-zero and exempt.
if !existed {
let count = self.repo.count_rows(group_id).await?;
if count >= self.max_rows {
return Err(GroupKvError::QuotaExceeded {
limit: usize::try_from(self.max_rows).unwrap_or(usize::MAX),
actual: usize::try_from(count).unwrap_or(usize::MAX),
});
}
}
self.repo
.set(group_id, collection, key, value.clone())
.await?;
self.emit_shared(
cx,
group_id,
if existed { "update" } else { "insert" },
collection,
key,
Some(value),
)
.await;
Ok(())
self.writer
.set(
cx,
GroupKvTarget {
group_id,
collection,
key,
},
value,
self.quota(),
)
.await
}
async fn set_if(
&self,
cx: &SdkCallCx,
collection: &str,
key: &str,
expected: Option<serde_json::Value>,
new: serde_json::Value,
) -> Result<bool, GroupKvError> {
let group_id = self.owning_group(cx, collection).await?;
self.check_value_size(&new)?;
self.check_write(cx, group_id).await?;
self.writer
.set_if(
cx,
GroupKvTarget {
group_id,
collection,
key,
},
expected,
new,
self.quota(),
)
.await
}
async fn delete(
@@ -242,12 +287,16 @@ impl GroupKvService for GroupKvServiceImpl {
) -> Result<bool, GroupKvError> {
let group_id = self.owning_group(cx, collection).await?;
self.check_write(cx, group_id).await?;
let deleted = self.repo.delete(group_id, collection, key).await?.is_some();
if deleted {
self.emit_shared(cx, group_id, "delete", collection, key, None)
.await;
}
Ok(deleted)
self.writer
.delete(
cx,
GroupKvTarget {
group_id,
collection,
key,
},
)
.await
}
async fn has(&self, cx: &SdkCallCx, collection: &str, key: &str) -> Result<bool, GroupKvError> {
@@ -319,6 +368,27 @@ mod tests {
.insert((group_id, collection.to_string(), key.to_string()), value))
}
async fn set_if(
&self,
group_id: GroupId,
collection: &str,
key: &str,
expected: Option<serde_json::Value>,
new: serde_json::Value,
) -> Result<bool, GroupKvRepoError> {
let mut data = self.data.lock().await;
let k = (group_id, collection.to_string(), key.to_string());
let matches = match (&expected, data.get(&k)) {
(None, None) => true,
(Some(exp), Some(cur)) => exp == cur,
_ => false,
};
if matches {
data.insert(k, new);
}
Ok(matches)
}
async fn delete(
&self,
group_id: GroupId,
@@ -370,6 +440,36 @@ mod tests {
let data = self.data.lock().await;
Ok(data.keys().filter(|(g, _, _)| *g == group_id).count() as u64)
}
async fn total_bytes(&self, group_id: GroupId) -> Result<u64, GroupKvRepoError> {
let data = self.data.lock().await;
Ok(data
.iter()
.filter(|((g, _, _), _)| *g == group_id)
.map(|(_, v)| serde_json::to_vec(v).map_or(0, |b| b.len() as u64))
.sum())
}
/// Same projection the Postgres impl computes in SQL, but with this
/// mock's own (compact) metric: every OTHER row in the group, plus the
/// new value — i.e. `used - old + new`, measured consistently.
async fn projected_total_bytes(
&self,
group_id: GroupId,
collection: &str,
key: &str,
new_value: &serde_json::Value,
) -> Result<u64, GroupKvRepoError> {
let data = self.data.lock().await;
let target = (group_id, collection.to_string(), key.to_string());
let others: u64 = data
.iter()
.filter(|(k, _)| k.0 == group_id && **k != target)
.map(|(_, v)| serde_json::to_vec(v).map_or(0, |b| b.len() as u64))
.sum();
let new_len = serde_json::to_vec(new_value).map_or(0, |b| b.len() as u64);
Ok(others + new_len)
}
}
/// Maps `(app_id, lowercased name) -> owning group`. Anything absent is
@@ -488,6 +588,56 @@ mod tests {
.expect("update of an existing key must be allowed at the cap");
}
#[tokio::test]
async fn per_group_byte_quota_uses_the_projected_total() {
// §11.6 M4: the byte ceiling is checked on the PROJECTED total, so a
// write that would push the group over is rejected, but a same/smaller
// update near the cap is still allowed (unlike a naive used+new check).
let app = AppId::new();
let group = GroupId::new();
let mut resolver = FakeResolver::default();
resolver.map.insert((app, "catalog".into()), group);
// A 20-byte ceiling; each `"xxxxx"` value encodes to 7 bytes ("\"xxxxx\"").
let kv = GroupKvServiceImpl::new(
Arc::new(InMemoryGroupKvRepo::default()),
Arc::new(resolver),
Arc::new(DenyingAuthzRepo),
)
.with_max_rows(1000)
.with_max_total_bytes(20);
let cx = cx_with(app, Some(owner()));
kv.set(&cx, "catalog", "a", serde_json::json!("xxxxx"))
.await
.unwrap(); // used = 7
kv.set(&cx, "catalog", "b", serde_json::json!("xxxxx"))
.await
.unwrap(); // used = 14
// A third 7-byte value → projected 21 > 20 → rejected.
let err = kv
.set(&cx, "catalog", "c", serde_json::json!("xxxxx"))
.await
.unwrap_err();
assert!(
matches!(err, GroupKvError::TotalBytesQuotaExceeded { limit: 20, .. }),
"got {err:?}"
);
// A same-size update of an existing key stays within budget (projected
// = 14 - 7 + 7 = 14 ≤ 20) — the delta rule, not used+new.
kv.set(&cx, "catalog", "a", serde_json::json!("yyyyy"))
.await
.expect("a same-size update near the cap must be allowed");
// Growing an existing value past the cap is still rejected.
let err = kv
.set(&cx, "catalog", "a", serde_json::json!("zzzzzzzzzzzzzzz"))
.await
.unwrap_err();
assert!(
matches!(err, GroupKvError::TotalBytesQuotaExceeded { .. }),
"got {err:?}"
);
}
#[tokio::test]
async fn unrelated_app_gets_collection_not_shared() {
// app_a's chain declares "catalog"; app_b's does not.
@@ -555,4 +705,56 @@ mod tests {
let err = kv.get(&cx, "", "k").await.unwrap_err();
assert!(matches!(err, GroupKvError::InvalidCollection));
}
#[tokio::test]
async fn set_if_is_compare_and_swap_and_writes_fail_closed() {
let app = AppId::new();
let group = GroupId::new();
let mut resolver = FakeResolver::default();
resolver.map.insert((app, "catalog".into()), group);
let kv = svc(resolver);
let cx = cx_with(app, Some(owner()));
// Insert-if-absent, then swap-if-equal (as the per-app path).
assert!(kv
.set_if(&cx, "catalog", "k", None, serde_json::json!(1))
.await
.unwrap());
assert!(!kv
.set_if(&cx, "catalog", "k", None, serde_json::json!(2))
.await
.unwrap());
assert!(!kv
.set_if(
&cx,
"catalog",
"k",
Some(serde_json::json!(9)),
serde_json::json!(3)
)
.await
.unwrap());
assert!(kv
.set_if(
&cx,
"catalog",
"k",
Some(serde_json::json!(1)),
serde_json::json!(3)
)
.await
.unwrap());
assert_eq!(
kv.get(&cx, "catalog", "k").await.unwrap(),
Some(serde_json::json!(3))
);
// A shared-collection CAS still fails closed for an anonymous writer.
let anon = cx_with(app, None);
let err = kv
.set_if(&anon, "catalog", "k", None, serde_json::json!(4))
.await
.unwrap_err();
assert!(matches!(err, GroupKvError::Forbidden));
}
}

View File

@@ -16,7 +16,10 @@
use std::sync::Arc;
use async_trait::async_trait;
use picloud_shared::{GroupId, GroupPubsubError, GroupPubsubService, SdkCallCx, TriggerEvent};
use picloud_shared::{
GroupId, GroupPubsubError, GroupPubsubService, RealtimeBroadcaster, RealtimeEvent, SdkCallCx,
TriggerEvent,
};
use crate::authz::{self, AuthzRepo, Capability};
use crate::group_collection_repo::GroupCollectionResolver;
@@ -31,6 +34,10 @@ pub struct GroupPubsubServiceImpl {
resolver: Arc<dyn GroupCollectionResolver>,
authz: Arc<dyn AuthzRepo>,
max_message_bytes: usize,
/// §11.6 D2: the realtime broadcaster, attached by the host via
/// [`Self::with_realtime`]. `None` in test bundles → no external SSE fan-out
/// (durable trigger fan-out is unaffected).
realtime: Option<Arc<dyn RealtimeBroadcaster>>,
}
impl GroupPubsubServiceImpl {
@@ -45,9 +52,19 @@ impl GroupPubsubServiceImpl {
resolver,
authz,
max_message_bytes: pubsub_max_message_bytes_from_env(),
realtime: None,
}
}
/// §11.6 D2: attach the realtime broadcaster so a shared-topic publish also
/// fans out to external SSE subscribers (in addition to the durable
/// `shared = true` trigger fan-out). Mirrors `PubsubServiceImpl::with_realtime`.
#[must_use]
pub fn with_realtime(mut self, broadcaster: Arc<dyn RealtimeBroadcaster>) -> Self {
self.realtime = Some(broadcaster);
self
}
/// The structural boundary: resolve the topic namespace to its owning group
/// on the calling app's chain. `CollectionNotShared` when no ancestor group
/// declares it — the same outcome a foreign app gets, by construction.
@@ -107,10 +124,11 @@ impl GroupPubsubService for GroupPubsubServiceImpl {
} else {
format!("{namespace}.{subtopic}")
};
let published_at = chrono::Utc::now();
let event = TriggerEvent::Pubsub {
topic: topic.clone(),
message,
published_at: chrono::Utc::now(),
message: message.clone(),
published_at,
};
let payload = serde_json::to_value(&event)
.map_err(|e| GroupPubsubError::Backend(format!("event serialize: {e}")))?;
@@ -120,9 +138,316 @@ impl GroupPubsubService for GroupPubsubServiceImpl {
trigger_depth: cx.trigger_depth,
root_execution_id: cx.root_execution_id,
};
self.repo
// Durable trigger fan-out FIRST (the authoritative delivery path).
let count = self
.repo
.fan_out_shared_publish(publish_ctx, group_id, &topic, payload)
.await
.map_err(|e| GroupPubsubError::Backend(e.to_string()))
.map_err(|e| GroupPubsubError::Backend(e.to_string()))?;
// Then the best-effort realtime broadcast to external SSE subscribers,
// keyed by the OWNING GROUP so every subtree subscriber shares one
// channel. A broadcast failure never fails the publish.
if let Some(realtime) = &self.realtime {
realtime
.publish_group(
group_id,
&topic,
RealtimeEvent {
topic: topic.clone(),
message,
published_at,
},
)
.await;
}
Ok(count)
}
}
#[cfg(test)]
mod tests {
use super::*;
use picloud_shared::{
AdminUserId, AppId, BroadcasterError, ExecutionId, InstanceRole, Principal,
RealtimeBroadcaster, RealtimeEvent, RequestId, ScriptId,
};
use std::sync::Mutex;
use tokio::sync::broadcast;
use crate::authz::AuthzError;
use crate::pubsub_repo::PubsubRepoError;
/// Records `publish_group` calls so the bridge can be asserted.
#[derive(Default)]
struct RecordingBroadcaster {
group_publishes: Mutex<Vec<(GroupId, String, serde_json::Value)>>,
}
#[async_trait]
impl RealtimeBroadcaster for RecordingBroadcaster {
async fn subscribe(
&self,
_: picloud_shared::AppId,
_: &str,
) -> Result<broadcast::Receiver<RealtimeEvent>, BroadcasterError> {
Ok(broadcast::channel(1).1)
}
async fn publish(&self, _: picloud_shared::AppId, _: &str, _: RealtimeEvent) {}
async fn drop_topic(&self, _: picloud_shared::AppId, _: &str) {}
async fn publish_group(&self, group_id: GroupId, topic: &str, event: RealtimeEvent) {
self.group_publishes
.lock()
.unwrap()
.push((group_id, topic.to_string(), event.message));
}
}
struct FakeRepo;
#[async_trait]
impl PubsubRepo for FakeRepo {
async fn fan_out_publish(
&self,
_: PublishCtx,
_: &str,
_: serde_json::Value,
) -> Result<u32, PubsubRepoError> {
Ok(0)
}
async fn fan_out_shared_publish(
&self,
_: PublishCtx,
_: GroupId,
_: &str,
_: serde_json::Value,
) -> Result<u32, PubsubRepoError> {
Ok(2) // pretend two shared triggers matched
}
}
struct FixedResolver(GroupId);
#[async_trait]
impl GroupCollectionResolver for FixedResolver {
async fn resolve_owning_group(
&self,
_: AppId,
_: &str,
kind: &str,
) -> Result<Option<GroupId>, sqlx::Error> {
Ok((kind == KIND_TOPIC).then_some(self.0))
}
}
struct DenyAuthz;
#[async_trait]
impl AuthzRepo for DenyAuthz {
async fn membership(
&self,
_: picloud_shared::UserId,
_: AppId,
) -> Result<Option<picloud_shared::AppRole>, AuthzError> {
Ok(None)
}
}
fn owner_cx(app_id: AppId) -> SdkCallCx {
SdkCallCx {
app_id,
script_id: ScriptId::new(),
// Instance Owner bypasses the capability check (as in the group_kv
// tests), so DenyAuthz is fine — we're testing the broadcast bridge.
principal: Some(Principal {
user_id: AdminUserId::new(),
instance_role: InstanceRole::Owner,
scopes: None,
app_binding: None,
}),
execution_id: ExecutionId::new(),
request_id: RequestId::new(),
trigger_depth: 0,
root_execution_id: ExecutionId::new(),
is_dead_letter_handler: false,
event: None,
}
}
#[tokio::test]
async fn publish_bridges_to_the_group_broadcaster() {
// §11.6 D2: a shared-topic publish fans out to the durable triggers AND,
// when a broadcaster is attached, to the OWNING GROUP's realtime channel
// with the full topic (`namespace.subtopic`).
let group = GroupId::new();
let app = AppId::new();
let recorder = Arc::new(RecordingBroadcaster::default());
let svc = GroupPubsubServiceImpl::new(
Arc::new(FakeRepo),
Arc::new(FixedResolver(group)),
Arc::new(DenyAuthz),
)
.with_realtime(recorder.clone());
let count = svc
.publish(
&owner_cx(app),
"events",
"created",
serde_json::json!({ "id": 1 }),
)
.await
.unwrap();
assert_eq!(count, 2, "durable fan-out count is returned");
let calls = recorder.group_publishes.lock().unwrap();
assert_eq!(calls.len(), 1, "one realtime broadcast");
assert_eq!(calls[0].0, group, "keyed by the owning group");
assert_eq!(calls[0].1, "events.created", "full topic");
assert_eq!(calls[0].2, serde_json::json!({ "id": 1 }));
}
#[tokio::test]
async fn publish_without_broadcaster_still_fans_out() {
// No broadcaster attached → durable fan-out only, no panic.
let group = GroupId::new();
let svc = GroupPubsubServiceImpl::new(
Arc::new(FakeRepo),
Arc::new(FixedResolver(group)),
Arc::new(DenyAuthz),
);
let count = svc
.publish(&owner_cx(AppId::new()), "events", "", serde_json::json!({}))
.await
.unwrap();
assert_eq!(count, 2);
}
// ------------------------------------------------------------------
// The two invariants the three older sibling services pin, and this one
// did not. Both existing tests above use `FixedResolver` (always resolves)
// and `owner_cx` (an instance Owner, who bypasses the capability check) —
// so neither the isolation boundary nor the anon gate was exercised at all.
// ------------------------------------------------------------------
/// Resolves a topic only for apps whose chain declares it — the fake stand-in
/// for the ancestor walk. The real walk is pinned against Postgres in
/// `tests/group_collection_isolation.rs`.
#[derive(Default)]
struct ChainResolver {
map: std::collections::HashMap<(AppId, String), GroupId>,
}
#[async_trait]
impl GroupCollectionResolver for ChainResolver {
async fn resolve_owning_group(
&self,
app_id: AppId,
name: &str,
kind: &str,
) -> Result<Option<GroupId>, sqlx::Error> {
if kind != KIND_TOPIC {
return Ok(None);
}
Ok(self.map.get(&(app_id, name.to_lowercase())).copied())
}
}
fn cx(app_id: AppId, principal: Option<Principal>) -> SdkCallCx {
SdkCallCx {
app_id,
script_id: ScriptId::new(),
principal,
execution_id: ExecutionId::new(),
request_id: RequestId::new(),
trigger_depth: 0,
root_execution_id: ExecutionId::new(),
is_dead_letter_handler: false,
event: None,
}
}
fn member_no_role() -> Principal {
Principal {
user_id: AdminUserId::new(),
instance_role: InstanceRole::Member,
scopes: None,
app_binding: None,
}
}
fn owner_principal() -> Principal {
Principal {
user_id: AdminUserId::new(),
instance_role: InstanceRole::Owner,
scopes: None,
app_binding: None,
}
}
#[tokio::test]
async fn unrelated_app_gets_collection_not_shared() {
let (app_a, app_b, group) = (AppId::new(), AppId::new(), GroupId::new());
let mut resolver = ChainResolver::default();
resolver.map.insert((app_a, "events".into()), group);
let svc = GroupPubsubServiceImpl::new(
Arc::new(FakeRepo),
Arc::new(resolver),
Arc::new(DenyAuthz),
);
// Positive control — app_a, on the chain, publishes fine.
svc.publish(&owner_cx(app_a), "events", "created", serde_json::json!({}))
.await
.expect("an app on the chain can publish");
// THE BOUNDARY — app_b is in another subtree.
let err = svc
.publish(
&cx(app_b, Some(owner_principal())),
"events",
"created",
serde_json::json!({}),
)
.await
.unwrap_err();
assert!(
matches!(&err, GroupPubsubError::CollectionNotShared(t) if t == "events"),
"a foreign app must not publish into another subtree's shared topic, got {err:?}"
);
}
#[tokio::test]
async fn publish_fails_closed_for_anon() {
let (app, group) = (AppId::new(), GroupId::new());
let mut resolver = ChainResolver::default();
resolver.map.insert((app, "events".into()), group);
let svc = GroupPubsubServiceImpl::new(
Arc::new(FakeRepo),
Arc::new(resolver),
Arc::new(DenyAuthz),
);
// A shared publish fans out to every `shared = true` trigger on the group,
// each handler running under the writer's app_id. An anonymous public route
// must not be able to drive that. This is the assertion that fires if
// `script_gate_require_principal` is swapped for `script_gate`.
let err = svc
.publish(&cx(app, None), "events", "created", serde_json::json!({}))
.await
.unwrap_err();
assert!(
matches!(err, GroupPubsubError::Forbidden),
"an ANONYMOUS shared publish must fail closed, got {err:?}"
);
// Authenticated but with no editor+ on the owning group: also denied.
let err = svc
.publish(
&cx(app, Some(member_no_role())),
"events",
"created",
serde_json::json!({}),
)
.await
.unwrap_err();
assert!(
matches!(err, GroupPubsubError::Forbidden),
"authentication alone is not authorization — editor+ on the owning group is required"
);
}
}

View File

@@ -9,9 +9,10 @@
//! all claiming this one store with `FOR UPDATE SKIP LOCKED`, so each message is
//! delivered at-most-once across the subtree).
//!
//! Deferred (documented): shared-queue dead-lettering an exhausted message is
//! nacked with backoff by the dispatcher (never silently dropped), but there is
//! no group dead-letter store yet.
//! §11.6 D3 (dead-lettering): an exhausted shared-queue message moves to the
//! group dead-letter store (`group_dead_letters`, 0068) via [`GroupQueueRepo::dead_letter`]
//! instead of being dropped — operator-visible at GET .../groups/{id}/dead-letters.
//! Deferred: fan-out to a *shared* dead-letter TRIGGER (needs a new trigger kind).
use async_trait::async_trait;
use chrono::{DateTime, Utc};
@@ -83,14 +84,35 @@ pub trait GroupQueueRepo: Send + Sync {
retry_delay: chrono::Duration,
) -> Result<bool, GroupQueueRepoError>;
/// Drop a message that exhausted its attempts (no group dead-letter store
/// yet — see the module deferral note). Deletes iff `claim_token` matches.
async fn drop_exhausted(
/// TRANSIENT release (handler never ran): re-queue AND undo `claim`'s
/// pre-increment of `attempt`, so a non-execution doesn't burn the retry
/// budget. Mirrors `queue_repo::release`. Floors `attempt` at 0.
async fn release(
&self,
message_id: QueueMessageId,
claim_token: Uuid,
retry_delay: chrono::Duration,
) -> Result<bool, GroupQueueRepoError>;
/// §11.6 D3: a message exhausted its attempts — move it to the group dead-
/// letter store (`group_dead_letters`) and delete it from the live queue, in
/// one transaction (mirrors `queue_repo::dead_letter`). Filtered by
/// `claim_token` so a lost lease can't dead-letter a re-claimed message.
/// Returns the new dead-letter id.
#[allow(clippy::too_many_arguments)]
async fn dead_letter(
&self,
message_id: QueueMessageId,
claim_token: Uuid,
group_id: GroupId,
collection: &str,
trigger_id: Option<picloud_shared::TriggerId>,
script_id: Option<picloud_shared::ScriptId>,
attempt: u32,
first_attempt_at: DateTime<Utc>,
last_error: &str,
) -> Result<picloud_shared::DeadLetterId, GroupQueueRepoError>;
/// Periodic safety net: clear claims older than a shared-queue consumer's
/// `visibility_timeout_secs`. Returns the number of rows reclaimed.
async fn reclaim_visibility_timeouts(&self) -> Result<u64, GroupQueueRepoError>;
@@ -209,20 +231,95 @@ impl GroupQueueRepo for PostgresGroupQueueRepo {
Ok(res.rows_affected() == 1)
}
async fn drop_exhausted(
async fn release(
&self,
message_id: QueueMessageId,
claim_token: Uuid,
retry_delay: chrono::Duration,
) -> Result<bool, GroupQueueRepoError> {
let res =
sqlx::query("DELETE FROM group_queue_messages WHERE id = $1 AND claim_token = $2")
.bind(message_id.into_inner())
.bind(claim_token)
.execute(&self.pool)
.await?;
// TRANSIENT release (handler never ran): re-queue AND undo `claim`'s
// pre-increment of `attempt` so a non-execution doesn't burn the retry
// budget. Mirrors `queue_repo::release`. Floors at 0.
let next = Utc::now() + retry_delay;
let res = sqlx::query(
"UPDATE group_queue_messages \
SET claim_token = NULL, claimed_at = NULL, deliver_after = $3, \
attempt = GREATEST(attempt - 1, 0) \
WHERE id = $1 AND claim_token = $2",
)
.bind(message_id.into_inner())
.bind(claim_token)
.bind(next)
.execute(&self.pool)
.await?;
Ok(res.rows_affected() == 1)
}
#[allow(clippy::too_many_arguments)]
async fn dead_letter(
&self,
message_id: QueueMessageId,
claim_token: Uuid,
group_id: GroupId,
collection: &str,
trigger_id: Option<picloud_shared::TriggerId>,
script_id: Option<picloud_shared::ScriptId>,
attempt: u32,
first_attempt_at: DateTime<Utc>,
last_error: &str,
) -> Result<picloud_shared::DeadLetterId, GroupQueueRepoError> {
let mut tx = self.pool.begin().await?;
// Pull the row inside the tx for its payload; filtered by claim_token so
// a lost lease can't dead-letter a re-claimed message.
let row: DeadLetterSourceRow = sqlx::query_as(
"SELECT id, payload, enqueued_at FROM group_queue_messages \
WHERE id = $1 AND claim_token = $2",
)
.bind(message_id.into_inner())
.bind(claim_token)
.fetch_one(&mut *tx)
.await?;
let payload = serde_json::json!({
"source": "queue",
"queue_name": collection,
"message": row.payload,
"enqueued_at": row.enqueued_at,
"attempt": attempt,
"message_id": row.id.to_string(),
});
let dl_id = picloud_shared::DeadLetterId::new();
sqlx::query(
"INSERT INTO group_dead_letters ( \
id, group_id, collection, original_event_id, source, op, \
trigger_id, script_id, payload, attempt_count, \
first_attempt_at, last_attempt_at, last_error \
) VALUES ($1, $2, $3, $4, 'queue', 'receive', $5, $6, $7, $8, $9, NOW(), $10)",
)
.bind(dl_id.into_inner())
.bind(group_id.into_inner())
.bind(collection)
.bind(row.id)
.bind(trigger_id.map(picloud_shared::TriggerId::into_inner))
.bind(script_id.map(picloud_shared::ScriptId::into_inner))
.bind(&payload)
.bind(i32::try_from(attempt).unwrap_or(0))
.bind(first_attempt_at)
.bind(last_error)
.execute(&mut *tx)
.await?;
sqlx::query("DELETE FROM group_queue_messages WHERE id = $1 AND claim_token = $2")
.bind(message_id.into_inner())
.bind(claim_token)
.execute(&mut *tx)
.await?;
tx.commit().await?;
Ok(dl_id)
}
async fn reclaim_visibility_timeouts(&self) -> Result<u64, GroupQueueRepoError> {
// A shared-queue consumer is a materialized app-owned trigger
// (`shared = TRUE`, `group_id` on the SOURCE template) whose
@@ -292,3 +389,11 @@ struct ClaimedRow {
attempt: i32,
max_attempts: i32,
}
/// Minimal projection of a to-be-dead-lettered message: its payload + timing.
#[derive(sqlx::FromRow)]
struct DeadLetterSourceRow {
id: Uuid,
payload: serde_json::Value,
enqueued_at: DateTime<Utc>,
}

View File

@@ -150,3 +150,251 @@ impl GroupQueueService for GroupQueueServiceImpl {
.map_err(|e| GroupQueueError::Backend(e.to_string()))
}
}
/// This service shipped with NO tests at all, while its three older siblings
/// (`group_kv_service`, `group_docs_service`, `group_files_service`) each pin the
/// same two invariants. Both matter more here than anywhere else: a shared-queue
/// consumer is materialized into EVERY descendant app, so an unauthorized enqueue
/// is attacker-controlled input fanned out as script execution across a whole
/// subtree.
///
/// The gap was concrete, not theoretical: swapping `script_gate_require_principal`
/// for `script_gate` (a one-token edit; the latter returns `Ok(())` for an
/// anonymous principal) would let an unauthenticated public HTTP route enqueue
/// into any ancestor group's shared queue — and nothing would have failed.
#[cfg(test)]
mod tests {
use super::*;
use crate::authz::AuthzError;
use crate::group_queue_repo::{ClaimedGroupMessage, GroupQueueRepoError};
use picloud_shared::{
AdminUserId, AppId, AppRole, ExecutionId, InstanceRole, Principal, RequestId, ScriptId,
UserId,
};
use std::collections::HashMap;
use uuid::Uuid;
/// Only `enqueue` / `depth` / `depth_pending` are reachable from the service —
/// consumption is the dispatcher's job — so the claim side is deliberately
/// unreachable rather than faked.
#[derive(Default)]
struct RecordingRepo {
enqueued: tokio::sync::Mutex<Vec<(GroupId, String)>>,
}
#[async_trait]
impl GroupQueueRepo for RecordingRepo {
async fn enqueue(
&self,
msg: NewGroupQueueMessage,
) -> Result<QueueMessageId, GroupQueueRepoError> {
self.enqueued
.lock()
.await
.push((msg.group_id, msg.collection));
Ok(QueueMessageId::new())
}
async fn claim(
&self,
_: GroupId,
_: &str,
) -> Result<Option<ClaimedGroupMessage>, GroupQueueRepoError> {
unreachable!("the service never consumes")
}
async fn ack(&self, _: QueueMessageId, _: Uuid) -> Result<bool, GroupQueueRepoError> {
unreachable!("the service never consumes")
}
async fn nack(
&self,
_: QueueMessageId,
_: Uuid,
_: chrono::Duration,
) -> Result<bool, GroupQueueRepoError> {
unreachable!("the service never consumes")
}
async fn release(
&self,
_: QueueMessageId,
_: Uuid,
_: chrono::Duration,
) -> Result<bool, GroupQueueRepoError> {
unreachable!("the service never consumes")
}
#[allow(clippy::too_many_arguments)]
async fn dead_letter(
&self,
_: QueueMessageId,
_: Uuid,
_: GroupId,
_: &str,
_: Option<picloud_shared::TriggerId>,
_: Option<ScriptId>,
_: u32,
_: chrono::DateTime<Utc>,
_: &str,
) -> Result<picloud_shared::DeadLetterId, GroupQueueRepoError> {
unreachable!("the service never consumes")
}
async fn reclaim_visibility_timeouts(&self) -> Result<u64, GroupQueueRepoError> {
unreachable!("the service never consumes")
}
async fn depth(&self, _: GroupId, _: &str) -> Result<u64, GroupQueueRepoError> {
Ok(7)
}
async fn depth_pending(&self, _: GroupId, _: &str) -> Result<u64, GroupQueueRepoError> {
Ok(3)
}
}
/// Models the ancestor-chain walk: a queue resolves only for apps whose chain
/// declares it, and only under `kind = "queue"`. The REAL walk is pinned
/// against Postgres in `tests/group_collection_isolation.rs`; this stands in
/// for it so the service's own plumbing can be tested without a DB.
#[derive(Default)]
struct FakeResolver {
map: HashMap<(AppId, String), GroupId>,
}
#[async_trait]
impl GroupCollectionResolver for FakeResolver {
async fn resolve_owning_group(
&self,
app_id: AppId,
name: &str,
kind: &str,
) -> Result<Option<GroupId>, sqlx::Error> {
if kind != KIND_QUEUE {
return Ok(None);
}
Ok(self.map.get(&(app_id, name.to_lowercase())).copied())
}
}
struct DenyingAuthzRepo;
#[async_trait]
impl AuthzRepo for DenyingAuthzRepo {
async fn membership(&self, _: UserId, _: AppId) -> Result<Option<AppRole>, AuthzError> {
Ok(None)
}
}
fn cx_with(app_id: AppId, principal: Option<Principal>) -> SdkCallCx {
SdkCallCx {
app_id,
script_id: ScriptId::new(),
principal,
execution_id: ExecutionId::new(),
request_id: RequestId::new(),
trigger_depth: 0,
root_execution_id: ExecutionId::new(),
is_dead_letter_handler: false,
event: None,
}
}
fn owner() -> Principal {
Principal {
user_id: AdminUserId::new(),
instance_role: InstanceRole::Owner,
scopes: None,
app_binding: None,
}
}
fn member_no_role() -> Principal {
Principal {
user_id: AdminUserId::new(),
instance_role: InstanceRole::Member,
scopes: None,
app_binding: None,
}
}
fn svc(resolver: FakeResolver) -> GroupQueueServiceImpl {
GroupQueueServiceImpl::new(
Arc::new(RecordingRepo::default()),
Arc::new(resolver),
Arc::new(DenyingAuthzRepo),
)
}
#[tokio::test]
async fn unrelated_app_gets_collection_not_shared() {
// app_a's chain declares "jobs"; app_b's does not.
let (app_a, app_b, group) = (AppId::new(), AppId::new(), GroupId::new());
let mut resolver = FakeResolver::default();
resolver.map.insert((app_a, "jobs".into()), group);
let q = svc(resolver);
// Positive control: app_a really can enqueue and read the depth. Without
// this, a service that rejected EVERYTHING would pass the negative below.
let cx_a = cx_with(app_a, Some(owner()));
q.enqueue(
&cx_a,
"jobs",
serde_json::json!({"n": 1}),
EnqueueOpts::default(),
)
.await
.expect("an app on the chain can enqueue");
assert_eq!(q.depth(&cx_a, "jobs").await.unwrap(), 7);
// THE BOUNDARY: app_b is off-chain, so the name does not resolve — on the
// write path and on the read path alike.
let cx_b = cx_with(app_b, Some(owner()));
let err = q
.enqueue(&cx_b, "jobs", serde_json::json!({}), EnqueueOpts::default())
.await
.unwrap_err();
assert!(
matches!(&err, GroupQueueError::CollectionNotShared(c) if c == "jobs"),
"a foreign app must not reach another subtree's shared queue, got {err:?}"
);
let err = q.depth(&cx_b, "jobs").await.unwrap_err();
assert!(matches!(&err, GroupQueueError::CollectionNotShared(c) if c == "jobs"));
}
#[tokio::test]
async fn enqueue_fails_closed_for_anon_while_depth_stays_open() {
let (app, group) = (AppId::new(), GroupId::new());
let mut resolver = FakeResolver::default();
resolver.map.insert((app, "jobs".into()), group);
let q = svc(resolver);
// Reads are open — the declaration IS the grant, anonymous included.
let anon = cx_with(app, None);
assert_eq!(
q.depth(&anon, "jobs").await.unwrap(),
7,
"depth is a read; reads are open to any subtree script"
);
// Writes fail closed. This is the assertion that fires if
// `script_gate_require_principal` is ever swapped for `script_gate`.
let err = q
.enqueue(&anon, "jobs", serde_json::json!({}), EnqueueOpts::default())
.await
.unwrap_err();
assert!(
matches!(err, GroupQueueError::Forbidden),
"an ANONYMOUS enqueue must fail closed — a public route must not be able to \
inject work that every descendant app's consumer will execute; got {err:?}"
);
// Authenticated but without an editor+ role on the owning group: also denied.
let member = cx_with(app, Some(member_no_role()));
let err = q
.enqueue(
&member,
"jobs",
serde_json::json!({}),
EnqueueOpts::default(),
)
.await
.unwrap_err();
assert!(
matches!(err, GroupQueueError::Forbidden),
"authentication alone is not authorization — editor+ on the owning group is required"
);
}
}

View File

@@ -1,44 +0,0 @@
//! §11.6 per-group quotas — global env-var ceilings on a group's shared
//! collections, enforced in the group write path (mirrors the per-value
//! `PICLOUD_*_MAX_*_BYTES` caps). One default applies to every group; per-group
//! configurable limits are deferred.
//!
//! - KV / docs: a per-group ROW-count ceiling (across the group's collections of
//! that kind). An update of an existing key/doc is net-zero and exempt.
//! - files: a per-group TOTAL-BYTES ceiling (sum of stored blob sizes).
/// Default per-group shared-KV row ceiling. Override `PICLOUD_GROUP_KV_MAX_ROWS`.
pub const DEFAULT_GROUP_KV_MAX_ROWS: u64 = 100_000;
/// Default per-group shared-docs row ceiling. Override `PICLOUD_GROUP_DOCS_MAX_ROWS`.
pub const DEFAULT_GROUP_DOCS_MAX_ROWS: u64 = 100_000;
/// Default per-group shared-files total-bytes ceiling (10 GiB). Override
/// `PICLOUD_GROUP_FILES_MAX_TOTAL_BYTES`.
pub const DEFAULT_GROUP_FILES_MAX_TOTAL_BYTES: u64 = 10 * 1024 * 1024 * 1024;
fn u64_from_env(var: &str, default: u64) -> u64 {
if let Ok(v) = std::env::var(var) {
match v.trim().parse::<u64>() {
Ok(n) if n > 0 => return n,
_ => tracing::warn!(value = %v, "ignoring invalid {var} (want a positive integer)"),
}
}
default
}
#[must_use]
pub fn group_kv_max_rows_from_env() -> u64 {
u64_from_env("PICLOUD_GROUP_KV_MAX_ROWS", DEFAULT_GROUP_KV_MAX_ROWS)
}
#[must_use]
pub fn group_docs_max_rows_from_env() -> u64 {
u64_from_env("PICLOUD_GROUP_DOCS_MAX_ROWS", DEFAULT_GROUP_DOCS_MAX_ROWS)
}
#[must_use]
pub fn group_files_max_total_bytes_from_env() -> u64 {
u64_from_env(
"PICLOUD_GROUP_FILES_MAX_TOTAL_BYTES",
DEFAULT_GROUP_FILES_MAX_TOTAL_BYTES,
)
}

View File

@@ -318,8 +318,15 @@ async fn reparent_group(
}
// §4.5 M5: the moved subtree inherits a different set of ancestor-group
// stateful templates — reconcile its materialized copies.
if let Err(e) = crate::materialize::rematerialize_stateful_templates(&s.pool).await {
tracing::warn!(error = %e, "groups: stateful-template materialization after reparent failed; it will self-heal");
match crate::materialize::rematerialize_stateful_templates(&s.pool).await {
Ok(warnings) => {
for w in warnings {
tracing::warn!(warning = %w, "groups: stateful-template materialization warning after reparent");
}
}
Err(e) => {
tracing::warn!(error = %e, "groups: stateful-template materialization after reparent failed; it will self-heal");
}
}
Ok(Json(moved))
}

View File

@@ -0,0 +1,320 @@
//! §9.4 Service-interceptor markers — the `interceptors` table (0073).
//!
//! A marker `(owner, service, op) -> interceptor_script` declares a before-op
//! allow/deny hook. Pure declaration (like `extension_points`, 0051); the
//! interceptor's behaviour is a normal script resolved + run through `invoke()`
//! re-entry. This module holds the read + transactional-write helpers (free
//! functions over `&PgPool` / `&mut Transaction`, keyed by [`ScriptOwner`]),
//! plus the runtime [`resolve_chain`] walk (all markers on the app's chain).
use chrono::{DateTime, Utc};
use picloud_shared::{AppId, ScriptOwner};
use sqlx::{PgPool, Postgres, Transaction};
use uuid::Uuid;
use crate::config_resolver::CHAIN_LEVELS_CTE;
/// One interceptor marker at an owner: which `(service, op)` it guards and the
/// interceptor script name.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct InterceptorMarker {
pub service: String,
pub op: String,
/// `before` (allow/deny + transform) or `after` (observe; §9.4 M3).
pub phase: String,
pub script: String,
/// §9.4 M5: per-hook timeout, or `None` for the instance default.
pub timeout_ms: Option<i32>,
}
/// List the markers declared **directly at** `owner` (not inherited), ordered
/// deterministically. Used by `load_current` (apply diff) and `interceptors ls`.
pub async fn list_for_owner(
pool: &PgPool,
owner: ScriptOwner,
) -> Result<Vec<InterceptorMarker>, sqlx::Error> {
let rows: Vec<(String, String, String, String, Option<i32>)> = match owner {
ScriptOwner::App(a) => {
sqlx::query_as(
"SELECT service, op, phase, interceptor_script, timeout_ms FROM interceptors \
WHERE app_id = $1 ORDER BY service, op, phase",
)
.bind(a.into_inner())
.fetch_all(pool)
.await?
}
ScriptOwner::Group(g) => {
sqlx::query_as(
"SELECT service, op, phase, interceptor_script, timeout_ms FROM interceptors \
WHERE group_id = $1 ORDER BY service, op, phase",
)
.bind(g.into_inner())
.fetch_all(pool)
.await?
}
};
Ok(rows
.into_iter()
.map(
|(service, op, phase, script, timeout_ms)| InterceptorMarker {
service,
op,
phase,
script,
timeout_ms,
},
)
.collect())
}
/// All markers **visible to an app** — declared at the app or any ancestor
/// group (each `(service, op)` resolved nearest-owner-wins). Used by
/// `interceptors ls --app` and by the resolver's chain view.
pub async fn list_on_app_chain(
pool: &PgPool,
app_id: AppId,
) -> Result<Vec<InterceptorMarker>, sqlx::Error> {
let rows: Vec<(String, String, String, String, Option<i32>)> = sqlx::query_as(&format!(
"{CHAIN_LEVELS_CTE} \
SELECT DISTINCT ON (i.service, i.op, i.phase) \
i.service, i.op, i.phase, i.interceptor_script, i.timeout_ms \
FROM chain c \
JOIN interceptors i ON (i.app_id = c.app_owner OR i.group_id = c.group_owner) \
ORDER BY i.service, i.op, i.phase, c.depth ASC",
))
.bind(app_id.into_inner())
.fetch_all(pool)
.await?;
Ok(rows
.into_iter()
.map(
|(service, op, phase, script, timeout_ms)| InterceptorMarker {
service,
op,
phase,
script,
timeout_ms,
},
)
.collect())
}
/// The nearest interceptor marker on an app's chain, with its script resolved
/// **at the declaring owner** (the seal). `script` is `None` when the marker
/// names a script that is missing or disabled at that owner — a dangling hook
/// the caller must fail closed on.
#[derive(Debug, Clone)]
pub struct SealedInterceptor {
pub marker_app: Option<Uuid>,
pub marker_group: Option<Uuid>,
pub script_name: String,
/// §9.4 M5: per-hook wall-clock timeout, or `None` for the instance default.
pub timeout_ms: Option<i32>,
/// `(script_id, source, updated_at)` of the sealed script, or `None` if it
/// is missing/disabled at the declaring owner.
pub script: Option<(Uuid, String, DateTime<Utc>)>,
}
impl SealedInterceptor {
/// The owner that DECLARED the marker (and therefore owns the resolved
/// script) — the lexical origin for the interceptor's own `import`s (§5.5).
#[must_use]
pub fn sealing_owner(&self) -> ScriptOwner {
if let Some(g) = self.marker_group {
ScriptOwner::Group(g.into())
} else {
ScriptOwner::App(
self.marker_app
.expect("marker XOR: app when not group")
.into(),
)
}
}
}
/// The ordered before + after interceptors guarding a `(service, op)` for a
/// calling app, each sealed to its declaring owner (see [`SealedInterceptor`]).
#[derive(Debug, Clone, Default)]
pub struct SealedChain {
/// Ordered ancestor→app (outermost/group guard first).
pub before: Vec<SealedInterceptor>,
/// Ordered app→ancestor.
pub after: Vec<SealedInterceptor>,
}
/// Resolve ALL interceptor markers guarding `(service, op)` for a calling app —
/// every marker visible on the app's chain (app + each ancestor group), each
/// with its script **sealed to the owner that declared the marker** (a
/// descendant app cannot shadow it with a same-named local script). Split by
/// `phase` and ordered: `before` ancestor→app (an outer/group guard runs first,
/// so a group denial cannot be bypassed by a descendant), `after` app→ancestor.
/// **This chain walk is the resolution boundary** — a sibling-subtree app never
/// sees another subtree's interceptor. The script join is `LEFT` so a marker
/// whose script is missing/disabled still returns an entry (with `script:
/// None`) the caller fails closed on, rather than being silently dropped.
///
/// Per the `(owner, service, op, phase)` partial-unique index there is at most
/// one marker per owner per phase, so no per-owner dedup is needed.
pub async fn resolve_chain(
pool: &PgPool,
app_id: AppId,
service: &str,
op: &str,
) -> Result<SealedChain, sqlx::Error> {
#[allow(clippy::type_complexity)]
let rows: Vec<(
Option<Uuid>, // marker_app
Option<Uuid>, // marker_group
String, // script_name
String, // phase
Option<i32>, // timeout_ms
i32, // depth (0 = app, larger = ancestor)
Option<Uuid>, // script id
Option<String>, // script source
Option<DateTime<Utc>>, // script updated_at
)> = sqlx::query_as(&format!(
"{CHAIN_LEVELS_CTE}, \
markers AS ( \
SELECT i.interceptor_script AS script_name, \
i.app_id AS marker_app, i.group_id AS marker_group, \
i.phase AS phase, i.timeout_ms AS timeout_ms, c.depth AS depth \
FROM chain c \
JOIN interceptors i ON (i.app_id = c.app_owner OR i.group_id = c.group_owner) \
WHERE i.service = $2 AND i.op = $3 \
) \
SELECT m.marker_app, m.marker_group, m.script_name, m.phase, m.timeout_ms, m.depth, \
s.id, s.source, s.updated_at \
FROM markers m \
LEFT JOIN scripts s ON ( \
(m.marker_app IS NOT NULL AND s.app_id = m.marker_app \
AND s.name = m.script_name AND s.enabled) \
OR (m.marker_group IS NOT NULL AND s.group_id = m.marker_group \
AND s.name = m.script_name AND s.enabled) \
)",
))
.bind(app_id.into_inner())
.bind(service)
.bind(op)
.fetch_all(pool)
.await?;
// Split by phase and order: `before` ancestor→app = depth DESC (larger
// depth = ancestor runs first); `after` app→ancestor = depth ASC.
let mut before: Vec<(i32, SealedInterceptor)> = Vec::new();
let mut after: Vec<(i32, SealedInterceptor)> = Vec::new();
for (marker_app, marker_group, script_name, phase, timeout_ms, depth, sid, src, upd) in rows {
let sealed = SealedInterceptor {
marker_app,
marker_group,
script_name,
timeout_ms,
script: match (sid, src, upd) {
(Some(id), Some(source), Some(updated_at)) => Some((id, source, updated_at)),
_ => None,
},
};
if phase == "after" {
after.push((depth, sealed));
} else {
before.push((depth, sealed));
}
}
before.sort_by(|a, b| b.0.cmp(&a.0)); // depth DESC → ancestor first
after.sort_by(|a, b| a.0.cmp(&b.0)); // depth ASC → app first
Ok(SealedChain {
before: before.into_iter().map(|(_, s)| s).collect(),
after: after.into_iter().map(|(_, s)| s).collect(),
})
}
/// Upsert a marker at `owner` in the apply transaction. A re-apply that changes
/// only the interceptor script updates it in place (no version churn on the
/// `(service, op)` identity).
pub async fn insert_interceptor_tx(
tx: &mut Transaction<'_, Postgres>,
owner: ScriptOwner,
service: &str,
op: &str,
phase: &str,
script: &str,
timeout_ms: Option<i32>,
) -> Result<(), sqlx::Error> {
match owner {
ScriptOwner::App(a) => {
sqlx::query(
// The conflict arbiter includes `phase` to match the
// per-(owner, service, op, phase) index (§9.4 M3). A re-apply
// that changes only the script or timeout updates in place.
"INSERT INTO interceptors (app_id, service, op, phase, interceptor_script, timeout_ms) \
VALUES ($1, $2, $3, $4, $5, $6) \
ON CONFLICT (app_id, service, op, phase) WHERE app_id IS NOT NULL \
DO UPDATE SET interceptor_script = EXCLUDED.interceptor_script, \
timeout_ms = EXCLUDED.timeout_ms, updated_at = NOW()",
)
.bind(a.into_inner())
.bind(service)
.bind(op)
.bind(phase)
.bind(script)
.bind(timeout_ms)
.execute(&mut **tx)
.await?;
}
ScriptOwner::Group(g) => {
sqlx::query(
"INSERT INTO interceptors (group_id, service, op, phase, interceptor_script, timeout_ms) \
VALUES ($1, $2, $3, $4, $5, $6) \
ON CONFLICT (group_id, service, op, phase) WHERE group_id IS NOT NULL \
DO UPDATE SET interceptor_script = EXCLUDED.interceptor_script, \
timeout_ms = EXCLUDED.timeout_ms, updated_at = NOW()",
)
.bind(g.into_inner())
.bind(service)
.bind(op)
.bind(phase)
.bind(script)
.bind(timeout_ms)
.execute(&mut **tx)
.await?;
}
}
Ok(())
}
/// Delete a marker at `owner` (by `(service, op, phase)`), in the apply
/// transaction. Used by `--prune` when the manifest stops declaring it.
pub async fn delete_interceptor_tx(
tx: &mut Transaction<'_, Postgres>,
owner: ScriptOwner,
service: &str,
op: &str,
phase: &str,
) -> Result<(), sqlx::Error> {
match owner {
ScriptOwner::App(a) => {
sqlx::query(
"DELETE FROM interceptors \
WHERE app_id = $1 AND service = $2 AND op = $3 AND phase = $4",
)
.bind(a.into_inner())
.bind(service)
.bind(op)
.bind(phase)
.execute(&mut **tx)
.await?;
}
ScriptOwner::Group(g) => {
sqlx::query(
"DELETE FROM interceptors \
WHERE group_id = $1 AND service = $2 AND op = $3 AND phase = $4",
)
.bind(g.into_inner())
.bind(service)
.bind(op)
.bind(phase)
.execute(&mut **tx)
.await?;
}
}
Ok(())
}

View File

@@ -0,0 +1,82 @@
//! `InterceptorServiceImpl` — the Postgres-backed §9.4 interceptor resolver
//! injected into `Services`. It maps `(cx.app_id, service, op)` to the nearest
//! interceptor marker on the calling app's chain and materializes its script
//! **sealed to the declaring owner** (via [`crate::interceptor_repo`]). Running
//! that script is the executor's job (the `invoke()` re-entry core), which keeps
//! `executor-core` Postgres-free.
use async_trait::async_trait;
use picloud_shared::{
InterceptorChain, InterceptorEntry, InterceptorService, ResolvedInterceptor, ResolvedScript,
ScriptId, SdkCallCx,
};
use sqlx::PgPool;
use crate::interceptor_repo::SealedInterceptor;
pub struct InterceptorServiceImpl {
pool: PgPool,
}
impl InterceptorServiceImpl {
#[must_use]
pub fn new(pool: PgPool) -> Self {
Self { pool }
}
}
impl InterceptorServiceImpl {
/// Map one sealed marker to a chain entry: a runnable [`ResolvedScript`]
/// (sealed to the declaring owner — the lexical `import` origin, §5.5) or a
/// [`Dangling`](InterceptorEntry::Dangling) entry when the marker's script
/// is missing/disabled. The execution boundary is always the CALLER's app
/// (`cx.app_id`); only the `owner` is the sealing owner.
fn map_entry(app_id: picloud_shared::AppId, sealed: SealedInterceptor) -> InterceptorEntry {
let owner = sealed.sealing_owner();
// A negative/oversized DB value is impossible (CHECK > 0), but clamp
// defensively to `None` (→ instance default) rather than panic.
let timeout_ms = sealed.timeout_ms.and_then(|v| u32::try_from(v).ok());
match sealed.script {
Some((sid, source, updated_at)) => InterceptorEntry::Run(ResolvedInterceptor {
script: Box::new(ResolvedScript {
script_id: ScriptId::from(sid),
app_id,
owner: Some(owner),
source,
updated_at,
name: sealed.script_name,
}),
timeout_ms,
}),
None => InterceptorEntry::Dangling(sealed.script_name),
}
}
}
#[async_trait]
impl InterceptorService for InterceptorServiceImpl {
async fn resolve(
&self,
cx: &SdkCallCx,
service: &str,
op: &str,
) -> Result<InterceptorChain, String> {
// `app_id` derives from `cx` (never a script arg) — the isolation
// boundary; the chain walk then scopes resolution to this app's subtree.
let chain = crate::interceptor_repo::resolve_chain(&self.pool, cx.app_id, service, op)
.await
.map_err(|e| e.to_string())?;
Ok(InterceptorChain {
before: chain
.before
.into_iter()
.map(|s| Self::map_entry(cx.app_id, s))
.collect(),
after: chain
.after
.into_iter()
.map(|s| Self::map_entry(cx.app_id, s))
.collect(),
})
}
}

View File

@@ -319,6 +319,9 @@ mod tests {
}
#[async_trait]
impl OutboxRepo for CapturingOutbox {
async fn reclaim_stale_claims(&self, _timeout_secs: u32) -> Result<u64, OutboxRepoError> {
Ok(0)
}
async fn insert(&self, row: NewOutboxRow) -> Result<uuid::Uuid, OutboxRepoError> {
let id = uuid::Uuid::new_v4();
*self.last.lock().await = Some(row);

View File

@@ -39,6 +39,19 @@ pub trait KvRepo: Send + Sync {
value: serde_json::Value,
) -> Result<Option<serde_json::Value>, KvRepoError>;
/// Compare-and-swap. Atomically writes `new` iff the current value equals
/// `expected` (`expected = None` → iff the key is ABSENT). Returns `true`
/// when the write happened. A single SQL statement, so it is atomic without a
/// transaction.
async fn set_if(
&self,
app_id: AppId,
collection: &str,
key: &str,
expected: Option<serde_json::Value>,
new: serde_json::Value,
) -> Result<bool, KvRepoError>;
/// Returns the deleted value if present, `None` if the row didn't
/// exist. The caller turns the `bool was-present` part into the
/// SDK's return value; the `Option<value>` part feeds the
@@ -78,6 +91,118 @@ impl PostgresKvRepo {
const KV_LIST_MAX_LIMIT: u32 = 1_000;
const KV_LIST_DEFAULT_LIMIT: u32 = 100;
// ----------------------------------------------------------------------------
// Connection-scoped mutations
//
// Generic over the executor so the same SQL serves the pooled `KvRepo` methods
// above and `crate::atomic_write`, which runs the write and the resulting
// trigger fan-out on ONE connection inside ONE transaction.
// ----------------------------------------------------------------------------
/// Upsert. Returns the previous value, so the caller knows whether this was an
/// `insert` or an `update` for the emitted `ServiceEvent`.
pub(crate) async fn set_on<'c, E>(
exec: E,
app_id: AppId,
collection: &str,
key: &str,
value: serde_json::Value,
) -> Result<Option<serde_json::Value>, KvRepoError>
where
E: sqlx::PgExecutor<'c>,
{
// `RETURNING` after `ON CONFLICT DO UPDATE` can't see the old value, so
// capture the prior value in a CTE alongside the upsert.
let row: Option<(Option<serde_json::Value>,)> = sqlx::query_as(
"WITH prev AS (\
SELECT value FROM kv_entries \
WHERE app_id = $1 AND collection = $2 AND key = $3\
), \
upserted AS (\
INSERT INTO kv_entries (app_id, collection, key, value) \
VALUES ($1, $2, $3, $4) \
ON CONFLICT (app_id, collection, key) DO UPDATE \
SET value = EXCLUDED.value, updated_at = NOW() \
RETURNING 1\
) \
SELECT (SELECT value FROM prev) FROM upserted",
)
.bind(app_id.into_inner())
.bind(collection)
.bind(key)
.bind(value)
.fetch_optional(exec)
.await?;
Ok(row.and_then(|(v,)| v))
}
/// Compare-and-swap. Writes `new` iff the current value equals `expected`
/// (`expected = None` → iff the key is ABSENT). Returns whether it wrote.
pub(crate) async fn set_if_on<'c, E>(
exec: E,
app_id: AppId,
collection: &str,
key: &str,
expected: Option<serde_json::Value>,
new: serde_json::Value,
) -> Result<bool, KvRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let affected = match expected {
// Swap iff the current value matches (JSONB `=` is semantic equality).
Some(exp) => sqlx::query(
"UPDATE kv_entries SET value = $4, updated_at = NOW() \
WHERE app_id = $1 AND collection = $2 AND key = $3 AND value = $5",
)
.bind(app_id.into_inner())
.bind(collection)
.bind(key)
.bind(new)
.bind(exp)
.execute(exec)
.await?
.rows_affected(),
// Insert iff absent.
None => sqlx::query(
"INSERT INTO kv_entries (app_id, collection, key, value) \
VALUES ($1, $2, $3, $4) \
ON CONFLICT (app_id, collection, key) DO NOTHING",
)
.bind(app_id.into_inner())
.bind(collection)
.bind(key)
.bind(new)
.execute(exec)
.await?
.rows_affected(),
};
Ok(affected == 1)
}
/// Returns the deleted value if present, `None` if the row didn't exist.
pub(crate) async fn delete_on<'c, E>(
exec: E,
app_id: AppId,
collection: &str,
key: &str,
) -> Result<Option<serde_json::Value>, KvRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let row: Option<(serde_json::Value,)> = sqlx::query_as(
"DELETE FROM kv_entries \
WHERE app_id = $1 AND collection = $2 AND key = $3 \
RETURNING value",
)
.bind(app_id.into_inner())
.bind(collection)
.bind(key)
.fetch_optional(exec)
.await?;
Ok(row.map(|(v,)| v))
}
#[async_trait]
impl KvRepo for PostgresKvRepo {
async fn get(
@@ -105,30 +230,18 @@ impl KvRepo for PostgresKvRepo {
key: &str,
value: serde_json::Value,
) -> Result<Option<serde_json::Value>, KvRepoError> {
// `RETURNING` after `ON CONFLICT DO UPDATE` exposes the old
// value via the `xmax`/old-row trick: capture the prior value
// with a CTE so callers know whether this was insert vs update.
let row: Option<(Option<serde_json::Value>,)> = sqlx::query_as(
"WITH prev AS (\
SELECT value FROM kv_entries \
WHERE app_id = $1 AND collection = $2 AND key = $3\
), \
upserted AS (\
INSERT INTO kv_entries (app_id, collection, key, value) \
VALUES ($1, $2, $3, $4) \
ON CONFLICT (app_id, collection, key) DO UPDATE \
SET value = EXCLUDED.value, updated_at = NOW() \
RETURNING 1\
) \
SELECT (SELECT value FROM prev) FROM upserted",
)
.bind(app_id.into_inner())
.bind(collection)
.bind(key)
.bind(value)
.fetch_optional(&self.pool)
.await?;
Ok(row.and_then(|(v,)| v))
set_on(&self.pool, app_id, collection, key, value).await
}
async fn set_if(
&self,
app_id: AppId,
collection: &str,
key: &str,
expected: Option<serde_json::Value>,
new: serde_json::Value,
) -> Result<bool, KvRepoError> {
set_if_on(&self.pool, app_id, collection, key, expected, new).await
}
async fn delete(
@@ -137,17 +250,7 @@ impl KvRepo for PostgresKvRepo {
collection: &str,
key: &str,
) -> Result<Option<serde_json::Value>, KvRepoError> {
let row: Option<(serde_json::Value,)> = sqlx::query_as(
"DELETE FROM kv_entries \
WHERE app_id = $1 AND collection = $2 AND key = $3 \
RETURNING value",
)
.bind(app_id.into_inner())
.bind(collection)
.bind(key)
.fetch_optional(&self.pool)
.await?;
Ok(row.map(|(v,)| v))
delete_on(&self.pool, app_id, collection, key).await
}
async fn has(&self, app_id: AppId, collection: &str, key: &str) -> Result<bool, KvRepoError> {
@@ -221,3 +324,16 @@ fn decode_cursor(cursor: &str) -> Result<String, KvRepoError> {
.map_err(|_| KvRepoError::InvalidCursor)?;
String::from_utf8(bytes).map_err(|_| KvRepoError::InvalidCursor)
}
/// Total key count for the app, across all its collections. Backs the per-app
/// row ceiling; an index-only count, and only run when a write ADDS a key.
pub(crate) async fn count_rows_on<'c, E>(exec: E, app_id: AppId) -> Result<u64, KvRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let (n,): (i64,) = sqlx::query_as("SELECT COUNT(*) FROM kv_entries WHERE app_id = $1")
.bind(app_id.into_inner())
.fetch_one(exec)
.await?;
Ok(u64::try_from(n).unwrap_or(0))
}

View File

@@ -12,16 +12,17 @@
//! Cross-app isolation isn't affected — every query is keyed by
//! `cx.app_id`, never an argument.
//! 3. `ServiceEvent` emission after each mutation (`insert` / `update`
//! / `delete`). v1.1.0 ships a `NoopEventEmitter` so this is a
//! no-op until the outbox emitter lands later in v1.1.1.
//! / `delete`), via the injected `KvWriter` — which for the host is
//! transactional, so a mutation and the trigger fan-out it produces
//! commit together. See `crate::atomic_write`.
use std::sync::Arc;
use async_trait::async_trait;
use picloud_shared::{
KvError, KvListPage, KvService, SdkCallCx, ServiceEvent, ServiceEventEmitter,
};
use picloud_shared::{KvError, KvListPage, KvService, SdkCallCx, ServiceEventEmitter};
use sqlx::PgPool;
use crate::atomic_write::{BestEffortKvWriter, KvWriter, PostgresKvWriter};
use crate::authz::{self, AuthzRepo, Capability};
use crate::kv_repo::{KvRepo, KvRepoError};
@@ -47,9 +48,11 @@ pub fn kv_max_value_bytes_from_env() -> usize {
}
pub struct KvServiceImpl {
/// Reads only. Mutations go through `writer`, which owns the write AND the
/// trigger fan-out so the two can share a transaction.
repo: Arc<dyn KvRepo>,
authz: Arc<dyn AuthzRepo>,
events: Arc<dyn ServiceEventEmitter>,
writer: Arc<dyn KvWriter>,
max_value_bytes: usize,
}
@@ -71,13 +74,38 @@ impl KvServiceImpl {
max_value_bytes: usize,
) -> Self {
Self {
writer: Arc::new(BestEffortKvWriter::new(repo.clone(), events)),
repo,
authz,
events,
max_value_bytes,
}
}
/// Swap the best-effort writer for the transactional one: the row write and
/// its trigger fan-out then commit together, so an outbox failure rolls the
/// write back instead of silently losing the event. The host always calls
/// this; the in-memory unit tests do not (they have no Postgres).
#[must_use]
pub fn with_atomic_writes(mut self, pool: PgPool, max_rows: u64) -> Self {
self.writer = Arc::new(PostgresKvWriter::new(pool, max_rows));
self
}
/// Encode `value` and enforce the per-key byte cap. Runs before authz so an
/// anonymous public script can't push an oversized payload at Postgres.
fn check_value_size(&self, value: &serde_json::Value) -> Result<(), KvError> {
let encoded_len = serde_json::to_vec(value)
.map(|v| v.len())
.map_err(|e| KvError::Backend(format!("encode value: {e}")))?;
if encoded_len > self.max_value_bytes {
return Err(KvError::ValueTooLarge {
limit: self.max_value_bytes,
actual: encoded_len,
});
}
Ok(())
}
async fn check_read(&self, cx: &SdkCallCx) -> Result<(), KvError> {
authz::script_gate(
&*self.authz,
@@ -135,82 +163,30 @@ impl KvService for KvServiceImpl {
value: serde_json::Value,
) -> Result<(), KvError> {
validate_collection(collection)?;
let encoded_len = serde_json::to_vec(&value)
.map(|v| v.len())
.map_err(|e| KvError::Backend(format!("encode value: {e}")))?;
if encoded_len > self.max_value_bytes {
return Err(KvError::ValueTooLarge {
limit: self.max_value_bytes,
actual: encoded_len,
});
}
self.check_value_size(&value)?;
self.check_write(cx).await?;
let previous = self
.repo
.set(cx.app_id, collection, key, value.clone())
.await?;
let op = if previous.is_some() {
"update"
} else {
"insert"
};
// Emit unconditionally; the noop emitter drops it, the outbox
// emitter persists it.
//
// Audit finding (Medium): this is non-transactional with the
// data write — the row above has already committed by the time
// emit() runs, so an emit failure means triggers silently
// don't fire. Logged at `error` level (with
// `event_emit_failure = true` for grepability); operators see
// the gap. The full transactional refactor — extending the
// ServiceEventEmitter trait with `emit_in_tx` and the KvRepo
// surface with `set_with_tx` — is deferred to a v1.2 design
// pass (pubsub_repo::fan_out_publish is the reference shape).
if let Err(e) = self
.events
.emit(
cx,
ServiceEvent {
source: "kv",
op,
collection: Some(collection.to_string()),
key: Some(key.to_string()),
payload: Some(value),
old_payload: previous,
},
)
.await
{
tracing::error!(error = %e, source = "kv", op, event_emit_failure = true, "event emit failed");
}
self.writer.set(cx, collection, key, value).await?;
Ok(())
}
async fn set_if(
&self,
cx: &SdkCallCx,
collection: &str,
key: &str,
expected: Option<serde_json::Value>,
new: serde_json::Value,
) -> Result<bool, KvError> {
validate_collection(collection)?;
self.check_value_size(&new)?;
self.check_write(cx).await?;
self.writer.set_if(cx, collection, key, expected, new).await
}
async fn delete(&self, cx: &SdkCallCx, collection: &str, key: &str) -> Result<bool, KvError> {
validate_collection(collection)?;
self.check_write(cx).await?;
let previous = self.repo.delete(cx.app_id, collection, key).await?;
let was_present = previous.is_some();
if was_present {
if let Err(e) = self
.events
.emit(
cx,
ServiceEvent {
source: "kv",
op: "delete",
collection: Some(collection.to_string()),
key: Some(key.to_string()),
payload: None,
old_payload: previous,
},
)
.await
{
tracing::error!(error = %e, source = "kv", op = "delete", event_emit_failure = true, "event emit failed");
}
}
Ok(was_present)
Ok(self.writer.delete(cx, collection, key).await?.is_some())
}
async fn has(&self, cx: &SdkCallCx, collection: &str, key: &str) -> Result<bool, KvError> {
@@ -283,6 +259,28 @@ mod tests {
.insert((app_id, collection.to_string(), key.to_string()), value))
}
async fn set_if(
&self,
app_id: AppId,
collection: &str,
key: &str,
expected: Option<serde_json::Value>,
new: serde_json::Value,
) -> Result<bool, KvRepoError> {
// Holding the map lock makes the compare-and-set atomic in-memory.
let mut data = self.data.lock().await;
let k = (app_id, collection.to_string(), key.to_string());
let matches = match (&expected, data.get(&k)) {
(None, None) => true, // insert-if-absent
(Some(exp), Some(cur)) => exp == cur, // swap-if-equal
_ => false,
};
if matches {
data.insert(k, new);
}
Ok(matches)
}
async fn delete(
&self,
app_id: AppId,
@@ -464,6 +462,63 @@ mod tests {
assert!(matches!(err, KvError::InvalidCollection));
}
#[tokio::test]
async fn set_if_is_compare_and_swap() {
let kv = svc();
let cx = anon_cx(AppId::new());
// Insert-if-absent: expected () swaps only when the key is missing.
assert!(
kv.set_if(&cx, "cas", "k", None, serde_json::json!(1))
.await
.unwrap(),
"insert-if-absent must succeed on a missing key"
);
assert!(
!kv.set_if(&cx, "cas", "k", None, serde_json::json!(2))
.await
.unwrap(),
"insert-if-absent must fail once the key exists"
);
assert_eq!(
kv.get(&cx, "cas", "k").await.unwrap(),
Some(serde_json::json!(1))
);
// Swap-if-equal: wrong expected → no-op; correct expected → swaps.
assert!(
!kv.set_if(
&cx,
"cas",
"k",
Some(serde_json::json!(99)),
serde_json::json!(3)
)
.await
.unwrap(),
"a mismatched expected must not swap"
);
assert_eq!(
kv.get(&cx, "cas", "k").await.unwrap(),
Some(serde_json::json!(1))
);
assert!(
kv.set_if(
&cx,
"cas",
"k",
Some(serde_json::json!(1)),
serde_json::json!(3)
)
.await
.unwrap(),
"a matching expected must swap"
);
assert_eq!(
kv.get(&cx, "cas", "k").await.unwrap(),
Some(serde_json::json!(3))
);
}
/// Load-bearing: a script with `cx.app_id = A` must NOT see
/// entries inserted under `cx.app_id = B`. This is the cross-app
/// isolation boundary; getting this wrong is a security
@@ -506,6 +561,48 @@ mod tests {
// No panic, no Forbidden.
}
/// `PICLOUD_KV_MAX_VALUE_BYTES` is called out in CLAUDE.md as an anti-DoS rail:
/// oversized values are rejected "before authz so anonymous public scripts
/// can't DoS Postgres JSONB columns." Nothing asserted it. This does, and it
/// pins the ORDERING — the security-relevant part.
///
/// The cx is an AUTHENTICATED member with no role, which `DenyingAuthzRepo`
/// rejects (see `authed_cx_with_no_role_is_forbidden`). That is what makes the
/// ordering observable: an anon cx passes `script_gate` regardless, so it
/// could not tell the two orders apart. With a denied cx, a size-check-first
/// service returns `ValueTooLarge` and an authz-first one returns `Forbidden`.
#[tokio::test]
async fn oversized_value_is_rejected_before_authz() {
let kv = KvServiceImpl::with_max_value_bytes(
Arc::new(InMemoryKvRepo::default()),
Arc::new(DenyingAuthzRepo),
Arc::new(NoopEventEmitter),
16,
);
let cx = member_no_role_cx(AppId::new());
let err = kv
.set(&cx, "widgets", "k", serde_json::json!("x".repeat(100)))
.await
.unwrap_err();
assert!(
matches!(err, KvError::ValueTooLarge { limit: 16, .. }),
"an oversized value must be rejected as ValueTooLarge BEFORE authz; an \
authz-first order would return Forbidden for this denied cx. got {err:?}"
);
// Control: an under-cap value with the SAME denied cx returns Forbidden —
// proving the cx really is denied, so the case above genuinely bypassed authz.
let err = kv
.set(&cx, "widgets", "k", serde_json::json!("x"))
.await
.unwrap_err();
assert!(
matches!(err, KvError::Forbidden),
"the control confirms this cx is authz-denied; got {err:?}"
);
}
/// Authenticated principal with no role on the app: the
/// `DenyingAuthzRepo` returns no membership, so the capability
/// check denies. Set must surface KvError::Forbidden.

View File

@@ -26,6 +26,7 @@ pub mod app_user_verification_repo;
pub mod apply_api;
pub mod apply_service;
pub mod apps_api;
pub mod atomic_write;
pub mod auth;
pub mod auth_api;
pub mod auth_bootstrap;
@@ -39,6 +40,7 @@ pub mod dead_letter_service;
pub mod dead_letters_api;
pub mod dev_email_api;
pub mod dispatcher;
pub mod docs_api;
pub mod docs_filter;
pub mod docs_repo;
pub mod docs_service;
@@ -52,6 +54,7 @@ pub mod files_sweep;
pub mod gc;
pub mod group_blobs_api;
pub mod group_collection_repo;
pub mod group_dead_letter_repo;
pub mod group_docs_repo;
pub mod group_docs_service;
pub mod group_files_repo;
@@ -62,11 +65,12 @@ pub mod group_members_repo;
pub mod group_pubsub_service;
pub mod group_queue_repo;
pub mod group_queue_service;
pub mod group_quota;
pub mod group_repo;
pub mod group_scripts_api;
pub mod groups_api;
pub mod http_service;
pub mod interceptor_repo;
pub mod interceptor_service;
pub mod invoke_service;
pub mod kv_api;
pub mod kv_repo;
@@ -74,6 +78,7 @@ pub mod kv_service;
pub mod log_sink;
pub mod login_rate_limit;
pub mod materialize;
pub mod metrics_api;
pub mod migrations;
pub mod module_source;
pub mod outbox_event_emitter;
@@ -86,6 +91,7 @@ pub mod pubsub_service;
pub mod queue_repo;
pub mod queue_service;
pub mod queues_api;
pub mod quota;
pub mod realtime_authority;
pub mod repo;
pub mod route_admin;
@@ -107,6 +113,12 @@ pub mod users_service;
pub mod vars_api;
pub mod vars_repo;
pub mod vars_service;
pub mod workflow_expr;
pub mod workflow_orchestrator;
pub mod workflow_repo;
pub mod workflow_service;
pub mod workflow_template;
pub mod workflows_api;
pub use abandoned_repo::{
AbandonedRepo, AbandonedRepoError, NewAbandonedExecution, PostgresAbandonedRepo,
@@ -178,6 +190,7 @@ pub use dead_letter_service::PostgresDeadLetterService;
pub use dead_letters_api::{dead_letters_router, DeadLettersApiError, DeadLettersState};
pub use dev_email_api::{dev_emails_router, DevEmailState};
pub use dispatcher::{compute_backoff, Dispatcher, DispatcherError};
pub use docs_api::{docs_admin_router, DocsAdminState};
pub use docs_repo::{DocsRepo, DocsRepoError, PostgresDocsRepo};
pub use docs_service::DocsServiceImpl;
pub use email_inbound_api::{
@@ -219,6 +232,7 @@ pub use kv_api::{kv_admin_router, KvAdminState};
pub use kv_repo::{KvRepo, KvRepoError, PostgresKvRepo};
pub use kv_service::KvServiceImpl;
pub use log_sink::PostgresExecutionLogSink;
pub use metrics_api::{metrics_router, MetricsState};
pub use module_source::PostgresModuleSource;
pub use outbox_event_emitter::OutboxEventEmitter;
pub use outbox_repo::{
@@ -255,7 +269,8 @@ pub use trigger_repo::{
collection_matches, CreateDeadLetterTrigger, CreateDocsTrigger, CreateEmailTrigger,
CreateFilesTrigger, CreateKvTrigger, CreatePubsubTrigger, DeadLetterTriggerMatch,
DocsTriggerMatch, EmailInboundTarget, FilesTriggerMatch, KvTriggerMatch, PostgresTriggerRepo,
Trigger, TriggerDetails, TriggerDispatchMode, TriggerKind, TriggerRepo, TriggerRepoError,
SmtpInboundTarget, Trigger, TriggerDetails, TriggerDispatchMode, TriggerKind, TriggerRepo,
TriggerRepoError,
};
pub use triggers_api::{triggers_router, TriggersApiError, TriggersState};
pub use users_admin_api::{app_users_router, AppUsersApiError, AppUsersState};
@@ -263,3 +278,6 @@ pub use users_service::{UsersServiceConfig, UsersServiceImpl};
pub use vars_api::{vars_router, VarsApiError, VarsApiState};
pub use vars_repo::{PostgresVarsRepo, VarOwner, VarRow, VarsRepo, VarsRepoError};
pub use vars_service::VarsServiceImpl;
pub use workflow_orchestrator::{spawn_workflow_orchestrator, WorkflowOrchestrator};
pub use workflow_service::WorkflowServiceImpl;
pub use workflows_api::{workflows_router, WorkflowsState};

View File

@@ -14,7 +14,10 @@
//!
//! Full-live like `rebuild_route_table`: called at the same chokepoints (apply,
//! app create/delete, group reparent). A precise create/delete diff (not
//! delete-all-then-recreate) preserves `last_fired_at` on unchanged cron copies.
//! delete-all-then-recreate) preserves `last_fired_at` on unchanged cron copies;
//! a DISABLED cron template keeps its copies (their `enabled` is synced off)
//! rather than deleting them, so a later re-enable resumes from the preserved
//! `last_fired_at` instead of firing from a fresh reference.
//!
//! Returns per-app WARNINGS (e.g. a queue whose consumer slot is already taken)
//! — the caller surfaces them; materialization of the other pairs still
@@ -22,6 +25,7 @@
use std::collections::HashSet;
use picloud_shared::AppId;
use sqlx::PgPool;
use uuid::Uuid;
@@ -86,7 +90,8 @@ pub async fn rematerialize_stateful_templates(pool: &PgPool) -> Result<Vec<Strin
SELECT DISTINCT ac.effective_app_id, t.id AS template_id, t.kind, t.shared \
FROM app_chain ac \
JOIN triggers t ON t.group_id = ac.owner_group \
WHERE t.group_id IS NOT NULL AND t.enabled = TRUE AND t.kind = ANY($1)",
WHERE t.group_id IS NOT NULL AND t.kind = ANY($1) \
AND (t.enabled = TRUE OR t.kind = 'cron')",
)
.bind(&kinds)
.fetch_all(&mut *tx)
@@ -135,6 +140,21 @@ pub async fn rematerialize_stateful_templates(pool: &PgPool) -> Result<Vec<Strin
}
}
// Sync each CRON copy's `enabled` to its template's, in place. A cron
// template's copies are kept across a disable (the `should` query includes
// disabled cron templates), so a disable→re-enable toggles this flag rather
// than delete+recreate — preserving each copy's per-app `last_fired_at` (and
// its `created_at` reference). Without this, a re-enabled cron would reset to
// a fresh reference and lose/shift its schedule state. Cron-only: queue/email
// keep delete-on-disable so the queue one-consumer slot is freed on disable.
sqlx::query(
"UPDATE triggers c SET enabled = t.enabled \
FROM triggers t \
WHERE c.materialized_from = t.id AND t.kind = 'cron' AND c.enabled <> t.enabled",
)
.execute(&mut *tx)
.await?;
tx.commit().await?;
Ok(warnings)
}
@@ -156,22 +176,39 @@ async fn materialize_one(
// as a competing consumer (a different store), so the slot check is skipped
// — every descendant intentionally gets a consumer.
if kind == "queue" && !shared {
let taken: Option<(Uuid,)> = sqlx::query_as(
"SELECT t.id FROM triggers t \
JOIN queue_trigger_details d ON d.trigger_id = t.id \
WHERE t.app_id = $1 AND t.kind = 'queue' \
AND d.queue_name = ( \
SELECT queue_name FROM queue_trigger_details WHERE trigger_id = $2)",
)
.bind(app_id)
.bind(template_id)
.fetch_optional(&mut **tx)
.await?;
if taken.is_some() {
return Ok(Some(format!(
"queue template not materialized for one app — it already has a \
consumer on that queue (app_id={app_id})"
)));
// Resolve the queue name so we can take the SAME per-(app, queue)
// advisory lock the interactive `create_queue_trigger` uses. Without it,
// this check-then-insert (serialized only against other reconcilers by
// REMATERIALIZE_LOCK) could race an interactive consumer-create and
// double-fill the slot — the two paths held different locks.
let queue_name: Option<(String,)> =
sqlx::query_as("SELECT queue_name FROM queue_trigger_details WHERE trigger_id = $1")
.bind(template_id)
.fetch_optional(&mut **tx)
.await?;
if let Some((queue_name,)) = queue_name {
sqlx::query("SELECT pg_advisory_xact_lock($1)")
.bind(crate::trigger_repo::advisory_lock_key(
AppId::from(app_id),
&queue_name,
))
.execute(&mut **tx)
.await?;
let taken: Option<(Uuid,)> = sqlx::query_as(
"SELECT t.id FROM triggers t \
JOIN queue_trigger_details d ON d.trigger_id = t.id \
WHERE t.app_id = $1 AND t.kind = 'queue' AND d.queue_name = $2",
)
.bind(app_id)
.bind(&queue_name)
.fetch_optional(&mut **tx)
.await?;
if taken.is_some() {
return Ok(Some(format!(
"queue template not materialized for one app — it already has a \
consumer on that queue (app_id={app_id})"
)));
}
}
}
@@ -201,7 +238,11 @@ async fn materialize_one(
match kind {
"cron" => {
// A fresh copy starts with last_fired_at NULL → fires on next due.
// A first-ever copy starts with last_fired_at NULL → its reference
// is created_at (now), so it fires at the next scheduled slot, not
// immediately. Re-enabling a previously-materialized template does
// NOT hit this path — that copy is kept and its `enabled` synced, so
// last_fired_at survives (see the enabled-sync UPDATE above).
sqlx::query(
"INSERT INTO cron_trigger_details (trigger_id, schedule, timezone) \
SELECT $1, schedule, timezone FROM cron_trigger_details WHERE trigger_id = $2",
@@ -226,14 +267,17 @@ async fn materialize_one(
"email" => {
// §M5.5: the group template sealed its inbound HMAC secret against
// the GROUP's own store once at apply (shared-group-secret model).
// The secret is v0 (no AAD) — bound only to the master key, not the
// owning row so a verbatim byte-copy is a valid per-app secret; no
// master key is needed here. Each copy has its own trigger_id (its
// own inbound webhook URL).
// The secret's AAD (§M5.5 / audit H-D1) is bound to the GROUP owner,
// NOT this copy's row, so a verbatim byte-copy stays openable — the
// inbound path recovers the group via `materialized_from`. Copy the
// version verbatim too. Each copy has its own trigger_id (its own
// inbound webhook URL).
sqlx::query(
"INSERT INTO email_trigger_details \
(trigger_id, inbound_secret_encrypted, inbound_secret_nonce) \
SELECT $1, inbound_secret_encrypted, inbound_secret_nonce \
(trigger_id, inbound_secret_encrypted, inbound_secret_nonce, \
inbound_secret_version) \
SELECT $1, inbound_secret_encrypted, inbound_secret_nonce, \
inbound_secret_version \
FROM email_trigger_details WHERE trigger_id = $2",
)
.bind(new_id.0)

View File

@@ -0,0 +1,118 @@
//! `/api/v1/admin/apps/{id}/metrics` — read-only observability rollup (A2).
//!
//! Aggregates the existing `execution_logs` table (counts, error rate, latency
//! percentiles, an hourly series) for one app over a trailing window. No
//! hot-path instrumentation: the data plane already persists `duration_ms` /
//! `status` / `source` / `created_at` per run, so this is a pure read model
//! behind the dashboard's Metrics tab.
//!
//! Capability: `AppLogRead` (the same tier as the per-script log view),
//! resolved against the app loaded from the path.
use std::sync::Arc;
use axum::extract::{Path, Query, State};
use axum::response::{IntoResponse, Json, Response};
use axum::routing::get;
use axum::{Extension, Router};
use picloud_shared::{AppId, Principal};
use serde::Deserialize;
use serde_json::json;
use crate::app_repo::AppRepository;
use crate::authz::{require, AuthzDenied, AuthzRepo, Capability};
use crate::repo::{ExecutionLogRepository, MetricsSummary};
#[derive(Clone)]
pub struct MetricsState {
pub logs: Arc<dyn ExecutionLogRepository>,
pub apps: Arc<dyn AppRepository>,
pub authz: Arc<dyn AuthzRepo>,
}
pub fn metrics_router(state: MetricsState) -> Router {
Router::new()
.route("/apps/{app_id}/metrics", get(summary))
.with_state(state)
}
/// Default rollup window when `?window=` is absent (last 24 hours).
const DEFAULT_WINDOW_HOURS: i32 = 24;
#[derive(Debug, Deserialize)]
pub struct MetricsQuery {
/// Trailing window in hours (clamped to `[1, 2160]` by the repo).
#[serde(default)]
pub window: Option<i32>,
}
async fn summary(
State(s): State<MetricsState>,
Extension(principal): Extension<Principal>,
Path(id_or_slug): Path<String>,
Query(q): Query<MetricsQuery>,
) -> Result<Json<MetricsSummary>, MetricsApiError> {
let app_id = resolve_app(&*s.apps, &id_or_slug).await?;
require(s.authz.as_ref(), &principal, Capability::AppLogRead(app_id)).await?;
let window = q.window.unwrap_or(DEFAULT_WINDOW_HOURS);
let summary = s
.logs
.summarize_for_app(app_id, window)
.await
.map_err(|e| MetricsApiError::Backend(e.to_string()))?;
Ok(Json(summary))
}
async fn resolve_app(apps: &dyn AppRepository, ident: &str) -> Result<AppId, MetricsApiError> {
crate::app_repo::resolve_app(apps, ident)
.await
.map_err(|e| MetricsApiError::Backend(e.to_string()))?
.map(|l| l.app.id)
.ok_or(MetricsApiError::AppNotFound)
}
#[derive(Debug, thiserror::Error)]
pub enum MetricsApiError {
#[error("app not found")]
AppNotFound,
#[error("forbidden")]
Forbidden,
#[error("authorization repo error: {0}")]
AuthzRepo(String),
#[error("metrics backend: {0}")]
Backend(String),
}
impl From<AuthzDenied> for MetricsApiError {
fn from(d: AuthzDenied) -> Self {
match d {
AuthzDenied::Denied => Self::Forbidden,
AuthzDenied::Repo(e) => Self::AuthzRepo(e.to_string()),
}
}
}
impl IntoResponse for MetricsApiError {
fn into_response(self) -> Response {
use axum::http::StatusCode;
let (status, body) = match &self {
Self::AppNotFound => (StatusCode::NOT_FOUND, json!({ "error": self.to_string() })),
Self::Forbidden => (StatusCode::FORBIDDEN, json!({ "error": self.to_string() })),
Self::AuthzRepo(e) => {
tracing::error!(error = %e, "metrics admin authz error");
(
StatusCode::INTERNAL_SERVER_ERROR,
json!({ "error": "internal error" }),
)
}
Self::Backend(e) => {
tracing::error!(error = %e, "metrics admin backend error");
(
StatusCode::INTERNAL_SERVER_ERROR,
json!({ "error": "internal error" }),
)
}
};
(status, Json(body)).into_response()
}
}

View File

@@ -1,139 +1,135 @@
//! `OutboxEventEmitter` — the real `ServiceEventEmitter` that replaces
//! v1.1.0's `NoopEventEmitter` once the triggers framework lands.
//! `OutboxEventEmitter` — the real `ServiceEventEmitter` behind every stateful
//! service. On each collection mutation (KV / docs / files) it looks up the
//! triggers the event matches and writes one outbox row per match; the
//! dispatcher reads them back out-of-band.
//!
//! On each `emit` (a KV mutation, future doc/file/pubsub event, etc.):
//! 1. Look up matching triggers for the event's (app_id, source, op,
//! collection) tuple via `TriggerRepo::list_matching_*`.
//! 2. For each match, write one outbox row carrying the event payload
//! serialized as a `TriggerEvent`.
//! **The fan-out is connection-scoped, not pool-scoped.** [`emit_on`] and
//! [`emit_shared_on`] take a `&mut PgConnection`, so a caller that already holds
//! a transaction can pass `&mut *tx` and have the outbox rows commit *with* the
//! data write that produced them — the transactional-outbox pattern. That closes
//! the window where a row committed but its trigger never fired because the
//! outbox insert failed (or the process died) immediately afterwards. Services
//! that write through `crate::atomic_write` take that path; the
//! `ServiceEventEmitter` trait impl below is the non-transactional entry point,
//! kept for the callers (and the tests) that only need best-effort emission.
//!
//! Defaults applied at write time so `OutboxRow.payload` carries
//! everything the dispatcher needs to reconstruct the executor
//! invocation without joining back to the trigger row.
//!
//! Non-KV `ServiceEvent` sources are silently dropped in v1.1.1 — the
//! dispatcher only knows how to fire KV triggers this release. Future
//! sources (docs/files/pubsub) add their own dispatch arm.
use std::sync::Arc;
//! Non-collection `ServiceEvent` sources are silently dropped: the SDK calls
//! `events.emit(...)` unconditionally for forward compat, and every source the
//! dispatcher has an arm for is handled here.
use async_trait::async_trait;
use picloud_shared::{
DocsEventOp, EmitError, FileMeta, FilesEventOp, GroupId, KvEventOp, SdkCallCx, ServiceEvent,
ServiceEventEmitter, TriggerEvent,
};
use sqlx::{PgConnection, PgPool};
use crate::outbox_repo::{NewOutboxRow, OutboxRepo, OutboxSourceKind};
use crate::trigger_repo::TriggerRepo;
use crate::outbox_repo::{insert_on, NewOutboxRow, OutboxSourceKind};
use crate::trigger_repo::{list_matching_on, list_matching_shared_on, KvMatchRow};
pub struct OutboxEventEmitter {
triggers: Arc<dyn TriggerRepo>,
outbox: Arc<dyn OutboxRepo>,
pool: PgPool,
}
impl OutboxEventEmitter {
#[must_use]
pub fn new(triggers: Arc<dyn TriggerRepo>, outbox: Arc<dyn OutboxRepo>) -> Self {
Self { triggers, outbox }
pub fn new(pool: PgPool) -> Self {
Self { pool }
}
}
#[async_trait]
impl ServiceEventEmitter for OutboxEventEmitter {
async fn emit(&self, cx: &SdkCallCx, event: ServiceEvent) -> Result<(), EmitError> {
match event.source {
"kv" => self.emit_kv(cx, event).await,
"docs" => self.emit_docs(cx, event).await,
"files" => self.emit_files(cx, event).await,
// Future sources land here. For now, silently drop — the
// SDK calls `events.emit(...)` unconditionally for forward
// compat, so swallowing without an error is correct.
_ => Ok(()),
}
let mut conn = self
.pool
.acquire()
.await
.map_err(|e| EmitError::Unavailable(format!("acquire connection: {e}")))?;
emit_on(&mut conn, cx, &event).await
}
#[allow(clippy::too_many_lines)] // one match arm per source kind (kv/docs/files)
async fn emit_shared(
&self,
cx: &SdkCallCx,
owning_group: GroupId,
event: ServiceEvent,
) -> Result<(), EmitError> {
// §11.6: a shared-collection write. Match `shared = true` triggers on
// the OWNING group; each fires under the WRITER app (`cx.app_id`), the
// same "group template runs under the firing app" model.
let source_kind = match event.source {
"kv" => OutboxSourceKind::Kv,
"docs" => OutboxSourceKind::Docs,
"files" => OutboxSourceKind::Files,
_ => return Ok(()),
};
let Some(collection) = event.collection.clone() else {
return Ok(());
};
let (matches, trigger_event) = match event.source {
"kv" => {
let Some(op) = KvEventOp::from_wire(event.op) else {
return Ok(());
};
let m = self
.triggers
.list_matching_shared_kv(owning_group, &collection, op)
.await
.map_err(|e| EmitError::Unavailable(format!("trigger lookup: {e}")))?;
let ev = TriggerEvent::Kv {
let mut conn = self
.pool
.acquire()
.await
.map_err(|e| EmitError::Unavailable(format!("acquire connection: {e}")))?;
emit_shared_on(&mut conn, cx, owning_group, &event).await
}
}
/// Everything the fan-out needs, derived from a `ServiceEvent` with no I/O: the
/// trigger-table coordinates to match on, and the `TriggerEvent` the handler
/// will see as `ctx.event`. `None` for a source or op the dispatcher has no arm
/// for — the caller drops the event quietly.
struct EventPlan {
source_kind: OutboxSourceKind,
/// `triggers.kind` discriminator. A literal — never user data, since it is
/// interpolated into the match SQL.
kind: &'static str,
/// Per-kind detail table. A literal, for the same reason.
detail_table: &'static str,
collection: String,
op_str: &'static str,
trigger_event: TriggerEvent,
}
fn plan(event: &ServiceEvent) -> Option<EventPlan> {
// Collection events always carry a collection; defensively skip if not.
let collection = event.collection.clone()?;
let key = event.key.clone().unwrap_or_default();
match event.source {
"kv" => {
let op = KvEventOp::from_wire(event.op)?;
Some(EventPlan {
source_kind: OutboxSourceKind::Kv,
kind: "kv",
detail_table: "kv_trigger_details",
collection: collection.clone(),
op_str: op.as_str(),
trigger_event: TriggerEvent::Kv {
op,
collection,
key: event.key.clone().unwrap_or_default(),
key,
value: event.payload.clone(),
};
(
m.into_iter()
.map(|x| (x.trigger_id, x.script_id))
.collect::<Vec<_>>(),
ev,
)
}
"docs" => {
let Some(op) = DocsEventOp::from_wire(event.op) else {
return Ok(());
};
let m = self
.triggers
.list_matching_shared_docs(owning_group, &collection, op)
.await
.map_err(|e| EmitError::Unavailable(format!("trigger lookup: {e}")))?;
let ev = TriggerEvent::Docs {
},
})
}
"docs" => {
let op = DocsEventOp::from_wire(event.op)?;
Some(EventPlan {
source_kind: OutboxSourceKind::Docs,
kind: "docs",
detail_table: "docs_trigger_details",
collection: collection.clone(),
op_str: op.as_str(),
trigger_event: TriggerEvent::Docs {
op,
collection,
id: event.key.clone().unwrap_or_default(),
id: key,
data: event.payload.clone(),
prev_data: event.old_payload.clone(),
};
(
m.into_iter()
.map(|x| (x.trigger_id, x.script_id))
.collect::<Vec<_>>(),
ev,
)
}
"files" => {
let Some(op) = FilesEventOp::from_wire(event.op) else {
return Ok(());
};
let Some(meta) = event
.payload
.clone()
.and_then(|v| serde_json::from_value::<FileMeta>(v).ok())
else {
return Ok(());
};
let m = self
.triggers
.list_matching_shared_files(owning_group, &collection, op)
.await
.map_err(|e| EmitError::Unavailable(format!("trigger lookup: {e}")))?;
let ev = TriggerEvent::Files {
},
})
}
"files" => {
let op = FilesEventOp::from_wire(event.op)?;
// The payload is the `FileMeta` JSON the files service emitted —
// never the blob bytes.
let meta: FileMeta = serde_json::from_value(event.payload.clone()?).ok()?;
Some(EventPlan {
source_kind: OutboxSourceKind::Files,
kind: "files",
detail_table: "files_trigger_details",
collection: collection.clone(),
op_str: op.as_str(),
trigger_event: TriggerEvent::Files {
op,
collection,
id: meta.id.to_string(),
@@ -142,206 +138,87 @@ impl ServiceEventEmitter for OutboxEventEmitter {
size: meta.size,
checksum: meta.checksum,
prev: event.old_payload.clone(),
};
(
m.into_iter()
.map(|x| (x.trigger_id, x.script_id))
.collect::<Vec<_>>(),
ev,
)
}
_ => return Ok(()),
};
if matches.is_empty() {
return Ok(());
},
})
}
let payload = serde_json::to_value(&trigger_event)
.map_err(|e| EmitError::Rejected(format!("event serialize: {e}")))?;
for (trigger_id, script_id) in matches {
self.outbox
.insert(NewOutboxRow {
app_id: cx.app_id,
source_kind,
trigger_id: Some(trigger_id),
script_id: Some(script_id),
reply_to: None,
payload: payload.clone(),
origin_principal: cx.principal.as_ref().map(|p| p.user_id),
trigger_depth: cx.trigger_depth.saturating_add(1),
root_execution_id: Some(cx.root_execution_id),
})
.await
.map_err(|e| EmitError::Unavailable(format!("outbox insert: {e}")))?;
}
Ok(())
_ => None,
}
}
impl OutboxEventEmitter {
async fn emit_kv(&self, cx: &SdkCallCx, event: ServiceEvent) -> Result<(), EmitError> {
let Some(op) = KvEventOp::from_wire(event.op) else {
return Ok(()); // unknown op — drop quietly
};
let Some(collection) = event.collection.clone() else {
return Ok(()); // KV events always carry a collection — defensively skip
};
let key = event.key.clone().unwrap_or_default();
let matches = self
.triggers
.list_matching_kv(cx.app_id, &collection, op)
.await
.map_err(|e| EmitError::Unavailable(format!("trigger lookup: {e}")))?;
if matches.is_empty() {
return Ok(());
}
// Serialize the originating event as a TriggerEvent so the
// dispatcher can hand it to the script as `ctx.event` without
// round-tripping back to the trigger row.
let trigger_event = TriggerEvent::Kv {
op,
collection,
key,
value: event.payload.clone(),
};
let payload = serde_json::to_value(&trigger_event)
.map_err(|e| EmitError::Rejected(format!("event serialize: {e}")))?;
for m in matches {
self.outbox
.insert(NewOutboxRow {
app_id: cx.app_id,
source_kind: OutboxSourceKind::Kv,
trigger_id: Some(m.trigger_id),
script_id: Some(m.script_id),
reply_to: None,
payload: payload.clone(),
origin_principal: cx.principal.as_ref().map(|p| p.user_id),
trigger_depth: cx.trigger_depth.saturating_add(1),
root_execution_id: Some(cx.root_execution_id),
})
.await
.map_err(|e| EmitError::Unavailable(format!("outbox insert: {e}")))?;
}
Ok(())
}
/// v1.1.2. Mirrors `emit_kv` — fan out a docs mutation across
/// matching docs triggers + write one outbox row each. The
/// `prev_data` change-data-capture surface is preserved from the
/// `ServiceEvent.old_payload` field (set by `DocsServiceImpl` on
/// update and delete; `None` for create).
async fn emit_docs(&self, cx: &SdkCallCx, event: ServiceEvent) -> Result<(), EmitError> {
let Some(op) = DocsEventOp::from_wire(event.op) else {
return Ok(());
};
let Some(collection) = event.collection.clone() else {
return Ok(());
};
let id = event.key.clone().unwrap_or_default();
let matches = self
.triggers
.list_matching_docs(cx.app_id, &collection, op)
.await
.map_err(|e| EmitError::Unavailable(format!("trigger lookup: {e}")))?;
if matches.is_empty() {
return Ok(());
}
let trigger_event = TriggerEvent::Docs {
op,
collection,
id,
data: event.payload.clone(),
prev_data: event.old_payload.clone(),
};
let payload = serde_json::to_value(&trigger_event)
.map_err(|e| EmitError::Rejected(format!("event serialize: {e}")))?;
for m in matches {
self.outbox
.insert(NewOutboxRow {
app_id: cx.app_id,
source_kind: OutboxSourceKind::Docs,
trigger_id: Some(m.trigger_id),
script_id: Some(m.script_id),
reply_to: None,
payload: payload.clone(),
origin_principal: cx.principal.as_ref().map(|p| p.user_id),
trigger_depth: cx.trigger_depth.saturating_add(1),
root_execution_id: Some(cx.root_execution_id),
})
.await
.map_err(|e| EmitError::Unavailable(format!("outbox insert: {e}")))?;
}
Ok(())
}
/// v1.1.5. Fan out a files mutation across matching files triggers.
/// The `ServiceEvent.payload` is the file **metadata** (never the
/// blob bytes); `old_payload` is the prior metadata (the deleted
/// row's metadata on delete). The `TriggerEvent::Files` carries the
/// metadata fields explicitly + `prev` for the change-data-capture
/// surface.
async fn emit_files(&self, cx: &SdkCallCx, event: ServiceEvent) -> Result<(), EmitError> {
let Some(op) = FilesEventOp::from_wire(event.op) else {
return Ok(());
};
let Some(collection) = event.collection.clone() else {
return Ok(());
};
// The payload is the FileMeta JSON the FilesServiceImpl emitted.
let Some(meta) = event
.payload
.clone()
.and_then(|v| serde_json::from_value::<FileMeta>(v).ok())
else {
return Ok(());
};
let matches = self
.triggers
.list_matching_files(cx.app_id, &collection, op)
.await
.map_err(|e| EmitError::Unavailable(format!("trigger lookup: {e}")))?;
if matches.is_empty() {
return Ok(());
}
let trigger_event = TriggerEvent::Files {
op,
collection,
id: meta.id.to_string(),
name: meta.name,
content_type: meta.content_type,
size: meta.size,
checksum: meta.checksum,
prev: event.old_payload.clone(),
};
let payload = serde_json::to_value(&trigger_event)
.map_err(|e| EmitError::Rejected(format!("event serialize: {e}")))?;
for m in matches {
self.outbox
.insert(NewOutboxRow {
app_id: cx.app_id,
source_kind: OutboxSourceKind::Files,
trigger_id: Some(m.trigger_id),
script_id: Some(m.script_id),
reply_to: None,
payload: payload.clone(),
origin_principal: cx.principal.as_ref().map(|p| p.user_id),
trigger_depth: cx.trigger_depth.saturating_add(1),
root_execution_id: Some(cx.root_execution_id),
})
.await
.map_err(|e| EmitError::Unavailable(format!("outbox insert: {e}")))?;
}
Ok(())
}
/// Fan a collection mutation out over the writing app's own + inherited
/// triggers. Pass `&mut *tx` to have the outbox rows commit with the write.
pub(crate) async fn emit_on(
conn: &mut PgConnection,
cx: &SdkCallCx,
event: &ServiceEvent,
) -> Result<(), EmitError> {
let Some(p) = plan(event) else { return Ok(()) };
let matches = list_matching_on(
&mut *conn,
p.kind,
p.detail_table,
cx.app_id,
&p.collection,
p.op_str,
)
.await
.map_err(|e| EmitError::Unavailable(format!("trigger lookup: {e}")))?;
write_outbox_rows(conn, cx, &p, matches).await
}
/// §11.6: fan a SHARED-collection write out over the `shared = true` triggers on
/// the OWNING group. Each fires under the WRITER app (`cx.app_id`) — the same
/// "a group template runs under the firing app" model as an inherited trigger.
pub(crate) async fn emit_shared_on(
conn: &mut PgConnection,
cx: &SdkCallCx,
owning_group: GroupId,
event: &ServiceEvent,
) -> Result<(), EmitError> {
let Some(p) = plan(event) else { return Ok(()) };
let matches = list_matching_shared_on(
&mut *conn,
p.kind,
p.detail_table,
owning_group,
&p.collection,
p.op_str,
)
.await
.map_err(|e| EmitError::Unavailable(format!("trigger lookup: {e}")))?;
write_outbox_rows(conn, cx, &p, matches).await
}
async fn write_outbox_rows(
conn: &mut PgConnection,
cx: &SdkCallCx,
p: &EventPlan,
matches: Vec<KvMatchRow>,
) -> Result<(), EmitError> {
if matches.is_empty() {
return Ok(());
}
// Serialize the originating event once so the dispatcher can hand it to the
// handler as `ctx.event` without joining back to the trigger row.
let payload = serde_json::to_value(&p.trigger_event)
.map_err(|e| EmitError::Rejected(format!("event serialize: {e}")))?;
for m in matches {
insert_on(
&mut *conn,
NewOutboxRow {
app_id: cx.app_id,
source_kind: p.source_kind,
trigger_id: Some(m.id.into()),
script_id: Some(m.script_id.into()),
reply_to: None,
payload: payload.clone(),
origin_principal: cx.principal.as_ref().map(|pr| pr.user_id),
trigger_depth: cx.trigger_depth.saturating_add(1),
root_execution_id: Some(cx.root_execution_id),
},
)
.await
.map_err(|e| EmitError::Unavailable(format!("outbox insert: {e}")))?;
}
Ok(())
}

View File

@@ -120,6 +120,26 @@ pub trait OutboxRepo: Send + Sync {
limit: i64,
) -> Result<Vec<OutboxRow>, OutboxRepoError>;
/// Return rows whose claim is older than `timeout_secs` to the queue.
///
/// A dispatcher claims a row, executes it, then either deletes it (success)
/// or reschedules it (failure) — both of which clear the claim. If the
/// PROCESS DIES in between, neither runs, and since `claim_due` only selects
/// `claimed_at IS NULL`, that row is stranded forever: its trigger never
/// fires and no retry can ever notice. Every other claim-based store (queue,
/// group queue, workflow steps) has had this safety net; the outbox did not,
/// so a crash or restart mid-dispatch permanently lost every in-flight event.
///
/// Returns how many rows were reclaimed. `attempt_count` is deliberately NOT
/// bumped: the handler never ran, so a reclaim must not consume the row's
/// retry budget (same reasoning as the transient queue `release`).
///
/// SYNCHRONOUS rows (`reply_to IS NOT NULL`) are NOT reclaimed — the schema
/// makes `reply_to.is_some()` the "never retry" signal, and their caller is
/// long gone. They are deleted instead: nobody is waiting, they can never be
/// dispatched, and leaving them would leak a row per crashed sync dispatch.
async fn reclaim_stale_claims(&self, timeout_secs: u32) -> Result<u64, OutboxRepoError>;
/// Remove a row after a terminal outcome (success or dead-letter).
async fn delete(&self, id: Uuid) -> Result<(), OutboxRepoError>;
@@ -145,28 +165,39 @@ impl PostgresOutboxRepo {
}
}
/// Insert one outbox row on an arbitrary executor — a pooled connection, or a
/// `&mut *tx` so the row commits atomically with the data write that produced
/// it (the transactional outbox; see `outbox_event_emitter`). The `insert`
/// trait method delegates here, so the INSERT has exactly one home.
pub(crate) async fn insert_on<'c, E>(exec: E, row: NewOutboxRow) -> Result<Uuid, OutboxRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let (id,): (Uuid,) = sqlx::query_as(
"INSERT INTO outbox ( \
app_id, source_kind, trigger_id, script_id, reply_to, \
payload, origin_principal, trigger_depth, root_execution_id \
) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9) \
RETURNING id",
)
.bind(row.app_id.into_inner())
.bind(row.source_kind.as_str())
.bind(row.trigger_id.map(TriggerId::into_inner))
.bind(row.script_id.map(ScriptId::into_inner))
.bind(row.reply_to)
.bind(row.payload)
.bind(row.origin_principal.map(AdminUserId::into_inner))
.bind(i32::try_from(row.trigger_depth).unwrap_or(0))
.bind(row.root_execution_id.map(ExecutionId::into_inner))
.fetch_one(exec)
.await?;
Ok(id)
}
#[async_trait]
impl OutboxRepo for PostgresOutboxRepo {
async fn insert(&self, row: NewOutboxRow) -> Result<Uuid, OutboxRepoError> {
let (id,): (Uuid,) = sqlx::query_as(
"INSERT INTO outbox ( \
app_id, source_kind, trigger_id, script_id, reply_to, \
payload, origin_principal, trigger_depth, root_execution_id \
) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9) \
RETURNING id",
)
.bind(row.app_id.into_inner())
.bind(row.source_kind.as_str())
.bind(row.trigger_id.map(TriggerId::into_inner))
.bind(row.script_id.map(ScriptId::into_inner))
.bind(row.reply_to)
.bind(row.payload)
.bind(row.origin_principal.map(AdminUserId::into_inner))
.bind(i32::try_from(row.trigger_depth).unwrap_or(0))
.bind(row.root_execution_id.map(ExecutionId::into_inner))
.fetch_one(&self.pool)
.await?;
Ok(id)
insert_on(&self.pool, row).await
}
async fn claim_due(
@@ -204,6 +235,38 @@ impl OutboxRepo for PostgresOutboxRepo {
Ok(())
}
async fn reclaim_stale_claims(&self, timeout_secs: u32) -> Result<u64, OutboxRepoError> {
// `reply_to IS NULL` is load-bearing: a row with a reply_to is a SYNCHRONOUS
// HTTP dispatch, and the schema (0009_outbox.sql) makes reply_to.is_some()
// the "never retry" signal. Returning one to the queue would re-execute a
// handler whose caller left minutes ago and whose inbox oneshot no longer
// exists — running its side effects a second time for nobody.
let res = sqlx::query(
"UPDATE outbox SET claimed_at = NULL, claimed_by = NULL \
WHERE claimed_at IS NOT NULL \
AND reply_to IS NULL \
AND claimed_at < NOW() - ($1 || ' seconds')::INTERVAL",
)
.bind(i64::from(timeout_secs))
.execute(&self.pool)
.await?;
// A stale SYNC row can never be dispatched again (never-retry) and nobody
// is waiting on it, so it is pure garbage — drop it rather than leak a row
// per crashed sync dispatch forever.
sqlx::query(
"DELETE FROM outbox \
WHERE claimed_at IS NOT NULL \
AND reply_to IS NOT NULL \
AND claimed_at < NOW() - ($1 || ' seconds')::INTERVAL",
)
.bind(i64::from(timeout_secs))
.execute(&self.pool)
.await?;
Ok(res.rows_affected())
}
async fn reschedule(
&self,
id: Uuid,

View File

@@ -26,6 +26,24 @@ pub trait ProjectRepository: Send + Sync {
async fn list_with_counts(&self) -> Result<Vec<(Project, i64)>, ProjectRepositoryError>;
async fn get_by_id(&self, id: ProjectId) -> Result<Option<Project>, ProjectRepositoryError>;
async fn get_by_slug(&self, slug: &str) -> Result<Option<Project>, ProjectRepositoryError>;
/// §3 M3 (hermetic gate): the persisted per-env approval policy for the
/// project with this slug — `env_name -> confirm`. Empty when the project is
/// unregistered or declares no gated envs. The apply/plan path unions this
/// with the request's declared policy so an omitted policy can't bypass a
/// gate a prior apply persisted (see `apply_api::env_gate_check`).
async fn get_environments_by_slug(
&self,
slug: &str,
) -> Result<std::collections::BTreeMap<String, bool>, ProjectRepositoryError>;
/// §3 M3 (hermetic gate, server-authoritative): the persisted per-env
/// approval policy for the project with this id. Same shape as
/// `get_environments_by_slug`, but keyed by the id the server RESOLVED from
/// the target node's nearest-claimed ancestor — so a client that omits or
/// spoofs `[project]` can't dodge a gate the owning project established.
async fn get_environments_by_id(
&self,
id: ProjectId,
) -> Result<std::collections::BTreeMap<String, bool>, ProjectRepositoryError>;
}
pub struct PostgresProjectRepository {
@@ -86,6 +104,35 @@ impl ProjectRepository for PostgresProjectRepository {
.await?;
Ok(row.map(Into::into))
}
async fn get_environments_by_slug(
&self,
slug: &str,
) -> Result<std::collections::BTreeMap<String, bool>, ProjectRepositoryError> {
let rows = sqlx::query_as::<_, (String, bool)>(
"SELECT pe.env_name, pe.confirm \
FROM project_environments pe \
JOIN projects p ON p.id = pe.project_id \
WHERE p.slug = $1",
)
.bind(slug)
.fetch_all(&self.pool)
.await?;
Ok(rows.into_iter().collect())
}
async fn get_environments_by_id(
&self,
id: ProjectId,
) -> Result<std::collections::BTreeMap<String, bool>, ProjectRepositoryError> {
let rows = sqlx::query_as::<_, (String, bool)>(
"SELECT env_name, confirm FROM project_environments WHERE project_id = $1",
)
.bind(id.into_inner())
.fetch_all(&self.pool)
.await?;
Ok(rows.into_iter().collect())
}
}
#[derive(sqlx::FromRow)]

View File

@@ -775,4 +775,47 @@ mod tests {
other => panic!("expected SubscriberToken, got {other:?}"),
}
}
/// `PICLOUD_PUBSUB_MAX_MESSAGE_BYTES` — CLAUDE.md: "Prevents one publish from
/// amplifying into N outbox rows × M MB." Pins that an oversized message is
/// rejected BEFORE authz. The cx is an authenticated member that
/// `DenyingAuthzRepo` rejects, so the ordering is observable: size-first
/// returns `MessageTooLarge`, authz-first would return `Forbidden`.
#[tokio::test]
async fn oversized_message_is_rejected_before_authz() {
let repo = Arc::new(InMemoryPubsubRepo::new(vec![]));
let svc = svc(repo.clone(), Arc::new(DenyingAuthzRepo)).with_max_message_bytes(16);
let cx = member_cx(AppId::new());
let err = svc
.publish_durable(
&cx,
"events",
serde_json::json!({ "blob": "x".repeat(100) }),
)
.await
.unwrap_err();
assert!(
matches!(err, PubsubError::MessageTooLarge { limit: 16, .. }),
"an oversized publish must be MessageTooLarge before authz; an authz-first \
order would return Forbidden for this denied cx. got {err:?}"
);
// The oversized publish never reached the repo — no outbox amplification.
assert_eq!(
repo.written_count(),
0,
"a rejected publish must not write any fan-out rows"
);
// Control: an under-cap publish with the same cx is Forbidden — confirming
// the cx is authz-denied, so the case above genuinely bypassed authz.
let err = svc
.publish_durable(&cx, "events", serde_json::json!({ "ok": 1 }))
.await
.unwrap_err();
assert!(
matches!(err, PubsubError::Forbidden),
"the control confirms this cx is authz-denied; got {err:?}"
);
}
}

View File

@@ -112,6 +112,18 @@ pub trait QueueRepo: Send + Sync {
retry_delay: chrono::Duration,
) -> Result<bool, QueueRepoError>;
/// TRANSIENT release: the handler never ran (gate saturated, or the script
/// was disabled at fire time), so re-queue AND undo `claim`'s pre-increment
/// of `attempt` — a non-execution must not burn the retry budget (else a
/// message could be dead-lettered under sustained overload having executed
/// zero times). Floors `attempt` at 0. IFF the claim_token matches.
async fn release(
&self,
message_id: QueueMessageId,
claim_token: Uuid,
retry_delay: chrono::Duration,
) -> Result<bool, QueueRepoError>;
/// Periodic safety net: clear claims whose `claimed_at` exceeds the
/// per-queue `visibility_timeout_secs` (joined from
/// `queue_trigger_details`). Returns the number of rows reclaimed.
@@ -251,6 +263,27 @@ impl QueueRepo for PostgresQueueRepo {
Ok(res.rows_affected() == 1)
}
async fn release(
&self,
message_id: QueueMessageId,
claim_token: Uuid,
retry_delay: chrono::Duration,
) -> Result<bool, QueueRepoError> {
let next = Utc::now() + retry_delay;
let res = sqlx::query(
"UPDATE queue_messages \
SET claim_token = NULL, claimed_at = NULL, deliver_after = $3, \
attempt = GREATEST(attempt - 1, 0) \
WHERE id = $1 AND claim_token = $2",
)
.bind(message_id.into_inner())
.bind(claim_token)
.bind(next)
.execute(&self.pool)
.await?;
Ok(res.rows_affected() == 1)
}
async fn reclaim_visibility_timeouts(&self) -> Result<u64, QueueRepoError> {
let res = sqlx::query(
"UPDATE queue_messages m \

View File

@@ -210,6 +210,14 @@ mod tests {
) -> Result<bool, crate::queue_repo::QueueRepoError> {
Ok(true)
}
async fn release(
&self,
_message_id: QueueMessageId,
_claim_token: uuid::Uuid,
_retry_delay: chrono::Duration,
) -> Result<bool, crate::queue_repo::QueueRepoError> {
Ok(true)
}
async fn reclaim_visibility_timeouts(
&self,
) -> Result<u64, crate::queue_repo::QueueRepoError> {
@@ -267,6 +275,30 @@ mod tests {
}
}
/// An AUTHENTICATED member with no app role. `DenyAllAuthz` denies it — unlike
/// an anon cx, which passes `script_gate` regardless, so only an authenticated-
/// denied cx can distinguish "size check ran before authz" from "after".
fn member_cx() -> SdkCallCx {
use picloud_shared::{InstanceRole, Principal, UserId};
let exec = ExecutionId::new();
SdkCallCx {
app_id: AppId::new(),
script_id: ScriptId::new(),
principal: Some(Principal {
user_id: UserId::new(),
instance_role: InstanceRole::Member,
scopes: None,
app_binding: None,
}),
execution_id: exec,
request_id: RequestId::new(),
trigger_depth: 0,
root_execution_id: exec,
is_dead_letter_handler: false,
event: None,
}
}
#[tokio::test]
async fn empty_queue_name_rejects() {
let repo = Arc::new(CapturingRepo {
@@ -358,11 +390,15 @@ mod tests {
#[tokio::test]
async fn oversized_payload_is_rejected_before_authz() {
// DenyAllAuthz + an authenticated cx makes the ORDERING observable: a
// size-first service returns PayloadTooLarge, an authz-first one would
// return Forbidden. (The previous version used AlwaysAllow + anon, which
// caught a DROPPED cap but not a REORDERED one.)
let repo = Arc::new(CapturingRepo {
last: tokio::sync::Mutex::new(None),
});
let svc = QueueServiceImpl::with_max_payload_bytes(repo, Arc::new(AlwaysAllowAuthz), 16);
let cx = anon_cx();
let svc = QueueServiceImpl::with_max_payload_bytes(repo, Arc::new(DenyAllAuthz), 16);
let cx = member_cx();
let payload = serde_json::json!({ "blob": "x".repeat(200) });
let err = svc
.enqueue(&cx, "jobs", payload, EnqueueOpts::default())
@@ -370,7 +406,23 @@ mod tests {
.unwrap_err();
assert!(
matches!(err, QueueError::PayloadTooLarge { .. }),
"expected PayloadTooLarge, got {err:?}"
"an oversized payload must be PayloadTooLarge before authz; an authz-first \
order would return Forbidden for this denied cx. got {err:?}"
);
// Control: an under-cap payload with the same cx is Forbidden — confirming
// the cx is authz-denied, so the case above genuinely bypassed authz.
let err = svc
.enqueue(
&cx,
"jobs",
serde_json::json!({ "ok": 1 }),
EnqueueOpts::default(),
)
.await
.unwrap_err();
assert!(
matches!(err, QueueError::Forbidden),
"the control confirms this cx is authz-denied; got {err:?}"
);
}
@@ -388,30 +440,17 @@ mod tests {
#[tokio::test]
async fn authed_principal_denied_returns_forbidden_variant() {
use picloud_shared::{InstanceRole, Principal, UserId};
let repo = Arc::new(CapturingRepo {
last: tokio::sync::Mutex::new(None),
});
let svc = QueueServiceImpl::new(repo, Arc::new(DenyAllAuthz));
let exec = ExecutionId::new();
let cx = SdkCallCx {
app_id: AppId::new(),
script_id: ScriptId::new(),
principal: Some(Principal {
user_id: UserId::new(),
instance_role: InstanceRole::Member,
scopes: None,
app_binding: None,
}),
execution_id: exec,
request_id: RequestId::new(),
trigger_depth: 0,
root_execution_id: exec,
is_dead_letter_handler: false,
event: None,
};
let err = svc
.enqueue(&cx, "x", serde_json::Value::Null, EnqueueOpts::default())
.enqueue(
&member_cx(),
"x",
serde_json::Value::Null,
EnqueueOpts::default(),
)
.await
.unwrap_err();
assert!(matches!(err, QueueError::Forbidden));

View File

@@ -0,0 +1,212 @@
//! Storage ceilings — per-app and §11.6 per-group.
//!
//! # Why the two are enforced DIFFERENTLY
//!
//! The group ceilings serialize (a `pg_advisory_xact_lock` across the check and
//! the write) because they must be exact: a group's shared collections are the
//! cross-app blast radius, and with a small ceiling the check-then-write race
//! overshoots wildly (measured: 19 rows stored against a ceiling of 5). Group
//! writes are comparatively rare, so the lock costs little.
//!
//! The per-app KV/docs ceilings deliberately do **not** lock, and check only a
//! ROW count, on INSERT only:
//!
//! * `kv::set` is the hottest write path in the system. Taking a per-app lock
//! on every set would serialize an app's entire data plane; a `SUM(...)` scan
//! on every set would make write cost grow with the app's stored size. Both
//! are far worse trades than what the lock buys here.
//! * What it buys is small: without the lock, overshoot is bounded by write
//! concurrency (`PICLOUD_MAX_CONCURRENT_EXECUTIONS`, 32 by default) against a
//! ceiling of 100k — roughly 0.03%. These are anti-DoS rails, not billing.
//! * A byte ceiling is unnecessary: every value is already capped at
//! `PICLOUD_KV_MAX_VALUE_BYTES`, so `max_rows × max_value_bytes` is ALREADY a
//! finite bound on an app's stored bytes. Add a second scan for nothing.
//! * An update adds no row and cannot exceed the per-value cap, so it is exempt
//! and pays nothing at all — the COUNT runs only when a key/doc is created.
//!
//! Per-app FILES are the exception and DO lock: a single blob may be 100 MB, so
//! 32 racing uploads could overshoot by gigabytes of real disk, and a file write
//! is heavy enough that a lock and a `SUM(size_bytes)` are lost in the noise.
//!
//! §11.6 per-group quotas — global env-var ceilings on a group's shared
//! collections, enforced in the group write path (mirrors the per-value
//! `PICLOUD_*_MAX_*_BYTES` caps). One default applies to every group; per-group
//! configurable limits are deferred.
//!
//! - KV / docs: a per-group ROW-count ceiling AND a per-group TOTAL-BYTES ceiling
//! (across the group's collections of that kind). Row count exempts an update
//! of an existing key/doc (net-zero rows); the byte ceiling is checked on the
//! projected total (old value's bytes subtracted, new value's added) so a
//! same-or-smaller update near the cap is still allowed.
//! - files: a per-group TOTAL-BYTES ceiling (sum of stored blob sizes).
/// Default per-group shared-KV row ceiling. Override `PICLOUD_GROUP_KV_MAX_ROWS`.
pub const DEFAULT_GROUP_KV_MAX_ROWS: u64 = 100_000;
/// Default per-group shared-docs row ceiling. Override `PICLOUD_GROUP_DOCS_MAX_ROWS`.
pub const DEFAULT_GROUP_DOCS_MAX_ROWS: u64 = 100_000;
/// Default per-group shared-KV total-bytes ceiling (256 MiB). Override
/// `PICLOUD_GROUP_KV_MAX_TOTAL_BYTES`.
pub const DEFAULT_GROUP_KV_MAX_TOTAL_BYTES: u64 = 256 * 1024 * 1024;
/// Default per-group shared-docs total-bytes ceiling (256 MiB). Override
/// `PICLOUD_GROUP_DOCS_MAX_TOTAL_BYTES`.
pub const DEFAULT_GROUP_DOCS_MAX_TOTAL_BYTES: u64 = 256 * 1024 * 1024;
/// Default per-group shared-files total-bytes ceiling (10 GiB). Override
/// `PICLOUD_GROUP_FILES_MAX_TOTAL_BYTES`.
pub const DEFAULT_GROUP_FILES_MAX_TOTAL_BYTES: u64 = 10 * 1024 * 1024 * 1024;
/// Default per-app KV key ceiling. Override `PICLOUD_APP_KV_MAX_ROWS`.
pub const DEFAULT_APP_KV_MAX_ROWS: u64 = 100_000;
/// Default per-app doc ceiling. Override `PICLOUD_APP_DOCS_MAX_ROWS`.
pub const DEFAULT_APP_DOCS_MAX_ROWS: u64 = 100_000;
/// Default per-app stored-blob ceiling (10 GiB). Override
/// `PICLOUD_APP_FILES_MAX_TOTAL_BYTES`.
pub const DEFAULT_APP_FILES_MAX_TOTAL_BYTES: u64 = 10 * 1024 * 1024 * 1024;
fn u64_from_env(var: &str, default: u64) -> u64 {
if let Ok(v) = std::env::var(var) {
match v.trim().parse::<u64>() {
Ok(n) if n > 0 => return n,
_ => tracing::warn!(value = %v, "ignoring invalid {var} (want a positive integer)"),
}
}
default
}
#[must_use]
pub fn app_kv_max_rows_from_env() -> u64 {
u64_from_env("PICLOUD_APP_KV_MAX_ROWS", DEFAULT_APP_KV_MAX_ROWS)
}
#[must_use]
pub fn app_docs_max_rows_from_env() -> u64 {
u64_from_env("PICLOUD_APP_DOCS_MAX_ROWS", DEFAULT_APP_DOCS_MAX_ROWS)
}
#[must_use]
pub fn app_files_max_total_bytes_from_env() -> u64 {
u64_from_env(
"PICLOUD_APP_FILES_MAX_TOTAL_BYTES",
DEFAULT_APP_FILES_MAX_TOTAL_BYTES,
)
}
#[must_use]
pub fn group_kv_max_rows_from_env() -> u64 {
u64_from_env("PICLOUD_GROUP_KV_MAX_ROWS", DEFAULT_GROUP_KV_MAX_ROWS)
}
#[must_use]
pub fn group_docs_max_rows_from_env() -> u64 {
u64_from_env("PICLOUD_GROUP_DOCS_MAX_ROWS", DEFAULT_GROUP_DOCS_MAX_ROWS)
}
#[must_use]
pub fn group_kv_max_total_bytes_from_env() -> u64 {
u64_from_env(
"PICLOUD_GROUP_KV_MAX_TOTAL_BYTES",
DEFAULT_GROUP_KV_MAX_TOTAL_BYTES,
)
}
#[must_use]
pub fn group_docs_max_total_bytes_from_env() -> u64 {
u64_from_env(
"PICLOUD_GROUP_DOCS_MAX_TOTAL_BYTES",
DEFAULT_GROUP_DOCS_MAX_TOTAL_BYTES,
)
}
#[must_use]
pub fn group_files_max_total_bytes_from_env() -> u64 {
u64_from_env(
"PICLOUD_GROUP_FILES_MAX_TOTAL_BYTES",
DEFAULT_GROUP_FILES_MAX_TOTAL_BYTES,
)
}
// ----------------------------------------------------------------------------
// The row/byte decision, shared by every backend
// ----------------------------------------------------------------------------
/// The ceilings one shared-collection write must fit under.
#[derive(Debug, Clone, Copy)]
pub struct GroupWriteQuota {
pub max_rows: u64,
pub max_total_bytes: u64,
/// The per-value cap. Not a ceiling of its own here — it is what makes the
/// fast path below sound.
pub max_value_bytes: usize,
}
/// The two usage reads a quota decision needs. Implemented over the writing
/// transaction's connection (the real path — see `crate::atomic_write`) and over
/// the in-memory test store, so the POLICY below has one home regardless of
/// backend.
///
/// **The reads must come from the same transaction as the write they gate.**
/// Reading usage on one connection and writing on another is the check-then-write
/// race that lets N concurrent writers each observe the pre-write total, each
/// pass, and together blow the ceiling.
#[async_trait::async_trait]
pub trait GroupUsage: Send {
/// Current row count for the group, across its collections of this kind.
async fn count_rows(&mut self) -> Result<u64, String>;
/// Total stored bytes for the group AFTER this write — the current total,
/// minus the bytes of the row being replaced (if any), plus the new value's.
async fn projected_total_bytes(&mut self) -> Result<u64, String>;
}
/// What the ceilings say about a pending write.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum QuotaOutcome {
Allowed,
RowsExceeded { limit: u64, actual: u64 },
BytesExceeded { limit: u64, actual: u64 },
}
/// Decide whether a shared-collection write fits. `adds_row` = the write will
/// ADD a row rather than replace one (a new key, or a doc create) — a
/// compare-and-swap against an absent key with a `Some` precondition adds
/// nothing, because it cannot swap at all.
///
/// # Errors
/// Propagates a backend failure from either usage read, as a string.
pub async fn check_group_write(
q: GroupWriteQuota,
adds_row: bool,
usage: &mut dyn GroupUsage,
) -> Result<QuotaOutcome, String> {
// One cheap COUNT backs BOTH the row ceiling and the byte fast path below.
let count = usage.count_rows().await?;
// Row ceiling: only a write that adds a row consumes one. An update is
// net-zero and exempt.
if adds_row && count >= q.max_rows {
return Ok(QuotaOutcome::RowsExceeded {
limit: q.max_rows,
actual: count,
});
}
// Byte ceiling, on the PROJECTED total — so an update that is the same size
// or smaller still goes through when the group is sitting near its cap.
//
// Fast path: every stored value is capped at `max_value_bytes`, so the group
// can hold at most `rows_after * max_value_bytes`. When even that upper bound
// fits under the ceiling, no write can breach it — skip the
// O(collection-size) `SUM(octet_length(...))` scan entirely. The scan runs
// only for a group actually near its byte cap.
let rows_after = if adds_row { count + 1 } else { count };
let upper_bound = rows_after.saturating_mul(q.max_value_bytes as u64);
if upper_bound > q.max_total_bytes {
let projected = usage.projected_total_bytes().await?;
if projected > q.max_total_bytes {
return Ok(QuotaOutcome::BytesExceeded {
limit: q.max_total_bytes,
actual: projected,
});
}
}
Ok(QuotaOutcome::Allowed)
}

View File

@@ -22,11 +22,18 @@ use std::collections::HashMap;
use std::sync::{Arc, Mutex};
use async_trait::async_trait;
use picloud_shared::{subscriber_token, AppId, RealtimeAuthority, SubscribeDenied, UsersService};
use picloud_shared::{
subscriber_token, AppId, GroupId, RealtimeAuthority, SubscribeDenied, UsersService,
};
use crate::app_secrets_repo::AppSecretsRepo;
use crate::group_collection_repo::GroupCollectionResolver;
use crate::topic_repo::{TopicAuthMode, TopicRepo};
/// The registry `kind` a shared topic is declared under (mirrors
/// `group_pubsub_service::KIND_TOPIC`).
const KIND_TOPIC: &str = "topic";
/// F-S-008: TTL on cached signing-key entries. Once key rotation
/// lands, every running process keeps accepting tokens signed by the
/// old key until restart — bounded eviction is the simplest defence.
@@ -37,6 +44,9 @@ pub struct RealtimeAuthorityImpl {
topics: Arc<dyn TopicRepo>,
secrets: Arc<dyn AppSecretsRepo>,
users: Arc<dyn UsersService>,
/// §11.6 D2: resolves a shared-topic name to its owning group on the
/// subscriber app's ancestor chain (the isolation boundary).
collections: Arc<dyn GroupCollectionResolver>,
key_cache: Mutex<HashMap<AppId, (std::time::Instant, Vec<u8>)>>,
}
@@ -46,11 +56,13 @@ impl RealtimeAuthorityImpl {
topics: Arc<dyn TopicRepo>,
secrets: Arc<dyn AppSecretsRepo>,
users: Arc<dyn UsersService>,
collections: Arc<dyn GroupCollectionResolver>,
) -> Self {
Self {
topics,
secrets,
users,
collections,
key_cache: Mutex::new(HashMap::new()),
}
}
@@ -152,6 +164,24 @@ impl RealtimeAuthority for RealtimeAuthorityImpl {
}
}
}
async fn authorize_subscribe_shared(
&self,
app_id: AppId,
topic: &str,
) -> Result<GroupId, SubscribeDenied> {
// The declared collection is the topic's ROOT segment (`events.created`
// → `events`), matching the publish-side validation. Resolving it against
// the subscriber app's chain (kind=`topic`) is both the existence check
// and the authorization — reads are open to the subtree; a foreign app's
// chain never reaches the owning group, so it 404s (isolation boundary).
let root = topic.split('.').next().unwrap_or(topic);
self.collections
.resolve_owning_group(app_id, root, KIND_TOPIC)
.await
.map_err(|e| SubscribeDenied::Backend(e.to_string()))?
.ok_or(SubscribeDenied::NotFound)
}
}
#[cfg(test)]
@@ -222,6 +252,26 @@ mod tests {
}
}
/// Fake shared-topic resolver: `Some((declared_name, owning_group))` resolves
/// that one topic name (kind=topic) to the group; everything else → None.
#[derive(Default)]
struct FakeCollections(Option<(String, GroupId)>);
#[async_trait]
impl GroupCollectionResolver for FakeCollections {
async fn resolve_owning_group(
&self,
_app_id: AppId,
name: &str,
kind: &str,
) -> Result<Option<GroupId>, sqlx::Error> {
Ok(self
.0
.as_ref()
.filter(|(n, _)| n == name && kind == KIND_TOPIC)
.map(|(_, g)| *g))
}
}
fn authority(
topics: Vec<(AppId, Topic)>,
key_app: AppId,
@@ -231,9 +281,32 @@ mod tests {
Arc::new(FakeTopics(topics)),
Arc::new(FakeSecrets(key_app, key)),
Arc::new(picloud_shared::NoopUsersService),
Arc::new(FakeCollections::default()),
)
}
#[tokio::test]
async fn shared_topic_resolves_owning_group_or_404() {
let app = AppId::new();
let group = GroupId::new();
let auth = RealtimeAuthorityImpl::new(
Arc::new(FakeTopics(vec![])),
Arc::new(FakeSecrets(app, vec![0u8; 32])),
Arc::new(picloud_shared::NoopUsersService),
Arc::new(FakeCollections(Some(("events".into(), group)))),
);
// The full topic's ROOT segment resolves; returns the OWNING group.
assert_eq!(
auth.authorize_subscribe_shared(app, "events.created").await,
Ok(group)
);
// An undeclared topic name → NotFound (the isolation/existence boundary).
assert_eq!(
auth.authorize_subscribe_shared(app, "secrets").await,
Err(SubscribeDenied::NotFound)
);
}
#[tokio::test]
async fn missing_topic_is_not_found() {
let app = AppId::new();
@@ -590,6 +663,7 @@ mod tests {
app_id,
user: stub_user(app_id),
}),
Arc::new(FakeCollections::default()),
)
}
@@ -652,6 +726,7 @@ mod tests {
app_id: app_a,
user: stub_user(app_a),
}),
Arc::new(FakeCollections::default()),
);
assert_eq!(
a.authorize_subscribe(app_b, "chat", Some("app-a-token"))

View File

@@ -731,6 +731,40 @@ impl ExecutionLogCursor {
}
}
/// One `(key, count)` row of a metrics breakdown (by status or by source).
#[derive(Debug, Clone, serde::Serialize)]
pub struct MetricsCount {
pub key: String,
pub count: i64,
}
/// One hour-bucket of the metrics time series.
#[derive(Debug, Clone, serde::Serialize)]
pub struct MetricsBucket {
pub ts: chrono::DateTime<chrono::Utc>,
pub total: i64,
pub errors: i64,
}
/// Aggregate execution metrics for one app over a trailing window — the read
/// model behind the observability dashboard. Sourced entirely from
/// `execution_logs` (no hot-path instrumentation): counts + error rate +
/// latency percentiles + an hourly series.
#[derive(Debug, Clone, serde::Serialize)]
pub struct MetricsSummary {
pub window_hours: i32,
pub total: i64,
pub errors: i64,
/// Errors / total in `[0, 1]` (0 when there are no executions).
pub error_rate: f64,
pub avg_duration_ms: f64,
pub p50_duration_ms: f64,
pub p95_duration_ms: f64,
pub by_status: Vec<MetricsCount>,
pub by_source: Vec<MetricsCount>,
pub series: Vec<MetricsBucket>,
}
#[async_trait]
pub trait ExecutionLogRepository: Send + Sync {
/// F-P-005: keyset cursor on `(created_at, id)` — `cursor` is the
@@ -743,6 +777,14 @@ pub trait ExecutionLogRepository: Send + Sync {
cursor: Option<ExecutionLogCursor>,
source: Option<ExecutionSource>,
) -> Result<Vec<ExecutionLog>, ScriptRepositoryError>;
/// A2: aggregate metrics for `app_id` over the trailing `window_hours`.
/// Read-only rollup over `execution_logs`.
async fn summarize_for_app(
&self,
app_id: AppId,
window_hours: i32,
) -> Result<MetricsSummary, ScriptRepositoryError>;
}
pub struct PostgresExecutionLogRepository {
@@ -813,6 +855,129 @@ impl ExecutionLogRepository for PostgresExecutionLogRepository {
Ok(rows.into_iter().map(Into::into).collect())
}
async fn summarize_for_app(
&self,
app_id: AppId,
window_hours: i32,
) -> Result<MetricsSummary, ScriptRepositoryError> {
// Clamp the window to something sane (1 hour … 90 days) so a crafted
// `?window=` can't ask for an unbounded scan.
let window_hours = window_hours.clamp(1, 24 * 90);
let app = app_id.into_inner();
// Headline aggregate: totals, error count, latency mean + percentiles.
// `<> 'success'` counts every non-success terminal state as an error.
let agg = sqlx::query_as::<_, AggRow>(
"SELECT count(*)::int8 AS total, \
count(*) FILTER (WHERE status <> 'success')::int8 AS errors, \
COALESCE(avg(duration_ms), 0)::float8 AS avg_ms, \
COALESCE(percentile_cont(0.5) WITHIN GROUP (ORDER BY duration_ms), 0)::float8 AS p50, \
COALESCE(percentile_cont(0.95) WITHIN GROUP (ORDER BY duration_ms), 0)::float8 AS p95 \
FROM execution_logs \
WHERE app_id = $1 AND created_at >= now() - ($2 * interval '1 hour')",
)
.bind(app)
.bind(window_hours)
.fetch_one(&self.pool)
.await?;
let by_status = sqlx::query_as::<_, CountRow>(
"SELECT status AS key, count(*)::int8 AS count \
FROM execution_logs \
WHERE app_id = $1 AND created_at >= now() - ($2 * interval '1 hour') \
GROUP BY status ORDER BY count DESC",
)
.bind(app)
.bind(window_hours)
.fetch_all(&self.pool)
.await?;
let by_source = sqlx::query_as::<_, CountRow>(
"SELECT source AS key, count(*)::int8 AS count \
FROM execution_logs \
WHERE app_id = $1 AND created_at >= now() - ($2 * interval '1 hour') \
GROUP BY source ORDER BY count DESC",
)
.bind(app)
.bind(window_hours)
.fetch_all(&self.pool)
.await?;
let series = sqlx::query_as::<_, BucketRow>(
"SELECT date_trunc('hour', created_at) AS ts, count(*)::int8 AS total, \
count(*) FILTER (WHERE status <> 'success')::int8 AS errors \
FROM execution_logs \
WHERE app_id = $1 AND created_at >= now() - ($2 * interval '1 hour') \
GROUP BY ts ORDER BY ts",
)
.bind(app)
.bind(window_hours)
.fetch_all(&self.pool)
.await?;
// Counts are execution tallies over a bounded window — always far below
// f64's 2^53 exact-integer range, so the cast is lossless in practice.
#[allow(clippy::cast_precision_loss)]
let error_rate = if agg.total > 0 {
agg.errors as f64 / agg.total as f64
} else {
0.0
};
Ok(MetricsSummary {
window_hours,
total: agg.total,
errors: agg.errors,
error_rate,
avg_duration_ms: agg.avg_ms,
p50_duration_ms: agg.p50,
p95_duration_ms: agg.p95,
by_status: by_status
.into_iter()
.map(|r| MetricsCount {
key: r.key,
count: r.count,
})
.collect(),
by_source: by_source
.into_iter()
.map(|r| MetricsCount {
key: r.key,
count: r.count,
})
.collect(),
series: series
.into_iter()
.map(|r| MetricsBucket {
ts: r.ts,
total: r.total,
errors: r.errors,
})
.collect(),
})
}
}
#[derive(sqlx::FromRow)]
struct AggRow {
total: i64,
errors: i64,
avg_ms: f64,
p50: f64,
p95: f64,
}
#[derive(sqlx::FromRow)]
struct CountRow {
key: String,
count: i64,
}
#[derive(sqlx::FromRow)]
struct BucketRow {
ts: chrono::DateTime<chrono::Utc>,
total: i64,
errors: i64,
}
#[derive(sqlx::FromRow)]

View File

@@ -409,13 +409,19 @@ pub async fn rebuild_route_table(
table: &RouteTable,
) -> Result<(), ScriptRepositoryError> {
let effective = routes.list_effective().await?;
// §11 tail: `(app_id, path)` route suppressions — an inherited route at a
// suppressed path is dropped from the app's slice (404).
let suppressed: std::collections::HashSet<(AppId, String)> = routes
.list_route_suppressions()
.await?
.into_iter()
.collect();
// §11 tail: route suppressions keyed `(app_id, path)` → the SMALLEST
// suppressor depth seen for that key. An inherited route is dropped only
// when its owner is above the suppressor (`route_depth > suppressor_depth`),
// so a group's decline of an ancestor template never clobbers a nearer
// descendant's own route at the same path.
let mut suppressed: std::collections::HashMap<(AppId, String), i32> =
std::collections::HashMap::new();
for (app, path, depth) in routes.list_route_suppressions().await? {
suppressed
.entry((app, path))
.and_modify(|d| *d = (*d).min(depth))
.or_insert(depth);
}
let compiled = compile_effective_routes(&effective, &suppressed);
table.replace_all(compiled);
Ok(())
@@ -443,10 +449,15 @@ pub async fn rebuild_route_table(
/// inherited route, a descendant SUPPRESSES its path (§11 tail, the
/// `suppressed_paths` set) — the deliberate per-app opt-out.
///
/// `suppressed_paths` holds `(app_id, path)` route suppressions: an INHERITED
/// row (`depth > 0`) whose `(effective_app_id, path)` is present is skipped
/// entirely (the binding is absent → 404). Gated to `depth > 0` so an app can
/// only decline what it inherits, never its own route.
/// `suppressed_paths` maps `(app_id, path)` → the smallest suppressor depth for
/// that key. An INHERITED route (`depth > 0`) is skipped only when its owner is
/// strictly ABOVE the suppressor (`route.depth > suppressor_depth`) — so an
/// app's own suppression (depth 0) declines anything inherited, a group's
/// suppression declines only what that group itself inherits (owned above it),
/// and a nearer descendant's own re-declaration at the same path SURVIVES an
/// ancestor's suppression. The `route.depth > suppressor_depth` test also
/// subsumes the "inherited only" gate (a suppressor at depth `d` can only touch
/// rows at depth `> d ≥ 0`, never its own depth-0 route).
///
/// Requires `rows` ordered by `(effective_app_id, depth ASC)` — which
/// [`RouteRepository::list_effective`] guarantees — so first-seen is nearest.
@@ -454,7 +465,7 @@ pub async fn rebuild_route_table(
#[allow(clippy::implicit_hasher)] // every caller uses the default hasher
pub fn compile_effective_routes(
rows: &[EffectiveRoute],
suppressed_paths: &std::collections::HashSet<(AppId, String)>,
suppressed_paths: &std::collections::HashMap<(AppId, String), i32>,
) -> Vec<CompiledRoute> {
let mut seen: std::collections::HashSet<(AppId, String)> = std::collections::HashSet::new();
let mut out = Vec::new();
@@ -465,12 +476,14 @@ pub fn compile_effective_routes(
continue;
}
// §11 tail: an inherited route whose path this app suppresses is
// dropped — the binding 404s. Gated to inherited rows (`depth > 0`).
// A `sealed` template is non-suppressible: the descendant's opt-out is
// ignored, so it stays in the slice regardless of the suppression.
if er.depth > 0
&& !er.route.sealed
&& suppressed_paths.contains(&(er.effective_app_id, er.route.path.clone()))
// dropped — but ONLY if the route's owner is above the suppressor
// (`route.depth > suppressor_depth`), so a descendant group's own
// re-declaration at the same path isn't over-declined by an ancestor's
// suppression. A `sealed` template is non-suppressible.
if !er.route.sealed
&& suppressed_paths
.get(&(er.effective_app_id, er.route.path.clone()))
.is_some_and(|&suppressor_depth| er.depth > suppressor_depth)
{
continue;
}
@@ -770,8 +783,18 @@ mod tests {
}
/// No route suppressions — the common case for these compile tests.
fn no_suppress() -> std::collections::HashSet<(AppId, String)> {
std::collections::HashSet::new()
fn no_suppress() -> std::collections::HashMap<(AppId, String), i32> {
std::collections::HashMap::new()
}
/// Build an `EffectiveRoute` at an explicit chain depth (for suppression
/// tests that need an inherited row).
fn eff_at(route: &Route, effective_app_id: AppId, depth: i32) -> EffectiveRoute {
EffectiveRoute {
effective_app_id,
depth,
route: route.clone(),
}
}
#[test]
@@ -853,4 +876,42 @@ mod tests {
"a non-identical /y template must coexist"
);
}
#[test]
fn group_suppression_does_not_over_decline_a_nearer_descendants_own_route() {
// Audit fix #3: a suppression declared at depth 2 (an ancestor group G)
// must decline only routes inherited from ABOVE it (a far ancestor GG's
// template at depth 3), NOT a nearer descendant group H's own
// re-declaration of the same path at depth 1. Both /x routes carry
// DIFFERENT bindings (GET vs POST) so shadowing doesn't hide the bug —
// the suppression, keyed by path, is what would wrongly drop H's route.
let app = AppId::from(Uuid::new_v4());
let mut gg = route_with_path("/x"); // far ancestor's template
gg.method = Some("GET".into());
gg.app_id = None;
gg.group_id = Some(picloud_shared::GroupId::from(Uuid::new_v4()));
let mut h = route_with_path("/x"); // nearer descendant group's OWN route
h.method = Some("POST".into());
h.app_id = None;
h.group_id = Some(picloud_shared::GroupId::from(Uuid::new_v4()));
// Ordered nearest-first, as list_effective guarantees.
let rows = [eff_at(&h, app, 1), eff_at(&gg, app, 3)];
// G suppresses /x at depth 2.
let mut suppressed = std::collections::HashMap::new();
suppressed.insert((app, "/x".to_string()), 2);
let ids: Vec<Uuid> = compile_effective_routes(&rows, &suppressed)
.iter()
.map(|c| c.route_id)
.collect();
assert!(
!ids.contains(&gg.id),
"the far-ancestor template (depth 3 > 2) must be declined"
);
assert!(
ids.contains(&h.id),
"the nearer descendant's own /x (depth 1 < 2) must NOT be over-declined"
);
}
}

View File

@@ -64,10 +64,17 @@ pub trait RouteRepository: Send + Sync {
) -> Result<Vec<Route>, ScriptRepositoryError> {
Ok(Vec::new())
}
/// §11 tail per-app opt-out: all `(app_id, path)` ROUTE suppressions. The
/// route-table rebuild drops an inherited route at a suppressed path.
/// §11 tail per-app opt-out: all ROUTE suppressions as
/// `(effective_app_id, path, suppressor_depth)` — the depth is the chain
/// distance from the app to the owner that DECLARED the suppression (0 for
/// the app's own suppression, >0 for an ancestor group's). The route-table
/// rebuild drops an inherited route at a suppressed path only when the
/// route's owner is ABOVE the suppressor (`route_depth > suppressor_depth`),
/// so a nearer descendant's own re-declaration is never over-declined.
/// Defaults empty (non-Postgres impls have no suppressions).
async fn list_route_suppressions(&self) -> Result<Vec<(AppId, String)>, ScriptRepositoryError> {
async fn list_route_suppressions(
&self,
) -> Result<Vec<(AppId, String, i32)>, ScriptRepositoryError> {
Ok(Vec::new())
}
/// §11 tail: every route resolved for every app — each app's own routes
@@ -166,14 +173,18 @@ impl RouteRepository for PostgresRouteRepository {
Ok(rows.into_iter().map(Into::into).collect())
}
async fn list_route_suppressions(&self) -> Result<Vec<(AppId, String)>, ScriptRepositoryError> {
async fn list_route_suppressions(
&self,
) -> Result<Vec<(AppId, String, i32)>, ScriptRepositoryError> {
// §11 tail M1: a route suppression is owned by an app (declines for
// itself) OR a group (declines for its whole subtree). Expand each
// group-owned suppression across every descendant app via the all-apps
// `app_chain` CTE (the same one `list_effective` uses): the result is
// `(effective_app_id, path)` pairs the rebuild consumes unchanged — a
// group suppression appears once per descendant app.
let rows: Vec<(Uuid, String)> = sqlx::query_as(
// `app_chain` CTE (the same one `list_effective` uses). We also return
// `ac.depth` — the chain distance from the effective app to the owner
// that DECLARED the suppression — so the rebuild only declines routes
// inherited from ABOVE that owner (`route_depth > suppressor_depth`),
// not a nearer descendant's own re-declaration at the same path.
let rows: Vec<(Uuid, String, i32)> = sqlx::query_as(
"WITH RECURSIVE app_chain AS ( \
SELECT a.id AS effective_app_id, a.id AS owner_app, \
NULL::uuid AS owner_group, a.group_id AS next_group, 0 AS depth \
@@ -183,7 +194,7 @@ impl RouteRepository for PostgresRouteRepository {
FROM groups g JOIN app_chain ac ON g.id = ac.next_group \
WHERE ac.depth < 64 \
) \
SELECT DISTINCT ac.effective_app_id, ts.reference \
SELECT ac.effective_app_id, ts.reference, ac.depth \
FROM app_chain ac \
JOIN template_suppressions ts \
ON (ts.app_id = ac.owner_app OR ts.group_id = ac.owner_group) \
@@ -191,7 +202,7 @@ impl RouteRepository for PostgresRouteRepository {
)
.fetch_all(&self.pool)
.await?;
Ok(rows.into_iter().map(|(a, r)| (a.into(), r)).collect())
Ok(rows.into_iter().map(|(a, r, d)| (a.into(), r, d)).collect())
}
async fn list_effective(&self) -> Result<Vec<EffectiveRoute>, ScriptRepositoryError> {

View File

@@ -167,13 +167,11 @@ pub fn open(
serde_json::from_slice(&plaintext).map_err(|_| SecretsError::Corrupted)
}
/// Legacy v0 seal (no AAD, no version) used by the email-trigger
/// inbound-secret path. The `email_trigger_details` table has no
/// `version` column yet, so the AAD-binding upgrade for that surface is
/// **deferred to v1.2's key-versioning pass** (audit 2026-06-11 H-D1
/// classifies the email-trigger AAD gap as Medium). Both this and
/// [`open_legacy`] preserve the exact pre-audit wire format so existing
/// email-trigger secrets keep decrypting.
/// Legacy v0 seal (no AAD, no version). The email-trigger path now seals v1 via
/// [`seal_email`] (Track A M3 / audit 2026-06-11 H-D1 closed — `email_trigger_details`
/// gained an `inbound_secret_version` column in 0069). This and [`open_legacy`]
/// remain to READ pre-M3 v0 rows that haven't been re-applied, preserving the
/// exact pre-audit wire format so existing email-trigger secrets keep decrypting.
///
/// # Errors
///
@@ -211,6 +209,75 @@ pub fn open_legacy(
serde_json::from_slice(&plaintext).map_err(|_| SecretsError::Corrupted)
}
/// §M5.5 / audit 2026-06-11 H-D1 — AAD for an email-trigger inbound secret.
/// Binds to the SEALING OWNER only (no per-row trigger id / name), so a group
/// template's sealed bytes stay valid when the materializer copies them verbatim
/// to each descendant (all share the group AAD). The `email:` prefix keeps this
/// namespace disjoint from the per-app `secrets` table's `secret:` AAD, so an
/// email ciphertext can't be swapped in for a same-owner secrets-table entry.
/// A swap ACROSS owners (app↔app, group↔group) fails the GCM tag — the
/// cross-tenant relocation the audit flags. (A swap between two email triggers
/// of the SAME owner is not distinguished — an accepted, low-severity residual.)
fn email_secret_aad(owner: SecretOwner) -> Vec<u8> {
match owner {
SecretOwner::App(app_id) => format!("email:{app_id}"),
SecretOwner::Group(group_id) => format!("email:group:{group_id}"),
}
.into_bytes()
}
/// Seal an email-trigger inbound secret at envelope v1 (AAD = the sealing
/// owner). Returns `(ciphertext, nonce, version)`. The value is a JSON string
/// (the HMAC key). Replaces the email path's former [`seal_legacy`] (v0).
///
/// # Errors
///
/// [`SecretsError::TooLarge`] / [`SecretsError::Backend`] as for [`seal`].
pub fn seal_email(
master_key: &MasterKey,
owner: SecretOwner,
value: &serde_json::Value,
max_value_bytes: usize,
) -> Result<(Vec<u8>, [u8; crypto::NONCE_LEN], i16), SecretsError> {
let plaintext = serde_json::to_vec(value)
.map_err(|e| SecretsError::Backend(format!("encode secret value: {e}")))?;
if plaintext.len() > max_value_bytes {
return Err(SecretsError::TooLarge {
limit: max_value_bytes,
actual: plaintext.len(),
});
}
let aad = email_secret_aad(owner);
let enc = crypto::encrypt_with_aad(&plaintext, &aad, master_key.as_bytes());
Ok((enc.ciphertext, enc.nonce, SECRET_ENVELOPE_V1))
}
/// Open an email-trigger inbound secret, dispatching on `version`: v0 is the
/// legacy no-AAD layout ([`open_legacy`], owner ignored); v1 binds AAD to the
/// sealing `owner`. The caller recovers the sealing owner (a materialized copy's
/// source-template group, else the app) before calling.
///
/// # Errors
///
/// [`SecretsError::Corrupted`] when decryption or JSON decoding fails.
pub fn open_email(
master_key: &MasterKey,
owner: SecretOwner,
ciphertext: &[u8],
nonce: &[u8],
version: i16,
) -> Result<serde_json::Value, SecretsError> {
let plaintext = match version {
SECRET_ENVELOPE_V1 => {
let aad = email_secret_aad(owner);
crypto::decrypt_with_aad(ciphertext, nonce, &aad, master_key.as_bytes())
}
_ => crypto::decrypt(ciphertext, nonce, master_key.as_bytes()),
}
.map_err(|_| SecretsError::Corrupted)?;
serde_json::from_slice(&plaintext).map_err(|_| SecretsError::Corrupted)
}
pub struct SecretsServiceImpl {
repo: Arc<dyn SecretsRepo>,
authz: Arc<dyn AuthzRepo>,

View File

@@ -69,6 +69,24 @@ pub struct TriggerConfig {
/// the message.
pub queue_reclaim_interval_ms: u32,
/// How long an OUTBOX claim may be held before the reclaim task assumes the
/// dispatcher that took it is gone and returns the row to the queue
/// (`PICLOUD_OUTBOX_CLAIM_TIMEOUT_SEC`, default 600).
///
/// Without this, a process that dies between claiming a row and finishing it
/// strands that row FOREVER — `claim_due` only selects `claimed_at IS NULL`,
/// and nothing else ever clears the claim. Every other claim-based store
/// (queue, group queue, workflow steps) already had a reclaimer; the outbox
/// did not, so a crash or restart mid-dispatch permanently lost every
/// in-flight trigger event.
///
/// The default is deliberately generous: a script may run for up to 300s
/// (`scripts.timeout_seconds` CHECK), so a claim older than twice that is
/// abandoned rather than slow. Reclaiming a row a LIVE dispatcher is still
/// working on would double-execute it — dispatch is at-least-once, so that is
/// survivable, but not something to court by tuning this down.
pub outbox_claim_timeout_secs: u32,
/// Default per-queue visibility-timeout in seconds (v1.1.9). Used
/// at queue-trigger creation when the request omits an explicit
/// value. Default 30. The per-trigger column overrides this.
@@ -88,6 +106,7 @@ impl TriggerConfig {
abandoned_retention_days: 7,
cron_tick_interval_ms: 30_000,
queue_reclaim_interval_ms: 30_000,
outbox_claim_timeout_secs: 600,
queue_default_visibility_timeout_secs: 30,
}
}
@@ -123,6 +142,10 @@ impl TriggerConfig {
&mut c.queue_default_visibility_timeout_secs,
"PICLOUD_QUEUE_DEFAULT_VISIBILITY_TIMEOUT_SECS",
);
load_u32(
&mut c.outbox_claim_timeout_secs,
"PICLOUD_OUTBOX_CLAIM_TIMEOUT_SEC",
);
c
}
}

View File

@@ -26,6 +26,12 @@ use crate::trigger_config::BackoffShape;
/// The `t.group_id IS NOT NULL` guard keeps an app's OWN trigger unsuppressable
/// (an owner can only decline what it inherits); `t.sealed = FALSE` keeps a
/// mandatory template non-declinable.
///
/// `sc.depth < c.depth` is the over-decline guard: a suppressor at depth
/// `sc.depth` may only decline a trigger whose owner is strictly ABOVE it
/// (`c.depth` is the trigger owner's depth in the outer query's `chain c`), so
/// a suppression of a far-ancestor template never clobbers a NEARER descendant
/// group's OWN trigger that happens to bind a same-named handler.
pub(crate) const TRIGGER_SUPPRESSION_ANTIJOIN: &str = " \
AND NOT EXISTS ( \
SELECT 1 FROM template_suppressions ts \
@@ -34,7 +40,8 @@ pub(crate) const TRIGGER_SUPPRESSION_ANTIJOIN: &str = " \
WHERE ts.target_kind = 'trigger' \
AND LOWER(ts.reference) = LOWER(s.name) \
AND t.group_id IS NOT NULL \
AND t.sealed = FALSE)";
AND t.sealed = FALSE \
AND sc.depth < c.depth)";
#[derive(Debug, thiserror::Error)]
pub enum TriggerRepoError {
@@ -306,6 +313,24 @@ pub struct CreateEmailTrigger {
pub script_id: ScriptId,
pub inbound_secret_encrypted: Option<Vec<u8>>,
pub inbound_secret_nonce: Option<Vec<u8>>,
/// Envelope version of the sealed inbound secret (0 = legacy no-AAD, 1 =
/// AAD-bound to the sealing owner). `0` when there is no secret.
pub inbound_secret_version: i16,
/// A3: the mailbox this trigger receives at over the native SMTP listener.
/// `None` = webhook-only (addressed by trigger UUID in the URL).
pub inbound_address: Option<String>,
pub registered_by_principal: AdminUserId,
}
/// A3: the SMTP-ingress lookup result — the app-owned email trigger claiming an
/// `RCPT TO` address, plus what an `Email` outbox row needs. No secret: the
/// SMTP path has no HMAC (the mailbox address + per-app isolation are the
/// boundary), unlike the webhook path.
#[derive(Debug, Clone)]
pub struct SmtpInboundTarget {
pub trigger_id: TriggerId,
pub app_id: AppId,
pub script_id: ScriptId,
pub registered_by_principal: AdminUserId,
}
@@ -358,6 +383,13 @@ pub struct EmailInboundTarget {
/// trigger (accepts any POST).
pub inbound_secret_encrypted: Option<Vec<u8>>,
pub inbound_secret_nonce: Option<Vec<u8>>,
/// Envelope version of the sealed secret (0 = legacy no-AAD, 1 = AAD-bound).
pub inbound_secret_version: i16,
/// §M5.5 / audit H-D1: the group this secret was sealed under, when this row
/// is a materialized copy of a group EMAIL TEMPLATE (`materialized_from ->
/// template.group_id`). `None` for a standalone per-app trigger (sealed under
/// the app). Drives the AAD owner at open time for a v1 secret.
pub sealing_group: Option<GroupId>,
}
/// One match for the dispatcher's "which KV triggers fire on this
@@ -458,6 +490,17 @@ pub trait TriggerRepo: Send + Sync {
trigger_id: TriggerId,
) -> Result<Option<EmailInboundTarget>, TriggerRepoError>;
/// A3: resolve an inbound SMTP `RCPT TO` address to the app-owned email
/// trigger claiming it (case-insensitive). `None` when no enabled app
/// trigger claims the address. Defaults to `None` so non-Postgres impls
/// (tests) need not provide it.
async fn email_inbound_target_by_address(
&self,
_address: &str,
) -> Result<Option<SmtpInboundTarget>, TriggerRepoError> {
Ok(None)
}
async fn list_for_app(&self, app_id: AppId) -> Result<Vec<Trigger>, TriggerRepoError>;
/// Group-owned trigger TEMPLATES (§11 tail) declared directly at `group_id`.
@@ -546,6 +589,23 @@ pub trait TriggerRepo: Send + Sync {
script_id: Option<ScriptId>,
) -> Result<Vec<DeadLetterTriggerMatch>, TriggerRepoError>;
/// §11.6 B2 shared dead-letter fan-out: enabled `shared = true`
/// `dead_letter` triggers on the OWNING group — fired when a message in
/// that group's SHARED queue is dead-lettered. Same source/trigger/script
/// filter logic as `list_matching_dead_letter`, but keyed on the owning
/// group instead of a chain walk. Default empty so non-Postgres impls
/// degrade to "no shared dead-letter triggers".
async fn list_matching_shared_dead_letter(
&self,
owning_group: GroupId,
source: &str,
trigger_id: Option<TriggerId>,
script_id: Option<ScriptId>,
) -> Result<Vec<DeadLetterTriggerMatch>, TriggerRepoError> {
let _ = (owning_group, source, trigger_id, script_id);
Ok(Vec::new())
}
/// v1.1.9. Create a queue:receive trigger. Enforces exactly one
/// consumer per `(app_id, queue_name)` via `pg_advisory_xact_lock`
/// + SELECT-then-INSERT (a partial unique index across the parent
@@ -588,39 +648,95 @@ impl PostgresTriggerRepo {
pub fn new(pool: PgPool) -> Self {
Self { pool }
}
}
/// §11.6 shared-collection match: enabled `shared = true` triggers on the
/// OWNING group, glob + op filtered in Rust (like the per-app path). `kind`
/// and `detail_table` are hard-coded literals from the three call sites (no
/// injection). No chain / no suppression anti-join — a shared trigger is
/// declared on the group that owns the collection and fires under the writer.
async fn shared_match_rows(
&self,
kind: &str,
detail_table: &str,
owning_group: GroupId,
collection: &str,
op_str: &str,
) -> Result<Vec<KvMatchRow>, TriggerRepoError> {
let rows: Vec<KvMatchRow> = sqlx::query_as(&format!(
"SELECT t.id, t.script_id, t.dispatch_mode, \
t.retry_max_attempts, t.retry_backoff, t.retry_base_ms, \
t.registered_by_principal, \
d.collection_glob, d.ops \
FROM triggers t \
JOIN {detail_table} d ON d.trigger_id = t.id \
WHERE t.kind = '{kind}' AND t.enabled = TRUE \
AND t.shared = TRUE AND t.group_id = $1"
))
.bind(owning_group.into_inner())
.fetch_all(&self.pool)
.await?;
Ok(rows
.into_iter()
.filter(|r| collection_matches(&r.collection_glob, collection))
.filter(|r| r.ops.is_empty() || r.ops.iter().any(|o| o == op_str))
.collect())
}
// ----------------------------------------------------------------------------
// Connection-scoped event matching
//
// The three collection-event kinds (kv / docs / files) match identically apart
// from the `triggers.kind` discriminator and their detail table, so both shapes
// — the per-app CHAIN walk and the §11.6 SHARED-collection lookup — live here
// once, parameterized on an executor.
//
// Taking an executor rather than `&self.pool` is what lets the transactional
// outbox work: `outbox_event_emitter` runs the match on the SAME connection
// (`&mut *tx`) as the data write it is fanning out, so the write and its outbox
// rows commit together or not at all. The `list_matching_*` trait methods below
// pass `&self.pool` and behave exactly as before.
//
// `kind` and `detail_table` are interpolated into the SQL, so they must stay
// hard-coded literals from the call sites below — never user data.
// ----------------------------------------------------------------------------
/// Glob + op filtering, in Rust rather than SQL. **Critical**: an empty `ops`
/// array means "any op", which a SQL `$op = ANY(ops)` predicate would silently
/// exclude.
fn filter_match_rows(rows: Vec<KvMatchRow>, collection: &str, op_str: &str) -> Vec<KvMatchRow> {
rows.into_iter()
.filter(|r| collection_matches(&r.collection_glob, collection))
.filter(|r| r.ops.is_empty() || r.ops.iter().any(|o| o == op_str))
.collect()
}
/// Per-app match: the firing app's OWN triggers plus any inherited from an
/// ancestor group, minus the ones a suppression on its chain declines.
pub(crate) async fn list_matching_on<'c, E>(
exec: E,
kind: &str,
detail_table: &str,
app_id: AppId,
collection: &str,
op_str: &str,
) -> Result<Vec<KvMatchRow>, TriggerRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let rows: Vec<KvMatchRow> = sqlx::query_as(&format!(
"{CHAIN_LEVELS_CTE} \
SELECT t.id, t.script_id, t.dispatch_mode, \
t.retry_max_attempts, t.retry_backoff, t.retry_base_ms, \
t.registered_by_principal, \
d.collection_glob, d.ops \
FROM triggers t \
JOIN {detail_table} d ON d.trigger_id = t.id \
JOIN chain c ON (t.app_id = c.app_owner OR t.group_id = c.group_owner) \
WHERE t.kind = '{kind}' AND t.enabled = TRUE \
AND t.shared = FALSE{TRIGGER_SUPPRESSION_ANTIJOIN}"
))
.bind(app_id.into_inner())
.fetch_all(exec)
.await?;
Ok(filter_match_rows(rows, collection, op_str))
}
/// §11.6 shared-collection match: enabled `shared = true` triggers on the
/// OWNING group. No chain and no suppression anti-join — a shared trigger is
/// declared on the group that owns the collection and fires under the writer.
pub(crate) async fn list_matching_shared_on<'c, E>(
exec: E,
kind: &str,
detail_table: &str,
owning_group: GroupId,
collection: &str,
op_str: &str,
) -> Result<Vec<KvMatchRow>, TriggerRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let rows: Vec<KvMatchRow> = sqlx::query_as(&format!(
"SELECT t.id, t.script_id, t.dispatch_mode, \
t.retry_max_attempts, t.retry_backoff, t.retry_base_ms, \
t.registered_by_principal, \
d.collection_glob, d.ops \
FROM triggers t \
JOIN {detail_table} d ON d.trigger_id = t.id \
WHERE t.kind = '{kind}' AND t.enabled = TRUE \
AND t.shared = TRUE AND t.group_id = $1"
))
.bind(owning_group.into_inner())
.fetch_all(exec)
.await?;
Ok(filter_match_rows(rows, collection, op_str))
}
/// Insert a trigger (parent row + per-kind detail) within an existing
@@ -652,7 +768,9 @@ pub(crate) async fn insert_trigger_tx(
TriggerDetails::Cron { .. } => "cron",
TriggerDetails::Pubsub { .. } => "pubsub",
TriggerDetails::Queue { .. } => "queue",
TriggerDetails::DeadLetter { .. } | TriggerDetails::Email { .. } => {
// §11.6 B2: a group-owned, shared dead_letter template is declarative.
TriggerDetails::DeadLetter { .. } => "dead_letter",
TriggerDetails::Email { .. } => {
return Err(TriggerRepoError::Invalid(
"trigger kind not supported by declarative apply".into(),
));
@@ -797,7 +915,27 @@ pub(crate) async fn insert_trigger_tx(
.execute(&mut **tx)
.await?;
}
TriggerDetails::DeadLetter { .. } | TriggerDetails::Email { .. } => {
// §11.6 B2: mirror `create_dead_letter_trigger`'s detail insert. For the
// declarative (bundle) path the trigger_id/script_id filters are always
// None — a group shared dead_letter matches by source only.
TriggerDetails::DeadLetter {
source_filter,
trigger_id_filter,
script_id_filter,
} => {
sqlx::query(
"INSERT INTO dead_letter_trigger_details \
(trigger_id, source_filter, trigger_id_filter, script_id_filter) \
VALUES ($1, $2, $3, $4)",
)
.bind(tid)
.bind(source_filter.as_deref())
.bind(trigger_id_filter.map(TriggerId::into_inner))
.bind(script_id_filter.map(ScriptId::into_inner))
.execute(&mut **tx)
.await?;
}
TriggerDetails::Email { .. } => {
unreachable!("guarded above")
}
}
@@ -820,6 +958,7 @@ pub(crate) async fn insert_email_trigger_tx(
registered_by: AdminUserId,
inbound_secret_encrypted: &[u8],
inbound_secret_nonce: &[u8],
inbound_secret_version: i16,
) -> Result<TriggerId, TriggerRepoError> {
let (owner_app_id, owner_group_id) = match owner {
ScriptOwner::App(a) => (Some(a.into_inner()), None),
@@ -840,12 +979,13 @@ pub(crate) async fn insert_email_trigger_tx(
.await?;
sqlx::query(
"INSERT INTO email_trigger_details \
(trigger_id, inbound_secret_encrypted, inbound_secret_nonce) \
VALUES ($1, $2, $3)",
(trigger_id, inbound_secret_encrypted, inbound_secret_nonce, inbound_secret_version) \
VALUES ($1, $2, $3, $4)",
)
.bind(row.0)
.bind(inbound_secret_encrypted)
.bind(inbound_secret_nonce)
.bind(inbound_secret_version)
.execute(&mut **tx)
.await?;
Ok(row.0.into())
@@ -1306,12 +1446,15 @@ impl TriggerRepo for PostgresTriggerRepo {
sqlx::query(
"INSERT INTO email_trigger_details \
(trigger_id, inbound_secret_encrypted, inbound_secret_nonce) \
VALUES ($1, $2, $3)",
(trigger_id, inbound_secret_encrypted, inbound_secret_nonce, \
inbound_secret_version, inbound_address) \
VALUES ($1, $2, $3, $4, $5)",
)
.bind(parent.id)
.bind(req.inbound_secret_encrypted.as_deref())
.bind(req.inbound_secret_nonce.as_deref())
.bind(req.inbound_secret_version)
.bind(req.inbound_address.as_deref())
.execute(&mut *tx)
.await?;
@@ -1349,11 +1492,17 @@ impl TriggerRepo for PostgresTriggerRepo {
// row — only its materialized per-app copies are directly invocable
// (mirrors the cron-scheduler / queue-consumer guards). A template
// has no app to run under, so hitting its webhook URL must 404.
// §M5.5 / audit H-D1: `tmpl.group_id` (via `materialized_from`) is the
// SEALING owner for a materialized copy of a group email template — the
// AAD owner needed to open a v1 secret. NULL for a standalone app
// trigger (sealed under the app).
"SELECT t.app_id, t.script_id, t.enabled, t.dispatch_mode, \
t.registered_by_principal, \
d.inbound_secret_encrypted, d.inbound_secret_nonce \
d.inbound_secret_encrypted, d.inbound_secret_nonce, \
d.inbound_secret_version, tmpl.group_id AS sealing_group \
FROM triggers t \
JOIN email_trigger_details d ON d.trigger_id = t.id \
LEFT JOIN triggers tmpl ON tmpl.id = t.materialized_from \
WHERE t.id = $1 AND t.kind = 'email' AND t.app_id IS NOT NULL",
)
.bind(trigger_id.into_inner())
@@ -1367,9 +1516,38 @@ impl TriggerRepo for PostgresTriggerRepo {
registered_by_principal: r.registered_by_principal.into(),
inbound_secret_encrypted: r.inbound_secret_encrypted,
inbound_secret_nonce: r.inbound_secret_nonce,
inbound_secret_version: r.inbound_secret_version,
sealing_group: r.sealing_group.map(Into::into),
}))
}
async fn email_inbound_target_by_address(
&self,
address: &str,
) -> Result<Option<SmtpInboundTarget>, TriggerRepoError> {
// `t.app_id IS NOT NULL` excludes group templates (only their
// materialized per-app copies are invocable); `t.enabled` skips inert
// triggers. The address is matched case-insensitively (mailboxes are).
let row: Option<(Uuid, Uuid, Uuid, Uuid)> = sqlx::query_as(
"SELECT t.id, t.app_id, t.script_id, t.registered_by_principal \
FROM triggers t \
JOIN email_trigger_details d ON d.trigger_id = t.id \
WHERE lower(d.inbound_address) = lower($1) \
AND t.kind = 'email' AND t.app_id IS NOT NULL AND t.enabled",
)
.bind(address)
.fetch_optional(&self.pool)
.await?;
Ok(
row.map(|(tid, app_id, script_id, principal)| SmtpInboundTarget {
trigger_id: tid.into(),
app_id: app_id.into(),
script_id: script_id.into(),
registered_by_principal: principal.into(),
}),
)
}
async fn list_for_app(&self, app_id: AppId) -> Result<Vec<Trigger>, TriggerRepoError> {
let parents: Vec<TriggerRow> = sqlx::query_as(
"SELECT id, app_id, script_id, name, kind, enabled, sealed, shared, materialized_from, \
@@ -1455,48 +1633,16 @@ impl TriggerRepo for PostgresTriggerRepo {
collection: &str,
op: KvEventOp,
) -> Result<Vec<KvTriggerMatch>, TriggerRepoError> {
// Fetch all enabled KV triggers for the app — glob matching
// happens in Rust so we don't have to teach the query about
// `*` and `prefix:*`. Sets are tiny in practice (one app's
// worth of triggers, usually a handful).
let rows: Vec<KvMatchRow> = sqlx::query_as(&format!(
"{CHAIN_LEVELS_CTE} \
SELECT t.id, t.script_id, t.dispatch_mode, \
t.retry_max_attempts, t.retry_backoff, t.retry_base_ms, \
t.registered_by_principal, \
d.collection_glob, d.ops \
FROM triggers t \
JOIN kv_trigger_details d ON d.trigger_id = t.id \
JOIN chain c ON (t.app_id = c.app_owner OR t.group_id = c.group_owner) \
WHERE t.kind = 'kv' AND t.enabled = TRUE \
AND t.shared = FALSE{TRIGGER_SUPPRESSION_ANTIJOIN}"
))
.bind(app_id.into_inner())
.fetch_all(&self.pool)
let rows = list_matching_on(
&self.pool,
"kv",
"kv_trigger_details",
app_id,
collection,
op.as_str(),
)
.await?;
let op_str = op.as_str();
let mut out = Vec::new();
for r in rows {
if !collection_matches(&r.collection_glob, collection) {
continue;
}
let any_op = r.ops.is_empty();
if !any_op && !r.ops.iter().any(|o| o == op_str) {
continue;
}
out.push(KvTriggerMatch {
trigger_id: r.id.into(),
script_id: r.script_id.into(),
dispatch_mode: dispatch_from_str(&r.dispatch_mode),
retry_max_attempts: u32::try_from(r.retry_max_attempts).unwrap_or(3),
retry_backoff: BackoffShape::from_wire(&r.retry_backoff)
.unwrap_or(BackoffShape::Exponential),
retry_base_ms: u32::try_from(r.retry_base_ms).unwrap_or(1000),
registered_by_principal: r.registered_by_principal.into(),
});
}
Ok(out)
Ok(rows.into_iter().map(KvMatchRow::into_kv).collect())
}
async fn list_matching_docs(
@@ -1505,49 +1651,16 @@ impl TriggerRepo for PostgresTriggerRepo {
collection: &str,
op: DocsEventOp,
) -> Result<Vec<DocsTriggerMatch>, TriggerRepoError> {
// Mirrors list_matching_kv: pull every enabled docs trigger,
// filter glob + ops in Rust. **Critical**: do NOT push the
// ops check into SQL (`WHERE $op = ANY(ops)`) — that would
// exclude rows with `ops = '{}'` from the results, breaking
// the empty-array-means-any-op semantic.
let rows: Vec<KvMatchRow> = sqlx::query_as(&format!(
"{CHAIN_LEVELS_CTE} \
SELECT t.id, t.script_id, t.dispatch_mode, \
t.retry_max_attempts, t.retry_backoff, t.retry_base_ms, \
t.registered_by_principal, \
d.collection_glob, d.ops \
FROM triggers t \
JOIN docs_trigger_details d ON d.trigger_id = t.id \
JOIN chain c ON (t.app_id = c.app_owner OR t.group_id = c.group_owner) \
WHERE t.kind = 'docs' AND t.enabled = TRUE \
AND t.shared = FALSE{TRIGGER_SUPPRESSION_ANTIJOIN}"
))
.bind(app_id.into_inner())
.fetch_all(&self.pool)
let rows = list_matching_on(
&self.pool,
"docs",
"docs_trigger_details",
app_id,
collection,
op.as_str(),
)
.await?;
let op_str = op.as_str();
let mut out = Vec::new();
for r in rows {
if !collection_matches(&r.collection_glob, collection) {
continue;
}
let any_op = r.ops.is_empty();
if !any_op && !r.ops.iter().any(|o| o == op_str) {
continue;
}
out.push(DocsTriggerMatch {
trigger_id: r.id.into(),
script_id: r.script_id.into(),
dispatch_mode: dispatch_from_str(&r.dispatch_mode),
retry_max_attempts: u32::try_from(r.retry_max_attempts).unwrap_or(3),
retry_backoff: BackoffShape::from_wire(&r.retry_backoff)
.unwrap_or(BackoffShape::Exponential),
retry_base_ms: u32::try_from(r.retry_base_ms).unwrap_or(1000),
registered_by_principal: r.registered_by_principal.into(),
});
}
Ok(out)
Ok(rows.into_iter().map(KvMatchRow::into_docs).collect())
}
async fn list_matching_files(
@@ -1556,46 +1669,16 @@ impl TriggerRepo for PostgresTriggerRepo {
collection: &str,
op: FilesEventOp,
) -> Result<Vec<FilesTriggerMatch>, TriggerRepoError> {
// Mirrors list_matching_kv: pull every enabled files trigger,
// filter glob + ops in Rust (empty ops array means "any op").
let rows: Vec<KvMatchRow> = sqlx::query_as(&format!(
"{CHAIN_LEVELS_CTE} \
SELECT t.id, t.script_id, t.dispatch_mode, \
t.retry_max_attempts, t.retry_backoff, t.retry_base_ms, \
t.registered_by_principal, \
d.collection_glob, d.ops \
FROM triggers t \
JOIN files_trigger_details d ON d.trigger_id = t.id \
JOIN chain c ON (t.app_id = c.app_owner OR t.group_id = c.group_owner) \
WHERE t.kind = 'files' AND t.enabled = TRUE \
AND t.shared = FALSE{TRIGGER_SUPPRESSION_ANTIJOIN}"
))
.bind(app_id.into_inner())
.fetch_all(&self.pool)
let rows = list_matching_on(
&self.pool,
"files",
"files_trigger_details",
app_id,
collection,
op.as_str(),
)
.await?;
let op_str = op.as_str();
let mut out = Vec::new();
for r in rows {
if !collection_matches(&r.collection_glob, collection) {
continue;
}
let any_op = r.ops.is_empty();
if !any_op && !r.ops.iter().any(|o| o == op_str) {
continue;
}
out.push(FilesTriggerMatch {
trigger_id: r.id.into(),
script_id: r.script_id.into(),
dispatch_mode: dispatch_from_str(&r.dispatch_mode),
retry_max_attempts: u32::try_from(r.retry_max_attempts).unwrap_or(3),
retry_backoff: BackoffShape::from_wire(&r.retry_backoff)
.unwrap_or(BackoffShape::Exponential),
retry_base_ms: u32::try_from(r.retry_base_ms).unwrap_or(1000),
registered_by_principal: r.registered_by_principal.into(),
});
}
Ok(out)
Ok(rows.into_iter().map(KvMatchRow::into_files).collect())
}
async fn list_matching_shared_kv(
@@ -1604,15 +1687,15 @@ impl TriggerRepo for PostgresTriggerRepo {
collection: &str,
op: KvEventOp,
) -> Result<Vec<KvTriggerMatch>, TriggerRepoError> {
let rows = self
.shared_match_rows(
"kv",
"kv_trigger_details",
owning_group,
collection,
op.as_str(),
)
.await?;
let rows = list_matching_shared_on(
&self.pool,
"kv",
"kv_trigger_details",
owning_group,
collection,
op.as_str(),
)
.await?;
Ok(rows.into_iter().map(KvMatchRow::into_kv).collect())
}
@@ -1622,15 +1705,15 @@ impl TriggerRepo for PostgresTriggerRepo {
collection: &str,
op: DocsEventOp,
) -> Result<Vec<DocsTriggerMatch>, TriggerRepoError> {
let rows = self
.shared_match_rows(
"docs",
"docs_trigger_details",
owning_group,
collection,
op.as_str(),
)
.await?;
let rows = list_matching_shared_on(
&self.pool,
"docs",
"docs_trigger_details",
owning_group,
collection,
op.as_str(),
)
.await?;
Ok(rows.into_iter().map(KvMatchRow::into_docs).collect())
}
@@ -1640,15 +1723,15 @@ impl TriggerRepo for PostgresTriggerRepo {
collection: &str,
op: FilesEventOp,
) -> Result<Vec<FilesTriggerMatch>, TriggerRepoError> {
let rows = self
.shared_match_rows(
"files",
"files_trigger_details",
owning_group,
collection,
op.as_str(),
)
.await?;
let rows = list_matching_shared_on(
&self.pool,
"files",
"files_trigger_details",
owning_group,
collection,
op.as_str(),
)
.await?;
Ok(rows.into_iter().map(KvMatchRow::into_files).collect())
}
@@ -1665,6 +1748,7 @@ impl TriggerRepo for PostgresTriggerRepo {
FROM triggers t \
JOIN dead_letter_trigger_details d ON d.trigger_id = t.id \
WHERE t.app_id = $1 AND t.kind = 'dead_letter' AND t.enabled = TRUE \
AND t.shared = FALSE \
AND (d.source_filter IS NULL OR d.source_filter = $2) \
AND (d.trigger_id_filter IS NULL OR d.trigger_id_filter = $3) \
AND (d.script_id_filter IS NULL OR d.script_id_filter = $4)",
@@ -1687,6 +1771,42 @@ impl TriggerRepo for PostgresTriggerRepo {
.collect())
}
async fn list_matching_shared_dead_letter(
&self,
owning_group: GroupId,
source: &str,
trigger_id: Option<TriggerId>,
script_id: Option<ScriptId>,
) -> Result<Vec<DeadLetterTriggerMatch>, TriggerRepoError> {
let rows: Vec<DlMatchRow> = sqlx::query_as(
"SELECT t.id, t.script_id, t.dispatch_mode, t.registered_by_principal, \
d.source_filter, d.trigger_id_filter, d.script_id_filter \
FROM triggers t \
JOIN dead_letter_trigger_details d ON d.trigger_id = t.id \
WHERE t.group_id = $1 AND t.kind = 'dead_letter' AND t.enabled = TRUE \
AND t.shared = TRUE \
AND (d.source_filter IS NULL OR d.source_filter = $2) \
AND (d.trigger_id_filter IS NULL OR d.trigger_id_filter = $3) \
AND (d.script_id_filter IS NULL OR d.script_id_filter = $4)",
)
.bind(owning_group.into_inner())
.bind(source)
.bind(trigger_id.map(TriggerId::into_inner))
.bind(script_id.map(ScriptId::into_inner))
.fetch_all(&self.pool)
.await?;
Ok(rows
.into_iter()
.map(|r| DeadLetterTriggerMatch {
trigger_id: r.id.into(),
script_id: r.script_id.into(),
dispatch_mode: dispatch_from_str(&r.dispatch_mode),
registered_by_principal: r.registered_by_principal.into(),
})
.collect())
}
async fn create_queue_trigger(
&self,
app_id: AppId,
@@ -1850,7 +1970,13 @@ impl TriggerRepo for PostgresTriggerRepo {
/// Stable per-(app_id, queue_name) i64 derived from the UUID bytes +
/// the queue name. Used as the key for `pg_advisory_xact_lock` so
/// concurrent `create_queue_trigger` calls for the same queue serialize.
fn advisory_lock_key(app_id: AppId, queue_name: &str) -> i64 {
/// Stable pg-advisory-lock key for the one-consumer-per-`(app_id, queue_name)`
/// invariant. `pub(crate)` so `materialize` can take the SAME lock the
/// interactive `create_queue_trigger` uses — otherwise a tree-apply
/// rematerialize could race an interactive consumer-create and double-fill the
/// slot (there is no DB unique index: the parent's `app_id` and the detail's
/// `queue_name` live in different tables).
pub(crate) fn advisory_lock_key(app_id: AppId, queue_name: &str) -> i64 {
use std::collections::hash_map::DefaultHasher;
use std::hash::{Hash, Hasher};
let mut h = DefaultHasher::new();
@@ -2122,6 +2248,8 @@ struct EmailInboundRow {
registered_by_principal: Uuid,
inbound_secret_encrypted: Option<Vec<u8>>,
inbound_secret_nonce: Option<Vec<u8>>,
inbound_secret_version: i16,
sealing_group: Option<Uuid>,
}
#[derive(sqlx::FromRow)]
@@ -2133,9 +2261,9 @@ struct DlDetailRow {
}
#[derive(sqlx::FromRow)]
struct KvMatchRow {
id: Uuid,
script_id: Uuid,
pub(crate) struct KvMatchRow {
pub(crate) id: Uuid,
pub(crate) script_id: Uuid,
dispatch_mode: String,
retry_max_attempts: i32,
retry_backoff: String,

View File

@@ -26,7 +26,7 @@ use serde_json::json;
use crate::app_repo::AppRepository;
use crate::authz::{require, AuthzDenied, AuthzError, AuthzRepo, Capability};
use crate::repo::{ScriptRepository, ScriptRepositoryError};
use crate::secrets_service::seal_legacy;
use crate::secrets_service::{seal_email, SecretOwner};
use crate::trigger_config::{BackoffShape, TriggerConfig};
use crate::trigger_repo::{
CreateCronTrigger, CreateDeadLetterTrigger, CreateDocsTrigger, CreateEmailTrigger,
@@ -576,6 +576,10 @@ struct CreateEmailTriggerRequest {
/// (or absent) means the trigger accepts unsigned POSTs.
#[serde(default)]
inbound_secret: Option<String>,
/// A3: the mailbox this trigger receives at over the native SMTP listener
/// (`RCPT TO`). Absent = webhook-only. Case-insensitively unique.
#[serde(default)]
inbound_address: Option<String>,
}
async fn create_email_trigger(
@@ -610,13 +614,12 @@ async fn create_email_trigger(
))
}
};
// 64 KB cap is irrelevant for a signing secret, but `seal_legacy`
// takes one; reuse the secrets default. Audit 2026-06-11 H-D1: the
// email-trigger secret is still sealed v0 (no AAD) — the AAD upgrade
// for this surface is deferred to v1.2 since email_trigger_details
// has no version column yet. See secrets_service::seal_legacy.
let (ct, nonce) = seal_legacy(
// 64 KB cap is irrelevant for a signing secret, but `seal_email` takes one;
// reuse the secrets default. §M5.5 / audit H-D1: sealed v1 with AAD bound to
// the app owner (this is a standalone per-app trigger — never materialized).
let (ct, nonce, version) = seal_email(
&s.master_key,
SecretOwner::App(app_id),
&serde_json::Value::String(secret),
crate::secrets_service::DEFAULT_SECRET_MAX_VALUE_BYTES,
)
@@ -628,6 +631,11 @@ async fn create_email_trigger(
script_id: input.script_id,
inbound_secret_encrypted,
inbound_secret_nonce,
inbound_secret_version: version,
inbound_address: input
.inbound_address
.map(|a| a.trim().to_string())
.filter(|a| !a.is_empty()),
registered_by_principal: principal.user_id,
};
let created = s.triggers.create_email_trigger(app_id, req).await?;
@@ -1043,6 +1051,8 @@ mod tests {
registered_by_principal: t.registered_by_principal,
inbound_secret_encrypted: None,
inbound_secret_nonce: None,
inbound_secret_version: 0,
sealing_group: None,
}))
}
async fn create_cron_trigger(

View File

@@ -56,6 +56,14 @@ pub fn app_users_router(state: AppUsersState) -> Router {
"/apps/{id_or_slug}/users/{user_id}/revoke-sessions",
post(revoke_sessions),
)
.route(
"/apps/{id_or_slug}/users/{user_id}/roles",
post(add_role_handler),
)
.route(
"/apps/{id_or_slug}/users/{user_id}/roles/{role}",
delete(remove_role_handler),
)
.route(
"/apps/{id_or_slug}/invitations",
get(list_invitations_handler).post(create_invitation_handler),
@@ -100,6 +108,11 @@ pub struct PatchUserRequest {
pub display_name: Option<Option<String>>,
}
#[derive(Debug, Deserialize)]
pub struct AddRoleRequest {
pub role: String,
}
#[derive(Debug, Serialize)]
pub struct ResetPasswordResponse {
/// The one-shot raw token. Returned exactly once — the admin
@@ -304,6 +317,50 @@ async fn revoke_sessions(
Ok(Json(RevokeSessionsResponse { revoked }))
}
/// Grant an app-user a role (F-012). Role assignment was previously reachable
/// only from a Rhai script via `users::add_role` — so bootstrapping the first
/// admin needed a bespoke setup-token endpoint. Idempotent (the repo upsert is
/// `ON CONFLICT DO NOTHING`). Authz (`AppUsersAdmin`) is enforced inside the
/// service. Returns the refreshed user so the caller sees the new role set.
async fn add_role_handler(
State(s): State<AppUsersState>,
Extension(principal): Extension<Principal>,
Path((id_or_slug, user_id)): Path<(String, Uuid)>,
Json(input): Json<AddRoleRequest>,
) -> Result<Json<AppUser>, AppUsersApiError> {
let app = resolve_app(&*s.apps, &id_or_slug).await?;
let cx = synth_cx(app.id, &principal);
let uid = AppUserId::from(user_id);
s.users.add_role(&cx, uid, &input.role).await?;
let user = s
.users
.get(&cx, uid)
.await?
.ok_or(AppUsersApiError::UserNotFound)?;
Ok(Json(user))
}
/// Revoke an app-user role (F-012). Idempotent — a `DELETE` of a role the user
/// doesn't hold still succeeds (200) and returns the current user.
async fn remove_role_handler(
State(s): State<AppUsersState>,
Extension(principal): Extension<Principal>,
Path((id_or_slug, user_id, role)): Path<(String, Uuid, String)>,
) -> Result<Json<AppUser>, AppUsersApiError> {
let app = resolve_app(&*s.apps, &id_or_slug).await?;
let cx = synth_cx(app.id, &principal);
let uid = AppUserId::from(user_id);
// The bool (whether a row was actually removed) is intentionally ignored —
// DELETE is idempotent.
let _ = s.users.remove_role(&cx, uid, &role).await?;
let user = s
.users
.get(&cx, uid)
.await?
.ok_or(AppUsersApiError::UserNotFound)?;
Ok(Json(user))
}
async fn list_invitations_handler(
State(s): State<AppUsersState>,
Extension(principal): Extension<Principal>,

View File

@@ -46,7 +46,7 @@ use crate::app_user_verification_repo::{AppUserVerificationRepo, AppUserVerifica
use crate::auth::{
generate_session_token, hash_password, hash_token, verify_password, TIMING_FLAT_DUMMY_HASH,
};
use crate::authz::{self, AuthzDenied, AuthzRepo, Capability};
use crate::authz::{self, AuthzRepo, Capability};
/// Runtime configuration. Defaults match the v1.1.8 brief — overridable
/// via env vars wired in the picloud binary.
@@ -268,17 +268,18 @@ impl UsersServiceImpl {
// Public HTTP execution — script-as-gate; not denied here.
return Ok(());
};
match authz::require(&*self.authz, p, cap).await {
Ok(()) => Ok(()),
Err(AuthzDenied::Denied) => Err(UsersError::Forbidden),
Err(AuthzDenied::Repo(e)) => Err(UsersError::Backend(e.to_string())),
}
authz::require_mapped(
&*self.authz,
p,
cap,
|| UsersError::Forbidden,
UsersError::Backend,
)
.await
}
/// Compose the public `AppUser` shape from a `AppUserRow`. Roles
/// are always empty in commit 4 of v1.1.8; commit 8 (per-app
/// roles) replaces this stub with an actual `app_user_roles`
/// query.
/// Compose the public `AppUser` shape from an `AppUserRow`, with the caller's
/// already-resolved per-app `roles` (see [`Self::fetch_roles`]).
fn to_app_user(row: AppUserRow, roles: Vec<String>) -> AppUser {
AppUser {
id: row.id,
@@ -301,6 +302,19 @@ impl UsersServiceImpl {
Ok(self.roles.list_for_user(app_id, user_id).await?)
}
/// Emit a `users` lifecycle event.
///
/// **Currently inert, and deliberately so.** `triggers.kind` has no `users`
/// value, so `outbox_event_emitter::plan()` has no arm for this source and
/// drops the event. These calls exist so the plumbing is already in place if
/// a users trigger kind lands.
///
/// **If you add one, do NOT wire it up here.** This is the non-transactional
/// write-then-emit shape that audit #6 removed from every other stateful
/// service: the row commits, and if the outbox insert then fails the trigger
/// silently never fires (note the `let _ =` — this one does not even log).
/// Route it through a `crate::atomic_write` writer instead, so the row and
/// its fan-out commit together. See the write-path invariant in CLAUDE.md.
async fn emit(&self, cx: &SdkCallCx, op: &'static str, user_id: AppUserId) {
let _ = self
.events

Some files were not shown because too many files have changed in this diff Show More