Compare commits

..

10 Commits

Author SHA1 Message Date
MechaCat02
aa2831da90 docs+test(hierarchies): end-to-end review follow-ups (M5/M6/M7)
Findings from a 3-lens end-to-end review (correctness, security, tests).
Security review came back clean (cross-repo authz sound, isolation intact,
M5 gate sound modulo the documented manifest-trust boundary). No behavior
changes here — only accuracy + coverage:

- apply_service Phase B2: correct the descendant-expansion comment. The chain
  is COMMITTED ancestry (ancestors() reads the pool), so a reparent in the same
  apply takes effect next apply — same "one more apply" shape the in-tree path
  and group-create reparenting already have. Authz stays sound: the gate and the
  expansion consume the SAME chain (checked == written).
- doc §4.5: document the two known "one more apply / no data risk" limitations
  — reparent-in-same-apply template resolution, and the `{env}` source split
  (declared apps use --env; cross-repo descendants use apps.environment).
- approval journey: give the app real (`[vars]`) content so the editor member's
  AppVarsWrite is actually exercised — the admin-gate test now proves approval
  is ABOVE editor-write, not just "any non-admin is refused". (Vars cascade with
  the app; scripts are ON DELETE RESTRICT and would break AppGuard teardown.)
- templates journey: assert descendant expansions are idempotent (stable row
  ids on re-apply — no churn), matching the in-tree guarantee.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 19:57:21 +02:00
MechaCat02
22d6c33d0b feat(hierarchies): per-env approval-policy gating (M5)
A project's root manifest can mark environments confirm-required; applying to
one needs an explicit `pic apply --dir --env <e> --approve <e>` (a blanket
`--yes` does NOT cover it), the act is admin-gated and audited, and the policy
folds into the bound-plan token. The last unbuilt piece of the project-tool
track (§4.2/§6).

- manifest: `[project]` block → `ManifestProject { environments[{name,confirm}] }`,
  valid only on the tree's ROOT manifest (rejected elsewhere by build_tree).
- discover: build_tree emits `project` into the bundle from the root manifest.
- apply_service: `TreeBundle.project` + `ProjectPolicy`; policy folds into the
  state_token (a policy edit between plan and apply trips StateMoved); plan
  surfaces `approvals_required`; new `ApplyError::ApprovalRequired` → 409.
- apply_api: `enforce_env_approval` (after authz_tree) — refuses a gated env
  not in `approved_envs`, and on approval requires admin (AppAdmin/GroupAdmin)
  on every declared node + audits. The server re-derives the policy from the
  bundle (CLI check is convenience; server is authoritative).
- CLI: `--approve <env>` (repeatable); `resolve_approvals` refuses a gated env
  non-interactively, prompts on a TTY (retype the env name); plan renders gated
  envs. Single-node `apply --file --env <e>` REFUSES a confirm-required env
  (can't carry the admin-gated approval) and directs to `--dir` — closing the
  bypass found in review rather than silently skipping the gate.
- approval journey + manifest/ProjectPolicy unit tests. No migration (policy
  lives in the manifest, like takeover/blast-radius).
- doc §11 Phase 5 + §12: approval gating shipped; gate is a `--dir` feature.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 19:06:27 +02:00
MechaCat02
c04c684a2e feat(hierarchies): cross-repo template expansion (M7)
Route/trigger templates declared at a group now fan out to EVERY descendant
app in the DB subtree, not just the app nodes present in the current
`pic apply --dir`. A template change at group N reaches subtree(N) across
repos, and a removed/disabled template reaps its expansions on those
descendants too.

- group_repo: recursive `descendant_app_ids` (+ `_tx`) — inverse of the
  ancestor chain CTE, app_id-ordered, depth-bounded.
- apply_service: Phase B2 expands templates into descendants of every in-tree
  group not already handled in Phase B. All descendant locks are taken up
  front in the single sorted batch (their union is invariant under reparenting
  in-tree groups), so there is no out-of-order mid-tx locking / deadlock.
  Per-recipient write caps (AppWriteRoute / AppManageTriggers /
  AppSecretsRead-for-email) are required AUTHORITATIVELY IN-TX against the
  post-reparent, already-locked app — checked set == written set, no TOCTOU.
  expand_*_templates_tx are now self-contained (load hand-declared identities
  from the tx), so collisions are caught on descendants too. Blast radius now
  counts the true DB subtree. Descendant expansion errors are tagged with the
  app slug (e.g. an {env} template on a NULL-environment descendant names it).
- apply_api: the pre-tx descendant authz pass is removed in favour of the
  in-tx gate (sound via the hierarchy-aware effective_app_role).
- templates journey: out-of-tree descendant at depth 1 AND 2 receives the
  expansion and is reaped on template removal.
- doc §4.5: strike the in-apply-only scope limitation.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 18:38:33 +02:00
MechaCat02
7c17d6e363 feat(hierarchies): template {var:NAME} resolves in-transaction (M6)
Route/trigger template `{var:NAME}` placeholders now resolve against vars
applied in the SAME `pic apply` transaction, not committed-only state — so a
var and a template referencing it converge in one apply instead of two.

- config_resolver: add tx-scoped `fetch_var_candidates_tx`, sharing the SQL
  tail (`VAR_CANDIDATES_TAIL`) with the pool variant so they can't drift.
- apply_service: `expand_route_templates_tx`/`expand_trigger_templates_tx`
  read vars via the tx (vars are reconciled in Phase A / the app's own
  reconcile, both before Phase B expansion).
- templates journey: `route_template_var_resolves_in_same_apply` — would fail
  against the old committed-only read.
- doc §4.5: strike the `{var:NAME}` committed-only limitation.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 18:11:44 +02:00
MechaCat02
4d2eed4e81 feat(hierarchies): trigger templates — per-app fan-out (M4b)
Completes §4.5 templates (M4a shipped routes). A group declares a trigger once;
the tree apply fans it out into one concrete per-app_id trigger on each
descendant app, reusing the M4a expansion engine over the trigger insert path.

- Migration 0054: `trigger_templates` table (group-owned; the whole
  BundleTrigger wire object stored as `spec` JSONB, placeholders unresolved) +
  `triggers.from_template` provenance column + index.
- `template_repo.rs`: trigger-template CRUD + chain-load.
- Manifest: `[group]` `[[trigger_templates]]` (flat: name, kind, script, + kind
  params); build_bundle emits them; rejected on an app node.
- Apply: group nodes reconcile trigger-template rows; tree Phase B expands each
  chain trigger template into a concrete trigger per descendant app —
  placeholders resolved in every spec string leaf, then the typed trigger is
  rebuilt and inserted through the shared `insert_bundle_trigger_tx` (email
  secrets resolved + re-sealed per recipient app). Idempotent via from_template
  (one trigger per template per app; semantic-identity compare → no-op /
  delete+recreate / reap). Collision with a hand-declared trigger or between
  templates is a hard error. Each resolved trigger is re-validated with the
  per-kind shape check (cron/queue/etc.) so a {var:}-injected bad
  schedule/timeout can't slip in.
- Authz: declaring → GroupScriptsWrite; receiving → AppManageTriggers, plus
  AppSecretsRead for email-trigger recipients (parity with hand-declared email).
- Plan/token/report extended for trigger templates; blast radius covers both.
- Tests: tests/templates.rs trigger fan-out (kv + {app_slug}, stable-ids,
  prune-reap); resolve_placeholders_in_json unit test; schema golden reblessed.

Reviewed (no CRITICAL; isolation + per-app email-secret sealing verified
sound). Closed a HIGH validation-parity gap (re-validate resolved triggers) and
a MEDIUM authz gap (AppSecretsRead for email expansions). v1.2 Hierarchies
template work (M4) complete.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 17:38:04 +02:00
MechaCat02
0396866698 feat(hierarchies): route templates — group-declared, per-app fan-out (M4a)
§4.5 of the groups/project-tool design. A group declares a route ONCE; the
tree apply fans it out into one concrete per-app_id route on every descendant
app — instantiation, not inheritance (the cross-app isolation boundary is
unchanged; expanded rows stay app-owned).

- Migration 0053: `route_templates` table (group-owned) + `routes.from_template`
  provenance column + index.
- `template_repo.rs`: tx-aware CRUD + chain-load (mirrors group_repo/route_repo).
- Manifest: `[group]` `[[route_templates]]`; build_bundle emits them; rejected
  on an app node (422).
- Apply: group nodes reconcile template rows (create/update/delete via the
  plan); tree Phase B expands each chain template into a concrete route per
  descendant app node — placeholders {app_slug}/{env}/{var:NAME} resolved per
  app (unknown var/placeholder = hard error), script bound nearest-owner-wins.
  Idempotent via from_template (diffed create/update-as-replace/delete; re-apply
  is no-churn, --prune reaps expansions). Collision with a hand-declared route,
  or between two templates, is a hard error. Each RESOLVED expansion is
  validated like a hand-declared route (reserved-path, path/host parse,
  host-claim) so a {var:}-injected path or unclaimed strict host can't slip in.
- Plan: route-template diff at the group node + blast-radius (descendant app
  count in this apply); state_token folds each template (edit trips StateMoved).
- Authz: declaring → GroupScriptsWrite; an app receiving expansions →
  AppWriteRoute (gated even with no hand-declared routes).
- Tests: tests/templates.rs (fan-out + per-slug + idempotent-stable-ids +
  prune-reap + collision-rejected); placeholder/validation unit tests; schema
  golden reblessed.

Reviewed (no CRITICAL/HIGH; isolation core verified sound). Closed the two
validation-parity MEDIUMs (host-claim + reserved-path/pattern on expansions)
and added an out-of-apply-descendant prune warning. Scope: expansion targets
app nodes in the tree apply (not yet cross-repo descendants); {var:NAME} uses
committed vars. M4b (trigger templates) reuses this engine next.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:58:41 +02:00
MechaCat02
193336a8a6 feat(hierarchies): multi-repo single-owner ownership + structural prune (M3)
§7 of the groups/project-tool design. A group node is now authoritatively
managed by exactly one project-root; `pic apply --dir` claims, conflicts,
takes over, and (with --prune) structurally reaps owned nodes.

- Migration 0052: `projects(id, key UNIQUE)` + promote the inert
  `groups.owner_project` (0047) to a real FK (ON DELETE SET NULL) + index.
- CLI mints a stable, gitignored project key in `.picloud/project.json`
  (`pic init`, or lazily on first tree plan/apply) and presents it on every
  tree request; `pic apply --dir --takeover` flag.
- Server: prepare_tree resolves ownership read-only (plan surfaces conflicts
  + prune candidates; token folds each group's owner key). apply_tree upserts
  the project in-tx, claims created groups on insert, reconciles existing-group
  ownership under the per-node advisory lock (first-commit-wins), and prunes
  owned-but-undeclared groups leaf-first (delete=RESTRICT, never another repo's
  or a UI-owned node).
- Authz (§7.4, ownership ⟂ RBAC): takeover requires GroupAdmin per contested
  node — enforced in authz_tree (pre-tx) AND re-verified in-tx at the ownership
  decision, so --force (which waives the staleness token) can't open a
  takeover-without-admin window. The attacker-supplied project key is
  length/charset-validated server-side.
- Tests: tests/ownership.rs (claim → conflict → takeover → flip; non-admin
  takeover → 403; prune-owned-only); format_conflicts unit test; schema golden
  reblessed. tree_shape M2 journey updated to reparent in-place (a fresh dir is
  now a distinct project and would correctly conflict).

Closes the M3 milestone; reviewed (4 findings: 1 authz-bypass via --force +
key validation + 2 LOW, all fixed). M4 (trigger/route templates) remains.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 23:04:53 +02:00
MechaCat02
c18ce7c2c4 feat(hierarchies): declarative group create/reparent — tree shape (M2)
M2 of the remaining-hierarchies work. `pic apply --dir` now owns the org-tree
SHAPE, not just node content: a `[group]` manifest for a group that doesn't
exist is CREATED under its declared parent, and an existing group whose declared
parent changed is REPARENTED — all inside the single tree-apply transaction
(create + reparent; structural prune is deferred to M3 with the ownership layer).

group_repo: extract transaction-aware structural mutations — `create_group_tx`,
`reparent_group_tx` (the ancestor-walk cycle guard, now reading through the tx so
it sees in-tx writes), `delete_group_tx`, and `acquire_structural_lock_tx`. The
trait `create`/`reparent`/`delete` delegate to them (one SQL definition,
behavior-preserving: the coarse structural advisory lock + structure_version
bump + delete=RESTRICT all preserved).

apply_service: `TreeNode` gains `parent_slug` + `name`. `prepare_tree` classifies
group nodes into existing (resolved id, maybe reparent) vs to-create (absent →
deferred), returning a `PreparedTree { prepared, creates, reparents, token }`.
`apply_tree` adds Phase 0 (`reconcile_tree_structure_tx`): create absent groups
parent-first (topo loop with cycle/unresolved-parent detection) and reparent
existing ones, then Phase A/B reconcile content as before. A to-create group
reconciles against an empty CurrentState (all-Create) — so it needs no DB read
and sidesteps the pool-vs-tx coupling. The bound-plan token folds a
declared-absent marker, so a group created out-of-band between plan and apply
trips StateMoved.

authz (apply_api `authz_tree`): a to-create group mirrors the interactive create
gate — root-level needs `InstanceCreateGroup`, a subgroup under an existing
parent needs only `GroupAdmin(parent)`; a reparent needs `GroupAdmin` at the
group, the SOURCE parent, and the DESTINATION parent (§5.6, parity with
`reparent_group`). No 404 on a to-create node.

CLI: `[group] parent` manifest key (else the parent is inferred from the nearest
ancestor directory's group); `build_tree` emits `parent_slug` + `name` per group
node; the apply report shows a `groups +N reparented M` line.

Reviewed by subagent (core mechanism verified sound: topo termination, empty-
CurrentState reconcile, in-tx cycle guard, atomicity, StateMoved, idempotency);
fixed the two authz-parity gaps it found (missing source-parent check on
reparent; over-gated subgroup create). Tests: a new `tree_shape` journey
(create + reparent + no-op re-plan); 393 manager-core lib + the Phase-5 tree/
group/ext journeys all green; clippy -D warnings clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 22:18:27 +02:00
MechaCat02
dc8c69ac6f docs(design): mark extension points (§5.5) shipped
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 21:36:11 +02:00
MechaCat02
4dcb8d1136 feat(hierarchies): extension points — opt-in dynamic module resolution (§5.5)
M1 of the remaining-hierarchies work. An extension point lets a PARENT/group
node opt a module name out of sealed lexical resolution: a parent script's
`import "x"` then resolves against the INHERITING app's module (with a declared
default body as fallback) instead of the parent's sealed chain — so each
descendant app supplies its own `x`. Only the declaring parent can open the
inversion; a descendant can never make a parent's sealed import dynamic or
hijack one (the "is this an extension point" decision keys on the importer's
trusted defining node, never a script-passed value).

Resolver (executor-core):
- `ModuleSource` gains `resolve_extension_point(origin, name)` and a
  chain-constrained `resolve_by_id(origin, id)`. The resolve seam checks for an
  ext point on the importer's chain first; on a hit it resolves against
  `App(cx.app_id)`, else the declared default, else ModuleNotFound. The Postgres
  query returns Some only when the NEAREST declaration of the name is an ext
  point (a peer/nearer concrete module shadows it). A group origin can't reach
  app-owned rows — the trust boundary holds. Cache stays id-keyed (no bleed).

Apply (manager-core, mirrors the `vars` pattern):
- migration 0051: owner-polymorphic `extension_points` table (group XOR app),
  optional `default_script_id` FK ON DELETE SET NULL, per-owner LOWER(name)
  unique indexes.
- Bundle/Plan/CurrentState gain extension_points; `diff_extension_points`
  (name-based, default change = Update), reconcile (upsert after scripts so the
  default resolves) + prune, `validate_bundle` (module-name shape; default must
  be a local module), `check_imports_resolve` (a declared/on-chain ext point
  satisfies an import), and the state_token fold. Writes gate on the
  script-write capability (AppWriteScript / GroupScriptsWrite).
- GET /apps|groups/{id}/extension-points so `pic pull` round-trips them — a
  re-applied pulled manifest is all-NoOp, so `--prune` can't silently drop one.

CLI: `[[extension_points]]` manifest table; plan/apply build + render; pull
exports them.

Tests: 5 resolver units (inversion, default fallback, no-provider error,
no-hijack of a sealed import, no cross-tenant cache bleed), 2 diff/state-token
units, 2 journeys (per-app resolution + default fallback via invoke; pull
round-trip). Schema golden re-blessed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 21:35:29 +02:00
343 changed files with 7858 additions and 50437 deletions

View File

@@ -37,25 +37,6 @@
"Bash(dig *)",
"Bash(host *)",
"Bash(./target/debug/pic plan *)",
"Bash(./target/debug/pic config *)",
"Bash(./target/debug/pic whoami)",
"Bash(./target/debug/pic pull *)",
"Bash(./target/debug/pic apps domains ls *)",
"Bash(./target/debug/pic routes ls *)",
"Bash(./target/debug/pic kv get *)",
"Bash(./target/debug/pic kv ls *)",
"Bash(./target/debug/pic dead-letters ls *)",
"Bash(/home/fabi/PiCloud/target/debug/pic plan *)",
"Bash(/home/fabi/PiCloud/target/debug/pic config *)",
"Bash(/home/fabi/PiCloud/target/debug/pic whoami)",
"Bash(/home/fabi/PiCloud/target/debug/pic pull *)",
"Bash(/home/fabi/PiCloud/target/debug/pic apps domains ls *)",
"Bash(/home/fabi/PiCloud/target/debug/pic routes ls *)",
"Bash(/home/fabi/PiCloud/target/debug/pic kv get *)",
"Bash(/home/fabi/PiCloud/target/debug/pic kv ls *)",
"Bash(/home/fabi/PiCloud/target/debug/pic dead-letters ls *)",
"Bash(ls)",
"Bash(ls *)",
"Bash(pwd)",

View File

@@ -10,11 +10,6 @@ env:
# Matches what docker-compose produces locally; the schema-snapshot
# guardrail and any other DB-backed tests run against this service.
DATABASE_URL: postgres://picloud:picloud@localhost:5432/picloud
# The DB-backed suites skip themselves when DATABASE_URL is unset. In CI it is
# always set, so this makes that skip a hard error: if the database ever goes
# missing (a broken service container, a lost env), the build goes RED instead
# of green-but-empty. Honored by picloud_test_support::abort_if_db_required.
PICLOUD_REQUIRE_DB: "1"
jobs:
rust:
@@ -52,49 +47,11 @@ jobs:
- name: Clippy
run: cargo clippy --all-targets --all-features -- -D warnings
# `--include-ignored` is load-bearing. Every DB-backed integration test is
# `#[ignore = "needs DATABASE_URL..."]`, and CI omitted the flag — so CI ran
# 927 tests and silently skipped 237, among them ALL of authz.rs and api.rs
# and the entire CLI journey suite. The isolation and RBAC tests existed but
# never executed (AUDIT.md F-Q-014). CI does provide Postgres, so run them.
#
# `--all-targets` (not a bare `cargo test`) is deliberate: it runs the lib,
# bins, and integration tests but NOT doctests. `-- --include-ignored`
# un-ignores not just `#[ignore]` tests but also ` ```ignore ` DOCTESTS,
# which are illustrative pseudocode that does not compile — so a bare
# `cargo test ... -- --include-ignored` fails on them. Doctests run in their
# own step below, without the flag. (Clippy already uses `--all-targets` for
# the same doctest-excluding reason.)
#
# The CLI journeys are a SEPARATE step, and deliberately not part of the
# workspace run: they spawn a real picloud whose dispatcher/orchestrator
# claim loops are global by design (one instance owns one database). Run
# concurrently with the manager-core suites — which share this database —
# it would claim their outbox and workflow rows out from under them. Keeping
# the steps sequential keeps that live server off the shared DB while the
# other suites are using it.
- name: Test (workspace, including DB-backed tests)
run: cargo test --workspace --exclude picloud-cli --all-targets -- --include-ignored
# Doctests, run WITHOUT --include-ignored so ` ```ignore ` snippets stay
# ignored. `--all-targets` above skips these, so nothing else covers them.
- name: Doctests
run: cargo test --workspace --doc
# The journey harness execs the prebuilt target/debug/picloud and does NOT
# rebuild it, so a stale binary would silently test old server code.
- name: Build picloud (the journey harness execs this binary)
run: cargo build -p picloud
# The spawned server inherits this env; without a secret key it aborts at
# startup and every journey fails as "/healthz never returned 200".
# `--all-targets` for the same reason as above (the journeys are `#[ignore]`
# integration tests; picloud-cli's doctests, if any, run in the Doctests step).
- name: Test (CLI journeys)
env:
PICLOUD_DEV_MODE: "true"
PICLOUD_DEV_INSECURE_KEY: i-understand-this-is-insecure
run: cargo test -p picloud-cli --all-targets -- --include-ignored
# Runs the whole workspace, including the schema-snapshot guardrail
# (it picks up DATABASE_URL from the env above and the postgres
# service; without a DB it would skip cleanly).
- name: Test
run: cargo test --workspace
dashboard:
name: Dashboard — check

5
.gitignore vendored
View File

@@ -26,11 +26,8 @@ docker-compose.override.yml
config.local.toml
/data
# Files-root blob storage created when integration tests run build_app
# from a crate dir (PICLOUD_FILES_ROOT default ./data). The CLI journey
# harness spawns the server from the picloud-cli dir, so it lands there too
# (the §11.6 group-files journey is the first CLI test to write blobs).
# from the picloud crate dir (PICLOUD_FILES_ROOT default ./data).
/crates/picloud/data
/crates/picloud-cli/data
/postgres-data
# Dashboard

View File

@@ -1,62 +1,5 @@
# PiCloud Changelog
## Unreleased — CMS proof-of-concept remediation
Fixes surfaced by building a full CMS on PiCloud (`examples/cms-poc/`). Pure
additive surface + one migration (`0077_apps_cors.sql`), no destructive changes.
### Security
- **502 info-leak closed on user routes.** An uncaught script error (incl. a
`throw`) on a user route returned HTTP 502 whose body embedded the app UUID,
script function names, and source line/col. The 2026-06-11 audit scrub had
only covered the execute-by-id path; the user-route (inbox) path still leaked.
Both paths now share `scrub_runtime_detail` — the raw detail is logged
server-side under a correlation id and the client sees only
`script execution error (ref: <uuid>)`.
### Added
- **`docs::find` `$contains` operator** — array-membership via JSONB `@>`, e.g.
`docs::find("posts", #{ tags: #{ "$contains": "rust" } })` finds every post
tagged `rust`. The inverse of `$in`. Covers per-app and group-shared docs.
- **App-user role management from the API/CLI** — `pic users add-role --app <app>
<user_id> <role>` / `rm-role`, backed by `POST`/`DELETE
/api/v1/admin/apps/{id}/users/{user_id}/roles[/{role}]` (`AppUsersAdmin`).
Role assignment was previously reachable only from a Rhai script.
- **Per-app CORS** — `pic apps cors set <app> <origins…>` (or `*`), stored on
`apps.cors_allowed_origins`. The orchestrator echoes an allowed `Origin` on
every response and answers the `OPTIONS` preflight. Empty = off (unchanged).
- **Non-JSON request bodies** — a user route no longer 400s on a non-JSON body.
`application/x-www-form-urlencoded` → `ctx.request.body` object, `text/*` →
string, other types → base64 string; `ctx.request.content_type` added as a
convenience. JSON (or unspecified) is unchanged; malformed JSON still 400s.
- **Binary responses** — a script may return `#{ statusCode, headers,
body_base64 }`; the platform decodes the base64 and returns raw bytes with the
script-set `Content-Type` (default `application/octet-stream`).
- **`workflow::run_status(run_id)` SDK** (F-038) — the script that started a run
can now poll it: returns `#{ status, output, error, steps: #{ name: status } }`
or `()` when no such run belongs to the app. Read-only, same-app scoped (the
run's `app_id` is the isolation boundary), same `AppInvoke` gate as
`workflow::start`. Closes the gap where run status was platform-admin-API only.
### Fixed
- **`pic apply` "app not found" is now actionable** — points at
`pic apps create <slug>` (apply reconciles groups but never creates an app).
- **Interceptor- and workflow-step-bound scripts no longer warned "no route or
trigger"** at plan time — an `[[interceptors]]` binding (F-021) *and* a
`[[workflows]]` step `function` (F-037) now count as reachability.
### Notes
- **Workflow `skipped` ≠ `failed` (F-039, by design).** A step whose `when`
evaluates false is `skipped` and counts as *satisfied-but-empty* for its
dependents — so a step downstream of a skipped one still runs. To gate a
dependent on an upstream actually having run, give it its own `when`
(e.g. `when = "steps.publish.output.published == true"`), or have the upstream
step return an explicit outcome the dependent checks.
## v1.1.9 — Durable Queues & Function Composition (unreleased)
Per-app durable queues + same-app function composition + caller-

File diff suppressed because one or more lines are too long

12
Cargo.lock generated
View File

@@ -1939,7 +1939,6 @@ dependencies = [
"picloud-manager-core",
"picloud-orchestrator-core",
"picloud-shared",
"picloud-test-support",
"serde",
"serde_json",
"sha2 0.10.9",
@@ -1965,7 +1964,6 @@ dependencies = [
"libc",
"percent-encoding",
"picloud-shared",
"picloud-test-support",
"postgres",
"predicates",
"reqwest",
@@ -2045,7 +2043,6 @@ dependencies = [
"picloud-executor-core",
"picloud-orchestrator-core",
"picloud-shared",
"picloud-test-support",
"rand 0.8.6",
"reqwest",
"serde",
@@ -2077,7 +2074,6 @@ version = "1.1.9"
dependencies = [
"async-trait",
"axum",
"base64",
"chrono",
"lru",
"picloud-executor-core",
@@ -2114,14 +2110,6 @@ dependencies = [
"uuid",
]
[[package]]
name = "picloud-test-support"
version = "1.1.9"
dependencies = [
"sqlx",
"tokio",
]
[[package]]
name = "pin-project-lite"
version = "0.2.17"

View File

@@ -10,7 +10,6 @@ members = [
"crates/picloud-orchestrator",
"crates/picloud-executor",
"crates/picloud-cli",
"crates/test-support",
]
[workspace.package]
@@ -64,7 +63,6 @@ futures = "0.3"
rhai = { version = "=1.24", features = ["sync", "serde"] }
# Postgres (manager-core only — others stay DB-free)
picloud-test-support = { path = "crates/test-support" }
sqlx = { version = "0.8", features = ["runtime-tokio-rustls", "postgres", "uuid", "chrono", "json", "macros", "migrate"] }
# Config

View File

@@ -44,14 +44,6 @@ fn current_deadline() -> Option<Instant> {
CURRENT_DEADLINE.with(Cell::get)
}
/// The current-thread execution deadline, if one is installed. Public so the
/// interceptor hook can compute a per-hook timeout as `min(ambient, now + t)` —
/// a hook must never be able to EXTEND its caller's deadline (§9.4 M5).
#[must_use]
pub fn ambient_deadline() -> Option<Instant> {
current_deadline()
}
use chrono::{DateTime, Utc};
use picloud_shared::{
ScriptId, ScriptValidator, SdkCallCx, Services, TriggerEvent, ValidatedScript, ValidationError,
@@ -65,9 +57,7 @@ use crate::module_resolver::{
};
use crate::sandbox::Limits;
use crate::sdk;
use crate::sdk::bridge::{
dynamic_to_json, dynamic_to_json_capped, json_to_dynamic, MAX_JSON_MATERIALIZE_BYTES,
};
use crate::sdk::bridge::{dynamic_to_json, json_to_dynamic};
use crate::types::{
ExecError, ExecRequest, ExecResponse, ExecStats, InvocationType, LogEntry, LogLevel,
};
@@ -288,17 +278,6 @@ impl Engine {
/// cache hands compiled ASTs in directly; this path skips the
/// per-call compile.
pub fn execute_ast(&self, ast: &Arc<AST>, req: ExecRequest) -> Result<ExecResponse, ExecError> {
// Audit #2: bound per-execution durable fan-out. The scope is
// re-entrancy aware — a fresh dispatched execution (a new task on a
// pooled thread) resets the budget, while a synchronous invoke() /
// interceptor re-entry nested in the SAME call stack shares it, so
// fan-out is counted across the whole synchronous chain. Held to the end
// of the call; its Drop re-zeroes the counter on the outermost exit.
let _emit_scope = crate::sdk::emit_budget::EmissionBudgetScope::enter();
// §9.4 M6: memoize interceptor chain resolution for this execution tree
// (same re-entrancy model as the emission budget); cleared at the
// outermost boundary so a pooled thread never reuses a foreign app's cache.
let _interceptor_cache = crate::sdk::interceptor::InterceptorCacheScope::enter();
let effective_limits = self.limits.with_overrides(&req.sandbox_overrides);
let logs: Arc<Mutex<Vec<LogEntry>>> = Arc::new(Mutex::new(Vec::new()));
let mut engine = build_engine(effective_limits, Some(logs.clone()));
@@ -358,13 +337,12 @@ impl Engine {
|m| m.into_inner().unwrap_or_default(),
);
let (status_code, headers, body, body_base64) = parse_response(value)?;
let (status_code, headers, body) = parse_response(value)?;
Ok(ExecResponse {
status_code,
headers,
body,
body_base64,
logs,
stats: ExecStats {
duration_ms: u64::try_from(duration.as_millis()).unwrap_or(u64::MAX),
@@ -511,14 +489,6 @@ fn build_ctx_map(req: &ExecRequest) -> Map {
"invocation_type".into(),
invocation_type_str(req.invocation_type).into(),
);
// Read-only depth of this call in a trigger/invoke chain: 0 for direct
// ingress, +1 per synchronous re-entry (`invoke`) or dispatched handler.
// Exposed so a script can observe where it sits in the chain (the same value
// the platform bounds via `trigger_depth_max`).
ctx.insert(
"trigger_depth".into(),
(i64::from(req.trigger_depth)).into(),
);
let mut request = Map::new();
request.insert("path".into(), req.path.clone().into());
@@ -532,20 +502,6 @@ fn build_ctx_map(req: &ExecRequest) -> Map {
request.insert("body".into(), json_to_dynamic(req.body.clone()));
// F-026 convenience: the request Content-Type (also present in `headers`).
// Lets a script branch on how to read `body` — parsed JSON for a JSON
// request, a raw string for `text/*`, or a base64 string for other binary
// types — without a case-insensitive header lookup. `()` when absent.
let content_type = req
.headers
.iter()
.find(|(k, _)| k.eq_ignore_ascii_case("content-type"))
.map(|(_, v)| v.clone());
request.insert(
"content_type".into(),
content_type.map_or(Dynamic::UNIT, Into::into),
);
// SDK 1.1 additions — route-captured params, query string, prefix
// tail. Empty when not applicable so scripts can always read them.
let mut params = Map::new();
@@ -783,9 +739,7 @@ fn invocation_type_str(it: InvocationType) -> &'static str {
// Response parsing
// ----------------------------------------------------------------------------
type ParsedResponse = (u16, BTreeMap<String, String>, Json, Option<String>);
fn parse_response(value: Dynamic) -> Result<ParsedResponse, ExecError> {
fn parse_response(value: Dynamic) -> Result<(u16, BTreeMap<String, String>, Json), ExecError> {
// Convention: a Map with a `statusCode` field is the structured shape.
// Anything else is treated as a 200 response with the value as body.
if value.is_map() {
@@ -795,14 +749,10 @@ fn parse_response(value: Dynamic) -> Result<ParsedResponse, ExecError> {
}
}
}
// The response body is otherwise an uncapped Rhai→JSON exit — bound it so a
// cheaply-aliased huge value can't OOM the node on materialization.
let body = dynamic_to_json_capped(&value, MAX_JSON_MATERIALIZE_BYTES)
.map_err(|e| ExecError::InvalidResponse(e.to_string()))?;
Ok((200, BTreeMap::new(), body, None))
Ok((200, BTreeMap::new(), dynamic_to_json(&value)))
}
fn parse_structured_response(map: Map) -> Result<ParsedResponse, ExecError> {
fn parse_structured_response(map: Map) -> Result<(u16, BTreeMap<String, String>, Json), ExecError> {
let status_dyn = map
.get("statusCode")
.ok_or_else(|| ExecError::InvalidResponse("missing statusCode".into()))?;
@@ -821,29 +771,9 @@ fn parse_structured_response(map: Map) -> Result<ParsedResponse, ExecError> {
}
}
// F-026: a `body_base64` string field returns raw bytes with the script-set
// `Content-Type`, instead of JSON-encoding `body`. When set, `body` is
// ignored (the bytes win) so an author can't accidentally send both.
let body_base64 = match map.get("body_base64") {
Some(b) => Some(
b.clone()
.into_string()
.map_err(|_| ExecError::InvalidResponse("body_base64 must be a string".into()))?,
),
None => None,
};
let body = map.get("body").map_or(Json::Null, dynamic_to_json);
let body = if body_base64.is_some() {
Json::Null
} else {
match map.get("body") {
Some(b) => dynamic_to_json_capped(b, MAX_JSON_MATERIALIZE_BYTES)
.map_err(|e| ExecError::InvalidResponse(e.to_string()))?,
None => Json::Null,
}
};
Ok((status_code, headers, body, body_base64))
Ok((status_code, headers, body))
}
// ----------------------------------------------------------------------------

View File

@@ -14,6 +14,14 @@
//! (a leaf can't shadow them — the trust boundary), while every SDK call
//! *inside* a module still scopes to the inheriting app via `cx.app_id`.
//!
//! **Extension points (§5.5).** A node may opt a module name OUT of sealing
//! by declaring it an extension point: an importer on that node's chain then
//! resolves the name against the *inheriting app's* effective view
//! (`App(cx.app_id)`), with the declared default body as fallback — so each
//! descendant app supplies its own impl. The "is this an extension point"
//! check keys on the importer's `origin` (trusted), so only the declaring
//! parent can open the inversion; a descendant can't hijack a sealed import.
//!
//! Three runtime invariants are enforced:
//!
//! 1. **Lexical isolation** — `ModuleSource::resolve` is keyed by the
@@ -377,40 +385,53 @@ impl ModuleResolver for PicloudModuleResolver {
let origin = importer_source
.and_then(decode_origin)
.unwrap_or(self.default_origin);
// §5.5 policy resolution: the source decides lexical (seal to `origin`)
// vs dynamic (an extension point resolves against the inheriting app,
// `cx.app_id`). The result is already the concrete module to bind, or a
// terminal NoProvider/NotFound.
let lookup_result: Result<picloud_shared::ModuleResolution, ModuleSourceError> =
tokio::task::block_in_place(|| {
handle.block_on(self.source.resolve_policy(origin, self.cx.app_id, path))
});
// Extension-point inversion (§5.5): if `path`'s nearest declaration on
// the importer's chain is an extension point (not a concrete module),
// resolve it against the INHERITING app's effective view instead of the
// sealed lexical chain — each descendant app supplies its own impl,
// falling back to the declared default body. The decision keys on
// `origin` (the trusted importer node), so only a declaration on the
// importer's own ancestry can open this; a descendant can never make a
// parent's sealed `import` dynamic, and a non-ext-point import never
// consults `App(cx.app_id)`. All branches funnel any backend error into
// the single redaction arm below.
let lookup_result: Result<Option<picloud_shared::ModuleScript>, ModuleSourceError> =
(|| {
let ep = tokio::task::block_in_place(|| {
handle.block_on(self.source.resolve_extension_point(origin, path))
})?;
let Some(ep) = ep else {
// Sealed lexical resolution (the default, unchanged).
return tokio::task::block_in_place(|| {
handle.block_on(self.source.resolve(origin, path))
});
};
let app_origin = ScriptOwner::App(self.cx.app_id);
if let Some(m) = tokio::task::block_in_place(|| {
handle.block_on(self.source.resolve(app_origin, path))
})? {
return Ok(Some(m));
}
match ep.default_script_id {
// The default body is a module declared at the node that
// opened the ext point, i.e. on the importer's (`origin`)
// chain — resolve it there, not against the app.
Some(id) => tokio::task::block_in_place(|| {
handle.block_on(self.source.resolve_by_id(origin, id))
}),
None => Ok(None),
}
})();
let module_row = match lookup_result {
Ok(picloud_shared::ModuleResolution::Module(m)) => m,
Ok(picloud_shared::ModuleResolution::NotFound) => {
Ok(Some(m)) => m,
Ok(None) => {
return Err(Box::new(EvalAltResult::ErrorModuleNotFound(
path.to_string(),
pos,
)));
}
Ok(picloud_shared::ModuleResolution::NoProvider) => {
// §5.5: an extension point with no provider for this app is a
// hard failure (the app must supply the module or a default
// body must exist up-chain). Distinct, actionable message.
return Err(Box::new(EvalAltResult::ErrorInModule(
path.to_string(),
Box::new(EvalAltResult::ErrorRuntime(
format!(
"extension point {path:?} has no provider for this app — \
define a module named {path:?} or a default body up-chain"
)
.into(),
pos,
)),
pos,
)));
}
Err(e) => {
// v1.1.4 §10a: redact the backend error before it
// reaches a script. In public-HTTP context (principal:

View File

@@ -43,15 +43,6 @@ where
})
}
/// Build a Rhai runtime error from a message. Returns the boxed error
/// directly because every caller needs a `Box<EvalAltResult>` (Rhai's error
/// type) — the shared home for the pattern the SDK bridges (secrets, email,
/// users, …) previously each copied.
#[allow(clippy::unnecessary_box_returns)]
pub(crate) fn runtime_err(msg: &str) -> Box<EvalAltResult> {
EvalAltResult::ErrorRuntime(msg.into(), rhai::Position::NONE).into()
}
/// Convert a `serde_json::Value` into a Rhai `Dynamic` suitable for
/// pushing into a script's scope. Numbers prefer the narrowest type
/// (`i64` over `f64`); anything that can't round-trip falls back to a
@@ -85,173 +76,35 @@ pub fn json_to_dynamic(value: Json) -> Dynamic {
}
}
/// Hard ceiling on the byte size of a single Rhai→JSON materialization.
///
/// The per-element Rhai sandbox caps (`max_string_size` 64 KiB,
/// `max_array_size`/`max_map_size` 10 000) do NOT bound the *materialized*
/// total: Rhai shares strings/arrays by `Rc`, so a 10 000-element array of one
/// aliased 64 KiB string is cheap to build (~10 000 ops) yet dealiases to
/// ~640 MiB of distinct JSON. Without a ceiling a handful of anonymous requests
/// could OOM the node. `dynamic_to_json_capped` charges a running byte budget
/// and bails as soon as it is exceeded, so the transient allocation is bounded
/// to roughly this ceiling regardless of aliasing. This is far above any
/// legitimate single value (the KV/docs value caps are 256 KiB); it is a
/// last-resort anti-OOM rail, not a business limit.
pub const MAX_JSON_MATERIALIZE_BYTES: usize = 16 * 1024 * 1024;
/// A Rhai value was too large to materialize into JSON within its byte budget.
#[derive(Debug, Clone, Copy)]
pub struct JsonSizeError {
pub limit: usize,
}
impl std::fmt::Display for JsonSizeError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(
f,
"value too large to serialize to JSON (exceeds the {}-byte limit)",
self.limit
)
}
}
impl std::error::Error for JsonSizeError {}
impl From<JsonSizeError> for Box<EvalAltResult> {
fn from(e: JsonSizeError) -> Self {
runtime_err(&e.to_string())
}
}
/// Convert a Rhai `Dynamic` back to a `serde_json::Value`, **infallibly** and
/// **bounded** by [`MAX_JSON_MATERIALIZE_BYTES`]. A value that would exceed the
/// ceiling collapses to a short marker string rather than OOMing — safe for
/// best-effort paths (logging) where a hard error is undesirable. Paths where
/// silently substituting a marker would be wrong (writes, the HTTP response
/// body, `invoke` args) must use [`dynamic_to_json_capped`] and surface the
/// error instead.
///
/// Custom Rhai types (timestamps, user-registered modules) fall back to their
/// `Display` form so they appear as strings rather than failing the build.
#[must_use]
/// Convert a Rhai `Dynamic` back to a `serde_json::Value`. Custom Rhai
/// types (timestamps, user-registered modules) fall back to their
/// `Display` form so they appear as strings in JSON output rather than
/// failing the response build.
pub fn dynamic_to_json(value: &Dynamic) -> Json {
dynamic_to_json_capped(value, MAX_JSON_MATERIALIZE_BYTES)
.unwrap_or_else(|_| Json::String("<value too large>".to_string()))
}
/// Bounded `Dynamic`→JSON conversion: materializes `value`, charging a running
/// byte budget starting at `max_bytes`, and returns [`JsonSizeError`] the moment
/// the budget is exceeded — so it never builds the full oversized tree.
///
/// # Errors
/// Returns [`JsonSizeError`] if the materialized JSON would exceed `max_bytes`.
pub fn dynamic_to_json_capped(value: &Dynamic, max_bytes: usize) -> Result<Json, JsonSizeError> {
let mut remaining = max_bytes;
to_json_bounded(value, &mut remaining, max_bytes)
}
/// Deduct `n` bytes from `remaining`, or fail if the budget is exhausted.
fn charge(remaining: &mut usize, n: usize, limit: usize) -> Result<(), JsonSizeError> {
match remaining.checked_sub(n) {
Some(left) => {
*remaining = left;
Ok(())
}
None => Err(JsonSizeError { limit }),
}
}
fn to_json_bounded(
value: &Dynamic,
remaining: &mut usize,
limit: usize,
) -> Result<Json, JsonSizeError> {
if value.is_unit() {
charge(remaining, 4, limit)?;
return Ok(Json::Null);
return Json::Null;
}
if let Ok(b) = value.as_bool() {
charge(remaining, 5, limit)?;
return Ok(Json::Bool(b));
return Json::Bool(b);
}
if let Ok(i) = value.as_int() {
charge(remaining, 8, limit)?;
return Ok(Json::Number(i.into()));
return Json::Number(i.into());
}
if let Ok(f) = value.as_float() {
charge(remaining, 8, limit)?;
return Ok(serde_json::Number::from_f64(f).map_or(Json::Null, Json::Number));
return serde_json::Number::from_f64(f).map_or(Json::Null, Json::Number);
}
if value.is_string() {
let s = value.clone().into_string().unwrap_or_default();
charge(remaining, s.len() + 2, limit)?;
return Ok(Json::String(s));
return Json::String(value.clone().into_string().unwrap_or_default());
}
if let Some(arr) = value.clone().try_cast::<rhai::Array>() {
charge(remaining, 2, limit)?; // "[]"
let mut out = Vec::with_capacity(arr.len().min(1024));
for v in &arr {
charge(remaining, 1, limit)?; // ","
out.push(to_json_bounded(v, remaining, limit)?);
}
return Ok(Json::Array(out));
return Json::Array(arr.iter().map(dynamic_to_json).collect());
}
if let Some(map) = value.clone().try_cast::<Map>() {
charge(remaining, 2, limit)?; // "{}"
let mut out = serde_json::Map::new();
for (k, v) in map {
charge(remaining, k.len() + 4, limit)?; // "key":,
out.insert(k.to_string(), to_json_bounded(&v, remaining, limit)?);
out.insert(k.to_string(), dynamic_to_json(&v));
}
return Ok(Json::Object(out));
}
let s = value.to_string();
charge(remaining, s.len() + 2, limit)?;
Ok(Json::String(s))
}
#[cfg(test)]
mod tests {
use super::*;
use rhai::{Array, Dynamic};
#[test]
fn small_value_round_trips_under_the_cap() {
let d = Dynamic::from("hello");
assert_eq!(
dynamic_to_json_capped(&d, MAX_JSON_MATERIALIZE_BYTES).unwrap(),
Json::String("hello".into())
);
}
#[test]
fn aliased_large_array_is_rejected_not_materialized() {
// The OOM regression: one 64 KiB string aliased 10 000×. Cheap to build
// (Rc-shared), ~640 MiB if dealiased. The bounded converter must bail on
// the byte budget instead of allocating the full tree.
let big = "x".repeat(64 * 1024);
let mut arr = Array::new();
for _ in 0..10_000 {
arr.push(Dynamic::from(big.clone())); // ImmutableString: shared, cheap
}
let d = Dynamic::from(arr);
// A 1 MiB budget is far below the ~640 MiB dealiased size → must error.
let err = dynamic_to_json_capped(&d, 1024 * 1024).unwrap_err();
assert_eq!(err.limit, 1024 * 1024);
}
#[test]
fn infallible_wrapper_yields_marker_instead_of_oom() {
// ~300 × 64 KiB aliased ≈ 19 MiB dealiased, just over the 16 MiB default
// ceiling — enough to trip the infallible wrapper's marker fallback
// without paying to build a pathologically huge test array.
let big = "x".repeat(64 * 1024);
let mut arr = Array::new();
for _ in 0..300 {
arr.push(Dynamic::from(big.clone()));
}
// The infallible path must NOT OOM — it collapses to a marker string.
let out = dynamic_to_json(&Dynamic::from(arr));
assert_eq!(out, Json::String("<value too large>".into()));
return Json::Object(out);
}
Json::String(value.to_string())
}

View File

@@ -23,14 +23,11 @@
use std::sync::Arc;
use picloud_shared::{DocId, DocRow, DocsService, GroupDocsService, SdkCallCx, Services};
use picloud_shared::{DocId, DocRow, DocsService, SdkCallCx, Services};
use rhai::{Array, Dynamic, Engine as RhaiEngine, EvalAltResult, Map, Module};
use uuid::Uuid;
use super::bridge::{
block_on, dynamic_to_json_capped, json_to_dynamic, MAX_JSON_MATERIALIZE_BYTES,
};
use super::interceptor::InterceptorCtx;
use super::bridge::{block_on, dynamic_to_json, json_to_dynamic};
/// Per-call handle captured by the Rhai SDK. Cheap to clone (two Arcs
/// plus an owned string).
@@ -39,37 +36,15 @@ pub struct DocsHandle {
collection: String,
service: Arc<dyn DocsService>,
cx: Arc<SdkCallCx>,
// §9.4: carried so `create`/`update`/`delete` run the before/after hook (M8).
ictx: InterceptorCtx,
}
/// §11.6 shared-collection handle, returned by `docs::shared_collection(name)`.
/// A distinct Rhai type from `DocsHandle` so a script's choice of
/// private-vs-shared scope is explicit. Routes through `GroupDocsService`, which
/// resolves the owning group from `cx.app_id` (the script never names a group).
#[derive(Clone)]
pub struct GroupDocsHandle {
collection: String,
service: Arc<dyn GroupDocsService>,
cx: Arc<SdkCallCx>,
// §9.4: carried so shared `create`/`update`/`delete` run the hook too (M8).
ictx: InterceptorCtx,
}
pub(super) fn register(
engine: &mut RhaiEngine,
services: &Services,
cx: Arc<SdkCallCx>,
ictx: InterceptorCtx,
) {
pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<SdkCallCx>) {
let docs_service = services.docs.clone();
let group_docs_service = services.group_docs.clone();
let mut module = Module::new();
{
let docs_service = docs_service.clone();
let cx = cx.clone();
let ictx = ictx.clone();
module.set_native_fn(
"collection",
move |name: &str| -> Result<DocsHandle, Box<EvalAltResult>> {
@@ -80,26 +55,6 @@ pub(super) fn register(
collection: name.to_string(),
service: docs_service.clone(),
cx: cx.clone(),
ictx: ictx.clone(),
})
},
);
}
{
let group_docs_service = group_docs_service.clone();
let cx = cx.clone();
let ictx = ictx.clone();
module.set_native_fn(
"shared_collection",
move |name: &str| -> Result<GroupDocsHandle, Box<EvalAltResult>> {
if name.is_empty() {
return Err("docs::shared_collection name must not be empty".into());
}
Ok(GroupDocsHandle {
collection: name.to_string(),
service: group_docs_service.clone(),
cx: cx.clone(),
ictx: ictx.clone(),
})
},
);
@@ -115,40 +70,18 @@ pub(super) fn register(
register_update(engine);
register_delete(engine);
register_list(engine);
// Same method names on GroupDocsHandle — Rhai dispatches by receiver type.
engine.register_type_with_name::<GroupDocsHandle>("GroupDocsHandle");
register_group_create(engine);
register_group_get(engine);
register_group_find(engine);
register_group_find_one(engine);
register_group_update(engine);
register_group_delete(engine);
register_group_list(engine);
}
fn register_create(engine: &mut RhaiEngine) {
engine.register_fn(
"create",
|handle: &mut DocsHandle, data: Map| -> Result<String, Box<EvalAltResult>> {
let json = dynamic_to_json_capped(&Dynamic::from(data), MAX_JSON_MATERIALIZE_BYTES)?;
// §9.4 before-op interceptor (allow/deny + M4 data-transform).
let write_val = run_before(handle, "create", "", Some(&json))?.unwrap_or(json);
let written = write_val.clone();
let h = handle.clone();
let json = dynamic_to_json(&Dynamic::from(data));
let id = block_on("docs", async move {
h.service.create(&h.cx, &h.collection, write_val).await
h.service.create(&h.cx, &h.collection, json).await
})?;
let id_str = id.to_string();
run_after(
handle,
"create",
"",
Some(&written),
serde_json::Value::String(id_str.clone()),
)?;
Ok(id_str)
Ok(id.to_string())
},
);
}
@@ -172,7 +105,7 @@ fn register_find(engine: &mut RhaiEngine) {
"find",
|handle: &mut DocsHandle, filter: Map| -> Result<Array, Box<EvalAltResult>> {
let h = handle.clone();
let json = dynamic_to_json_capped(&Dynamic::from(filter), MAX_JSON_MATERIALIZE_BYTES)?;
let json = dynamic_to_json(&Dynamic::from(filter));
let rows = block_on("docs", async move {
h.service.find(&h.cx, &h.collection, json).await
})?;
@@ -189,7 +122,7 @@ fn register_find_one(engine: &mut RhaiEngine) {
"find_one",
|handle: &mut DocsHandle, filter: Map| -> Result<Dynamic, Box<EvalAltResult>> {
let h = handle.clone();
let json = dynamic_to_json_capped(&Dynamic::from(filter), MAX_JSON_MATERIALIZE_BYTES)?;
let json = dynamic_to_json(&Dynamic::from(filter));
let row = block_on("docs", async move {
h.service.find_one(&h.cx, &h.collection, json).await
})?;
@@ -202,23 +135,14 @@ fn register_update(engine: &mut RhaiEngine) {
engine.register_fn(
"update",
|handle: &mut DocsHandle, id: &str, data: Map| -> Result<(), Box<EvalAltResult>> {
let parsed_id = parse_doc_id(id)?;
let json = dynamic_to_json_capped(&Dynamic::from(data), MAX_JSON_MATERIALIZE_BYTES)?;
let write_val = run_before(handle, "update", id, Some(&json))?.unwrap_or(json);
let written = write_val.clone();
let h = handle.clone();
let parsed_id = parse_doc_id(id)?;
let json = dynamic_to_json(&Dynamic::from(data));
block_on("docs", async move {
h.service
.update(&h.cx, &h.collection, parsed_id, write_val)
.update(&h.cx, &h.collection, parsed_id, json)
.await
})?;
run_after(
handle,
"update",
id,
Some(&written),
serde_json::Value::Null,
)
})
},
);
}
@@ -227,62 +151,15 @@ fn register_delete(engine: &mut RhaiEngine) {
engine.register_fn(
"delete",
|handle: &mut DocsHandle, id: &str| -> Result<bool, Box<EvalAltResult>> {
let parsed_id = parse_doc_id(id)?;
// Delete carries no value, so the before-hook never transforms.
run_before(handle, "delete", id, None)?;
let h = handle.clone();
let was_present = block_on("docs", async move {
let parsed_id = parse_doc_id(id)?;
block_on("docs", async move {
h.service.delete(&h.cx, &h.collection, parsed_id).await
})?;
run_after(
handle,
"delete",
id,
None,
serde_json::Value::Bool(was_present),
)?;
Ok(was_present)
})
},
);
}
/// §9.4: run the before-op interceptor for a per-app docs write.
fn run_before(
handle: &DocsHandle,
op: &'static str,
key: &str,
value: Option<&serde_json::Value>,
) -> Result<Option<serde_json::Value>, Box<EvalAltResult>> {
super::interceptor::run_before(
&handle.ictx,
&handle.cx,
"docs",
op,
&handle.collection,
key,
value,
)
}
fn run_after(
handle: &DocsHandle,
op: &'static str,
key: &str,
value: Option<&serde_json::Value>,
result: serde_json::Value,
) -> Result<(), Box<EvalAltResult>> {
super::interceptor::run_after(
&handle.ictx,
&handle.cx,
"docs",
op,
&handle.collection,
key,
value,
result,
)
}
fn register_list(engine: &mut RhaiEngine) {
// Zero-arg form: full page from the start.
engine.register_fn(
@@ -341,219 +218,6 @@ fn list_call(
Ok(m)
}
// --- GroupDocsHandle methods (§11.6 shared docs collections) ---------------
fn register_group_create(engine: &mut RhaiEngine) {
engine.register_fn(
"create",
|handle: &mut GroupDocsHandle, data: Map| -> Result<String, Box<EvalAltResult>> {
let json = dynamic_to_json_capped(&Dynamic::from(data), MAX_JSON_MATERIALIZE_BYTES)?;
let write_val = group_run_before(handle, "create", "", Some(&json))?.unwrap_or(json);
let written = write_val.clone();
let h = handle.clone();
let id = block_on("docs", async move {
h.service.create(&h.cx, &h.collection, write_val).await
})?;
let id_str = id.to_string();
group_run_after(
handle,
"create",
"",
Some(&written),
serde_json::Value::String(id_str.clone()),
)?;
Ok(id_str)
},
);
}
fn register_group_get(engine: &mut RhaiEngine) {
engine.register_fn(
"get",
|handle: &mut GroupDocsHandle, id: &str| -> Result<Dynamic, Box<EvalAltResult>> {
let h = handle.clone();
let parsed_id = parse_doc_id(id)?;
let row = block_on("docs", async move {
h.service.get(&h.cx, &h.collection, parsed_id).await
})?;
Ok(row.map_or(Dynamic::UNIT, |d| Dynamic::from(doc_to_map(&d))))
},
);
}
fn register_group_find(engine: &mut RhaiEngine) {
engine.register_fn(
"find",
|handle: &mut GroupDocsHandle, filter: Map| -> Result<Array, Box<EvalAltResult>> {
let h = handle.clone();
let json = dynamic_to_json_capped(&Dynamic::from(filter), MAX_JSON_MATERIALIZE_BYTES)?;
let rows = block_on("docs", async move {
h.service.find(&h.cx, &h.collection, json).await
})?;
Ok(rows
.iter()
.map(|d| Dynamic::from(doc_to_map(d)))
.collect::<Vec<Dynamic>>())
},
);
}
fn register_group_find_one(engine: &mut RhaiEngine) {
engine.register_fn(
"find_one",
|handle: &mut GroupDocsHandle, filter: Map| -> Result<Dynamic, Box<EvalAltResult>> {
let h = handle.clone();
let json = dynamic_to_json_capped(&Dynamic::from(filter), MAX_JSON_MATERIALIZE_BYTES)?;
let row = block_on("docs", async move {
h.service.find_one(&h.cx, &h.collection, json).await
})?;
Ok(row.map_or(Dynamic::UNIT, |d| Dynamic::from(doc_to_map(&d))))
},
);
}
fn register_group_update(engine: &mut RhaiEngine) {
engine.register_fn(
"update",
|handle: &mut GroupDocsHandle, id: &str, data: Map| -> Result<(), Box<EvalAltResult>> {
let parsed_id = parse_doc_id(id)?;
let json = dynamic_to_json_capped(&Dynamic::from(data), MAX_JSON_MATERIALIZE_BYTES)?;
let write_val = group_run_before(handle, "update", id, Some(&json))?.unwrap_or(json);
let written = write_val.clone();
let h = handle.clone();
block_on("docs", async move {
h.service
.update(&h.cx, &h.collection, parsed_id, write_val)
.await
})?;
group_run_after(
handle,
"update",
id,
Some(&written),
serde_json::Value::Null,
)
},
);
}
fn register_group_delete(engine: &mut RhaiEngine) {
engine.register_fn(
"delete",
|handle: &mut GroupDocsHandle, id: &str| -> Result<bool, Box<EvalAltResult>> {
let parsed_id = parse_doc_id(id)?;
group_run_before(handle, "delete", id, None)?;
let h = handle.clone();
let was_present = block_on("docs", async move {
h.service.delete(&h.cx, &h.collection, parsed_id).await
})?;
group_run_after(
handle,
"delete",
id,
None,
serde_json::Value::Bool(was_present),
)?;
Ok(was_present)
},
);
}
/// §9.4: before/after hooks for a shared (group) docs write — mirror the per-app
/// helpers but over `GroupDocsHandle`.
fn group_run_before(
handle: &GroupDocsHandle,
op: &'static str,
key: &str,
value: Option<&serde_json::Value>,
) -> Result<Option<serde_json::Value>, Box<EvalAltResult>> {
super::interceptor::run_before(
&handle.ictx,
&handle.cx,
"docs",
op,
&handle.collection,
key,
value,
)
}
fn group_run_after(
handle: &GroupDocsHandle,
op: &'static str,
key: &str,
value: Option<&serde_json::Value>,
result: serde_json::Value,
) -> Result<(), Box<EvalAltResult>> {
super::interceptor::run_after(
&handle.ictx,
&handle.cx,
"docs",
op,
&handle.collection,
key,
value,
result,
)
}
fn register_group_list(engine: &mut RhaiEngine) {
engine.register_fn(
"list",
|handle: &mut GroupDocsHandle| -> Result<Map, Box<EvalAltResult>> {
group_list_call(handle, None, 0)
},
);
engine.register_fn(
"list",
|handle: &mut GroupDocsHandle, args: Map| -> Result<Map, Box<EvalAltResult>> {
let cursor = match args.get("cursor") {
Some(d) if !d.is_unit() => {
Some(d.clone().into_string().map_err(|_| -> Box<EvalAltResult> {
"docs::list: 'cursor' must be a string or ()".into()
})?)
}
_ => None,
};
let limit = match args.get("limit") {
Some(d) if !d.is_unit() => {
let n = d.as_int().map_err(|_| -> Box<EvalAltResult> {
"docs::list: 'limit' must be an integer".into()
})?;
u32::try_from(n.max(0)).unwrap_or(0)
}
_ => 0,
};
group_list_call(handle, cursor, limit)
},
);
}
fn group_list_call(
handle: &GroupDocsHandle,
cursor: Option<String>,
limit: u32,
) -> Result<Map, Box<EvalAltResult>> {
let h = handle.clone();
let page = block_on("docs", async move {
h.service
.list(&h.cx, &h.collection, cursor.as_deref(), limit)
.await
})?;
let mut m = Map::new();
let docs: Array = page
.docs
.iter()
.map(|d| Dynamic::from(doc_to_map(d)))
.collect();
m.insert("docs".into(), docs.into());
m.insert(
"next_cursor".into(),
page.next_cursor.map_or(Dynamic::UNIT, Dynamic::from),
);
Ok(m)
}
/// Build the `{ id, data, created_at, updated_at }` envelope per
/// Decision D. Scripts read user fields via `doc.data.<field>`; `id`
/// and timestamps are direct children of the envelope.

View File

@@ -26,7 +26,7 @@
use std::sync::Arc;
use super::bridge::{block_on, runtime_err};
use super::bridge::block_on;
use picloud_shared::{OutboundEmail, SdkCallCx, Services};
use rhai::{Array, Engine as RhaiEngine, EvalAltResult, Map, Module};
@@ -124,3 +124,8 @@ fn addresses(opts: &Map, key: &str) -> Result<Vec<String>, Box<EvalAltResult>> {
}
}
}
#[allow(clippy::unnecessary_box_returns)]
fn runtime_err(msg: &str) -> Box<EvalAltResult> {
EvalAltResult::ErrorRuntime(msg.into(), rhai::Position::NONE).into()
}

View File

@@ -1,124 +0,0 @@
//! Per-execution fan-out ceiling (audit fix #2).
//!
//! `trigger_depth` bounds how DEEP a trigger/invoke chain can go, but nothing
//! bounded how WIDE a single execution could fan out: one anonymous request
//! running `for i in 0..1_000_000 { invoke_async("w", #{}) }` costs a few Rhai
//! ops per iteration plus one cheap outbox INSERT, so within the op / wall-clock
//! budget it could write ~10^5 durable rows — each dispatched as its own
//! execution — flooding the outbox/queue (a durable amplification DoS).
//!
//! This caps the number of DURABLE emissions (`invoke_async`, `publish_durable`,
//! `enqueue`, and their group-shared variants) a single execution may make.
//! The counter is a thread-local. [`EmissionBudgetScope`] is created around
//! EVERY `execute_ast` and is **re-entrancy aware**: only the OUTERMOST scope
//! on a thread resets the counter. A dispatched handler (queue/trigger/
//! `invoke_async`/cron/workflow) is a fresh task on a pooled thread, so its
//! scope is outermost → it resets and gets a full budget. A synchronous
//! `invoke()` / interceptor re-entry runs a nested `execute_ast` in the SAME
//! call stack, so its scope is inner → it does NOT reset, and the whole
//! synchronous chain shares one budget (fan-out counted across it). The
//! outermost Drop re-zeroes the counter so a pooled thread never leaks a count
//! into the next task.
use std::cell::Cell;
use std::sync::LazyLock;
use rhai::EvalAltResult;
use crate::sdk::bridge::runtime_err;
/// Default max durable emissions per execution. Generous for a legit batch
/// producer, far below the ~10^5 a runaway loop can reach within budget.
const DEFAULT_MAX_EMISSIONS: u32 = 1000;
static MAX_EMISSIONS: LazyLock<u32> = LazyLock::new(|| {
std::env::var("PICLOUD_MAX_EMISSIONS_PER_EXECUTION")
.ok()
.and_then(|v| v.parse::<u32>().ok())
.filter(|&n| n > 0)
.unwrap_or(DEFAULT_MAX_EMISSIONS)
});
thread_local! {
static EMISSIONS: Cell<u32> = const { Cell::new(0) };
/// Scope nesting depth on this thread. `0` when no execution is active.
static ACTIVE: Cell<u32> = const { Cell::new(0) };
}
/// Charge one durable emission against the current execution's budget. Returns
/// an `Err` (aborting the emit) once the ceiling is exceeded.
///
/// # Errors
/// Returns a Rhai runtime error when the per-execution emission ceiling is hit.
pub(super) fn charge_emission(service: &str) -> Result<(), Box<EvalAltResult>> {
EMISSIONS.with(|c| {
let next = c.get().saturating_add(1);
if next > *MAX_EMISSIONS {
return Err(runtime_err(&format!(
"{service}: per-execution emission limit exceeded (max {} durable \
invoke_async/publish/enqueue calls per execution)",
*MAX_EMISSIONS
)));
}
c.set(next);
Ok(())
})
}
/// RAII scope wrapping one `execute_ast`. Re-entrancy aware: the OUTERMOST scope
/// on a thread zeroes the counter on entry and on final exit; nested scopes (a
/// synchronous re-entry in the same call stack) leave it alone, so the chain
/// shares one budget.
pub(crate) struct EmissionBudgetScope;
impl EmissionBudgetScope {
pub(crate) fn enter() -> Self {
ACTIVE.with(|a| {
if a.get() == 0 {
EMISSIONS.with(|c| c.set(0));
}
a.set(a.get() + 1);
});
Self
}
}
impl Drop for EmissionBudgetScope {
fn drop(&mut self) {
ACTIVE.with(|a| {
let n = a.get().saturating_sub(1);
a.set(n);
if n == 0 {
EMISSIONS.with(|c| c.set(0));
}
});
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn outermost_scope_resets_but_nested_shares_budget() {
let outer = EmissionBudgetScope::enter();
for _ in 0..DEFAULT_MAX_EMISSIONS {
charge_emission("test").expect("under budget");
}
// A NESTED scope (synchronous re-entry) must NOT reset — the budget is
// already exhausted and stays that way.
{
let _nested = EmissionBudgetScope::enter();
assert!(
charge_emission("test").is_err(),
"a nested re-entry shares the exhausted budget"
);
}
// Still exhausted after the nested scope drops (outer still active).
assert!(charge_emission("test").is_err());
drop(outer);
// A fresh OUTERMOST scope resets.
let _fresh = EmissionBudgetScope::enter();
charge_emission("test").expect("reset by a fresh outermost scope");
}
}

View File

@@ -24,10 +24,7 @@
use std::sync::Arc;
use super::bridge::block_on;
use super::interceptor::InterceptorCtx;
use picloud_shared::{
FileMeta, FileUpdate, FilesService, GroupFilesService, NewFile, SdkCallCx, Services,
};
use picloud_shared::{FileMeta, FileUpdate, FilesService, NewFile, SdkCallCx, Services};
use rhai::{Array, Dynamic, Engine as RhaiEngine, EvalAltResult, Map, Module};
/// Per-call handle captured by the Rhai SDK. Cheap to clone (two Arcs
@@ -37,39 +34,15 @@ pub struct FilesHandle {
collection: String,
service: Arc<dyn FilesService>,
cx: Arc<SdkCallCx>,
// §9.4: carried so `create`/`update`/`delete` run the before/after hook (M9).
// Files carry bytes, so the hook sees service/op/collection/key only — the
// blob is never materialized into a `value`, and there is no data transform.
ictx: InterceptorCtx,
}
/// §11.6 shared-collection handle, returned by `files::shared_collection(name)`.
/// Same method surface as `FilesHandle`, but routes through the
/// `GroupFilesService` — the owning group is resolved from `cx.app_id`'s
/// ancestor chain inside the service (the isolation boundary).
#[derive(Clone)]
pub struct GroupFilesHandle {
collection: String,
service: Arc<dyn GroupFilesService>,
cx: Arc<SdkCallCx>,
// §9.4: carried so shared `create`/`update`/`delete` run the hook too (M9).
ictx: InterceptorCtx,
}
pub(super) fn register(
engine: &mut RhaiEngine,
services: &Services,
cx: Arc<SdkCallCx>,
ictx: InterceptorCtx,
) {
pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<SdkCallCx>) {
let files_service = services.files.clone();
let group_files_service = services.group_files.clone();
let mut module = Module::new();
{
let files_service = files_service.clone();
let cx = cx.clone();
let ictx = ictx.clone();
module.set_native_fn(
"collection",
move |name: &str| -> Result<FilesHandle, Box<EvalAltResult>> {
@@ -80,26 +53,6 @@ pub(super) fn register(
collection: name.to_string(),
service: files_service.clone(),
cx: cx.clone(),
ictx: ictx.clone(),
})
},
);
}
{
let group_files_service = group_files_service.clone();
let cx = cx.clone();
let ictx = ictx.clone();
module.set_native_fn(
"shared_collection",
move |name: &str| -> Result<GroupFilesHandle, Box<EvalAltResult>> {
if name.is_empty() {
return Err("files::shared_collection name must not be empty".into());
}
Ok(GroupFilesHandle {
collection: name.to_string(),
service: group_files_service.clone(),
cx: cx.clone(),
ictx: ictx.clone(),
})
},
);
@@ -114,15 +67,6 @@ pub(super) fn register(
register_update(engine);
register_delete(engine);
register_list(engine);
// Same method names on GroupFilesHandle — Rhai dispatches by receiver type.
engine.register_type_with_name::<GroupFilesHandle>("GroupFilesHandle");
register_group_create(engine);
register_group_head(engine);
register_group_get(engine);
register_group_update(engine);
register_group_delete(engine);
register_group_list(engine);
}
fn register_create(engine: &mut RhaiEngine) {
@@ -132,9 +76,6 @@ fn register_create(engine: &mut RhaiEngine) {
let name = require_string(&meta, "name")?;
let content_type = require_string(&meta, "content_type")?;
let data = require_blob(&meta, "data")?;
// §9.4 before-op interceptor (allow/deny only — no transform for
// files, the blob is not surfaced to the hook).
run_before(handle, "create", "")?;
let h = handle.clone();
let new = NewFile {
name,
@@ -144,14 +85,7 @@ fn register_create(engine: &mut RhaiEngine) {
let id = block_on("files", async move {
h.service.create(&h.cx, &h.collection, new).await
})?;
let id_str = id.to_string();
run_after(
handle,
"create",
"",
serde_json::Value::String(id_str.clone()),
)?;
Ok(id_str)
Ok(id.to_string())
},
);
}
@@ -191,18 +125,16 @@ fn register_update(engine: &mut RhaiEngine) {
let data = require_blob(&meta, "data")?;
let name = optional_string(&meta, "name")?;
let content_type = optional_string(&meta, "content_type")?;
run_before(handle, "update", id)?;
let h = handle.clone();
let id_owned = id.to_string();
let id = id.to_string();
let upd = FileUpdate {
data,
name,
content_type,
};
block_on("files", async move {
h.service.update(&h.cx, &h.collection, &id_owned, upd).await
})?;
run_after(handle, "update", id, serde_json::Value::Null)
h.service.update(&h.cx, &h.collection, &id, upd).await
})
},
);
}
@@ -211,52 +143,15 @@ fn register_delete(engine: &mut RhaiEngine) {
engine.register_fn(
"delete",
|handle: &mut FilesHandle, id: &str| -> Result<bool, Box<EvalAltResult>> {
run_before(handle, "delete", id)?;
let h = handle.clone();
let id_owned = id.to_string();
let was_present = block_on("files", async move {
h.service.delete(&h.cx, &h.collection, &id_owned).await
})?;
run_after(handle, "delete", id, serde_json::Value::Bool(was_present))?;
Ok(was_present)
let id = id.to_string();
block_on("files", async move {
h.service.delete(&h.cx, &h.collection, &id).await
})
},
);
}
/// §9.4: before/after hooks for a per-app files write. Files pass `value = None`
/// (the blob is never surfaced to the hook), so the before-hook only allows or
/// denies — it never transforms.
fn run_before(handle: &FilesHandle, op: &'static str, key: &str) -> Result<(), Box<EvalAltResult>> {
super::interceptor::run_before(
&handle.ictx,
&handle.cx,
"files",
op,
&handle.collection,
key,
None,
)
.map(|_| ())
}
fn run_after(
handle: &FilesHandle,
op: &'static str,
key: &str,
result: serde_json::Value,
) -> Result<(), Box<EvalAltResult>> {
super::interceptor::run_after(
&handle.ictx,
&handle.cx,
"files",
op,
&handle.collection,
key,
None,
result,
)
}
fn register_list(engine: &mut RhaiEngine) {
engine.register_fn(
"list",
@@ -325,213 +220,6 @@ fn list_call(
Ok(m)
}
// --- GroupFilesHandle methods (§11.6 shared files collections) -------------
// Bodies mirror the app handle's; only the receiver type and service differ
// (Rhai dispatches `create`/`head`/... by the handle's concrete type).
fn register_group_create(engine: &mut RhaiEngine) {
engine.register_fn(
"create",
|handle: &mut GroupFilesHandle, meta: Map| -> Result<String, Box<EvalAltResult>> {
let name = require_string(&meta, "name")?;
let content_type = require_string(&meta, "content_type")?;
let data = require_blob(&meta, "data")?;
group_run_before(handle, "create", "")?;
let h = handle.clone();
let new = NewFile {
name,
content_type,
data,
};
let id = block_on("files", async move {
h.service.create(&h.cx, &h.collection, new).await
})?;
let id_str = id.to_string();
group_run_after(
handle,
"create",
"",
serde_json::Value::String(id_str.clone()),
)?;
Ok(id_str)
},
);
}
fn register_group_head(engine: &mut RhaiEngine) {
engine.register_fn(
"head",
|handle: &mut GroupFilesHandle, id: &str| -> Result<Dynamic, Box<EvalAltResult>> {
let h = handle.clone();
let id = id.to_string();
let meta = block_on("files", async move {
h.service.head(&h.cx, &h.collection, &id).await
})?;
Ok(meta.map_or(Dynamic::UNIT, |m| file_meta_to_map(&m).into()))
},
);
}
fn register_group_get(engine: &mut RhaiEngine) {
engine.register_fn(
"get",
|handle: &mut GroupFilesHandle, id: &str| -> Result<Dynamic, Box<EvalAltResult>> {
let h = handle.clone();
let id = id.to_string();
let bytes = block_on("files", async move {
h.service.get(&h.cx, &h.collection, &id).await
})?;
Ok(bytes.map_or(Dynamic::UNIT, Dynamic::from_blob))
},
);
}
fn register_group_update(engine: &mut RhaiEngine) {
engine.register_fn(
"update",
|handle: &mut GroupFilesHandle, id: &str, meta: Map| -> Result<(), Box<EvalAltResult>> {
let data = require_blob(&meta, "data")?;
let name = optional_string(&meta, "name")?;
let content_type = optional_string(&meta, "content_type")?;
group_run_before(handle, "update", id)?;
let h = handle.clone();
let id_owned = id.to_string();
let upd = FileUpdate {
data,
name,
content_type,
};
block_on("files", async move {
h.service.update(&h.cx, &h.collection, &id_owned, upd).await
})?;
group_run_after(handle, "update", id, serde_json::Value::Null)
},
);
}
fn register_group_delete(engine: &mut RhaiEngine) {
engine.register_fn(
"delete",
|handle: &mut GroupFilesHandle, id: &str| -> Result<bool, Box<EvalAltResult>> {
group_run_before(handle, "delete", id)?;
let h = handle.clone();
let id_owned = id.to_string();
let was_present = block_on("files", async move {
h.service.delete(&h.cx, &h.collection, &id_owned).await
})?;
group_run_after(handle, "delete", id, serde_json::Value::Bool(was_present))?;
Ok(was_present)
},
);
}
/// §9.4: before/after hooks for a shared (group) files write — mirror the
/// per-app helpers but over `GroupFilesHandle`.
fn group_run_before(
handle: &GroupFilesHandle,
op: &'static str,
key: &str,
) -> Result<(), Box<EvalAltResult>> {
super::interceptor::run_before(
&handle.ictx,
&handle.cx,
"files",
op,
&handle.collection,
key,
None,
)
.map(|_| ())
}
fn group_run_after(
handle: &GroupFilesHandle,
op: &'static str,
key: &str,
result: serde_json::Value,
) -> Result<(), Box<EvalAltResult>> {
super::interceptor::run_after(
&handle.ictx,
&handle.cx,
"files",
op,
&handle.collection,
key,
None,
result,
)
}
fn register_group_list(engine: &mut RhaiEngine) {
engine.register_fn(
"list",
|handle: &mut GroupFilesHandle| -> Result<Map, Box<EvalAltResult>> {
group_list_call(handle, None, 0)
},
);
engine.register_fn(
"list",
|handle: &mut GroupFilesHandle, cursor: &str| -> Result<Map, Box<EvalAltResult>> {
group_list_call(handle, Some(cursor.to_string()), 0)
},
);
engine.register_fn(
"list",
|handle: &mut GroupFilesHandle,
cursor: &str,
limit: i64|
-> Result<Map, Box<EvalAltResult>> {
let limit = u32::try_from(limit.max(0)).unwrap_or(0);
group_list_call(handle, Some(cursor.to_string()), limit)
},
);
engine.register_fn(
"list",
|handle: &mut GroupFilesHandle, opts: Map| -> Result<Map, Box<EvalAltResult>> {
let cursor = match opts.get("cursor") {
Some(v) if !v.is_unit() => {
Some(v.clone().into_string().map_err(|_| -> Box<EvalAltResult> {
"files: list cursor must be a string".into()
})?)
}
_ => None,
};
let limit = match opts.get("limit") {
Some(v) if !v.is_unit() => {
u32::try_from(v.as_int().unwrap_or(0).max(0)).unwrap_or(0)
}
_ => 0,
};
group_list_call(handle, cursor, limit)
},
);
}
fn group_list_call(
handle: &GroupFilesHandle,
cursor: Option<String>,
limit: u32,
) -> Result<Map, Box<EvalAltResult>> {
let h = handle.clone();
let page = block_on("files", async move {
h.service
.list(&h.cx, &h.collection, cursor.as_deref(), limit)
.await
})?;
let mut m = Map::new();
let files: Array = page
.files
.iter()
.map(|meta| Dynamic::from(file_meta_to_map(meta)))
.collect();
m.insert("files".into(), files.into());
m.insert(
"next_cursor".into(),
page.next_cursor.map_or(Dynamic::UNIT, Dynamic::from),
);
Ok(m)
}
/// Render a `FileMeta` into the Rhai map shape scripts see from
/// `head` / `list`.
fn file_meta_to_map(meta: &FileMeta) -> Map {

View File

@@ -32,13 +32,11 @@
use std::collections::BTreeMap;
use std::sync::Arc;
use picloud_shared::{HttpRequest, HttpResponse, HttpService, SdkCallCx, Services};
use picloud_shared::{HttpError, HttpRequest, HttpResponse, HttpService, SdkCallCx, Services};
use rhai::{Dynamic, Engine as RhaiEngine, EvalAltResult, Map, Module};
use tokio::runtime::Handle as TokioHandle;
use super::bridge::{
block_on, dynamic_to_json_capped, json_to_dynamic, runtime_err, MAX_JSON_MATERIALIZE_BYTES,
};
use super::interceptor::InterceptorCtx;
use super::bridge::{dynamic_to_json, json_to_dynamic};
/// Bridge-side defaults (the service clamps server-side too). The
/// `MAX_*` ceilings stay `i64` because they're compared against the
@@ -51,31 +49,20 @@ const MAX_REDIRECTS: i64 = 10;
const ALLOWED_OPT_KEYS: [&str; 4] = ["headers", "timeout_ms", "follow_redirects", "max_redirects"];
// `http` registers its verbs as module native fns (no handle struct), so the
// interceptor ctx is captured into each request closure directly. Every verb
// funnels through `invoke` / `invoke_form`, which run the §9.4 before/after hook
// around `svc.request` (M11). The hook sees `service = "http"`, `op =
// "request"`, `collection = <METHOD>`, `key = <url>`; the body is never
// surfaced, so there is no data transform.
pub(super) fn register(
engine: &mut RhaiEngine,
services: &Services,
cx: Arc<SdkCallCx>,
ictx: InterceptorCtx,
) {
pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<SdkCallCx>) {
let svc = services.http.clone();
let mut module = Module::new();
// Bodyless verbs: (url) / (url, opts).
for verb in ["get", "head"] {
register_bodyless(&mut module, verb, &svc, &cx, &ictx);
register_bodyless(&mut module, verb, &svc, &cx);
}
// Body verbs: (url) / (url, body) / (url, body, opts).
for verb in ["post", "put", "patch", "delete"] {
register_body(&mut module, verb, &svc, &cx, &ictx);
register_body(&mut module, verb, &svc, &cx);
}
register_post_form(&mut module, &svc, &cx, &ictx);
register_request(&mut module, &svc, &cx, &ictx);
register_post_form(&mut module, &svc, &cx);
register_request(&mut module, &svc, &cx);
engine.register_static_module("http", module.into());
}
@@ -85,18 +72,17 @@ fn register_bodyless(
verb: &'static str,
svc: &Arc<dyn HttpService>,
cx: &Arc<SdkCallCx>,
ictx: &InterceptorCtx,
) {
{
let (svc, cx, ictx) = (svc.clone(), cx.clone(), ictx.clone());
let (svc, cx) = (svc.clone(), cx.clone());
module.set_native_fn(verb, move |url: &str| {
invoke(&ictx, &svc, &cx, verb, url, None, None)
invoke(&svc, &cx, verb, url, None, None)
});
}
{
let (svc, cx, ictx) = (svc.clone(), cx.clone(), ictx.clone());
let (svc, cx) = (svc.clone(), cx.clone());
module.set_native_fn(verb, move |url: &str, opts: Map| {
invoke(&ictx, &svc, &cx, verb, url, None, Some(&opts))
invoke(&svc, &cx, verb, url, None, Some(&opts))
});
}
}
@@ -106,72 +92,61 @@ fn register_body(
verb: &'static str,
svc: &Arc<dyn HttpService>,
cx: &Arc<SdkCallCx>,
ictx: &InterceptorCtx,
) {
{
let (svc, cx, ictx) = (svc.clone(), cx.clone(), ictx.clone());
let (svc, cx) = (svc.clone(), cx.clone());
module.set_native_fn(verb, move |url: &str| {
invoke(&ictx, &svc, &cx, verb, url, None, None)
invoke(&svc, &cx, verb, url, None, None)
});
}
{
let (svc, cx, ictx) = (svc.clone(), cx.clone(), ictx.clone());
let (svc, cx) = (svc.clone(), cx.clone());
module.set_native_fn(verb, move |url: &str, body: Dynamic| {
invoke(&ictx, &svc, &cx, verb, url, Some(body), None)
invoke(&svc, &cx, verb, url, Some(body), None)
});
}
{
let (svc, cx, ictx) = (svc.clone(), cx.clone(), ictx.clone());
let (svc, cx) = (svc.clone(), cx.clone());
module.set_native_fn(verb, move |url: &str, body: Dynamic, opts: Map| {
invoke(&ictx, &svc, &cx, verb, url, Some(body), Some(&opts))
invoke(&svc, &cx, verb, url, Some(body), Some(&opts))
});
}
}
fn register_post_form(
module: &mut Module,
svc: &Arc<dyn HttpService>,
cx: &Arc<SdkCallCx>,
ictx: &InterceptorCtx,
) {
fn register_post_form(module: &mut Module, svc: &Arc<dyn HttpService>, cx: &Arc<SdkCallCx>) {
{
let (svc, cx, ictx) = (svc.clone(), cx.clone(), ictx.clone());
let (svc, cx) = (svc.clone(), cx.clone());
module.set_native_fn("post_form", move |url: &str, form: Map| {
invoke_form(&ictx, &svc, &cx, url, &form, None)
invoke_form(&svc, &cx, url, &form, None)
});
}
{
let (svc, cx, ictx) = (svc.clone(), cx.clone(), ictx.clone());
let (svc, cx) = (svc.clone(), cx.clone());
module.set_native_fn("post_form", move |url: &str, form: Map, opts: Map| {
invoke_form(&ictx, &svc, &cx, url, &form, Some(&opts))
invoke_form(&svc, &cx, url, &form, Some(&opts))
});
}
}
fn register_request(
module: &mut Module,
svc: &Arc<dyn HttpService>,
cx: &Arc<SdkCallCx>,
ictx: &InterceptorCtx,
) {
fn register_request(module: &mut Module, svc: &Arc<dyn HttpService>, cx: &Arc<SdkCallCx>) {
{
let (svc, cx, ictx) = (svc.clone(), cx.clone(), ictx.clone());
let (svc, cx) = (svc.clone(), cx.clone());
module.set_native_fn("request", move |method: &str, url: &str| {
invoke(&ictx, &svc, &cx, method, url, None, None)
invoke(&svc, &cx, method, url, None, None)
});
}
{
let (svc, cx, ictx) = (svc.clone(), cx.clone(), ictx.clone());
let (svc, cx) = (svc.clone(), cx.clone());
module.set_native_fn("request", move |method: &str, url: &str, body: Dynamic| {
invoke(&ictx, &svc, &cx, method, url, Some(body), None)
invoke(&svc, &cx, method, url, Some(body), None)
});
}
{
let (svc, cx, ictx) = (svc.clone(), cx.clone(), ictx.clone());
let (svc, cx) = (svc.clone(), cx.clone());
module.set_native_fn(
"request",
move |method: &str, url: &str, body: Dynamic, opts: Map| {
invoke(&ictx, &svc, &cx, method, url, Some(body), Some(&opts))
invoke(&svc, &cx, method, url, Some(body), Some(&opts))
},
);
}
@@ -262,21 +237,20 @@ fn dispatch_body(body: Dynamic) -> Result<EncodedBody, Box<EvalAltResult>> {
return Ok((Some(s.into_bytes()), Some("text/plain".to_string())));
}
if body.is_map() || body.is_array() {
let json = dynamic_to_json_capped(&body, MAX_JSON_MATERIALIZE_BYTES)?;
let json = dynamic_to_json(&body);
let bytes = serde_json::to_vec(&json)
.map_err(|e| err(format!("could not encode JSON body: {e}")))?;
return Ok((Some(bytes), Some("application/json".to_string())));
}
// Scalars (int/float/bool) → JSON-encode for consistency.
let json = dynamic_to_json_capped(&body, MAX_JSON_MATERIALIZE_BYTES)?;
let json = dynamic_to_json(&body);
let bytes =
serde_json::to_vec(&json).map_err(|e| err(format!("could not encode body: {e}")))?;
Ok((Some(bytes), Some("application/json".to_string())))
}
#[allow(clippy::needless_pass_by_value, clippy::too_many_arguments)]
#[allow(clippy::needless_pass_by_value)]
fn invoke(
ictx: &InterceptorCtx,
svc: &Arc<dyn HttpService>,
cx: &Arc<SdkCallCx>,
method: &str,
@@ -286,9 +260,6 @@ fn invoke(
) -> Result<Dynamic, Box<EvalAltResult>> {
let opts = parse_opts(opts)?;
let method_uc = method.to_ascii_uppercase();
// §9.4 before-op interceptor (allow/deny only; the body is not surfaced to
// the hook). `collection` = the HTTP method, `key` = the URL.
super::interceptor::run_before(ictx, cx, "http", "request", &method_uc, url, None)?;
let bodyless = matches!(method_uc.as_str(), "GET" | "HEAD");
let (encoded, content_type) = if bodyless {
(None, None)
@@ -299,7 +270,7 @@ fn invoke(
};
let req = HttpRequest {
method: method_uc.clone(),
method: method_uc,
url: url.to_string(),
headers: opts.headers,
body: encoded,
@@ -309,27 +280,12 @@ fn invoke(
max_redirects: opts.max_redirects,
script_id: Some(cx.script_id.to_string()),
};
let resp = {
let svc = svc.clone();
let cx = cx.clone();
block_on("http", async move { svc.request(&cx, req).await })?
};
super::interceptor::run_after(
ictx,
cx,
"http",
"request",
&method_uc,
url,
None,
serde_json::Value::Null,
)?;
let resp = block_on(svc, cx, req)?;
Ok(response_to_dynamic(&resp))
}
#[allow(clippy::needless_pass_by_value)]
fn invoke_form(
ictx: &InterceptorCtx,
svc: &Arc<dyn HttpService>,
cx: &Arc<SdkCallCx>,
url: &str,
@@ -337,8 +293,6 @@ fn invoke_form(
opts: Option<&Map>,
) -> Result<Dynamic, Box<EvalAltResult>> {
let opts = parse_opts(opts)?;
// §9.4 before-op interceptor. `post_form` is always a POST.
super::interceptor::run_before(ictx, cx, "http", "request", "POST", url, None)?;
let mut serializer = url::form_urlencoded::Serializer::new(String::new());
for (k, v) in form {
serializer.append_pair(k.as_str(), &dyn_to_string(v));
@@ -356,21 +310,7 @@ fn invoke_form(
max_redirects: opts.max_redirects,
script_id: Some(cx.script_id.to_string()),
};
let resp = {
let svc = svc.clone();
let cx = cx.clone();
block_on("http", async move { svc.request(&cx, req).await })?
};
super::interceptor::run_after(
ictx,
cx,
"http",
"request",
"POST",
url,
None,
serde_json::Value::Null,
)?;
let resp = block_on(svc, cx, req)?;
Ok(response_to_dynamic(&resp))
}
@@ -416,10 +356,36 @@ fn dyn_to_string(v: &Dynamic) -> String {
}
}
// A validation error, prefixed like the service's runtime errors (the shared
// `bridge::block_on("http", …)` prefixes the same way, so messages are uniform).
// Delegates to `bridge::runtime_err`; the boxed return is Rhai's error channel.
// Rhai's native-fn error channel is `Box<EvalAltResult>`, so these
// helpers return the boxed form the call sites need.
#[allow(clippy::unnecessary_box_returns)]
fn err(msg: String) -> Box<EvalAltResult> {
runtime_err(&format!("http: {msg}"))
EvalAltResult::ErrorRuntime(format!("http: {msg}").into(), rhai::Position::NONE).into()
}
/// Run the async service call from the synchronous Rhai context. Same
/// pattern as `kv`/`docs`: the script runs under `spawn_blocking`, so a
/// runtime handle is reachable and blocking on it is correct.
fn block_on(
svc: &Arc<dyn HttpService>,
cx: &Arc<SdkCallCx>,
req: HttpRequest,
) -> Result<HttpResponse, Box<EvalAltResult>> {
let handle = TokioHandle::try_current().map_err(|e| -> Box<EvalAltResult> {
EvalAltResult::ErrorRuntime(
format!("http: no tokio runtime available: {e}").into(),
rhai::Position::NONE,
)
.into()
})?;
let svc = svc.clone();
let cx = cx.clone();
handle
.block_on(async move { svc.request(&cx, req).await })
.map_err(map_http_err)
}
#[allow(clippy::unnecessary_box_returns)]
fn map_http_err(e: HttpError) -> Box<EvalAltResult> {
EvalAltResult::ErrorRuntime(format!("http: {e}").into(), rhai::Position::NONE).into()
}

View File

@@ -1,417 +0,0 @@
//! §9.4 Service Interceptors — the executor-side before-op hook.
//!
//! MVP: `kv::set` / `kv::delete` (private AND group-shared collections) run an
//! allow/deny interceptor first. The hook resolves the nearest interceptor for
//! `(service, op)` on the calling app's chain via the injected
//! `InterceptorService` (a cheap indexed query; `None` = un-hooked → allow, no
//! further work). The resolver returns a script already **sealed to the owner
//! that declared the marker** and fully materialized, so we run it straight
//! through the SAME `invoke()` re-entry core (`run_resolved_blocking`) — no
//! second dispatch mechanism, no re-resolution by name (which would let a
//! descendant app shadow a group's guard).
//!
//! **Fail closed.** Everything but an explicit allow denies:
//! - a registered marker whose script is missing/disabled → deny;
//! - a registered marker with no runnable engine back-reference → deny;
//! - a return value that is not `#{ allowed: true }` → deny.
//!
//! Only a total absence of any marker allows without running anything.
//!
//! **Re-entrancy.** An interceptor that itself performs the guarded op (e.g.
//! an "allow + audit-log" hook that writes KV) must not re-trigger itself. A
//! thread-local guard, set for the duration of the interceptor's synchronous
//! execution, makes any nested op the interceptor performs bypass interception.
use std::cell::{Cell, RefCell};
use std::collections::HashMap;
use std::sync::{Arc, LazyLock};
use std::time::Duration;
use picloud_shared::{
AppId, InterceptorChain, InterceptorEntry, InterceptorService, ScriptId, SdkCallCx,
};
use rhai::EvalAltResult;
use serde_json::{json, Value as Json};
use tokio::runtime::Handle as TokioHandle;
use crate::engine::Engine;
use crate::sandbox::Limits;
use crate::sdk::bridge::runtime_err;
use crate::sdk::invoke::run_resolved_blocking_with_timeout;
/// Default per-interceptor wall-clock timeout when a marker sets no `timeout_ms`
/// (§9.4 M5). Read once from `PICLOUD_INTERCEPTOR_TIMEOUT_MS`; a runaway guard
/// (`loop {}`) is denied once this elapses rather than hanging the write path.
static DEFAULT_TIMEOUT_MS: LazyLock<u32> = LazyLock::new(|| {
std::env::var("PICLOUD_INTERCEPTOR_TIMEOUT_MS")
.ok()
.and_then(|s| s.parse::<u32>().ok())
.filter(|&v| v > 0)
.unwrap_or(5000)
});
/// Clone-cheap bundle of the §9.4 interceptor dependencies threaded into every
/// SDK service `register` fn (M1). Carries the resolver, the `invoke()` re-entry
/// engine back-reference, and the sandbox limits — everything `run_before`
/// (and, from M3, `run_after`) needs. A few `Arc`s plus a small `Limits` copy.
#[derive(Clone)]
pub(crate) struct InterceptorCtx {
pub interceptors: Arc<dyn InterceptorService>,
pub self_engine: Option<Arc<Engine>>,
pub limits: Limits,
}
thread_local! {
/// Re-entrancy depth: `> 0` while an interceptor script (and anything it
/// synchronously invokes) is executing on this thread. The whole re-entry
/// runs synchronously in the caller's `spawn_blocking` thread (same model
/// as `invoke()`), so a thread-local is sufficient and correct.
static IN_INTERCEPTOR: Cell<u32> = const { Cell::new(0) };
/// Identity cycle guard (M2): the `script_id`s of the interceptors currently
/// on the synchronous run stack. A chain that would run an interceptor whose
/// script is already executing is a cycle — we DENY rather than recurse.
/// Precise (keyed by script id) and independent of the binary
/// `IN_INTERCEPTOR` counter, which suppresses interception of an
/// interceptor's OWN nested writes but does not track identity.
static VISITED: RefCell<Vec<ScriptId>> = const { RefCell::new(Vec::new()) };
/// §9.4 M6: per-execution resolve cache. Keyed by `(app_id, service, op)` —
/// the chain a `(service, op)` resolves to is constant for one app across an
/// execution tree, so the FIRST hooked-eligible op of that kind pays the one
/// chain query and every later op reuses it. The dominant win is the
/// un-hooked case: an empty chain is cached once, so N `kv::set`s in a script
/// with no interceptors issue ONE resolve, not N. Cleared at the outermost
/// execution boundary so a pooled thread never serves a stale (or foreign)
/// app's cache.
static RESOLVE_CACHE: RefCell<HashMap<(AppId, String, String), InterceptorChain>> =
RefCell::new(HashMap::new());
/// Scope nesting depth for the resolve cache (mirrors the emission budget):
/// `0` when no execution is active on this thread.
static CACHE_ACTIVE: Cell<u32> = const { Cell::new(0) };
}
/// RAII scope wrapping one `execute_ast` for the §9.4 M6 resolve cache.
/// Re-entrancy aware: the OUTERMOST scope on a thread clears the cache on entry
/// and on final exit; a nested synchronous re-entry (invoke / interceptor)
/// shares the same cache, so a whole execution tree resolves each `(service,
/// op)` at most once.
pub(crate) struct InterceptorCacheScope;
impl InterceptorCacheScope {
pub(crate) fn enter() -> Self {
CACHE_ACTIVE.with(|a| {
if a.get() == 0 {
RESOLVE_CACHE.with(|c| c.borrow_mut().clear());
}
a.set(a.get() + 1);
});
Self
}
}
impl Drop for InterceptorCacheScope {
fn drop(&mut self) {
CACHE_ACTIVE.with(|a| {
let n = a.get().saturating_sub(1);
a.set(n);
if n == 0 {
RESOLVE_CACHE.with(|c| c.borrow_mut().clear());
}
});
}
}
fn currently_in_interceptor() -> bool {
IN_INTERCEPTOR.with(|c| c.get() > 0)
}
/// RAII: increment the re-entrancy counter while an interceptor runs, decrement
/// on drop (including on the `?`/panic-unwind paths).
struct ReentryGuard;
impl ReentryGuard {
fn enter() -> Self {
IN_INTERCEPTOR.with(|c| c.set(c.get().saturating_add(1)));
Self
}
}
impl Drop for ReentryGuard {
fn drop(&mut self) {
IN_INTERCEPTOR.with(|c| c.set(c.get().saturating_sub(1)));
}
}
/// RAII for the identity cycle guard: push a `script_id` on enter, pop on drop
/// (including the `?`/panic-unwind paths), so the visited set exactly tracks the
/// interceptors on the current synchronous run stack.
struct CycleGuard;
impl CycleGuard {
fn enter(id: ScriptId) -> Self {
VISITED.with(|v| v.borrow_mut().push(id));
Self
}
fn contains(id: ScriptId) -> bool {
VISITED.with(|v| v.borrow().contains(&id))
}
}
impl Drop for CycleGuard {
fn drop(&mut self) {
VISITED.with(|v| {
v.borrow_mut().pop();
});
}
}
/// The parsed outcome of ONE allowing hook: the operation passes, optionally
/// carrying a `data` transform (M4) the caller writes instead of the original.
struct HookAllow {
/// `Some` iff the hook returned a `data` key alongside `allowed: true`.
data: Option<Json>,
}
/// Run the before-op interceptor CHAIN for `(service, op)` if any are
/// registered. Returns `Ok(Some(value))` when a before-hook rewrote the payload
/// via a `data` transform (M4) — the caller writes THAT instead of the original
/// — or `Ok(None)` to write the original unchanged (un-hooked, or every hook
/// allowed without transforming). An `Err` denies the operation (the caller
/// must NOT perform the write); the FIRST deny short-circuits. `value` is the
/// original payload (`None` for delete, which never transforms).
#[allow(clippy::too_many_arguments)]
pub(super) fn run_before(
ictx: &InterceptorCtx,
cx: &Arc<SdkCallCx>,
service: &'static str,
op: &'static str,
collection: &str,
key: &str,
value: Option<&Json>,
) -> Result<Option<Json>, Box<EvalAltResult>> {
// Re-entrancy break: writes performed BY an interceptor (or anything it
// invokes) are not themselves intercepted — otherwise an "allow + write"
// hook would recurse to the depth cap and deny the original op.
if currently_in_interceptor() {
return Ok(None);
}
let chain = resolve_chain(ictx, cx, service, op)?;
if chain.before.is_empty() {
return Ok(None);
}
// Thread the (possibly rewritten) value through the chain: each before-hook
// sees the prior hook's transform in `value`, and the write uses the final.
let mut current: Option<Json> = value.cloned();
let mut transformed = false;
for entry in &chain.before {
let payload = op_payload(service, op, collection, key, current.as_ref(), cx, None);
let outcome = run_one_hook(ictx, cx, entry, service, op, &payload)?;
// M4 data-transform: honored ONLY on an allowing hook (the verdict parse
// already enforced fail-closed). Delete has no value to rewrite.
if let Some(data) = outcome.data {
if value.is_some() {
enforce_transform_size(service, op, &data)?;
current = Some(data);
transformed = true;
}
}
}
Ok(transformed.then(|| current.unwrap_or(Json::Null)))
}
/// Run the after-op interceptor CHAIN (M3) once the write has committed.
/// `result` is the write's outcome (serialized), surfaced to the hook so it can
/// observe/audit. After-hooks CANNOT roll back — the write already happened; a
/// deny here surfaces as an operation error to the caller but the write
/// persists. Returns `Ok(())` when every after-hook allowed (or none exist), an
/// `Err` on the first deny.
#[allow(clippy::too_many_arguments)]
pub(super) fn run_after(
ictx: &InterceptorCtx,
cx: &Arc<SdkCallCx>,
service: &'static str,
op: &'static str,
collection: &str,
key: &str,
value: Option<&Json>,
result: Json,
) -> Result<(), Box<EvalAltResult>> {
if currently_in_interceptor() {
return Ok(());
}
let chain = resolve_chain(ictx, cx, service, op)?;
if chain.after.is_empty() {
return Ok(());
}
for entry in &chain.after {
let payload = op_payload(service, op, collection, key, value, cx, Some(&result));
// After-hooks observe/deny; a returned `data` is ignored (the write is
// already durable, so there is nothing to rewrite for the kv slice).
run_one_hook(ictx, cx, entry, service, op, &payload)?;
}
Ok(())
}
/// Resolve the before+after chain for `(service, op)` — every marker visible on
/// the calling app's chain, ordered per phase. Memoized per execution tree
/// (§9.4 M6): the FIRST call for a `(app_id, service, op)` pays the chain query;
/// later calls reuse the cached result. An `Err` (no tokio runtime, or a backend
/// failure) makes the caller fail closed rather than silently allow.
fn resolve_chain(
ictx: &InterceptorCtx,
cx: &Arc<SdkCallCx>,
service: &'static str,
op: &'static str,
) -> Result<InterceptorChain, Box<EvalAltResult>> {
let cache_key = (cx.app_id, service.to_string(), op.to_string());
if let Some(hit) = RESOLVE_CACHE.with(|c| c.borrow().get(&cache_key).cloned()) {
return Ok(hit);
}
let handle = TokioHandle::try_current()
.map_err(|e| runtime_err(&format!("{service} interceptor: no tokio runtime: {e}")))?;
let interceptors = ictx.interceptors.clone();
let resolve_cx = cx.clone();
let chain = handle
.block_on(async move { interceptors.resolve(&resolve_cx, service, op).await })
.map_err(|e| runtime_err(&format!("{service}::{op} interceptor resolve: {e}")))?;
RESOLVE_CACHE.with(|c| c.borrow_mut().insert(cache_key, chain.clone()));
Ok(chain)
}
/// Build the operation-context payload handed to a hook. `result` is `Some` only
/// for an after-hook (the write's outcome).
fn op_payload(
service: &str,
op: &str,
collection: &str,
key: &str,
value: Option<&Json>,
cx: &SdkCallCx,
result: Option<&Json>,
) -> Json {
let mut m = json!({
"service": service,
"action": op,
"collection": collection,
"key": key,
"value": value,
"caller_script_id": cx.script_id.to_string(),
"caller_execution_id": cx.execution_id.to_string(),
});
if let (Json::Object(obj), Some(r)) = (&mut m, result) {
obj.insert("result".to_string(), r.clone());
}
m
}
/// Run ONE chain entry: fail-closed on a dangling marker, an identity cycle, a
/// missing engine back-reference, or a depth-limit breach; otherwise run the
/// sealed script and parse its verdict. Returns the allow outcome (with any
/// `data` transform) or an `Err` denying the operation.
fn run_one_hook(
ictx: &InterceptorCtx,
cx: &Arc<SdkCallCx>,
entry: &InterceptorEntry,
service: &str,
op: &str,
payload: &Json,
) -> Result<HookAllow, Box<EvalAltResult>> {
let resolved = match entry {
// A guard is declared but its script is missing/disabled — fail closed.
InterceptorEntry::Dangling(name) => {
return Err(runtime_err(&format!(
"{service}::{op} denied: interceptor script `{name}` is missing or disabled"
)));
}
InterceptorEntry::Run(r) => r,
};
let script = &resolved.script;
// Identity cycle guard: if this interceptor's script is already on the run
// stack, running it again would recurse forever — DENY.
if CycleGuard::contains(script.script_id) {
return Err(runtime_err(&format!(
"{service}::{op} denied: interceptor cycle detected: `{}`",
script.name
)));
}
// A guard is registered but the engine back-reference isn't installed (a bare
// test engine without `set_self_weak`): we cannot run it, so DENY — a
// resolvable guard we can't execute must not fail open. Production always
// installs the back-ref.
let Some(self_engine) = ictx.self_engine.as_ref() else {
return Err(runtime_err(&format!(
"{service}::{op} denied: interceptor `{}` cannot run (engine back-reference not installed)",
script.name
)));
};
// Depth bound (shared with invoke / trigger fan-out).
if cx.trigger_depth + 1 > ictx.limits.trigger_depth_max {
return Err(runtime_err(&format!(
"{service}::{op} interceptor `{}`: depth limit exceeded (max {})",
script.name, ictx.limits.trigger_depth_max
)));
}
// Run the sealed script with the operation context as its request body, under
// the re-entrancy guard (its own writes bypass interception), the identity
// cycle guard (its own script id is on the stack), and a per-hook timeout
// (§9.4 M5 — the marker's `timeout_ms` or the env default, tightened to at
// most the caller's remaining deadline). A runaway guard is denied, not hung.
let timeout = Duration::from_millis(u64::from(
resolved.timeout_ms.unwrap_or(*DEFAULT_TIMEOUT_MS),
));
let ret = {
let _reentry = ReentryGuard::enter();
let _cycle = CycleGuard::enter(script.script_id);
run_resolved_blocking_with_timeout(
self_engine,
cx,
script,
payload.clone(),
&format!("{service}::{op} interceptor `{}`", script.name),
Some(timeout),
)?
};
// Fail-closed verdict: allow ONLY on an explicit `#{ allowed: true }`. A bare
// bool, a typo'd key, a non-bool `allowed`, a bare unit, or any other shape
// DENIES — a control whose job is to deny must not allow by omission.
match ret {
Json::Object(mut m) if m.get("allowed") == Some(&Json::Bool(true)) => Ok(HookAllow {
data: m.remove("data"),
}),
Json::Object(m) => {
let reason = m
.get("reason")
.and_then(Json::as_str)
.unwrap_or("denied by interceptor");
Err(runtime_err(&format!(
"{service}::{op} denied by interceptor `{}`: {reason}",
script.name
)))
}
_ => Err(runtime_err(&format!(
"{service}::{op} denied by interceptor `{}`: no allow verdict returned",
script.name
))),
}
}
/// Cap a `data` transform at the same byte ceiling the SDK uses for a Rhai→JSON
/// materialization, so an interceptor can't rewrite a small write into an
/// unbounded blob before it reaches the service's own value cap.
fn enforce_transform_size(service: &str, op: &str, data: &Json) -> Result<(), Box<EvalAltResult>> {
let len = serde_json::to_vec(data)
.map(|v| v.len())
.unwrap_or(usize::MAX);
if len > crate::sdk::bridge::MAX_JSON_MATERIALIZE_BYTES {
return Err(runtime_err(&format!(
"{service}::{op} interceptor transform too large ({len} bytes exceeds the {}-byte limit)",
crate::sdk::bridge::MAX_JSON_MATERIALIZE_BYTES
)));
}
Ok(())
}

View File

@@ -42,7 +42,7 @@ use tokio::runtime::Handle as TokioHandle;
use crate::engine::Engine;
use crate::sandbox::Limits;
use crate::sdk::bridge::{json_to_dynamic, MAX_JSON_MATERIALIZE_BYTES};
use crate::sdk::bridge::json_to_dynamic;
use crate::types::{ExecRequest, InvocationType};
pub(super) fn register(
@@ -85,9 +85,6 @@ pub(super) fn register(
module.set_native_fn(
"invoke_async",
move |target: Dynamic, args: Dynamic| -> Result<String, Box<EvalAltResult>> {
// Audit #2: count this durable emission against the per-execution
// fan-out ceiling before doing any work.
crate::sdk::emit_budget::charge_emission("invoke_async")?;
let target = parse_target(target)?;
let args_json = args_to_json(&args)?;
let svc = svc.clone();
@@ -166,69 +163,9 @@ fn invoke_blocking(
.into()
})?;
// The callee's return is the response `body` JSON. Convert back to
// Dynamic for the caller. Status code + headers are dropped; the
// function-call mental model is "return value", not HTTP response.
let body = run_resolved_blocking(self_engine, cx, &resolved, args_json, &target_label)?;
Ok(json_to_dynamic(body))
}
/// Synchronous same-engine re-entry: build the callee `ExecRequest` (inheriting
/// the caller's app/principal/root/depth+1 and the callee's lexical owner),
/// compile through the per-Engine AST cache (F-P-004), execute, and return the
/// response `body` JSON. Shared by `invoke()` and the §9.4 interceptor hook —
/// the caller is responsible for the depth check (both do it before resolving).
/// `label` prefixes any compile/execute error.
/// Run a resolved script through the `invoke()` re-entry core, inheriting the
/// caller's ambient execution deadline.
pub(super) fn run_resolved_blocking(
self_engine: &Arc<Engine>,
cx: &Arc<SdkCallCx>,
resolved: &picloud_shared::ResolvedScript,
body_json: Json,
label: &str,
) -> Result<Json, Box<EvalAltResult>> {
run_resolved_core(
self_engine,
cx,
resolved,
body_json,
label,
crate::engine::ambient_deadline(),
)
}
/// Like [`run_resolved_blocking`] but bounds the run by a per-hook `timeout`
/// (§9.4 M5). The effective deadline is `min(ambient, now + timeout)` — a hook
/// can only ever TIGHTEN, never extend, its caller's deadline.
pub(super) fn run_resolved_blocking_with_timeout(
self_engine: &Arc<Engine>,
cx: &Arc<SdkCallCx>,
resolved: &picloud_shared::ResolvedScript,
body_json: Json,
label: &str,
timeout: Option<std::time::Duration>,
) -> Result<Json, Box<EvalAltResult>> {
let ambient = crate::engine::ambient_deadline();
let own = timeout.map(|d| std::time::Instant::now() + d);
let effective = match (ambient, own) {
(Some(a), Some(o)) => Some(a.min(o)),
(a, None) => a,
(None, o) => o,
};
run_resolved_core(self_engine, cx, resolved, body_json, label, effective)
}
fn run_resolved_core(
self_engine: &Arc<Engine>,
cx: &Arc<SdkCallCx>,
resolved: &picloud_shared::ResolvedScript,
body_json: Json,
label: &str,
deadline: Option<std::time::Instant>,
) -> Result<Json, Box<EvalAltResult>> {
let execution_id = ExecutionId::new();
let req = ExecRequest {
execution_id: ExecutionId::new(),
execution_id,
request_id: cx.request_id,
script_id: resolved.script_id,
script_name: resolved.name.clone(),
@@ -236,7 +173,7 @@ fn run_resolved_core(
path: "/invoke".into(),
method: String::new(),
headers: BTreeMap::new(),
body: body_json,
body: args_json,
params: BTreeMap::new(),
query: BTreeMap::new(),
rest: String::new(),
@@ -246,24 +183,42 @@ fn run_resolved_core(
// script's imports resolve from the group even when invoked by an
// app. `None` falls back to `App(cx.app_id)` in the engine.
script_owner: resolved.owner,
// Same-app re-entry is not a re-auth boundary — inherit the principal.
// Same-app invoke is a function call, not a re-auth boundary —
// inherit the caller's principal.
principal: cx.principal.clone(),
trigger_depth: cx.trigger_depth + 1,
root_execution_id: cx.root_execution_id,
is_dead_letter_handler: cx.is_dead_letter_handler,
event: None,
};
// F-P-004: synchronous re-entry — route through the per-Engine
// AST cache so each callee parses once per (script_id, updated_at),
// not once per invoke. Composed workflows multiply parse cost by
// depth; the cache cuts that to constant compile + N executions.
let ast = self_engine
.compile_for_identity(resolved.script_id, resolved.updated_at, &resolved.source)
.map_err(|e| -> Box<EvalAltResult> {
EvalAltResult::ErrorRuntime(format!("{label}: {e}").into(), rhai::Position::NONE).into()
EvalAltResult::ErrorRuntime(
format!("invoke({target_label}): {e}").into(),
rhai::Position::NONE,
)
.into()
})?;
let resp = self_engine
.execute_ast_with_deadline(&ast, req, deadline)
.execute_ast(&ast, req)
.map_err(|e| -> Box<EvalAltResult> {
EvalAltResult::ErrorRuntime(format!("{label}: {e}").into(), rhai::Position::NONE).into()
EvalAltResult::ErrorRuntime(
format!("invoke({target_label}): {e}").into(),
rhai::Position::NONE,
)
.into()
})?;
Ok(resp.body)
// The callee's return is the response `body` JSON. Convert back to
// Dynamic for the caller. Status code + headers are dropped; the
// function-call mental model is "return value", not HTTP response.
Ok(json_to_dynamic(resp.body))
}
/// Accept a string (route path OR script name) or a Rhai script-id
@@ -292,35 +247,8 @@ fn parse_target(target: Dynamic) -> Result<InvokeTarget, Box<EvalAltResult>> {
}
/// Convert Rhai Dynamic → JSON. Rejects FnPtr at any depth (closures
/// don't survive invoke boundaries) and is bounded by
/// [`MAX_JSON_MATERIALIZE_BYTES`] so a cheaply-aliased huge args value can't OOM
/// the node on materialization (same anti-OOM rail as `dynamic_to_json_capped`).
/// don't survive invoke boundaries).
fn args_to_json(value: &Dynamic) -> Result<Json, Box<EvalAltResult>> {
let mut remaining = MAX_JSON_MATERIALIZE_BYTES;
args_to_json_bounded(value, &mut remaining)
}
fn args_charge(remaining: &mut usize, n: usize) -> Result<(), Box<EvalAltResult>> {
match remaining.checked_sub(n) {
Some(left) => {
*remaining = left;
Ok(())
}
None => Err(EvalAltResult::ErrorRuntime(
format!(
"invoke: args too large to serialize (exceeds the {MAX_JSON_MATERIALIZE_BYTES}-byte limit)"
)
.into(),
rhai::Position::NONE,
)
.into()),
}
}
fn args_to_json_bounded(
value: &Dynamic,
remaining: &mut usize,
) -> Result<Json, Box<EvalAltResult>> {
if value.is::<rhai::FnPtr>() {
return Err(EvalAltResult::ErrorRuntime(
"invoke: args must not contain FnPtr / closures".into(),
@@ -330,52 +258,40 @@ fn args_to_json_bounded(
}
if value.is_blob() {
let blob = value.clone().into_blob().unwrap_or_default();
let encoded = STANDARD.encode(&blob);
args_charge(remaining, encoded.len() + 2)?;
return Ok(Json::String(encoded));
return Ok(Json::String(STANDARD.encode(&blob)));
}
if value.is_unit() {
args_charge(remaining, 4)?;
return Ok(Json::Null);
}
if let Ok(b) = value.as_bool() {
args_charge(remaining, 5)?;
return Ok(Json::Bool(b));
}
if let Ok(i) = value.as_int() {
args_charge(remaining, 8)?;
return Ok(Json::Number(i.into()));
}
if let Ok(f) = value.as_float() {
args_charge(remaining, 8)?;
return Ok(serde_json::Number::from_f64(f).map_or(Json::Null, Json::Number));
}
if value.is_string() {
let s = value.clone().into_string().unwrap_or_default();
args_charge(remaining, s.len() + 2)?;
return Ok(Json::String(s));
return Ok(Json::String(
value.clone().into_string().unwrap_or_default(),
));
}
if let Some(arr) = value.clone().try_cast::<Array>() {
args_charge(remaining, 2)?;
let mut out = Vec::with_capacity(arr.len().min(1024));
let mut out = Vec::with_capacity(arr.len());
for v in &arr {
args_charge(remaining, 1)?;
out.push(args_to_json_bounded(v, remaining)?);
out.push(args_to_json(v)?);
}
return Ok(Json::Array(out));
}
if let Some(map) = value.clone().try_cast::<Map>() {
args_charge(remaining, 2)?;
let mut out = serde_json::Map::new();
for (k, v) in map {
args_charge(remaining, k.len() + 4)?;
out.insert(k.to_string(), args_to_json_bounded(&v, remaining)?);
out.insert(k.to_string(), args_to_json(&v)?);
}
return Ok(Json::Object(out));
}
let s = value.to_string();
args_charge(remaining, s.len() + 2)?;
Ok(Json::String(s))
Ok(Json::String(value.to_string()))
}
/// `Limits` is `Copy`; passed by value into `register` so closures take

View File

@@ -5,8 +5,6 @@
//! widgets.set("k", #{ n: 1 });
//! let v = widgets.get("k"); // value or () if absent
//! if widgets.has("k") { ... }
//! widgets.set_if("k", (), #{ n: 0 }); // insert only if ABSENT -> bool
//! widgets.set_if("k", old, new); // swap only if current == old -> bool
//! widgets.delete("k"); // bool (was-present)
//! let page = widgets.list(); // returns #{ keys: [...], next_cursor: () }
//! ```
@@ -30,57 +28,29 @@
use std::sync::Arc;
use picloud_shared::{GroupKvService, KvService, SdkCallCx, Services};
use picloud_shared::{KvService, SdkCallCx, Services};
use rhai::{Array, Dynamic, Engine as RhaiEngine, EvalAltResult, Map, Module};
use super::bridge::{
block_on, dynamic_to_json_capped, json_to_dynamic, JsonSizeError, MAX_JSON_MATERIALIZE_BYTES,
};
use super::interceptor::InterceptorCtx;
use super::bridge::{block_on, dynamic_to_json, json_to_dynamic};
/// Per-call handle captured by the Rhai SDK. Cheap to clone (a few Arcs plus an
/// owned string). Carries the §9.4 interceptor ctx so `set`/`delete` can run a
/// before-op allow/deny hook (the resolver + the `invoke()` re-entry engine).
/// Per-call handle captured by the Rhai SDK. Cheap to clone (two Arcs
/// plus an owned string).
#[derive(Clone)]
pub struct KvHandle {
collection: String,
service: Arc<dyn KvService>,
cx: Arc<SdkCallCx>,
ictx: InterceptorCtx,
}
/// §11.6 shared-collection handle, returned by `kv::shared_collection(name)`. A distinct
/// Rhai type from `KvHandle` so the method set can diverge (e.g. shared
/// collections never grow triggers) and a script's choice of private-vs-shared
/// scope is explicit. Routes through the `GroupKvService`, which resolves the
/// owning group from `cx.app_id` (the script never names a group). Carries the
/// §9.4 interceptor ctx too so a `(kv, set/delete)` guard covers shared-
/// collection writes — otherwise the shared handle would be a silent bypass.
#[derive(Clone)]
pub struct GroupKvHandle {
collection: String,
service: Arc<dyn GroupKvService>,
cx: Arc<SdkCallCx>,
ictx: InterceptorCtx,
}
pub(super) fn register(
engine: &mut RhaiEngine,
services: &Services,
cx: Arc<SdkCallCx>,
ictx: InterceptorCtx,
) {
pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<SdkCallCx>) {
let kv_service = services.kv.clone();
let group_kv_service = services.group_kv.clone();
// `kv::collection(name)` / `kv::shared_collection(name)` — both constructors live in
// the `kv` static module so the script-visible calls are `kv::collection`
// and `kv::shared_collection` (`shared` alone is a Rhai reserved word).
// `kv::collection(name)` — handle constructor lives in the `kv`
// static module so the script-visible call is `kv::collection(...)`.
let mut module = Module::new();
{
let kv_service = kv_service.clone();
let cx = cx.clone();
let ictx = ictx.clone();
module.set_native_fn(
"collection",
move |name: &str| -> Result<KvHandle, Box<EvalAltResult>> {
@@ -91,26 +61,6 @@ pub(super) fn register(
collection: name.to_string(),
service: kv_service.clone(),
cx: cx.clone(),
ictx: ictx.clone(),
})
},
);
}
{
let group_kv_service = group_kv_service.clone();
let cx = cx.clone();
let ictx = ictx.clone();
module.set_native_fn(
"shared_collection",
move |name: &str| -> Result<GroupKvHandle, Box<EvalAltResult>> {
if name.is_empty() {
return Err("kv::shared_collection name must not be empty".into());
}
Ok(GroupKvHandle {
collection: name.to_string(),
service: group_kv_service.clone(),
cx: cx.clone(),
ictx: ictx.clone(),
})
},
);
@@ -124,34 +74,9 @@ pub(super) fn register(
register_get(engine);
register_set(engine);
register_set_if(engine);
register_has(engine);
register_delete(engine);
register_list(engine);
// Same method names on GroupKvHandle — Rhai dispatches by receiver type.
engine.register_type_with_name::<GroupKvHandle>("GroupKvHandle");
register_group_get(engine);
register_group_set(engine);
register_group_set_if(engine);
register_group_has(engine);
register_group_delete(engine);
register_group_list(engine);
}
/// Map a Rhai `expected` argument to the CAS precondition: Rhai unit `()` means
/// "expected ABSENT" (insert-if-absent); any other value is the expected current
/// value.
fn expected_from_dynamic(expected: &Dynamic) -> Result<Option<serde_json::Value>, JsonSizeError> {
if expected.is_unit() {
Ok(None)
} else {
Ok(Some(dynamic_to_json_capped(
expected,
MAX_JSON_MATERIALIZE_BYTES,
)?))
}
}
fn register_get(engine: &mut RhaiEngine) {
@@ -171,82 +96,11 @@ fn register_set(engine: &mut RhaiEngine) {
engine.register_fn(
"set",
|handle: &mut KvHandle, key: &str, value: Dynamic| -> Result<(), Box<EvalAltResult>> {
let json = dynamic_to_json_capped(&value, MAX_JSON_MATERIALIZE_BYTES)?;
// §9.4 before-op interceptor (allow/deny + M4 data-transform). A
// denial errors here; a returned `data` rewrites the written value.
let write_val = super::interceptor::run_before(
&handle.ictx,
&handle.cx,
"kv",
"set",
&handle.collection,
key,
Some(&json),
)?
.unwrap_or(json);
let written = write_val.clone();
let h = handle.clone();
let json = dynamic_to_json(&value);
block_on("kv", async move {
h.service.set(&h.cx, &h.collection, key, write_val).await
})?;
// §9.4 M3 after-hook (observe/audit; the write already committed).
super::interceptor::run_after(
&handle.ictx,
&handle.cx,
"kv",
"set",
&handle.collection,
key,
Some(&written),
serde_json::Value::Null,
)
},
);
}
fn register_set_if(engine: &mut RhaiEngine) {
// `handle.set_if(key, expected, new)` — CAS. `expected = ()` means "only if
// absent". Returns `true` if the swap happened, `false` if the precondition
// failed.
engine.register_fn(
"set_if",
|handle: &mut KvHandle,
key: &str,
expected: Dynamic,
new: Dynamic|
-> Result<bool, Box<EvalAltResult>> {
let exp = expected_from_dynamic(&expected)?;
let json = dynamic_to_json_capped(&new, MAX_JSON_MATERIALIZE_BYTES)?;
// §9.4 M12: CAS is a WRITE, so it runs the `(kv, set)` guard too —
// otherwise `set_if` would be a silent bypass of a `set` policy.
let write_val = super::interceptor::run_before(
&handle.ictx,
&handle.cx,
"kv",
"set",
&handle.collection,
key,
Some(&json),
)?
.unwrap_or(json);
let written = write_val.clone();
let h = handle.clone();
let swapped = block_on("kv", async move {
h.service
.set_if(&h.cx, &h.collection, key, exp, write_val)
.await
})?;
super::interceptor::run_after(
&handle.ictx,
&handle.cx,
"kv",
"set",
&handle.collection,
key,
Some(&written),
serde_json::Value::Bool(swapped),
)?;
Ok(swapped)
h.service.set(&h.cx, &h.collection, key, json).await
})
},
);
}
@@ -267,73 +121,14 @@ fn register_delete(engine: &mut RhaiEngine) {
engine.register_fn(
"delete",
|handle: &mut KvHandle, key: &str| -> Result<bool, Box<EvalAltResult>> {
// Delete carries no value, so the before-hook never transforms.
super::interceptor::run_before(
&handle.ictx,
&handle.cx,
"kv",
"delete",
&handle.collection,
key,
None,
)?;
let h = handle.clone();
let was_present = block_on("kv", async move {
block_on("kv", async move {
h.service.delete(&h.cx, &h.collection, key).await
})?;
super::interceptor::run_after(
&handle.ictx,
&handle.cx,
"kv",
"delete",
&handle.collection,
key,
None,
serde_json::Value::Bool(was_present),
)?;
Ok(was_present)
})
},
);
}
/// §9.4: run the before-op interceptor for a shared-collection write, so a
/// `(kv, set/delete)` guard covers the shared handle too (not just private KV).
fn group_run_before(
handle: &GroupKvHandle,
op: &'static str,
key: &str,
value: Option<&serde_json::Value>,
) -> Result<Option<serde_json::Value>, Box<EvalAltResult>> {
super::interceptor::run_before(
&handle.ictx,
&handle.cx,
"kv",
op,
&handle.collection,
key,
value,
)
}
fn group_run_after(
handle: &GroupKvHandle,
op: &'static str,
key: &str,
value: Option<&serde_json::Value>,
result: serde_json::Value,
) -> Result<(), Box<EvalAltResult>> {
super::interceptor::run_after(
&handle.ictx,
&handle.cx,
"kv",
op,
&handle.collection,
key,
value,
result,
)
}
fn register_list(engine: &mut RhaiEngine) {
// Zero-arg form — full page, no cursor.
engine.register_fn(
@@ -379,143 +174,3 @@ fn list_call(
);
Ok(m)
}
// --- GroupKvHandle methods (§11.6 shared collections) ----------------------
fn register_group_get(engine: &mut RhaiEngine) {
engine.register_fn(
"get",
|handle: &mut GroupKvHandle, key: &str| -> Result<Dynamic, Box<EvalAltResult>> {
let h = handle.clone();
block_on("kv", async move {
h.service.get(&h.cx, &h.collection, key).await
})
.map(|opt| opt.map_or(Dynamic::UNIT, json_to_dynamic))
},
);
}
fn register_group_set(engine: &mut RhaiEngine) {
engine.register_fn(
"set",
|handle: &mut GroupKvHandle, key: &str, value: Dynamic| -> Result<(), Box<EvalAltResult>> {
let json = dynamic_to_json_capped(&value, MAX_JSON_MATERIALIZE_BYTES)?;
// §9.4 before-op interceptor also guards shared-collection writes
// (allow/deny + M4 data-transform).
let write_val = group_run_before(handle, "set", key, Some(&json))?.unwrap_or(json);
let written = write_val.clone();
let h = handle.clone();
block_on("kv", async move {
h.service.set(&h.cx, &h.collection, key, write_val).await
})?;
group_run_after(handle, "set", key, Some(&written), serde_json::Value::Null)
},
);
}
fn register_group_set_if(engine: &mut RhaiEngine) {
engine.register_fn(
"set_if",
|handle: &mut GroupKvHandle,
key: &str,
expected: Dynamic,
new: Dynamic|
-> Result<bool, Box<EvalAltResult>> {
let exp = expected_from_dynamic(&expected)?;
let json = dynamic_to_json_capped(&new, MAX_JSON_MATERIALIZE_BYTES)?;
// §9.4 M12: CAS on a shared collection runs the `(kv, set)` guard too.
let write_val = group_run_before(handle, "set", key, Some(&json))?.unwrap_or(json);
let written = write_val.clone();
let h = handle.clone();
let swapped = block_on("kv", async move {
h.service
.set_if(&h.cx, &h.collection, key, exp, write_val)
.await
})?;
group_run_after(
handle,
"set",
key,
Some(&written),
serde_json::Value::Bool(swapped),
)?;
Ok(swapped)
},
);
}
fn register_group_has(engine: &mut RhaiEngine) {
engine.register_fn(
"has",
|handle: &mut GroupKvHandle, key: &str| -> Result<bool, Box<EvalAltResult>> {
let h = handle.clone();
block_on("kv", async move {
h.service.has(&h.cx, &h.collection, key).await
})
},
);
}
fn register_group_delete(engine: &mut RhaiEngine) {
engine.register_fn(
"delete",
|handle: &mut GroupKvHandle, key: &str| -> Result<bool, Box<EvalAltResult>> {
group_run_before(handle, "delete", key, None)?;
let h = handle.clone();
let was_present = block_on("kv", async move {
h.service.delete(&h.cx, &h.collection, key).await
})?;
group_run_after(
handle,
"delete",
key,
None,
serde_json::Value::Bool(was_present),
)?;
Ok(was_present)
},
);
}
fn register_group_list(engine: &mut RhaiEngine) {
engine.register_fn(
"list",
|handle: &mut GroupKvHandle| -> Result<Map, Box<EvalAltResult>> {
group_list_call(handle, None, 0)
},
);
engine.register_fn(
"list",
|handle: &mut GroupKvHandle, cursor: &str| -> Result<Map, Box<EvalAltResult>> {
group_list_call(handle, Some(cursor.to_string()), 0)
},
);
engine.register_fn(
"list",
|handle: &mut GroupKvHandle, cursor: &str, limit: i64| -> Result<Map, Box<EvalAltResult>> {
let limit = u32::try_from(limit.max(0)).unwrap_or(0);
group_list_call(handle, Some(cursor.to_string()), limit)
},
);
}
fn group_list_call(
handle: &GroupKvHandle,
cursor: Option<String>,
limit: u32,
) -> Result<Map, Box<EvalAltResult>> {
let h = handle.clone();
let page = block_on("kv", async move {
h.service
.list(&h.cx, &h.collection, cursor.as_deref(), limit)
.await
})?;
let mut m = Map::new();
let keys: Array = page.keys.into_iter().map(Dynamic::from).collect();
m.insert("keys".into(), keys.into());
m.insert(
"next_cursor".into(),
page.next_cursor.map_or(Dynamic::UNIT, Dynamic::from),
);
Ok(m)
}

View File

@@ -16,10 +16,8 @@ pub mod cx;
pub mod dead_letters;
pub mod docs;
pub mod email;
pub mod emit_budget;
pub mod files;
pub mod http;
pub mod interceptor;
pub mod invoke;
pub mod kv;
pub mod pubsub;
@@ -29,12 +27,8 @@ pub mod secrets;
pub mod stdlib;
pub mod users;
pub mod vars;
pub mod workflow;
pub use bridge::{
dynamic_to_json, dynamic_to_json_capped, json_to_dynamic, JsonSizeError,
MAX_JSON_MATERIALIZE_BYTES,
};
pub use bridge::{dynamic_to_json, json_to_dynamic};
pub use cx::SdkCallCx;
use std::sync::Arc;
@@ -60,26 +54,17 @@ pub fn register_all(
limits: Limits,
self_engine: Option<Arc<Engine>>,
) {
// §9.4 M1: build the interceptor ctx once and thread it into every SDK
// service that has (or will gain) a before/after hook. Only `kv` runs a
// hook today; docs/files/queue/pubsub/http carry it for M7M11.
let ictx = interceptor::InterceptorCtx {
interceptors: services.interceptors.clone(),
self_engine: self_engine.clone(),
limits,
};
kv::register(engine, services, cx.clone(), ictx.clone());
docs::register(engine, services, cx.clone(), ictx.clone());
kv::register(engine, services, cx.clone());
docs::register(engine, services, cx.clone());
dead_letters::register(engine, services, cx.clone());
http::register(engine, services, cx.clone(), ictx.clone());
files::register(engine, services, cx.clone(), ictx.clone());
pubsub::register(engine, services, cx.clone(), ictx.clone());
queue::register(engine, services, cx.clone(), ictx);
http::register(engine, services, cx.clone());
files::register(engine, services, cx.clone());
pubsub::register(engine, services, cx.clone());
queue::register(engine, services, cx.clone());
retry::register(engine, services, cx.clone());
secrets::register(engine, services, cx.clone());
vars::register(engine, services, cx.clone());
email::register(engine, services, cx.clone());
users::register(engine, services, cx.clone());
workflow::register(engine, services, cx.clone());
invoke::register(engine, services, cx, limits, self_engine);
}

View File

@@ -24,53 +24,23 @@ use rhai::{Array, Dynamic, Engine as RhaiEngine, EvalAltResult, Map, Module};
use serde_json::Value as Json;
use tokio::runtime::Handle as TokioHandle;
use super::bridge::{block_on, MAX_JSON_MATERIALIZE_BYTES};
use super::interceptor::InterceptorCtx;
use super::bridge::block_on;
pub(super) fn register(
engine: &mut RhaiEngine,
services: &Services,
cx: Arc<SdkCallCx>,
ictx: InterceptorCtx,
) {
pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<SdkCallCx>) {
let svc = services.pubsub.clone();
let mut module = Module::new();
{
let svc = svc.clone();
let cx = cx.clone();
let ictx = ictx.clone();
module.set_native_fn(
"publish_durable",
move |topic: &str, message: Dynamic| -> Result<(), Box<EvalAltResult>> {
crate::sdk::emit_budget::charge_emission("pubsub::publish_durable")?;
let json = message_to_json(&message)?;
// §9.4 before-op interceptor (allow/deny + M4 data-transform).
let write_val = super::interceptor::run_before(
&ictx,
&cx,
"pubsub",
"publish",
topic,
"",
Some(&json),
)?
.unwrap_or(json);
let written = write_val.clone();
let svc2 = svc.clone();
let cx2 = cx.clone();
let json = message_to_json(&message);
let svc = svc.clone();
let cx = cx.clone();
block_on("pubsub", async move {
svc2.publish_durable(&cx2, topic, write_val).await
})?;
super::interceptor::run_after(
&ictx,
&cx,
"pubsub",
"publish",
topic,
"",
Some(&written),
Json::Null,
)
svc.publish_durable(&cx, topic, json).await
})
},
);
}
@@ -99,101 +69,7 @@ pub(super) fn register(
},
);
}
// §11.6 D2: `pubsub::shared_topic("name")` → a handle whose `.publish(...)`
// publishes to the group's shared topic namespace (fans out to `shared`
// group pubsub triggers on the owning group). The owning group resolves from
// `cx.app_id` inside the service — never a script arg.
{
let group_svc = services.group_pubsub.clone();
let cx = cx.clone();
let ictx = ictx.clone();
module.set_native_fn(
"shared_topic",
move |name: &str| -> Result<GroupTopicHandle, Box<EvalAltResult>> {
if name.is_empty() {
return Err("pubsub::shared_topic name must not be empty".into());
}
Ok(GroupTopicHandle {
namespace: name.to_string(),
service: group_svc.clone(),
cx: cx.clone(),
ictx: ictx.clone(),
})
},
);
}
engine.register_static_module("pubsub", module.into());
engine.register_type_with_name::<GroupTopicHandle>("GroupTopicHandle");
register_shared_publish(engine);
}
/// §11.6 D2 handle returned by `pubsub::shared_topic("name")`. `.publish(sub,
/// msg)` publishes `name.sub` to the group shared topic; `.publish(msg)`
/// publishes to `name` directly.
#[derive(Clone)]
pub struct GroupTopicHandle {
namespace: String,
service: Arc<dyn picloud_shared::GroupPubsubService>,
cx: Arc<SdkCallCx>,
// §9.4: carried so shared `publish` runs the before/after hook too (M11).
ictx: InterceptorCtx,
}
fn shared_publish(
h: &mut GroupTopicHandle,
subtopic: &str,
message: Dynamic,
) -> Result<(), Box<EvalAltResult>> {
crate::sdk::emit_budget::charge_emission("pubsub::shared_topic")?;
let json = message_to_json(&message)?;
// §9.4 before-op interceptor also guards shared-topic publishes. The
// collection is the namespace; the subtopic is the key.
let write_val = super::interceptor::run_before(
&h.ictx,
&h.cx,
"pubsub",
"publish",
&h.namespace,
subtopic,
Some(&json),
)?
.unwrap_or(json);
let written = write_val.clone();
let service = h.service.clone();
let cx = h.cx.clone();
let namespace = h.namespace.clone();
let subtopic_owned = subtopic.to_string();
block_on("pubsub", async move {
service
.publish(&cx, &namespace, &subtopic_owned, write_val)
.await
})
// The fan-out count isn't surfaced to scripts (fire-and-forget, like the
// per-app publish).
.map(|_n: u32| ())?;
super::interceptor::run_after(
&h.ictx,
&h.cx,
"pubsub",
"publish",
&h.namespace,
subtopic,
Some(&written),
Json::Null,
)
}
fn register_shared_publish(engine: &mut RhaiEngine) {
engine.register_fn(
"publish",
|h: &mut GroupTopicHandle, subtopic: &str, message: Dynamic| {
shared_publish(h, subtopic, message)
},
);
// `.publish(msg)` with no subtopic → publish to the namespace directly.
engine.register_fn("publish", |h: &mut GroupTopicHandle, message: Dynamic| {
shared_publish(h, "", message)
});
}
/// Interpret the optional `ttl` argument: `()` → use the default,
@@ -249,81 +125,38 @@ fn mint_token(
/// Convert a Rhai `Dynamic` message into JSON, base64-encoding any
/// `Blob` (at any nesting depth). Mirrors `bridge::dynamic_to_json` but
/// adds the blob arm the pub/sub wire contract requires. Bounded by
/// [`MAX_JSON_MATERIALIZE_BYTES`] so a cheaply-aliased huge message can't OOM
/// the node on materialization (the per-message wire cap is enforced later, on
/// the already-materialized value).
fn message_to_json(value: &Dynamic) -> Result<Json, Box<EvalAltResult>> {
let mut remaining = MAX_JSON_MATERIALIZE_BYTES;
message_to_json_bounded(value, &mut remaining)
}
fn msg_charge(remaining: &mut usize, n: usize) -> Result<(), Box<EvalAltResult>> {
match remaining.checked_sub(n) {
Some(left) => {
*remaining = left;
Ok(())
}
None => Err(EvalAltResult::ErrorRuntime(
format!("pubsub: message too large to serialize (exceeds the {MAX_JSON_MATERIALIZE_BYTES}-byte limit)").into(),
rhai::Position::NONE,
)
.into()),
}
}
fn message_to_json_bounded(
value: &Dynamic,
remaining: &mut usize,
) -> Result<Json, Box<EvalAltResult>> {
/// adds the blob arm the pub/sub wire contract requires.
fn message_to_json(value: &Dynamic) -> Json {
// Blob must be checked before the generic array path (a Blob is a
// `Vec<u8>`, distinct from a Rhai `Array`).
if value.is_blob() {
let blob = value.clone().into_blob().unwrap_or_default();
let encoded = STANDARD.encode(&blob);
msg_charge(remaining, encoded.len() + 2)?;
return Ok(Json::String(encoded));
return Json::String(STANDARD.encode(&blob));
}
if value.is_unit() {
msg_charge(remaining, 4)?;
return Ok(Json::Null);
return Json::Null;
}
if let Ok(b) = value.as_bool() {
msg_charge(remaining, 5)?;
return Ok(Json::Bool(b));
return Json::Bool(b);
}
if let Ok(i) = value.as_int() {
msg_charge(remaining, 8)?;
return Ok(Json::Number(i.into()));
return Json::Number(i.into());
}
if let Ok(f) = value.as_float() {
msg_charge(remaining, 8)?;
return Ok(serde_json::Number::from_f64(f).map_or(Json::Null, Json::Number));
return serde_json::Number::from_f64(f).map_or(Json::Null, Json::Number);
}
if value.is_string() {
let s = value.clone().into_string().unwrap_or_default();
msg_charge(remaining, s.len() + 2)?;
return Ok(Json::String(s));
return Json::String(value.clone().into_string().unwrap_or_default());
}
if let Some(arr) = value.clone().try_cast::<Array>() {
msg_charge(remaining, 2)?;
let mut out = Vec::with_capacity(arr.len().min(1024));
for v in &arr {
msg_charge(remaining, 1)?;
out.push(message_to_json_bounded(v, remaining)?);
}
return Ok(Json::Array(out));
return Json::Array(arr.iter().map(message_to_json).collect());
}
if let Some(map) = value.clone().try_cast::<Map>() {
msg_charge(remaining, 2)?;
let mut out = serde_json::Map::new();
for (k, v) in map {
msg_charge(remaining, k.len() + 4)?;
out.insert(k.to_string(), message_to_json_bounded(&v, remaining)?);
out.insert(k.to_string(), message_to_json(&v));
}
return Ok(Json::Object(out));
return Json::Object(out);
}
let s = value.to_string();
msg_charge(remaining, s.len() + 2)?;
Ok(Json::String(s))
Json::String(value.to_string())
}

View File

@@ -20,22 +20,12 @@ use std::sync::Arc;
use base64::engine::general_purpose::STANDARD;
use base64::Engine as _;
use picloud_shared::{
EnqueueOpts, GroupQueueService, QueueError, QueueService, SdkCallCx, Services,
};
use picloud_shared::{EnqueueOpts, QueueError, QueueService, SdkCallCx, Services};
use rhai::{Array, Dynamic, Engine as RhaiEngine, EvalAltResult, Map, Module};
use serde_json::Value as Json;
use tokio::runtime::Handle as TokioHandle;
use super::bridge::MAX_JSON_MATERIALIZE_BYTES;
use super::interceptor::InterceptorCtx;
pub(super) fn register(
engine: &mut RhaiEngine,
services: &Services,
cx: Arc<SdkCallCx>,
ictx: InterceptorCtx,
) {
pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<SdkCallCx>) {
let svc = services.queue.clone();
let mut module = Module::new();
@@ -43,11 +33,11 @@ pub(super) fn register(
{
let svc = svc.clone();
let cx = cx.clone();
let ictx = ictx.clone();
module.set_native_fn(
"enqueue",
move |name: &str, message: Dynamic| -> Result<(), Box<EvalAltResult>> {
app_enqueue(&ictx, &svc, &cx, name, &message, EnqueueOpts::default())
let json = message_to_json(&message)?;
enqueue_blocking(&svc, &cx, name, json, EnqueueOpts::default())
},
);
}
@@ -57,12 +47,12 @@ pub(super) fn register(
{
let svc = svc.clone();
let cx = cx.clone();
let ictx = ictx.clone();
module.set_native_fn(
"enqueue",
move |name: &str, message: Dynamic, opts: Map| -> Result<(), Box<EvalAltResult>> {
let json = message_to_json(&message)?;
let opts = parse_opts(&opts)?;
app_enqueue(&ictx, &svc, &cx, name, &message, opts)
enqueue_blocking(&svc, &cx, name, json, opts)
},
);
}
@@ -97,176 +87,18 @@ pub(super) fn register(
);
}
// §11.6 D3: `queue::shared_collection("name")` → a handle over the group
// shared queue (any subtree app enqueues into one group-owned store;
// competing per-descendant consumers drain it). The owning group resolves
// from `cx.app_id` inside the service — never a script arg.
{
let group_svc = services.group_queue.clone();
let cx = cx.clone();
let ictx = ictx.clone();
module.set_native_fn(
"shared_collection",
move |name: &str| -> Result<GroupQueueHandle, Box<EvalAltResult>> {
if name.is_empty() {
return Err("queue::shared_collection name must not be empty".into());
}
Ok(GroupQueueHandle {
collection: name.to_string(),
service: group_svc.clone(),
cx: cx.clone(),
ictx: ictx.clone(),
})
},
);
}
engine.register_static_module("queue", module.into());
engine.register_type_with_name::<GroupQueueHandle>("GroupQueueHandle");
register_shared_queue_methods(engine);
}
/// §11.6 D3 handle returned by `queue::shared_collection("name")`.
#[derive(Clone)]
pub struct GroupQueueHandle {
collection: String,
service: Arc<dyn GroupQueueService>,
cx: Arc<SdkCallCx>,
// §9.4: carried so shared `enqueue` runs the before/after hook too (M10).
ictx: InterceptorCtx,
}
/// Per-app enqueue with the §9.4 interceptor around it: charge the emission
/// budget, run the before-hook (allow/deny + M4 data-transform), enqueue the
/// (possibly rewritten) message, then run the after-hook with the new message
/// id as the result. `key` is `""` (a queue has no key). `collection` = the
/// queue name.
fn app_enqueue(
ictx: &InterceptorCtx,
svc: &Arc<dyn QueueService>,
cx: &Arc<SdkCallCx>,
name: &str,
message: &Dynamic,
opts: EnqueueOpts,
) -> Result<(), Box<EvalAltResult>> {
crate::sdk::emit_budget::charge_emission("queue::enqueue")?;
let json = message_to_json(message)?;
let write_val =
super::interceptor::run_before(ictx, cx, "queue", "enqueue", name, "", Some(&json))?
.unwrap_or(json);
let written = write_val.clone();
let id = enqueue_blocking(svc, cx, name, write_val, opts)?;
super::interceptor::run_after(
ictx,
cx,
"queue",
"enqueue",
name,
"",
Some(&written),
Json::String(id.0.to_string()),
)
}
fn group_enqueue(
h: &mut GroupQueueHandle,
message: Dynamic,
opts: EnqueueOpts,
) -> Result<(), Box<EvalAltResult>> {
crate::sdk::emit_budget::charge_emission("queue::shared_collection")?;
let json = message_to_json(&message)?;
// §9.4 before-op interceptor also guards shared-queue enqueues.
let write_val = super::interceptor::run_before(
&h.ictx,
&h.cx,
"queue",
"enqueue",
&h.collection,
"",
Some(&json),
)?
.unwrap_or(json);
let written = write_val.clone();
let handle = TokioHandle::try_current().map_err(|e| -> Box<EvalAltResult> {
EvalAltResult::ErrorRuntime(
format!("queue: no tokio runtime available: {e}").into(),
rhai::Position::NONE,
)
.into()
})?;
let service = h.service.clone();
let cx = h.cx.clone();
let collection = h.collection.clone();
let id = handle
.block_on(async move { service.enqueue(&cx, &collection, write_val, opts).await })
.map_err(|err| -> Box<EvalAltResult> {
EvalAltResult::ErrorRuntime(format!("queue: {err}").into(), rhai::Position::NONE).into()
})?;
super::interceptor::run_after(
&h.ictx,
&h.cx,
"queue",
"enqueue",
&h.collection,
"",
Some(&written),
Json::String(id.0.to_string()),
)
}
fn group_depth<F, Fut>(h: &mut GroupQueueHandle, f: F) -> Result<i64, Box<EvalAltResult>>
where
F: FnOnce(Arc<dyn GroupQueueService>, Arc<SdkCallCx>, String) -> Fut,
Fut: std::future::Future<Output = Result<u64, picloud_shared::GroupQueueError>>,
{
let handle = TokioHandle::try_current().map_err(|e| -> Box<EvalAltResult> {
EvalAltResult::ErrorRuntime(
format!("queue: no tokio runtime available: {e}").into(),
rhai::Position::NONE,
)
.into()
})?;
let fut = f(h.service.clone(), h.cx.clone(), h.collection.clone());
let n = handle.block_on(fut).map_err(|err| -> Box<EvalAltResult> {
EvalAltResult::ErrorRuntime(format!("queue: {err}").into(), rhai::Position::NONE).into()
})?;
Ok(i64::try_from(n).unwrap_or(i64::MAX))
}
fn register_shared_queue_methods(engine: &mut RhaiEngine) {
engine.register_fn("enqueue", |h: &mut GroupQueueHandle, message: Dynamic| {
group_enqueue(h, message, EnqueueOpts::default())
});
engine.register_fn(
"enqueue",
|h: &mut GroupQueueHandle, message: Dynamic, opts: Map| {
let opts = parse_opts(&opts)?;
group_enqueue(h, message, opts)
},
);
engine.register_fn("depth", |h: &mut GroupQueueHandle| {
group_depth(
h,
|svc, cx, coll| async move { svc.depth(&cx, &coll).await },
)
});
engine.register_fn("depth_pending", |h: &mut GroupQueueHandle| {
group_depth(h, |svc, cx, coll| async move {
svc.depth_pending(&cx, &coll).await
})
});
}
/// Run the async enqueue inside the synchronous Rhai context. Mirrors
/// `pubsub::block_on` but for the queue's `Result<_, QueueError>`. Returns the
/// new message id (surfaced to the §9.4 after-hook, not to the script).
/// `pubsub::block_on` but for the queue's `Result<_, QueueError>`.
fn enqueue_blocking(
svc: &Arc<dyn QueueService>,
cx: &Arc<SdkCallCx>,
name: &str,
payload: Json,
opts: EnqueueOpts,
) -> Result<picloud_shared::QueueMessageId, Box<EvalAltResult>> {
) -> Result<(), Box<EvalAltResult>> {
let handle = TokioHandle::try_current().map_err(|e| -> Box<EvalAltResult> {
EvalAltResult::ErrorRuntime(
format!("queue: no tokio runtime available: {e}").into(),
@@ -279,6 +111,7 @@ fn enqueue_blocking(
let name = name.to_string();
handle
.block_on(async move { svc.enqueue(&cx, &name, payload, opts).await })
.map(|_id| ())
.map_err(|err| -> Box<EvalAltResult> {
EvalAltResult::ErrorRuntime(format!("queue: {err}").into(), rhai::Position::NONE).into()
})
@@ -328,33 +161,8 @@ fn parse_opts(opts: &Map) -> Result<EnqueueOpts, Box<EvalAltResult>> {
/// Convert a Rhai `Dynamic` into JSON. Mirrors `pubsub::message_to_json`
/// — blobs become base64, FnPtr / closures are rejected so a script
/// can't accidentally enqueue something that won't survive a trip
/// through Postgres + back through the bridge. Bounded by
/// [`MAX_JSON_MATERIALIZE_BYTES`] so a cheaply-aliased huge message can't OOM
/// the node on materialization (the per-message payload cap is enforced later,
/// on the already-materialized value).
/// through Postgres + back through the bridge.
fn message_to_json(value: &Dynamic) -> Result<Json, Box<EvalAltResult>> {
let mut remaining = MAX_JSON_MATERIALIZE_BYTES;
message_to_json_bounded(value, &mut remaining)
}
fn msg_charge(remaining: &mut usize, n: usize) -> Result<(), Box<EvalAltResult>> {
match remaining.checked_sub(n) {
Some(left) => {
*remaining = left;
Ok(())
}
None => Err(EvalAltResult::ErrorRuntime(
format!("queue::enqueue: message too large to serialize (exceeds the {MAX_JSON_MATERIALIZE_BYTES}-byte limit)").into(),
rhai::Position::NONE,
)
.into()),
}
}
fn message_to_json_bounded(
value: &Dynamic,
remaining: &mut usize,
) -> Result<Json, Box<EvalAltResult>> {
if value.is::<rhai::FnPtr>() {
return Err(EvalAltResult::ErrorRuntime(
"queue::enqueue: messages must not contain FnPtr / closures".into(),
@@ -364,52 +172,40 @@ fn message_to_json_bounded(
}
if value.is_blob() {
let blob = value.clone().into_blob().unwrap_or_default();
let encoded = STANDARD.encode(&blob);
msg_charge(remaining, encoded.len() + 2)?;
return Ok(Json::String(encoded));
return Ok(Json::String(STANDARD.encode(&blob)));
}
if value.is_unit() {
msg_charge(remaining, 4)?;
return Ok(Json::Null);
}
if let Ok(b) = value.as_bool() {
msg_charge(remaining, 5)?;
return Ok(Json::Bool(b));
}
if let Ok(i) = value.as_int() {
msg_charge(remaining, 8)?;
return Ok(Json::Number(i.into()));
}
if let Ok(f) = value.as_float() {
msg_charge(remaining, 8)?;
return Ok(serde_json::Number::from_f64(f).map_or(Json::Null, Json::Number));
}
if value.is_string() {
let s = value.clone().into_string().unwrap_or_default();
msg_charge(remaining, s.len() + 2)?;
return Ok(Json::String(s));
return Ok(Json::String(
value.clone().into_string().unwrap_or_default(),
));
}
if let Some(arr) = value.clone().try_cast::<Array>() {
msg_charge(remaining, 2)?;
let mut out = Vec::with_capacity(arr.len().min(1024));
let mut out = Vec::with_capacity(arr.len());
for v in &arr {
msg_charge(remaining, 1)?;
out.push(message_to_json_bounded(v, remaining)?);
out.push(message_to_json(v)?);
}
return Ok(Json::Array(out));
}
if let Some(map) = value.clone().try_cast::<Map>() {
msg_charge(remaining, 2)?;
let mut out = serde_json::Map::new();
for (k, v) in map {
msg_charge(remaining, k.len() + 4)?;
out.insert(k.to_string(), message_to_json_bounded(&v, remaining)?);
out.insert(k.to_string(), message_to_json(&v)?);
}
return Ok(Json::Object(out));
}
let s = value.to_string();
msg_charge(remaining, s.len() + 2)?;
Ok(Json::String(s))
Ok(Json::String(value.to_string()))
}
#[cfg(test)]

View File

@@ -21,9 +21,7 @@ use std::sync::Arc;
use picloud_shared::{SdkCallCx, SecretsListPage, Services};
use rhai::{Array, Dynamic, Engine as RhaiEngine, EvalAltResult, Map, Module};
use super::bridge::{
block_on, dynamic_to_json_capped, json_to_dynamic, runtime_err, MAX_JSON_MATERIALIZE_BYTES,
};
use super::bridge::{block_on, dynamic_to_json, json_to_dynamic};
pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<SdkCallCx>) {
let svc = services.secrets.clone();
@@ -36,7 +34,7 @@ pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<Sdk
module.set_native_fn(
"set",
move |name: &str, value: Dynamic| -> Result<(), Box<EvalAltResult>> {
let json = dynamic_to_json_capped(&value, MAX_JSON_MATERIALIZE_BYTES)?;
let json = dynamic_to_json(&value);
let svc = svc.clone();
let cx = cx.clone();
block_on("secrets", async move { svc.set(&cx, name, json).await })
@@ -126,3 +124,10 @@ fn list_page_to_map(page: SecretsListPage) -> Map {
);
m
}
// Returns the boxed error directly because every caller needs a
// `Box<EvalAltResult>` (Rhai's error type), matching the other bridges.
#[allow(clippy::unnecessary_box_returns)]
fn runtime_err(msg: &str) -> Box<EvalAltResult> {
EvalAltResult::ErrorRuntime(msg.into(), rhai::Position::NONE).into()
}

View File

@@ -4,7 +4,7 @@
use rhai::{Dynamic, Engine as RhaiEngine, EvalAltResult, Module};
use crate::sdk::bridge::{dynamic_to_json_capped, json_to_dynamic, MAX_JSON_MATERIALIZE_BYTES};
use crate::sdk::bridge::{dynamic_to_json, json_to_dynamic};
pub fn register(engine: &mut RhaiEngine) {
let mut module = Module::new();
@@ -26,7 +26,7 @@ fn register_stringify(module: &mut Module) {
module.set_native_fn(
"stringify",
|v: Dynamic| -> Result<String, Box<EvalAltResult>> {
serde_json::to_string(&dynamic_to_json_capped(&v, MAX_JSON_MATERIALIZE_BYTES)?)
serde_json::to_string(&dynamic_to_json(&v))
.map_err(|e| format!("json::stringify: {e}").into())
},
);
@@ -36,7 +36,7 @@ fn register_stringify_pretty(module: &mut Module) {
module.set_native_fn(
"stringify_pretty",
|v: Dynamic| -> Result<String, Box<EvalAltResult>> {
serde_json::to_string_pretty(&dynamic_to_json_capped(&v, MAX_JSON_MATERIALIZE_BYTES)?)
serde_json::to_string_pretty(&dynamic_to_json(&v))
.map_err(|e| format!("json::stringify_pretty: {e}").into())
},
);

View File

@@ -50,7 +50,7 @@
use std::sync::Arc;
use super::bridge::{block_on, runtime_err};
use super::bridge::block_on;
use picloud_shared::{
AppUser, AppUserId, CreateUserInput, EmailTemplateOpts, GeneratedAccept, GeneratedSession,
InviteOpts, SdkCallCx, Services, UpdateUserInput, UsersListOpts, UsersListPage, UsersService,
@@ -615,3 +615,8 @@ fn optional_string(opts: &Map, key: &str) -> Option<String> {
Some(d) => Some(d.to_string()),
}
}
#[allow(clippy::unnecessary_box_returns)]
fn runtime_err(msg: &str) -> Box<EvalAltResult> {
EvalAltResult::ErrorRuntime(msg.into(), rhai::Position::NONE).into()
}

View File

@@ -1,156 +0,0 @@
//! `workflow::` Rhai bridge — start a durable workflow run + read its status.
//!
//! ```rhai
//! let run_id = workflow::start("nightly_report", #{ date: "2026-07-12" });
//! let run_id = workflow::start("cleanup"); // no input
//! let st = workflow::run_status(run_id); // #{ status, output, error, steps } | ()
//! ```
//!
//! Returns the run id as a string. The owning workflow resolves from
//! `cx.app_id` inside the service — never a script arg (the isolation
//! boundary). The durable orchestrator advances the run asynchronously; the
//! script does not block on completion. `run_status` (F-038) lets the starter
//! poll progress without the platform-admin API — `()` when no such run
//! belongs to this app.
use std::sync::Arc;
use picloud_shared::{SdkCallCx, Services, WorkflowRunId, WorkflowRunView, WorkflowService};
use rhai::{Dynamic, Engine as RhaiEngine, EvalAltResult, Map, Module};
use tokio::runtime::Handle as TokioHandle;
use crate::sdk::bridge::{dynamic_to_json_capped, json_to_dynamic, MAX_JSON_MATERIALIZE_BYTES};
pub(super) fn register(engine: &mut RhaiEngine, services: &Services, cx: Arc<SdkCallCx>) {
let svc = services.workflow.clone();
let mut module = Module::new();
// `workflow::start(name, input)`
{
let svc = svc.clone();
let cx = cx.clone();
module.set_native_fn(
"start",
move |name: &str, input: Dynamic| -> Result<String, Box<EvalAltResult>> {
start_blocking(&svc, &cx, name, &input)
},
);
}
// `workflow::start(name)` — no input (passes JSON null).
{
let svc = svc.clone();
let cx = cx.clone();
module.set_native_fn(
"start",
move |name: &str| -> Result<String, Box<EvalAltResult>> {
start_blocking(&svc, &cx, name, &Dynamic::UNIT)
},
);
}
// `workflow::run_status(run_id)` — read-only poll (F-038). `()` if absent.
{
let svc = svc.clone();
let cx = cx.clone();
module.set_native_fn(
"run_status",
move |run_id: &str| -> Result<Dynamic, Box<EvalAltResult>> {
run_status_blocking(&svc, &cx, run_id)
},
);
}
engine.register_static_module("workflow", module.into());
}
fn start_blocking(
svc: &Arc<dyn WorkflowService>,
cx: &Arc<SdkCallCx>,
name: &str,
input: &Dynamic,
) -> Result<String, Box<EvalAltResult>> {
let input_json = if input.is_unit() {
serde_json::Value::Null
} else {
dynamic_to_json_capped(input, MAX_JSON_MATERIALIZE_BYTES)?
};
let handle = TokioHandle::try_current().map_err(|e| -> Box<EvalAltResult> {
EvalAltResult::ErrorRuntime(
format!("workflow::start: no tokio runtime available: {e}").into(),
rhai::Position::NONE,
)
.into()
})?;
let svc = svc.clone();
let cx = cx.clone();
let name = name.to_string();
let run_id = handle
.block_on(async move { svc.start(&cx, &name, input_json).await })
.map_err(|e| -> Box<EvalAltResult> {
EvalAltResult::ErrorRuntime(
format!("workflow::start: {e}").into(),
rhai::Position::NONE,
)
.into()
})?;
Ok(run_id.to_string())
}
fn run_status_blocking(
svc: &Arc<dyn WorkflowService>,
cx: &Arc<SdkCallCx>,
run_id: &str,
) -> Result<Dynamic, Box<EvalAltResult>> {
let run_id: WorkflowRunId = run_id
.parse::<uuid::Uuid>()
.map(WorkflowRunId::from)
.map_err(|_| -> Box<EvalAltResult> {
EvalAltResult::ErrorRuntime(
format!("workflow::run_status: invalid run id {run_id:?}").into(),
rhai::Position::NONE,
)
.into()
})?;
let handle = TokioHandle::try_current().map_err(|e| -> Box<EvalAltResult> {
EvalAltResult::ErrorRuntime(
format!("workflow::run_status: no tokio runtime available: {e}").into(),
rhai::Position::NONE,
)
.into()
})?;
let svc = svc.clone();
let cx = cx.clone();
let view = handle
.block_on(async move { svc.run_status(&cx, run_id).await })
.map_err(|e| -> Box<EvalAltResult> {
EvalAltResult::ErrorRuntime(
format!("workflow::run_status: {e}").into(),
rhai::Position::NONE,
)
.into()
})?;
// Absent run → `()` so a script can test `if status == ()`.
Ok(view.map_or(Dynamic::UNIT, |v| view_to_dynamic(&v)))
}
/// Convert a run view into the Rhai shape
/// `#{ status, output, error, steps: #{ name: status } }`.
fn view_to_dynamic(v: &WorkflowRunView) -> Dynamic {
let mut m = Map::new();
m.insert("status".into(), v.status.as_str().into());
m.insert(
"output".into(),
v.output.clone().map_or(Dynamic::UNIT, json_to_dynamic),
);
m.insert(
"error".into(),
v.error.clone().map_or(Dynamic::UNIT, Into::into),
);
let mut steps = Map::new();
for s in &v.steps {
steps.insert(s.name.clone().into(), s.status.as_str().into());
}
m.insert("steps".into(), steps.into());
Dynamic::from_map(m)
}

View File

@@ -123,13 +123,6 @@ pub struct ExecResponse {
pub status_code: u16,
pub headers: BTreeMap<String, String>,
pub body: serde_json::Value,
/// F-026 binary response. When a script returns an envelope with a
/// `body_base64` field, the HTTP layer decodes it and returns the raw
/// bytes with the script-set `Content-Type` (instead of JSON-encoding
/// `body`). `None` ⇒ the normal JSON `body` path. `#[serde(default)]`
/// keeps the cluster-mode wire format backward-compatible.
#[serde(default)]
pub body_base64: Option<String>,
pub logs: Vec<LogEntry>,
pub stats: ExecStats,
}

View File

@@ -93,26 +93,6 @@ fn returns_structured_response_when_status_code_present() {
assert_eq!(resp.body, json!({ "ok": true, "msg": "created" }));
}
#[test]
fn body_base64_envelope_sets_binary_response() {
// F-026: a `body_base64` field carries raw bytes; `body` is ignored.
let src = r#"
#{ statusCode: 200,
headers: #{ "content-type": "image/png" },
body: #{ ignored: true },
body_base64: "aGVsbG8=" }
"#;
let resp = engine().execute(src, req(json!(null))).unwrap();
assert_eq!(resp.status_code, 200);
assert_eq!(resp.body_base64.as_deref(), Some("aGVsbG8="));
// `body` is nulled out when body_base64 is present.
assert_eq!(resp.body, json!(null));
assert_eq!(
resp.headers.get("content-type").map(String::as_str),
Some("image/png")
);
}
#[test]
fn ctx_exposes_request_data() {
let src = r"

View File

@@ -34,6 +34,22 @@ impl ModuleSource for FailingSource {
) -> Result<Option<ModuleScript>, ModuleSourceError> {
Err(ModuleSourceError::Backend(SENTINEL.to_string()))
}
async fn resolve_extension_point(
&self,
_origin: ScriptOwner,
_name: &str,
) -> Result<Option<picloud_shared::ExtPointResolution>, ModuleSourceError> {
Err(ModuleSourceError::Backend(SENTINEL.to_string()))
}
async fn resolve_by_id(
&self,
_origin: ScriptOwner,
_script_id: picloud_shared::ScriptId,
) -> Result<Option<ModuleScript>, ModuleSourceError> {
Err(ModuleSourceError::Backend(SENTINEL.to_string()))
}
}
/// `MakeWriter` that appends to a shared buffer.

View File

@@ -8,7 +8,7 @@
//! verify the same code path the `picloud` binary runs at request
//! time.
use std::collections::{BTreeMap, HashMap, HashSet};
use std::collections::{BTreeMap, HashMap};
use std::sync::atomic::{AtomicUsize, Ordering};
use std::sync::Arc;
@@ -95,6 +95,30 @@ impl ModuleSource for CountingModuleSource {
.get(&(app_id, name.to_string()))
.cloned())
}
async fn resolve_extension_point(
&self,
_origin: ScriptOwner,
_name: &str,
) -> Result<Option<picloud_shared::ExtPointResolution>, ModuleSourceError> {
// This flat fake has no extension points; the inversion is covered by
// `LexicalModuleSource` below and the Postgres journey tests.
Ok(None)
}
async fn resolve_by_id(
&self,
_origin: ScriptOwner,
script_id: ScriptId,
) -> Result<Option<ModuleScript>, ModuleSourceError> {
Ok(self
.table
.lock()
.await
.values()
.find(|m| m.script_id == script_id)
.cloned())
}
}
fn services_with(modules: Arc<dyn ModuleSource>) -> Services {
@@ -623,6 +647,10 @@ fn validate_endpoint_skips_dynamic_imports_in_imports_list() {
#[derive(Default)]
struct LexicalModuleSource {
table: Mutex<HashMap<(ScriptOwner, String), ModuleScript>>,
/// Extension-point declarations, keyed by exact declaring owner →
/// optional default module id. Flat (exact-owner) — the chain-walk +
/// shadowing tie-break is covered by the Postgres journey tests.
ext_points: Mutex<HashMap<(ScriptOwner, String), Option<ScriptId>>>,
}
impl LexicalModuleSource {
@@ -630,15 +658,16 @@ impl LexicalModuleSource {
Arc::new(Self::default())
}
async fn put(self: &Arc<Self>, owner: ScriptOwner, name: &str, source: &str) {
async fn put(self: &Arc<Self>, owner: ScriptOwner, name: &str, source: &str) -> ScriptId {
let (app_id, group_id) = match owner {
ScriptOwner::App(a) => (Some(a), None),
ScriptOwner::Group(g) => (None, Some(g)),
};
let script_id = ScriptId::new();
self.table.lock().await.insert(
(owner, name.to_string()),
ModuleScript {
script_id: ScriptId::new(),
script_id,
app_id,
group_id,
name: name.to_string(),
@@ -646,6 +675,21 @@ impl LexicalModuleSource {
updated_at: Utc::now(),
},
);
script_id
}
/// Declare `name` an extension point at `owner`, with an optional default
/// module body id.
async fn put_ext_point(
self: &Arc<Self>,
owner: ScriptOwner,
name: &str,
default_script_id: Option<ScriptId>,
) {
self.ext_points
.lock()
.await
.insert((owner, name.to_string()), default_script_id);
}
}
@@ -663,6 +707,35 @@ impl ModuleSource for LexicalModuleSource {
.get(&(origin, name.to_string()))
.cloned())
}
async fn resolve_extension_point(
&self,
origin: ScriptOwner,
name: &str,
) -> Result<Option<picloud_shared::ExtPointResolution>, ModuleSourceError> {
Ok(self
.ext_points
.lock()
.await
.get(&(origin, name.to_string()))
.map(|default_script_id| picloud_shared::ExtPointResolution {
default_script_id: *default_script_id,
}))
}
async fn resolve_by_id(
&self,
_origin: ScriptOwner,
script_id: ScriptId,
) -> Result<Option<ModuleScript>, ModuleSourceError> {
Ok(self
.table
.lock()
.await
.values()
.find(|m| m.script_id == script_id)
.cloned())
}
}
fn req_with_owner(app_id: AppId, owner: ScriptOwner) -> ExecRequest {
@@ -738,148 +811,172 @@ async fn lexical_entry_origin_selects_app_module() {
}
// ---------------------------------------------------------------------------
// §5.5 extension points — resolver handling of the policy outcomes.
//
// `PolicyModuleSource` is a flat fake that overrides `resolve_policy`: an EP
// name resolves dynamically against the inheriting app; a non-EP name resolves
// lexically from the importing origin. This verifies the resolver maps
// Module/NoProvider/NotFound correctly. The full chain semantics (default body
// up-chain, nearest-declaration tie) are covered by the CLI journey tests.
// Extension points (§5.5) — the opt-in resolution inversion. A group declares
// a module name an extension point; an importer on the group's chain resolves
// it against the INHERITING APP, not the sealed lexical chain.
// ---------------------------------------------------------------------------
#[derive(Default)]
struct PolicyModuleSource {
modules: Mutex<HashMap<(ScriptOwner, String), ModuleScript>>,
eps: Mutex<HashSet<String>>,
}
impl PolicyModuleSource {
fn new() -> Arc<Self> {
Arc::new(Self::default())
}
async fn put(self: &Arc<Self>, owner: ScriptOwner, name: &str, source: &str) {
let (app_id, group_id) = match owner {
ScriptOwner::App(a) => (Some(a), None),
ScriptOwner::Group(g) => (None, Some(g)),
};
self.modules.lock().await.insert(
(owner, name.to_string()),
ModuleScript {
script_id: ScriptId::new(),
app_id,
group_id,
name: name.to_string(),
source: source.to_string(),
updated_at: Utc::now(),
},
);
}
async fn mark_ep(self: &Arc<Self>, name: &str) {
self.eps.lock().await.insert(name.to_string());
}
}
#[async_trait]
impl ModuleSource for PolicyModuleSource {
async fn resolve(
&self,
origin: ScriptOwner,
name: &str,
) -> Result<Option<ModuleScript>, ModuleSourceError> {
Ok(self
.modules
.lock()
.await
.get(&(origin, name.to_string()))
.cloned())
}
async fn resolve_policy(
&self,
origin: ScriptOwner,
inheriting_app: AppId,
name: &str,
) -> Result<picloud_shared::ModuleResolution, ModuleSourceError> {
use picloud_shared::ModuleResolution;
if self.eps.lock().await.contains(name) {
// Extension point → dynamic, resolved against the inheriting app.
return Ok(
match self.resolve(ScriptOwner::App(inheriting_app), name).await? {
Some(m) => ModuleResolution::Module(m),
None => ModuleResolution::NoProvider,
},
);
}
Ok(match self.resolve(origin, name).await? {
Some(m) => ModuleResolution::Module(m),
None => ModuleResolution::NotFound,
})
}
}
/// An extension-point import binds the inheriting APP's module even though the
/// importing script's defining node is the group (dynamic override — the
/// inverse of the Phase 4b sealed/lexical import).
/// A group script importing a declared extension point resolves it against the
/// executing app's own module — each tenant supplies its own body.
#[tokio::test(flavor = "multi_thread")]
async fn extension_point_resolves_app_override() {
let source = PolicyModuleSource::new();
async fn ext_point_resolves_to_app_module() {
let source = LexicalModuleSource::new();
let app = AppId::new();
let group = GroupId::new();
source.mark_ep("theme").await;
// App provides its own `theme`; group has none.
// The group opens "theme" as an extension point (no default).
source
.put(ScriptOwner::App(app), "theme", r#"fn color() { "red" }"#)
.put_ext_point(ScriptOwner::Group(group), "theme", None)
.await;
// The app provides its own "theme".
source
.put(
ScriptOwner::App(app),
"theme",
r#"fn name() { "app-theme" }"#,
)
.await;
let engine = engine_with(source.clone());
// Entry runs as the inherited GROUP endpoint importing the EP `theme`.
// Entry runs as the inherited GROUP script (default_origin = group), but
// the executing app is `app` (cx.app_id).
let resp = engine
.execute(
r#"import "theme" as t; t::color()"#,
r#"import "theme" as t; t::name()"#,
req_with_owner(app, ScriptOwner::Group(group)),
)
.expect("should execute");
assert_eq!(resp.body, serde_json::json!("red"));
assert_eq!(
resp.body,
serde_json::json!("app-theme"),
"extension point must resolve to the inheriting app's module"
);
}
/// An extension point with no provider for the app is a hard error.
/// When the app provides no module for the extension point, the declared
/// default body is used.
#[tokio::test(flavor = "multi_thread")]
async fn extension_point_without_provider_errors() {
let source = PolicyModuleSource::new();
async fn ext_point_falls_back_to_default() {
let source = LexicalModuleSource::new();
let app = AppId::new();
let group = GroupId::new();
source.mark_ep("theme").await; // declared, but no module anywhere.
// The group's default "theme" body, registered as a group module.
let default_id = source
.put(
ScriptOwner::Group(group),
"default-theme",
r#"fn name() { "default-theme" }"#,
)
.await;
source
.put_ext_point(ScriptOwner::Group(group), "theme", Some(default_id))
.await;
// The app provides NO "theme".
let engine = engine_with(source.clone());
let resp = engine
.execute(
r#"import "theme" as t; t::name()"#,
req_with_owner(app, ScriptOwner::Group(group)),
)
.expect("should execute");
assert_eq!(resp.body, serde_json::json!("default-theme"));
}
/// An extension point with neither an app provider nor a default is a clean
/// module-not-found at import time.
#[tokio::test(flavor = "multi_thread")]
async fn ext_point_no_provider_no_default_errors() {
let source = LexicalModuleSource::new();
let app = AppId::new();
let group = GroupId::new();
source
.put_ext_point(ScriptOwner::Group(group), "theme", None)
.await;
let engine = engine_with(source.clone());
let err = engine
.execute(
r#"import "theme" as t; t::color()"#,
r#"import "theme" as t; t::name()"#,
req_with_owner(app, ScriptOwner::Group(group)),
)
.expect_err("missing provider must error");
.expect_err("ext point with no provider/default should fail");
let msg = format!("{err:?}").to_lowercase();
assert!(
msg.contains("no provider") || msg.contains("extension point"),
"expected a no-provider error, got {msg}"
msg.contains("module") || msg.contains("not found") || msg.contains("theme"),
"expected module-not-found-flavoured error, got {msg}"
);
}
/// A non-extension-point name still resolves lexically from the origin.
/// A NORMAL (non-extension-point) group import is NOT hijacked by a leaf app's
/// module of the same name — the inversion only fires for declared extension
/// points, so the trust boundary holds.
#[tokio::test(flavor = "multi_thread")]
async fn non_extension_point_stays_lexical() {
let source = PolicyModuleSource::new();
async fn sealed_import_not_hijacked_by_leaf_when_not_ext_point() {
let source = LexicalModuleSource::new();
let app = AppId::new();
let group = GroupId::new();
// "auth" is a concrete group module — NOT an extension point.
source
.put(ScriptOwner::Group(group), "util", r#"fn v() { 7 }"#)
.put(
ScriptOwner::Group(group),
"auth",
r#"fn who() { "group-auth" }"#,
)
.await;
// The app has a trap "auth" that must NOT be selected.
source
.put(ScriptOwner::App(app), "auth", r#"fn who() { "leaf-trap" }"#)
.await;
let engine = engine_with(source.clone());
let resp = engine
.execute(
r#"import "util" as u; u::v()"#,
r#"import "auth" as a; a::who()"#,
req_with_owner(app, ScriptOwner::Group(group)),
)
.expect("should execute");
assert_eq!(resp.body, serde_json::json!(7));
assert_eq!(
resp.body,
serde_json::json!("group-auth"),
"a non-ext-point import must seal to the group, never the leaf's trap"
);
}
/// Two apps inheriting the same extension point get their OWN bodies — the
/// id-keyed module cache does not bleed one tenant's body into another.
#[tokio::test(flavor = "multi_thread")]
async fn ext_point_two_apps_get_distinct_bodies() {
let source = LexicalModuleSource::new();
let app1 = AppId::new();
let app2 = AppId::new();
let group = GroupId::new();
source
.put_ext_point(ScriptOwner::Group(group), "theme", None)
.await;
source
.put(ScriptOwner::App(app1), "theme", r#"fn name() { "one" }"#)
.await;
source
.put(ScriptOwner::App(app2), "theme", r#"fn name() { "two" }"#)
.await;
let engine = engine_with(source.clone());
let r1 = engine
.execute(
r#"import "theme" as t; t::name()"#,
req_with_owner(app1, ScriptOwner::Group(group)),
)
.expect("app1 executes");
let r2 = engine
.execute(
r#"import "theme" as t; t::name()"#,
req_with_owner(app2, ScriptOwner::Group(group)),
)
.expect("app2 executes");
assert_eq!(r1.body, serde_json::json!("one"));
assert_eq!(
r2.body,
serde_json::json!("two"),
"no cross-tenant cache bleed"
);
}

View File

@@ -501,27 +501,8 @@ async fn docs_bridge_preserves_cross_app_isolation() {
v == ()
"#
);
let body = run_script(engine.clone(), &reader_src, baseline_request(app_b)).await;
assert_eq!(body, json!(true), "app B must not see app A's document");
// Positive control — without it this test has no teeth (an execution_id-keyed
// bridge would pass the negative above). `baseline_request` mints a fresh
// execution_id/script_id per call, so this re-read under app_a is a different
// execution than the writer; it must still find A's own doc.
let reader_a = format!(
r#"
let c = docs::collection("shared");
let v = c.get("{id_a_str}");
if v == () {{ "MISSING" }} else {{ v.data.from }}
"#
);
let body = run_script(engine, &reader_a, baseline_request(app_a)).await;
assert_eq!(
body,
json!("a"),
"app A must read back its own document in a later execution — storage is \
keyed by app_id, not execution_id/script_id"
);
let body = run_script(engine, &reader_src, baseline_request(app_b)).await;
assert_eq!(body, json!(true));
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]

View File

@@ -194,23 +194,16 @@ async fn invoke_callee_sees_incremented_depth() {
#{ statusCode: 200, body: d }
"#
.to_string();
// The caller is direct ingress (depth 0); the callee, re-entered via
// `invoke`, must see depth 1. Previously this test asserted NOTHING (`let _ =
// resp`) because `ctx.trigger_depth` wasn't exposed — so it would have passed
// even if `invoke` forwarded the depth unchanged (never incrementing the
// chain-depth counter that bounds runaway trigger loops). `ctx.trigger_depth`
// is now surfaced in build_ctx_map, so we can pin the increment.
let req = baseline_request(app);
assert_eq!(req.trigger_depth, 0, "the caller is direct ingress");
let resp = tokio::task::spawn_blocking(move || engine.execute(&src, req))
.await
.unwrap()
.unwrap();
assert_eq!(
resp.body,
json!(1),
"the invoke callee must see trigger_depth = caller + 1"
);
// ctx exposes trigger_depth? — not yet (it's not in build_ctx_map).
// Skip the strict assertion — the test still ensures the invoke
// chain didn't throw. (See HANDBACK §11 for cx.trigger_depth surface
// exposure as a v1.2 follow-up.)
let _ = resp;
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]

View File

@@ -52,27 +52,6 @@ impl KvService for InMemoryKv {
Ok(())
}
async fn set_if(
&self,
cx: &SdkCallCx,
collection: &str,
key: &str,
expected: Option<Value>,
new: Value,
) -> Result<bool, KvError> {
let mut data = self.data.lock().await;
let k = (cx.app_id, collection.to_string(), key.to_string());
let matches = match (&expected, data.get(&k)) {
(None, None) => true,
(Some(exp), Some(cur)) => exp == cur,
_ => false,
};
if matches {
data.insert(k, new);
}
Ok(matches)
}
async fn delete(&self, cx: &SdkCallCx, collection: &str, key: &str) -> Result<bool, KvError> {
Ok(self
.data
@@ -188,27 +167,6 @@ async fn kv_set_then_get_round_trip() {
assert_eq!(body, json!({ "n": 1 }));
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn kv_set_if_compare_and_swap() {
let engine = make_engine();
let app = AppId::new();
// `set_if(key, (), new)` inserts only when absent; `set_if(key, expected,
// new)` swaps only on match. Returns a bool each time.
let src = r#"
let c = kv::collection("counters");
let first = c.set_if("n", (), 1); // absent -> inserts, true
let dup = c.set_if("n", (), 2); // present -> false
let bad = c.set_if("n", 99, 3); // mismatch -> false
let good = c.set_if("n", 1, 3); // match -> swaps, true
#{ first: first, dup: dup, bad: bad, good: good, final: c.get("n") }
"#;
let body = run_script(engine, src, baseline_request(app)).await;
assert_eq!(
body,
json!({ "first": true, "dup": false, "bad": false, "good": true, "final": 3 })
);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn kv_get_missing_returns_unit() {
let engine = make_engine();
@@ -311,99 +269,6 @@ async fn kv_bridge_preserves_cross_app_isolation() {
let c = kv::collection("shared");
c.get("k")
"#;
let body = run_script(engine.clone(), reader, baseline_request(app_b)).await;
assert_eq!(body, Value::Null, "app B must not see app A's value");
// Positive control — WITHOUT it this test has no teeth. `baseline_request`
// mints a fresh execution_id AND script_id each call, so this second run under
// app_a is a DIFFERENT execution/script than the writer. A bridge that keyed
// storage by execution_id or script_id (instead of app_id) would pass the
// negative above AND every single-execution round-trip, while app-scoped
// persistence was completely gone. This catches that: app A must still read
// back its own value across executions.
let body = run_script(engine, reader, baseline_request(app_a)).await;
assert_eq!(
body,
Value::from("from-a"),
"app A must read back its own value in a later execution — storage is keyed \
by app_id, not execution_id/script_id"
);
}
// --- §9.4 M6: per-execution interceptor resolve cache ---------------------
/// An `InterceptorService` that resolves to an EMPTY chain (nothing hooked) but
/// counts every `resolve` call, so a test can prove the per-execution cache
/// collapses N same-`(service, op)` resolves into one.
#[derive(Default)]
struct CountingInterceptors {
calls: std::sync::atomic::AtomicUsize,
}
#[async_trait]
impl picloud_shared::InterceptorService for CountingInterceptors {
async fn resolve(
&self,
_cx: &SdkCallCx,
_service: &str,
_op: &str,
) -> Result<picloud_shared::InterceptorChain, String> {
self.calls.fetch_add(1, std::sync::atomic::Ordering::SeqCst);
Ok(picloud_shared::InterceptorChain::default())
}
}
fn make_engine_with_interceptors(ic: Arc<CountingInterceptors>) -> Arc<Engine> {
let services = Services::new(
Arc::new(InMemoryKv::default()),
Arc::new(NoopDocsService),
Arc::new(NoopDeadLetterService),
Arc::new(NoopEventEmitter),
Arc::new(NoopModuleSource),
Arc::new(NoopHttpService),
Arc::new(picloud_shared::NoopFilesService),
Arc::new(picloud_shared::NoopPubsubService),
Arc::new(picloud_shared::NoopSecretsService),
Arc::new(picloud_shared::NoopEmailService),
Arc::new(picloud_shared::NoopUsersService),
Arc::new(picloud_shared::NoopQueueService),
Arc::new(picloud_shared::NoopInvokeService),
Arc::new(picloud_shared::NoopVarsService),
)
.with_interceptors(ic);
Arc::new(Engine::new(Limits::default(), services))
}
/// M6: within ONE execution, N `kv::set`s of the same `(service, op)` resolve
/// the interceptor chain at most once (the empty chain is cached), and a SECOND
/// execution starts fresh (the cache is cleared at the outermost boundary).
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn resolve_is_cached_per_execution() {
let ic = Arc::new(CountingInterceptors::default());
let engine = make_engine_with_interceptors(ic.clone());
let app = AppId::new();
// 25 sets in one execution — all the same (kv, set).
let src = r#"
let c = kv::collection("c");
let n = 0;
while n < 25 { c.set("k" + n, n); n += 1; }
"done"
"#;
let body = run_script(engine.clone(), src, baseline_request(app)).await;
assert_eq!(body, Value::from("done"));
assert_eq!(
ic.calls.load(std::sync::atomic::Ordering::SeqCst),
1,
"25 kv::sets in one execution must resolve the (kv, set) chain exactly once"
);
// A second execution resolves again (the cache cleared at the outermost exit).
let body = run_script(engine, src, baseline_request(app)).await;
assert_eq!(body, Value::from("done"));
assert_eq!(
ic.calls.load(std::sync::atomic::Ordering::SeqCst),
2,
"a fresh execution must not reuse the previous execution's resolve cache"
);
let body = run_script(engine, reader, baseline_request(app_b)).await;
assert_eq!(body, Value::Null);
}

View File

@@ -19,7 +19,6 @@ use serde_json::{json, Value};
#[derive(Default)]
struct RecordingPubsub {
last: Mutex<Option<(String, Value)>>,
count: std::sync::atomic::AtomicUsize,
}
#[async_trait]
@@ -33,7 +32,6 @@ impl PubsubService for RecordingPubsub {
if topic.trim().is_empty() {
return Err(PubsubError::EmptyTopic);
}
self.count.fetch_add(1, std::sync::atomic::Ordering::SeqCst);
*self.last.lock().unwrap() = Some((topic.to_string(), message));
Ok(())
}
@@ -165,40 +163,3 @@ async fn publish_empty_topic_throws() {
.expect("spawn_blocking should not panic");
assert!(res.is_err(), "empty topic should throw");
}
/// `PICLOUD_MAX_EMISSIONS_PER_EXECUTION` bounds a one-request outbox-amplification
/// DoS. Only the counter itself was unit-tested (`emit_budget`); this drives it
/// through the REAL `pubsub::publish_durable` SDK surface — i.e. it pins that the
/// charge_emission call is actually wired into publish, not just that the counter
/// works. Uses the DEFAULT budget (1000) so it needs no env mutation: a loop of
/// 1001 publishes must error, and no more than the budget may have reached the
/// service.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn publish_beyond_the_emission_budget_is_refused() {
let svc = Arc::new(RecordingPubsub::default());
let engine = make_engine(svc.clone());
// 1001 durable publishes in one execution — one past the default ceiling.
let src = r#"
let n = 0;
while n < 1001 { pubsub::publish_durable("t", #{ i: n }); n += 1; }
"#
.to_string();
let req = baseline_request(AppId::new());
let res = tokio::task::spawn_blocking(move || engine.execute(&src, req))
.await
.expect("spawn_blocking should not panic");
let err = res.expect_err("exceeding the per-execution emission budget must error");
assert!(
format!("{err:?}").to_lowercase().contains("emission"),
"the error must name the emission budget, got: {err:?}"
);
// The rail actually stopped the amplification: the service saw at most the
// budget, not all 1001.
let seen = svc.count.load(std::sync::atomic::Ordering::SeqCst);
assert!(
seen <= 1000,
"the service must not have received more than the 1000-emission budget, saw {seen}"
);
assert!(seen > 0, "some publishes should have landed before the cap");
}

View File

@@ -140,40 +140,12 @@ async fn retry_policy_clamps_visible_at_script_layer() {
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn retry_policy_rejects_bogus_backoff() {
let engine = build_engine();
// Pin the REASON. A bare `is_err()` would pass if `retry::policy` were not
// registered at all (ErrorFunctionNotFound), or on a typo in the script
// source, or if EVERY policy were rejected — none of which is "an invalid
// backoff is rejected". The message must name `backoff`.
let src = r#"retry::policy(#{ backoff: "bogus" });"#.to_string();
let req = baseline_request(AppId::new());
let err = tokio::task::spawn_blocking({
let engine = engine.clone();
move || engine.execute(&src, req)
})
.await
.unwrap()
.unwrap_err()
.to_string();
assert!(
err.to_lowercase().contains("backoff"),
"the rejection must be about the backoff value specifically, got: {err}"
);
// Control: the SAME call with a VALID backoff succeeds — proving `retry::policy`
// is registered and does not reject everything, so the failure above is
// specific to the bad value.
let ok_src = r#"
let p = retry::policy(#{ backoff: "constant", max_attempts: 2, base_ms: 1 });
retry::run(p, || 7)
"#
.to_string();
let req = baseline_request(AppId::new());
let resp = tokio::task::spawn_blocking(move || engine.execute(&ok_src, req))
let res = tokio::task::spawn_blocking(move || engine.execute(&src, req))
.await
.unwrap()
.expect("a valid backoff must be accepted");
assert_eq!(resp.body, serde_json::json!(7));
.unwrap();
assert!(res.is_err());
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]

View File

@@ -41,5 +41,4 @@ data-encoding.workspace = true
lettre.workspace = true
[dev-dependencies]
picloud-test-support.workspace = true
tokio.workspace = true

View File

@@ -1,40 +1,49 @@
-- §5.5 (v1.2 Hierarchies): extension-point markers — opt-in module polymorphism.
-- Phase 4b+1 (v1.2 Hierarchies): extension points (§5.5).
--
-- An extension point marks a module NAME at a node (app or group) as one
-- descendants are *expected* to provide or override. Imports of an EP name
-- resolve DYNAMICALLY against the inheriting app's view (the app can supply
-- its own module), instead of the Phase 4b lexical default (sealed to the
-- importing script's defining node). See docs §5.5.
-- An extension point opts a MODULE NAME into dynamic, per-tenant resolution.
-- Normally a group/parent script's `import "x"` resolves sealed-lexically
-- against the importer's own defining node — a leaf app cannot shadow it
-- (the trust boundary). When a node declares `x` an extension point, an
-- importer whose defining node is on that node's chain instead resolves `x`
-- against the INHERITING app's effective view, so each descendant app may
-- supply its own `x`. Only the declaring (parent) node can open the hole; a
-- descendant cannot make one of its parent's sealed imports dynamic, because
-- the declaration must live on the importer's own ancestry.
--
-- This table holds only the MARKER (owner, name) — it is pure declaration,
-- structurally identical to a `secrets` name. The optional DEFAULT BODY is
-- just a co-located `kind = 'module'` script of the same name at this node
-- (Phase 4b already stores/resolves/caches those); there is no body column
-- here. So a marker is config, not code → ON DELETE CASCADE (unlike the
-- module body's RESTRICT in 0050).
--
-- Ownership is polymorphic (mirrors vars/secrets/scripts): exactly one of
-- (app_id, group_id) is set, with per-owner partial-unique LOWER(name)
-- indexes for case-insensitive uniqueness.
-- Owner-polymorphic exactly like vars (0048): exactly one of group_id/app_id.
-- Optional `default_script_id` is the fallback module body used when the
-- inheriting app provides no module of `name`.
CREATE TABLE extension_points (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
group_id UUID REFERENCES groups(id) ON DELETE CASCADE,
app_id UUID REFERENCES apps(id) ON DELETE CASCADE,
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
-- Polymorphic owner: exactly one FK set (real FKs so ON DELETE CASCADE
-- works per owner kind and a dangling owner is impossible).
group_id UUID REFERENCES groups(id) ON DELETE CASCADE,
app_id UUID REFERENCES apps(id) ON DELETE CASCADE,
CONSTRAINT extension_points_owner_exactly_one
CHECK ((group_id IS NULL) <> (app_id IS NULL)),
name TEXT NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
name TEXT NOT NULL,
-- Optional fallback module (a module owned by the declaring node). SET
-- NULL on delete so deleting the module can never block — the fallback is
-- then absent and a non-providing app errors at import time (the next plan
-- shows the now-null default as an Update). A valid manifest apply can't
-- reach this: validation rejects a kept ext point whose default names a
-- module not in the bundle.
default_script_id UUID REFERENCES scripts(id) ON DELETE SET NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
-- One marker per (owner, name), case-insensitive. Partial because the owner
-- is split across two nullable columns.
-- One declaration per (owner, name), case-insensitive to match the module
-- name indexes (0050). Partial because the owner is split across two columns.
CREATE UNIQUE INDEX extension_points_group_uidx
ON extension_points (group_id, LOWER(name)) WHERE group_id IS NOT NULL;
CREATE UNIQUE INDEX extension_points_app_uidx
ON extension_points (app_id, LOWER(name)) WHERE app_id IS NOT NULL;
-- Lookup indexes for the resolver's chain join + list-by-owner.
CREATE INDEX extension_points_group_id_idx ON extension_points (group_id) WHERE group_id IS NOT NULL;
CREATE INDEX extension_points_app_id_idx ON extension_points (app_id) WHERE app_id IS NOT NULL;
CREATE INDEX extension_points_group_id_idx
ON extension_points (group_id) WHERE group_id IS NOT NULL;
CREATE INDEX extension_points_app_id_idx
ON extension_points (app_id) WHERE app_id IS NOT NULL;
CREATE INDEX extension_points_default_script_idx
ON extension_points (default_script_id) WHERE default_script_id IS NOT NULL;

View File

@@ -1,47 +0,0 @@
-- §11.6 (v1.2 Hierarchies): group-collection registry — shared cross-app data.
--
-- A row marks a collection NAME at a node as group-SHARED: every app in the
-- owning group's subtree may read it (and, with an authenticated editor+,
-- write it) via the explicit `kv::shared_collection("name")` SDK handle. Resolution is
-- nearest-ancestor-group-wins, walking the reading app's chain — that ancestry
-- walk is the isolation boundary (a foreign app's chain never contains the
-- owning group, so the name simply does not resolve). See docs §11.6.
--
-- This table holds only the MARKER (owner, name, kind) — the data itself lives
-- in a per-kind storage table (`group_kv_entries`, 0053, for kind='kv'). It is
-- pure declaration, structurally identical to an `extension_points` marker
-- (0051). A marker is config, not code → ON DELETE CASCADE.
--
-- Ownership is polymorphic (mirrors vars/secrets/scripts/extension_points):
-- exactly one of (app_id, group_id) is set. MVP authoring is restricted to
-- GROUP owners at the manifest layer (an app-declared shared collection is
-- degenerate — only that app would read it); the polymorphic shape keeps the
-- owner-generic reconcile path uniform with the other inheritable kinds.
--
-- `kind` is the storage discriminator. Only 'kv' ships in the MVP; the column
-- + CHECK exist now so docs/files/topics/queue slot in later without a
-- migration, and so a future `docs` collection of the same name is a distinct
-- row (the unique index covers `kind`).
CREATE TABLE group_collections (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
group_id UUID REFERENCES groups(id) ON DELETE CASCADE,
app_id UUID REFERENCES apps(id) ON DELETE CASCADE,
CONSTRAINT group_collections_owner_exactly_one
CHECK ((group_id IS NULL) <> (app_id IS NULL)),
name TEXT NOT NULL,
kind TEXT NOT NULL DEFAULT 'kv' CHECK (kind IN ('kv')),
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
-- One marker per (owner, name, kind), case-insensitive. Partial because the
-- owner is split across two nullable columns.
CREATE UNIQUE INDEX group_collections_group_uidx
ON group_collections (group_id, LOWER(name), kind) WHERE group_id IS NOT NULL;
CREATE UNIQUE INDEX group_collections_app_uidx
ON group_collections (app_id, LOWER(name), kind) WHERE app_id IS NOT NULL;
-- Lookup indexes for the resolver's chain join + list-by-owner.
CREATE INDEX group_collections_group_id_idx ON group_collections (group_id) WHERE group_id IS NOT NULL;
CREATE INDEX group_collections_app_id_idx ON group_collections (app_id) WHERE app_id IS NOT NULL;

View File

@@ -0,0 +1,39 @@
-- Phase 5 / M3 (v1.2 Hierarchies): multi-repo single-owner ownership (§7).
--
-- A group node is authoritatively MANAGED by at most one project-root (the
-- repo whose manifest declares it as something it applies, not merely
-- references). `groups.owner_project` has carried this seam since 0047 as an
-- inert, FK-less UUID. M3 makes it a real reference: a `projects` table keyed
-- by a stable, gitignored project key the CLI mints in `.picloud/`, and a
-- foreign key from `groups.owner_project` to it.
--
-- Ownership is recorded at GROUP granularity; apps inherit their owning project
-- from their group (apps are never claimed directly). A NULL `owner_project`
-- means the node is UI/API-owned — no manifest fights it (§7.5). First apply
-- claims; a second project's apply to an owned node is refused unless
-- `--takeover` (group-admin gated). Ownership ⟂ RBAC: owning the manifest does
-- NOT grant write — the actor still needs the usual group capabilities (§7.4).
--
-- ON DELETE SET NULL: deleting a project row (not something the platform does
-- today) reverts its nodes to UI-owned rather than cascading away real groups.
CREATE TABLE projects (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
-- Stable, opaque key minted by `pic init` and persisted (gitignored) in the
-- repo's `.picloud/`. The CLI presents it on every tree apply; the server
-- maps it to this row (upsert-by-key), so the same repo keeps the same
-- project identity across clones/CI without committing any server id.
key TEXT NOT NULL UNIQUE,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
-- Promote the inert `groups.owner_project` (0047:38) to a real FK now that the
-- referent exists. Existing rows are all NULL (no projects yet), so this adds
-- no validation burden.
ALTER TABLE groups
ADD CONSTRAINT groups_owner_project_fkey
FOREIGN KEY (owner_project) REFERENCES projects(id) ON DELETE SET NULL;
-- The ownership reconcile reads "which groups does project P own?" (to find
-- owned-but-undeclared nodes for structural prune) and "who owns group G?".
CREATE INDEX groups_owner_project_idx ON groups (owner_project);

View File

@@ -1,30 +0,0 @@
-- §11.6 (v1.2 Hierarchies): group-KV storage — the shared read/write data for
-- a collection declared group-shared in `group_collections` (0052, kind='kv').
--
-- Identity tuple `(group_id, collection, key)`. Unlike per-app `kv_entries`
-- (0007) this is keyed by the OWNING GROUP, not by app — the whole point is
-- that many apps in the group's subtree share one store. There is no `app_id`
-- column: a shared row belongs to the group, not to whichever app wrote it.
--
-- `value` is JSONB, mirroring `kv_entries`. The reading/writing app is resolved
-- to its owning group at the service layer (walking the app's ancestor chain);
-- this table only ever sees a concrete `group_id`.
--
-- ON DELETE CASCADE on group_id: the shared store is DATA, so it dies with its
-- owning group — like `vars`/`secrets` config CASCADE, deliberately UNLIKE the
-- `scripts` (0050) RESTRICT (code is not data). Deleting an app does NOT touch
-- this table (the data is the group's, and survives the app's departure).
CREATE TABLE group_kv_entries (
group_id UUID NOT NULL REFERENCES groups(id) ON DELETE CASCADE,
collection TEXT NOT NULL,
key TEXT NOT NULL,
value JSONB NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
PRIMARY KEY (group_id, collection, key)
);
-- Supports list-by-collection (keyset pagination); the PK already covers
-- (group_id, collection) as a prefix but the explicit index makes intent clear.
CREATE INDEX idx_group_kv_entries_group_collection ON group_kv_entries (group_id, collection);

View File

@@ -0,0 +1,49 @@
-- Phase 5 / M4a (v1.2 Hierarchies): ROUTE templates (§4.5).
--
-- Triggers and routes are app-scoped (never group-inherited — §5.1). At high
-- tenant cardinality that means 100 apps × N routes = 100N hand declarations.
-- A TEMPLATE fixes that: declare a route ONCE on a group, and the apply engine
-- fans it out into one concrete per-`app_id` row for every descendant app. This
-- is INSTANTIATION, not inheritance — expanded rows stay app-owned (the
-- isolation boundary is unchanged); the template is just a stamp.
--
-- Placeholders in template fields are a small, inert, documented set resolved
-- per descendant at expansion: `{app_slug}`, `{env}`, `{var:NAME}` (the app's
-- effective var). Provenance: each expanded `routes` row carries `from_template`
-- (the template id it was stamped from), so re-apply diffs idempotently and
-- `--prune` removes expansions WITHOUT touching a hand-declared route of the
-- same identity.
--
-- (Trigger templates reuse this engine in a follow-up: `trigger_templates` +
-- `triggers.from_template`.)
-- Route templates — one per (group, name). Mirrors the `routes` column shape
-- minus `app_id` (filled per descendant) plus `script_name` (resolved to the
-- nearest inherited endpoint at expansion, like declarative route bindings).
CREATE TABLE route_templates (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
group_id UUID NOT NULL REFERENCES groups(id) ON DELETE CASCADE,
-- Explicit identity/upsert key, unique per group (case-insensitive).
name TEXT NOT NULL,
script_name TEXT NOT NULL,
method TEXT,
host_kind TEXT NOT NULL CHECK (host_kind IN ('any', 'strict', 'wildcard')),
host TEXT NOT NULL DEFAULT '',
host_param_name TEXT,
path_kind TEXT NOT NULL CHECK (path_kind IN ('exact', 'prefix', 'param')),
path TEXT NOT NULL,
dispatch_mode TEXT NOT NULL DEFAULT 'sync' CHECK (dispatch_mode IN ('sync', 'async')),
enabled BOOLEAN NOT NULL DEFAULT TRUE,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE UNIQUE INDEX route_templates_group_name_idx
ON route_templates (group_id, LOWER(name));
-- Provenance on the expanded rows. NULL = hand-declared (the app's own
-- manifest); non-NULL = stamped from this template, managed by template
-- expansion/prune. No FK: the apply engine owns the expansion lifecycle (it
-- deletes orphaned expansions explicitly), and a dangling id after a raw
-- template delete is harmless (treated as "template gone → prune the row").
ALTER TABLE routes ADD COLUMN from_template UUID;
CREATE INDEX routes_from_template_idx ON routes (app_id, from_template);

View File

@@ -1,33 +0,0 @@
-- §11.6 (cont., v1.2 Hierarchies): group-shared DOCS collections.
--
-- Extends the shared-collection machinery (0052 registry + 0053 group_kv_entries)
-- from KV to the queryable-JSON `docs` store. The registry's `kind`
-- discriminator was built for exactly this — widen its CHECK to admit 'docs',
-- and add a group-keyed storage table mirroring `docs` (0013).
-- Widen the marker kind allow-list. The constraint was an inline column CHECK,
-- so Postgres auto-named it `group_collections_kind_check`.
ALTER TABLE group_collections
DROP CONSTRAINT group_collections_kind_check,
ADD CONSTRAINT group_collections_kind_check CHECK (kind IN ('kv', 'docs'));
-- Group-keyed docs store. Identity tuple `(group_id, collection, id)` — keyed
-- by the OWNING GROUP, not an app (the shared rows belong to the group). `id`
-- is a server-generated UUID, as in `docs`. CASCADE on group delete (data dies
-- with its group; an app delete leaves it), matching group_kv_entries (0053).
CREATE TABLE group_docs (
group_id UUID NOT NULL REFERENCES groups(id) ON DELETE CASCADE,
collection TEXT NOT NULL,
id UUID NOT NULL,
data JSONB NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
PRIMARY KEY (group_id, collection, id)
);
-- "all docs in group X / collection Y" — mirrors idx_docs_app_collection (0013).
CREATE INDEX idx_group_docs_group_collection ON group_docs (group_id, collection);
-- GIN on JSONB (jsonb_path_ops) for the find DSL's equality/containment, same
-- as idx_docs_data_gin (0013).
CREATE INDEX idx_group_docs_data_gin ON group_docs USING GIN (data jsonb_path_ops);

View File

@@ -0,0 +1,33 @@
-- Phase 5 / M4b (v1.2 Hierarchies): TRIGGER templates (§4.5).
--
-- The trigger half of templates (M4a shipped routes). A `[group]` manifest
-- declares `[[trigger_templates.<kind>]]`; the apply fans each one out into a
-- concrete per-`app_id` trigger on every descendant app, reusing the same
-- expansion engine, placeholder set (`{app_slug}`/`{env}`/`{var:NAME}`), and
-- `from_template` provenance as routes.
--
-- A trigger's parameters vary by kind (collection_glob/ops, schedule/timezone,
-- topic_pattern, queue_name, inbound_secret_ref, …), so the template stores the
-- whole `BundleTrigger` wire object as `spec` JSONB (kind tag included). The
-- expansion resolves placeholders in the spec's string leaves, then rebuilds the
-- typed trigger and inserts it through the normal trigger path (email secrets
-- are resolved + re-sealed per app, exactly like a hand-declared email trigger).
CREATE TABLE trigger_templates (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
group_id UUID NOT NULL REFERENCES groups(id) ON DELETE CASCADE,
-- Identity/upsert key, unique per group (case-insensitive).
name TEXT NOT NULL,
-- The full `BundleTrigger` wire object (internally tagged by `kind`), with
-- placeholders left unresolved. Rebuilt + resolved per descendant at apply.
spec JSONB NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE UNIQUE INDEX trigger_templates_group_name_idx
ON trigger_templates (group_id, LOWER(name));
-- Provenance on expanded triggers (mirrors routes.from_template, 0053). NULL =
-- hand-declared; non-NULL = stamped from this template, managed by expansion.
ALTER TABLE triggers ADD COLUMN from_template UUID;
CREATE INDEX triggers_from_template_idx ON triggers (app_id, from_template);

View File

@@ -1,37 +0,0 @@
-- §11.6 (cont., v1.2 Hierarchies): group-shared FILES collections.
--
-- Extends the shared-collection machinery (0052 registry + the per-kind stores
-- 0053 group_kv_entries / 0054 group_docs) from KV/docs to the filesystem-backed
-- `files` blob store. The registry's `kind` discriminator was built for exactly
-- this — widen its CHECK to admit 'files', and add a group-keyed metadata table
-- mirroring `files` (0018).
-- Widen the marker kind allow-list (auto-named `group_collections_kind_check`,
-- per 0052's inline column CHECK; 0054 widened it to ('kv','docs')).
ALTER TABLE group_collections
DROP CONSTRAINT group_collections_kind_check,
ADD CONSTRAINT group_collections_kind_check CHECK (kind IN ('kv', 'docs', 'files'));
-- Group-keyed file metadata. Identity tuple `(group_id, collection, id)` — keyed
-- by the OWNING GROUP, not an app (the shared blobs belong to the group). The
-- bytes live on disk at
-- <PICLOUD_FILES_ROOT>/files/groups/<group_id>/<collection>/<id[0:2]>/<id>
-- (a `groups/` infix disjoint from the per-app `files/<app_id>/...` subtree, so
-- the orphan sweeper covers both with one walk). CASCADE on group delete (the
-- metadata dies with its group; an app delete leaves it), matching 0053/0054.
CREATE TABLE group_files (
group_id UUID NOT NULL REFERENCES groups(id) ON DELETE CASCADE,
collection TEXT NOT NULL,
id UUID NOT NULL,
name TEXT NOT NULL,
content_type TEXT NOT NULL,
size_bytes BIGINT NOT NULL,
checksum_sha256 TEXT NOT NULL, -- hex, 64 chars, lowercase
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
PRIMARY KEY (group_id, collection, id)
);
-- List + cursor pagination scans by (group_id, collection) — mirrors
-- idx_files_app_collection (0018).
CREATE INDEX idx_group_files_group_collection ON group_files (group_id, collection);

View File

@@ -1,49 +0,0 @@
-- §11 tail (v1.2 Hierarchies): group TRIGGER templates.
--
-- Until now every trigger was owned by exactly one app (`app_id NOT NULL`).
-- A group-owned trigger is a TEMPLATE: it is never dispatched directly, but
-- the dispatcher's hot-path match queries UNION it in for every descendant app
-- via the ancestor-chain CTE (live resolution, like vars/secrets/scripts and
-- §11.6 collections — no per-app materialized rows). It binds a group-owned
-- handler script and fires under each firing app's `app_id`.
--
-- Scope: EVENT kinds only (kv/docs/files/pubsub) — those are stateless at
-- dispatch. cron/queue/email carry per-instance state (last_fired_at, the
-- one-consumer advisory lock, a sealed inbound secret) and would need
-- materialization; they are rejected on a [group] at the manifest layer and
-- deferred. The CHECK here is deliberately NOT narrowed to event kinds: the
-- column allows any kind for forward-compat, the authoring gate is upstream.
--
-- Reshape mirrors 0050_group_scripts exactly (RESTRICT, not CASCADE — a
-- trigger template references a group script; a group can't be deleted out
-- from under it):
-- * `group_id` — nullable FK→groups, ON DELETE RESTRICT.
-- * `app_id` — made nullable; the exactly-one CHECK enforces app XOR group.
-- * per-app unique name index + dispatch index become per-owner partials.
ALTER TABLE triggers
ADD COLUMN group_id UUID REFERENCES groups(id) ON DELETE RESTRICT;
ALTER TABLE triggers
ALTER COLUMN app_id DROP NOT NULL;
ALTER TABLE triggers
ADD CONSTRAINT triggers_owner_exactly_one
CHECK ((group_id IS NULL) <> (app_id IS NULL));
-- Per-owner name uniqueness (partial so each owner column only constrains its
-- own rows). Existing app rows keep the exact (app_id, name) uniqueness.
DROP INDEX triggers_app_name_uniq;
CREATE UNIQUE INDEX triggers_app_name_uniq
ON triggers (app_id, name) WHERE app_id IS NOT NULL;
CREATE UNIQUE INDEX triggers_group_name_uniq
ON triggers (group_id, name) WHERE group_id IS NOT NULL;
-- Per-owner dispatch hot-path index ("all enabled triggers of kind Y for
-- owner X"). The app index keeps its name + shape (now owner-partial); a
-- parallel group index serves the chain-union's group-template lookups.
DROP INDEX idx_triggers_app_kind_enabled;
CREATE INDEX idx_triggers_app_kind_enabled
ON triggers (app_id, kind) WHERE enabled = TRUE AND app_id IS NOT NULL;
CREATE INDEX idx_triggers_group_kind_enabled
ON triggers (group_id, kind) WHERE enabled = TRUE AND group_id IS NOT NULL;

View File

@@ -1,48 +0,0 @@
-- §11 tail (v1.2 Hierarchies): group ROUTE templates.
--
-- Until now every route was owned by exactly one app (`app_id NOT NULL`).
-- A group-owned route is a TEMPLATE: it is never served directly, but the
-- in-memory RouteTable rebuild EXPANDS it into every descendant app's slice
-- via the ancestor chain (live resolution, like vars/secrets/scripts, §11.6
-- collections, and §11 trigger templates — no per-app materialized rows). It
-- binds a group-owned handler script and runs under each firing app's
-- `app_id`.
--
-- Unlike triggers (a SQL-per-event dispatch), routes are served from an
-- in-memory cache, so the inheritance is resolved at table-rebuild time, not
-- per request. The reshape, however, is identical: a polymorphic owner column.
--
-- Reshape mirrors 0056_group_triggers exactly (RESTRICT, not CASCADE — a route
-- template references a group script; a group can't be deleted out from under
-- it):
-- * `group_id` — nullable FK→groups, ON DELETE RESTRICT.
-- * `app_id` — made nullable; the exactly-one CHECK enforces app XOR group.
-- * the per-app unique binding index becomes per-owner partials.
ALTER TABLE routes
ADD COLUMN group_id UUID REFERENCES groups(id) ON DELETE RESTRICT;
ALTER TABLE routes
ALTER COLUMN app_id DROP NOT NULL;
ALTER TABLE routes
ADD CONSTRAINT routes_owner_exactly_one
CHECK ((group_id IS NULL) <> (app_id IS NULL));
-- Per-owner binding uniqueness (partial so each owner column only constrains
-- its own rows). Existing app rows keep the exact binding-tuple uniqueness; a
-- group can't declare two identical templates. An app declaring a binding
-- identical to an inherited group template is a deliberate SHADOW, resolved
-- at rebuild (nearest-owner-wins) — not a DB conflict.
DROP INDEX routes_unique_binding_idx;
CREATE UNIQUE INDEX routes_unique_binding_idx
ON routes (app_id, host_kind, host, path_kind, path, COALESCE(method, ''))
WHERE app_id IS NOT NULL;
CREATE UNIQUE INDEX routes_group_binding_idx
ON routes (group_id, host_kind, host, path_kind, path, COALESCE(method, ''))
WHERE group_id IS NOT NULL;
-- The expansion join matches routes by owner (`r.app_id = ac.owner_app OR
-- r.group_id = ac.owner_group`). The app side already has routes_app_id_idx;
-- add the parallel group lookup index.
CREATE INDEX routes_group_id_idx ON routes (group_id) WHERE group_id IS NOT NULL;

View File

@@ -1,36 +0,0 @@
-- §11 tail (v1.2 Hierarchies): per-app opt-out of inherited group templates.
--
-- A group TRIGGER or ROUTE template inherits to every descendant app. Until now
-- a descendant could SHADOW a template (declare its own identical binding) but
-- not DECLINE one. This marker records that a specific app opts OUT of a
-- specific inherited template — the template then stops resolving for that app
-- (and only that app; a sibling subtree is unaffected).
--
-- Coarse by REFERENCE (not row id — template ids churn on re-apply): a
-- suppression names a handler SCRIPT NAME (target_kind='trigger') or a PATH
-- (target_kind='route'). A reference may decline more than one inherited
-- template (a group that bound several to the same script/path). The dispatch
-- filters gate to group-owned rows, so an app can only decline what it
-- INHERITS — never its own trigger/route, never a sibling's.
--
-- App-only: only an app suppresses (a group would just not declare the
-- template). So `app_id NOT NULL` — no polymorphic owner. It is pure app config
-- (like `extension_points`), so ON DELETE CASCADE (not the code-owner RESTRICT).
CREATE TABLE template_suppressions (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
app_id UUID NOT NULL REFERENCES apps(id) ON DELETE CASCADE,
target_kind TEXT NOT NULL CHECK (target_kind IN ('trigger', 'route')),
-- A handler script name (trigger) or a path (route). Matched
-- case-insensitively for triggers (as script-name resolution is
-- elsewhere) and exactly for routes; stored as authored.
reference TEXT NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
-- One marker per (app, kind, reference) — re-apply is a NoOp.
CREATE UNIQUE INDEX template_suppressions_uidx
ON template_suppressions (app_id, target_kind, reference);
-- The trigger anti-join / route rebuild both look up by app_id.
CREATE INDEX template_suppressions_app_idx ON template_suppressions (app_id);

View File

@@ -1,19 +0,0 @@
-- §11 tail (v1.2 Hierarchies): `sealed` (mandatory) group templates.
--
-- Per-app opt-out (0058_template_suppressions) made an inherited group TRIGGER
-- or ROUTE template advisory-by-default: it runs on a descendant *unless* that
-- descendant declares a `[suppress]` declining it. That is a compliance footgun
-- — a group's audit / security / rate-limit template can be silently opted out
-- of by the very tenant it is meant to police.
--
-- A `sealed` template closes the gap: the two suppression filters skip a sealed
-- row, so it fires / serves on every descendant regardless of any suppression.
-- The column is only meaningful on a group-owned template (`group_id IS NOT
-- NULL`); an app-owned row is never inherited, so sealing it is meaningless and
-- rejected at apply. Existing rows default to unsealed (the prior behaviour).
--
-- * triggers.sealed — a sealed group trigger template is never suppressible.
-- * routes.sealed — a sealed group route template is never suppressible.
ALTER TABLE triggers ADD COLUMN sealed BOOLEAN NOT NULL DEFAULT FALSE;
ALTER TABLE routes ADD COLUMN sealed BOOLEAN NOT NULL DEFAULT FALSE;

View File

@@ -1,38 +0,0 @@
-- §11 tail (v1.2 Hierarchies): GROUP-level template suppression.
--
-- 0058 let an APP decline an inherited group template (for that app only). A
-- group operator often wants the same opt-out for a whole subtree: "no
-- descendant of this group runs the ancestor's `audit` template." Until now
-- only an app could suppress; a group could not.
--
-- Reshape `template_suppressions` to a POLYMORPHIC owner, exactly like the
-- group triggers/routes reshape (0056/0057) and the config markers (0051/0052):
-- * `group_id` — nullable FK→groups, ON DELETE CASCADE (config, not code).
-- * `app_id` — made nullable; the exactly-one CHECK enforces app XOR group.
-- * the per-app unique index becomes per-owner partials.
--
-- A GROUP suppression declines a template the group INHERITS from a higher
-- ancestor, for the group's whole subtree. Inheritance-only still holds: the
-- dispatch filters gate to group-owned rows, and a suppression only matches on
-- the suppressing owner's ancestor chain — a group can decline what it
-- inherits, never a sibling subtree's, never its own descendants' own rows.
ALTER TABLE template_suppressions
ADD COLUMN group_id UUID REFERENCES groups(id) ON DELETE CASCADE;
ALTER TABLE template_suppressions
ALTER COLUMN app_id DROP NOT NULL;
ALTER TABLE template_suppressions
ADD CONSTRAINT template_suppressions_owner_exactly_one
CHECK ((group_id IS NULL) <> (app_id IS NULL));
DROP INDEX template_suppressions_uidx;
CREATE UNIQUE INDEX template_suppressions_app_uidx
ON template_suppressions (app_id, target_kind, reference) WHERE app_id IS NOT NULL;
CREATE UNIQUE INDEX template_suppressions_group_uidx
ON template_suppressions (group_id, target_kind, reference) WHERE group_id IS NOT NULL;
-- The trigger anti-join / route rebuild look up by both owners now.
CREATE INDEX template_suppressions_group_idx
ON template_suppressions (group_id) WHERE group_id IS NOT NULL;

View File

@@ -1,21 +0,0 @@
-- §11.6 / §4.5: SHARED-collection triggers.
--
-- A §11.6 shared collection is group-owned and written by any subtree app via
-- the `kv::shared_collection(...)` handles. Until now such writes fired NO
-- trigger (the "group trigger has no app to watch" deferral): the per-app
-- trigger dispatch keys on the writer app's OWN collections, and a shared
-- collection lives in a distinct group-keyed store.
--
-- A SHARED trigger closes that gap: a group-owned trigger template marked
-- `shared = true` watches the group's shared collection (not per-app
-- collections). On a shared-collection write the emitter matches these triggers
-- by the OWNING GROUP's chain and runs the handler under the WRITER app
-- (`cx.app_id`) — the same "group template runs under the firing app" model.
--
-- The `shared` marker is the namespace boundary: a `shared = false` trigger
-- (the default, incl. every existing row) watches per-app collections exactly
-- as before; a `shared = true` trigger watches shared collections only. The two
-- never cross (the per-app match queries add `AND t.shared = FALSE`, the shared
-- match queries `AND t.shared = TRUE`).
ALTER TABLE triggers ADD COLUMN shared BOOLEAN NOT NULL DEFAULT FALSE;

View File

@@ -1,25 +0,0 @@
-- §4.5 M5: STATEFUL group trigger templates (cron / queue / email) via
-- per-descendant MATERIALIZATION.
--
-- The event kinds (kv/docs/files/pubsub) resolve a group template LIVE at
-- dispatch via the chain CTE — no per-app rows. The stateful kinds can't:
-- * cron needs a per-app `last_fired_at` the scheduler advances;
-- * queue needs the one-consumer-per-(app_id, queue_name) advisory lock;
-- * email needs a per-app SEALED inbound secret.
-- So a group stateful template is instead MATERIALIZED into an app-owned
-- trigger row per descendant app (created/removed as the tree changes), and the
-- unchanged dispatch paths (scheduler, queue consumer, email webhook) only ever
-- see app-owned rows.
--
-- `materialized_from` links a managed app-owned row back to the group template
-- it was expanded from: NULL = a hand-authored trigger; set = a reconciler-owned
-- copy (dropped + re-created as descendants come and go). ON DELETE CASCADE so
-- deleting the group template (RESTRICT-guarded elsewhere) or the group cleans
-- up the copies; a plain app delete CASCADEs the row away via `triggers.app_id`.
ALTER TABLE triggers
ADD COLUMN materialized_from UUID REFERENCES triggers(id) ON DELETE CASCADE;
-- The reconciler looks up existing copies by their source template.
CREATE INDEX idx_triggers_materialized_from
ON triggers (materialized_from) WHERE materialized_from IS NOT NULL;

View File

@@ -1,12 +0,0 @@
-- §4.5 M5: exactly one materialized copy per (descendant app, source template).
--
-- The materialization reconciler (materialize::rematerialize_stateful_templates)
-- runs on every tree/apply mutation, and two of them can run concurrently (e.g.
-- two apps created in parallel each trigger a full reconcile). Both compute the
-- same "missing" (app, template) pair and both try to insert the copy — without
-- a constraint that races into a DUPLICATE. This partial unique index makes the
-- second insert a no-op (the reconciler uses `ON CONFLICT DO NOTHING`), so a
-- copy is idempotent under concurrency.
CREATE UNIQUE INDEX triggers_materialized_uidx
ON triggers (app_id, materialized_from) WHERE materialized_from IS NOT NULL;

View File

@@ -1,17 +0,0 @@
-- §11.6 (cont., v1.2 Hierarchies): admit 'topic' and 'queue' as shared-
-- collection kinds.
--
-- D2 (shared topics) + D3 (shared queues) extend the group shared-collection
-- machinery from data stores (kv/docs/files) to the two event/messaging kinds.
-- A shared TOPIC is storeless — it is a group-scoped publish NAMESPACE watched
-- by `shared = true` group pubsub triggers (no per-kind storage table). A shared
-- QUEUE gets its group-keyed message store in 0065 (D3). This migration only
-- widens the marker allow-list; both kinds route through the same owner-generic
-- reconcile + `resolve_owning_group` path as the data kinds.
-- Widen the marker kind allow-list (auto-named `group_collections_kind_check`,
-- an inline column CHECK), matching 0054/0055.
ALTER TABLE group_collections
DROP CONSTRAINT group_collections_kind_check,
ADD CONSTRAINT group_collections_kind_check
CHECK (kind IN ('kv', 'docs', 'files', 'topic', 'queue'));

View File

@@ -1,46 +0,0 @@
-- §11.6 D3: group-shared durable queues.
--
-- The queue counterpart to the group_kv/docs/files shared stores (0053-0055).
-- A group declares a `queue` shared collection (kind admitted in 0064); any
-- subtree app enqueues into ONE group-owned store via
-- `queue::shared_collection(name).enqueue(...)`. Consumption is by COMPETING
-- CONSUMERS: a group `[[triggers.queue]] shared = true` template materializes a
-- consumer copy per descendant app (like the M5 stateful templates), and all
-- copies claim from this shared store with FOR UPDATE SKIP LOCKED — each message
-- delivered at-most-once across the whole subtree, scaling horizontally.
--
-- Identity tuple: (group_id, collection). Keyed by the OWNING GROUP, not an app
-- (the shared rows belong to the group). CASCADE on group delete (data dies with
-- its group; an app delete leaves it), matching the other group stores. Mirrors
-- queue_messages (0034) column-for-column, swapping (app_id, queue_name) for
-- (group_id, collection).
CREATE TABLE group_queue_messages (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
group_id UUID NOT NULL REFERENCES groups(id) ON DELETE CASCADE,
collection TEXT NOT NULL,
payload JSONB NOT NULL,
enqueued_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
deliver_after TIMESTAMPTZ NULL,
claim_token UUID NULL,
claimed_at TIMESTAMPTZ NULL,
attempt INT NOT NULL DEFAULT 0,
max_attempts INT NOT NULL DEFAULT 3,
enqueued_by_principal UUID NULL
);
-- Dispatch hot path: scan for unclaimed messages in (group_id, collection)
-- order by enqueued_at (the NOW() deliver_after filter is applied on the small
-- partial result, as in 0034).
CREATE INDEX idx_group_queue_messages_dispatch
ON group_queue_messages (group_id, collection, enqueued_at)
WHERE claim_token IS NULL;
-- depth() / depth_pending() helpers.
CREATE INDEX idx_group_queue_messages_group_collection
ON group_queue_messages (group_id, collection);
-- Reclaim-task scan: currently-leased messages past their visibility timeout.
CREATE INDEX idx_group_queue_messages_claimed
ON group_queue_messages (claimed_at)
WHERE claim_token IS NOT NULL;

View File

@@ -1,41 +0,0 @@
-- §7 multi-repo ownership: the `projects` registry + wiring the inert
-- `owner_project` seam.
--
-- A "project" is a repo-root that declaratively MANAGES a slice of the shared
-- group tree (contains its manifest as something it applies, not merely
-- references). §7's model is SINGLE-OWNER-PER-NODE: each group node is owned by
-- exactly one project; the first `pic apply` that declares a `[project]` claims
-- the node, and a second repo applying to an owned node is refused unless it
-- passes a group-admin-gated `--takeover`. Ownership ⟂ RBAC — it records which
-- manifest is authoritative, not whether a principal may act.
--
-- Identity is a committed `[project] slug` (stable across clones); the server
-- assigns the UUID (server-internal). The `owner_project UUID` column already
-- exists on `groups` (0047, inert) — this migration creates the table it was
-- always meant to reference and wires the FK. Apps carry NO owner column: an
-- app inherits ownership from its NEAREST claimed ancestor group (the ancestor
-- walk is the isolation boundary), faithful to §7's "don't co-own a node —
-- split config downward" corollary.
CREATE TABLE projects (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
-- Instance-global identity, declared in the repo's committed root manifest
-- and frozen. First apply with a new slug registers the project.
slug TEXT NOT NULL UNIQUE,
name TEXT NOT NULL,
-- The admin who first registered it. SET NULL (not CASCADE) — losing the
-- creator's account must not orphan-delete the project or un-claim its tree.
created_by UUID REFERENCES admin_users(id) ON DELETE SET NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
-- Wire the inert 0047 seam. ON DELETE SET NULL un-claims the owned nodes (they
-- fall back to UI/API-owned) rather than cascading a destructive tree delete —
-- deleting a project must never destroy the groups/apps it happened to manage.
ALTER TABLE groups
ADD CONSTRAINT groups_owner_project_fk
FOREIGN KEY (owner_project) REFERENCES projects(id) ON DELETE SET NULL;
-- Ownership lookups: "what does project X own?" (pic projects ls) and the
-- LEFT JOIN behind pic groups ls' owner column.
CREATE INDEX groups_owner_project_idx ON groups (owner_project);

View File

@@ -1,25 +0,0 @@
-- §3 M3 (hermetic gate): persist the per-env approval policy server-side.
--
-- Before this table the policy lived ONLY in the apply request
-- (`ProjectDecl.environments`), so a hand-crafted request that omitted it was
-- ungated — the server had nothing authoritative to check against. This makes
-- the gate hermetic: the confirm-required set is stored per project and the
-- server enforces against `persisted declared` (an env is gated if EITHER the
-- stored row OR the request says so), so an omitted or flipped-to-false policy
-- can no longer bypass a gate a prior apply established.
--
-- One owner (the project), a single boolean policy per environment name — no
-- polymorphic owner and no environment_scope (unlike `vars`/`secrets`, which
-- are data the apply targets; this is metadata ABOUT the apply). CASCADE on the
-- project mirrors 0066's philosophy: un-claiming/deleting a project drops its
-- policy, it does not linger to gate a tree the project no longer owns.
CREATE TABLE project_environments (
project_id UUID NOT NULL REFERENCES projects(id) ON DELETE CASCADE,
-- The environment name as declared in `[project.environments]` (e.g.
-- "production"); matched against the apply's `--env`.
env_name TEXT NOT NULL,
-- TRUE => applying to this env is gated (needs `--approve <env>` + admin).
confirm BOOLEAN NOT NULL,
PRIMARY KEY (project_id, env_name)
);

View File

@@ -1,43 +0,0 @@
-- §11.6 D3 (dead-letter store for group SHARED durable queues).
--
-- Previously an exhausted shared-queue message was `drop_exhausted()`-ed with a
-- warning — silent data loss. Per-app queues persist to `dead_letters` (0010);
-- this is the symmetric group store, keyed by `group_id` (the shared-queue owner)
-- + `collection` (a group has many shared queues) instead of `app_id`.
--
-- CASCADE on the group mirrors `group_queue_messages` (0065): deleting the group
-- drops its dead-letters; an app delete leaves them (the data belongs to the
-- group, not the consuming app). No `app_id` — a shared-queue message is not
-- owned by whichever descendant happened to consume it.
--
-- Fan-out to a *shared* dead-letter TRIGGER is deferred (would need a new shared
-- dead-letter trigger kind); M2 stops the data loss + makes exhausted messages
-- operator-visible (GET /api/v1/admin/groups/{id}/dead-letters).
CREATE TABLE group_dead_letters (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
group_id UUID NOT NULL REFERENCES groups(id) ON DELETE CASCADE,
-- The shared-queue collection name the exhausted message came from.
collection TEXT NOT NULL,
-- The group_queue_messages.id row that exhausted retries (already deleted).
original_event_id UUID NOT NULL,
source TEXT NOT NULL,
op TEXT NOT NULL,
-- The materialized consumer copy's trigger/script (nullable, mirrors 0010).
trigger_id UUID,
script_id UUID,
payload JSONB NOT NULL,
attempt_count INT NOT NULL,
first_attempt_at TIMESTAMPTZ NOT NULL,
last_attempt_at TIMESTAMPTZ NOT NULL,
last_error TEXT NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
resolved_at TIMESTAMPTZ,
resolution TEXT
CHECK (resolution IN
('replayed', 'ignored', 'handled_by_script', 'handler_failed'))
);
-- Operator listing: newest-first per group (GET .../groups/{id}/dead-letters).
CREATE INDEX idx_group_dead_letters_group
ON group_dead_letters (group_id, created_at DESC);

View File

@@ -1,16 +0,0 @@
-- §M5.5 / audit 2026-06-11 H-D1: AAD-bind the email-trigger inbound secret.
--
-- `email_trigger_details.inbound_secret_encrypted` was sealed v0 (no AAD, bound
-- only to the master key) because this table had no version column — so a
-- ciphertext could in principle be relocated between rows. The per-app `secrets`
-- table gained AAD versioning in 0042; this brings the same to email secrets.
--
-- The AAD binds to the SEALING OWNER (the group for a template, the app for a
-- standalone trigger) — deliberately NOT the per-row trigger_id, so a group
-- template's sealed bytes stay valid when the materializer copies them verbatim
-- to each descendant app (all share the group AAD). At open time the sealing
-- owner is recovered from `materialized_from -> template.group_id` (else the
-- app). v0 rows keep their exact pre-audit no-AAD read path (default 0).
ALTER TABLE email_trigger_details
ADD COLUMN inbound_secret_version SMALLINT NOT NULL DEFAULT 0;

View File

@@ -1,19 +0,0 @@
-- Audit 2026-07-11 (LOW C1): admin sessions had only a SLIDING expiry, so a
-- continuously-used (or stolen-but-warm) admin token never self-expired. The
-- data-plane app-user sessions (0027) already carry an absolute hard cap; give
-- the higher-privilege admin session the same. `touch` clamps the sliding bump
-- at this value, and `lookup` filters it, so an admin session dies at the later
-- of its idle window or this absolute ceiling — whichever comes first.
ALTER TABLE admin_sessions ADD COLUMN absolute_expires_at TIMESTAMPTZ;
-- Backfill live rows: cap them 30 days from creation (the new default absolute
-- TTL). A row already older than that will be swept on its next lookup.
UPDATE admin_sessions
SET absolute_expires_at = created_at + INTERVAL '30 days'
WHERE absolute_expires_at IS NULL;
ALTER TABLE admin_sessions ALTER COLUMN absolute_expires_at SET NOT NULL;
-- Backs the absolute-expiry filter on lookup / prune.
CREATE INDEX admin_sessions_absolute_expiry_idx
ON admin_sessions (absolute_expires_at);

View File

@@ -1,120 +0,0 @@
-- v1.2 Workflows (blueprint §9.1/§9.2): a durable DAG orchestration engine.
--
-- A `workflow` is a declarative graph of steps; each step invokes a function
-- (a script, by name, resolved in the app's scope) or a nested sub-workflow,
-- with `depends_on` edges, a per-step `when` condition, input mapping, and a
-- per-step retry + `on_error` policy. A `workflow::start(name, input)` (or the
-- admin run API) creates a RUN; a dedicated background orchestrator advances it
-- step-by-step, durably, surviving restarts.
--
-- Three tables:
-- * workflows — the definition (owner-polymorphic like scripts/0050:
-- app XOR group; M1 authors app-owned only, the group_id
-- column ships unused so group-owned templates stay a
-- door-open follow-up).
-- * workflow_runs — one row per start(); the durable run record.
-- * workflow_run_steps — one row per step per run; carries the competing-
-- consumer lease (claim_token/claimed_at) exactly like
-- queue_messages (0034), so parallel steps complete
-- lock-free and a crashed worker's step is reclaimable.
-- ---------------------------------------------------------------------------
-- workflows: the definition (owner-polymorphic, mirrors scripts/0050).
-- ---------------------------------------------------------------------------
CREATE TABLE workflows (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
-- exactly one of app_id / group_id (the exactly-one CHECK below). Code is
-- not data, so RESTRICT (a group can't be deleted out from under a workflow
-- it owns), mirroring scripts.
app_id UUID NULL REFERENCES apps(id) ON DELETE CASCADE,
group_id UUID NULL REFERENCES groups(id) ON DELETE RESTRICT,
name TEXT NOT NULL,
-- the parsed DAG: { "steps": [ { name, function|workflow, input, depends_on,
-- when, retry, on_error } ] }. Validated server-side (acyclic, unique step
-- names, deps exist) before it is ever written.
definition JSONB NOT NULL,
enabled BOOLEAN NOT NULL DEFAULT TRUE,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
CONSTRAINT workflows_owner_exactly_one CHECK ((group_id IS NULL) <> (app_id IS NULL))
);
-- Per-owner, case-insensitive name uniqueness (partial, one per owner column).
CREATE UNIQUE INDEX workflows_app_name_uidx
ON workflows (app_id, LOWER(name)) WHERE app_id IS NOT NULL;
CREATE UNIQUE INDEX workflows_group_name_uidx
ON workflows (group_id, LOWER(name)) WHERE group_id IS NOT NULL;
CREATE INDEX workflows_group_id_idx ON workflows (group_id) WHERE group_id IS NOT NULL;
-- ---------------------------------------------------------------------------
-- workflow_runs: one row per start(). Carries app_id NOT NULL (the data-plane
-- invariant — every run-listing query filters by app without a join) even
-- though it is derivable from the workflow.
-- ---------------------------------------------------------------------------
CREATE TABLE workflow_runs (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
workflow_id UUID NOT NULL REFERENCES workflows(id) ON DELETE RESTRICT,
app_id UUID NOT NULL REFERENCES apps(id) ON DELETE CASCADE,
status TEXT NOT NULL DEFAULT 'pending'
CHECK (status IN ('pending', 'running', 'succeeded', 'failed', 'canceled')),
input JSONB NOT NULL DEFAULT '{}'::jsonb,
output JSONB NULL,
error TEXT NULL,
-- correlates every step execution in execution_logs under one id.
root_execution_id UUID NOT NULL,
-- nesting: a sub-workflow run links back to the parent step that spawned it.
workflow_depth INT NOT NULL DEFAULT 0,
parent_run_id UUID NULL REFERENCES workflow_runs(id) ON DELETE SET NULL,
parent_step_id UUID NULL,
started_at TIMESTAMPTZ NULL,
finished_at TIMESTAMPTZ NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE INDEX workflow_runs_app_status_idx ON workflow_runs (app_id, status);
CREATE INDEX workflow_runs_workflow_idx ON workflow_runs (workflow_id, created_at DESC);
CREATE INDEX workflow_runs_parent_idx ON workflow_runs (parent_run_id) WHERE parent_run_id IS NOT NULL;
-- ---------------------------------------------------------------------------
-- workflow_run_steps: one row per step per run. The claim_token/claimed_at
-- lease is the queue_messages (0034) competing-consumer pattern — the
-- orchestrator claims a `ready` step FOR UPDATE SKIP LOCKED, and the reclaim
-- task frees a stale lease so a crashed worker's step is retried.
-- ---------------------------------------------------------------------------
CREATE TABLE workflow_run_steps (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
run_id UUID NOT NULL REFERENCES workflow_runs(id) ON DELETE CASCADE,
step_name TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'pending'
CHECK (status IN ('pending', 'ready', 'running', 'succeeded', 'failed', 'skipped')),
attempt INT NOT NULL DEFAULT 0,
max_attempts INT NOT NULL DEFAULT 1,
output JSONB NULL,
error TEXT NULL,
-- competing-consumer lease (mirrors queue_messages).
claim_token UUID NULL,
claimed_at TIMESTAMPTZ NULL,
-- retry backoff / initial dispatch gate (NULL = due immediately once ready).
next_attempt_at TIMESTAMPTZ NULL,
-- set when this step is a nested sub-workflow: the child run it is waiting on.
child_run_id UUID NULL REFERENCES workflow_runs(id) ON DELETE SET NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
UNIQUE (run_id, step_name)
);
-- The orchestrator's claim scan: ready steps whose backoff gate has elapsed.
-- (deliver-gate NOW() comparison is applied as a filter on the small partial
-- set, as in queue_messages.)
CREATE INDEX workflow_run_steps_claimable_idx
ON workflow_run_steps (next_attempt_at)
WHERE status = 'ready';
-- Advance re-eval + reclaim scans by run.
CREATE INDEX workflow_run_steps_run_idx ON workflow_run_steps (run_id);
-- Reclaim-task scan: leased steps whose claim is older than the visibility
-- timeout (bounded by in-flight step count).
CREATE INDEX workflow_run_steps_claimed_idx
ON workflow_run_steps (claimed_at)
WHERE claim_token IS NOT NULL;

View File

@@ -1,18 +0,0 @@
-- v1.2 Workflows M2: the orchestrator executes each workflow step as a normal
-- function invocation and logs it, so `pic logs` can surface (and filter by)
-- workflow-driven runs — exactly as it does for cron/queue/invoke triggers.
--
-- Add 'workflow' to the execution_logs.source CHECK (extends 0043). The column
-- default stays 'http'; this only widens the allowed set. Keep this list in
-- sync with `shared::ExecutionSource` (which gains the matching `Workflow`
-- variant) and `manager-core::OutboxSourceKind`.
--
-- Postgres has no ALTER … ALTER CONSTRAINT for a CHECK, so drop + re-add.
ALTER TABLE execution_logs DROP CONSTRAINT execution_logs_source_check;
ALTER TABLE execution_logs
ADD CONSTRAINT execution_logs_source_check
CHECK (source IN (
'http', 'kv', 'docs', 'dead_letter', 'cron',
'files', 'pubsub', 'email', 'invoke', 'queue', 'workflow'
));

View File

@@ -1,42 +0,0 @@
-- §9.4 Service Interceptors (v1.2) — before-op allow/deny hooks.
--
-- A marker `(owner, service, op) -> interceptor_script` declares that a script
-- runs BEFORE a data-plane operation and may deny it. MVP scope: `service='kv'`,
-- `op IN ('set','delete')`, allow/deny only (no data transform, no chaining,
-- no after-hooks). The interceptor is itself a script owned by the same node
-- (or an ancestor group); it is resolved + run through the existing `invoke()`
-- re-entry path, so this table holds only the MARKER — pure declaration,
-- structurally like an `extension_points` row (0051): config, not code, so
-- ON DELETE CASCADE.
--
-- Ownership is polymorphic (mirrors extension_points/vars/secrets/scripts):
-- exactly one of (app_id, group_id) is set. Resolution walks the calling app's
-- chain (app, then nearest ancestor group) and picks the nearest declaration
-- for a (service, op) — nearest-owner-wins, so an app overrides a group's
-- interceptor, the deliberate inverse of a sealed import.
CREATE TABLE interceptors (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
group_id UUID REFERENCES groups(id) ON DELETE CASCADE,
app_id UUID REFERENCES apps(id) ON DELETE CASCADE,
CONSTRAINT interceptors_owner_exactly_one
CHECK ((group_id IS NULL) <> (app_id IS NULL)),
-- The intercepted operation. MVP: service='kv', op IN ('set','delete').
service TEXT NOT NULL,
op TEXT NOT NULL,
-- Name of the interceptor script (resolved on the owner's chain at run time).
interceptor_script TEXT NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
-- One marker per (owner, service, op). Partial because the owner is split
-- across two nullable columns.
CREATE UNIQUE INDEX interceptors_group_uidx
ON interceptors (group_id, service, op) WHERE group_id IS NOT NULL;
CREATE UNIQUE INDEX interceptors_app_uidx
ON interceptors (app_id, service, op) WHERE app_id IS NOT NULL;
-- Lookup indexes for the resolver's chain join + list-by-owner.
CREATE INDEX interceptors_group_id_idx ON interceptors (group_id) WHERE group_id IS NOT NULL;
CREATE INDEX interceptors_app_id_idx ON interceptors (app_id) WHERE app_id IS NOT NULL;

View File

@@ -1,20 +0,0 @@
-- §9.4 Service Interceptors M2 — before + after phases.
--
-- The 0073 marker guarded only the BEFORE-op allow/deny hook. M2 introduces an
-- explicit `phase` so an owner can declare a `before` AND an `after` interceptor
-- for one `(service, op)`. Existing markers default to `phase='before'` — the
-- 0073 behaviour is unchanged. The partial-unique indexes gain `phase` so the
-- two phases are distinct rows per owner (one before + one after each).
ALTER TABLE interceptors
ADD COLUMN phase TEXT NOT NULL DEFAULT 'before'
CHECK (phase IN ('before', 'after'));
-- Recreate the per-owner uniqueness to be per (owner, service, op, PHASE).
DROP INDEX interceptors_group_uidx;
DROP INDEX interceptors_app_uidx;
CREATE UNIQUE INDEX interceptors_group_uidx
ON interceptors (group_id, service, op, phase) WHERE group_id IS NOT NULL;
CREATE UNIQUE INDEX interceptors_app_uidx
ON interceptors (app_id, service, op, phase) WHERE app_id IS NOT NULL;

View File

@@ -1,10 +0,0 @@
-- §9.4 Service Interceptors M5 — per-interceptor wall-clock timeout.
--
-- An optional per-marker `timeout_ms` bounds how long ONE interceptor hook may
-- run before it is denied (a runaway guard, e.g. `loop {}`, must not hang the
-- guarded write path). NULL means "use the instance default"
-- (`PICLOUD_INTERCEPTOR_TIMEOUT_MS`). The effective deadline is always tightened
-- to at most the caller's remaining budget — a hook can never EXTEND it.
ALTER TABLE interceptors
ADD COLUMN timeout_ms INTEGER
CHECK (timeout_ms IS NULL OR timeout_ms > 0);

View File

@@ -1,13 +0,0 @@
-- A3 raw SMTP ingress — an inbound email ADDRESS an email trigger listens on.
--
-- The HMAC-webhook path (v1.1.7) addresses a trigger by its UUID in the URL. A
-- native SMTP listener instead receives `RCPT TO:<address>` and must resolve
-- that address to the bound trigger. `inbound_address` is the mailbox the
-- trigger claims; NULL for a webhook-only trigger. Case-insensitively unique
-- among app-owned triggers so an address resolves to exactly one script.
ALTER TABLE email_trigger_details
ADD COLUMN inbound_address TEXT;
CREATE UNIQUE INDEX email_trigger_details_inbound_address_uidx
ON email_trigger_details (lower(inbound_address))
WHERE inbound_address IS NOT NULL;

View File

@@ -1,11 +0,0 @@
-- F-030 per-app CORS. A decoupled browser SPA on a different origin cannot
-- call an app's user routes today: the orchestrator emits no
-- `Access-Control-Allow-Origin` header and never answers an OPTIONS preflight.
--
-- This adds a per-app allow-list of origins. The orchestrator (which has no DB)
-- reads it into its in-memory cache on each domain-cache rebuild, echoes the
-- request `Origin` when it matches, and short-circuits preflights. Empty array
-- (the default) = CORS off, preserving today's behaviour. `["*"]` allows any
-- origin.
ALTER TABLE apps
ADD COLUMN cors_allowed_origins JSONB NOT NULL DEFAULT '[]'::jsonb;

View File

@@ -17,15 +17,13 @@ pub enum AdminSessionRepositoryError {
Db(#[from] sqlx::Error),
}
/// Result of a session lookup. Includes the user id (for auth context),
/// the existing `expires_at` (so the middleware can decide whether the
/// sliding-window bump is worth a write), and the `absolute_expires_at`
/// hard cap the bump must clamp at (C1).
/// Result of a session lookup. Includes the user id (for auth context)
/// and the existing `expires_at` so the middleware can decide whether
/// the sliding window bump is worth a write.
#[derive(Debug, Clone)]
pub struct AdminSessionLookup {
pub user_id: AdminUserId,
pub expires_at: DateTime<Utc>,
pub absolute_expires_at: DateTime<Utc>,
}
#[async_trait]
@@ -35,11 +33,9 @@ pub trait AdminSessionRepository: Send + Sync {
user_id: AdminUserId,
token_hash: &str,
expires_at: DateTime<Utc>,
absolute_expires_at: DateTime<Utc>,
) -> Result<(), AdminSessionRepositoryError>;
/// Look up a session by token hash. Returns `None` for missing or
/// already-expired rows — either the sliding `expires_at` OR the
/// absolute cap having passed (the query filters both).
/// already-expired rows (the query filters them).
async fn lookup(
&self,
token_hash: &str,
@@ -82,16 +78,14 @@ impl AdminSessionRepository for PostgresAdminSessionRepository {
user_id: AdminUserId,
token_hash: &str,
expires_at: DateTime<Utc>,
absolute_expires_at: DateTime<Utc>,
) -> Result<(), AdminSessionRepositoryError> {
sqlx::query(
"INSERT INTO admin_sessions (token_hash, user_id, expires_at, absolute_expires_at) \
VALUES ($1, $2, $3, $4)",
"INSERT INTO admin_sessions (token_hash, user_id, expires_at) \
VALUES ($1, $2, $3)",
)
.bind(token_hash)
.bind(user_id.into_inner())
.bind(expires_at)
.bind(absolute_expires_at)
.execute(&self.pool)
.await?;
Ok(())
@@ -101,17 +95,16 @@ impl AdminSessionRepository for PostgresAdminSessionRepository {
&self,
token_hash: &str,
) -> Result<Option<AdminSessionLookup>, AdminSessionRepositoryError> {
let row: Option<(uuid::Uuid, DateTime<Utc>, DateTime<Utc>)> = sqlx::query_as(
"SELECT user_id, expires_at, absolute_expires_at FROM admin_sessions \
WHERE token_hash = $1 AND expires_at > NOW() AND absolute_expires_at > NOW()",
let row: Option<(uuid::Uuid, DateTime<Utc>)> = sqlx::query_as(
"SELECT user_id, expires_at FROM admin_sessions \
WHERE token_hash = $1 AND expires_at > NOW()",
)
.bind(token_hash)
.fetch_optional(&self.pool)
.await?;
Ok(row.map(|(uid, exp, abs)| AdminSessionLookup {
Ok(row.map(|(uid, exp)| AdminSessionLookup {
user_id: uid.into(),
expires_at: exp,
absolute_expires_at: abs,
}))
}
@@ -151,12 +144,9 @@ impl AdminSessionRepository for PostgresAdminSessionRepository {
}
async fn prune_expired(&self) -> Result<u64, AdminSessionRepositoryError> {
let res = sqlx::query(
"DELETE FROM admin_sessions \
WHERE expires_at <= NOW() OR absolute_expires_at <= NOW()",
)
.execute(&self.pool)
.await?;
let res = sqlx::query("DELETE FROM admin_sessions WHERE expires_at <= NOW()")
.execute(&self.pool)
.await?;
Ok(res.rows_affected())
}
}

View File

@@ -18,7 +18,7 @@
use std::sync::Arc;
use axum::extract::{Path, State};
use axum::http::{HeaderMap, StatusCode};
use axum::http::StatusCode;
use axum::response::{IntoResponse, Json, Response};
use axum::routing::{delete, get};
use axum::{Extension, Router};
@@ -116,7 +116,7 @@ async fn mint_key(
State(state): State<ApiKeysState>,
Extension(principal): Extension<Principal>,
Json(input): Json<MintApiKeyRequest>,
) -> Result<(StatusCode, HeaderMap, Json<MintApiKeyResponse>), ApiKeysError> {
) -> Result<(StatusCode, Json<MintApiKeyResponse>), ApiKeysError> {
validate_name(&input.name)?;
validate_scopes(&input.scopes, input.app_id)?;
@@ -135,8 +135,6 @@ async fn mint_key(
.await?;
Ok((
StatusCode::CREATED,
// C2: the response carries the raw key exactly once — never cache it.
crate::auth_api::no_store_headers(),
Json(MintApiKeyResponse {
key: row.into(),
raw_token: minted.raw,

View File

@@ -6,7 +6,7 @@
use std::sync::Arc;
use picloud_shared::{App, AppId, HostKind, PathKind, ScriptOwner};
use picloud_shared::{App, AppId, HostKind, PathKind};
use crate::app_repo::AppRepository;
use crate::repo::{NewScript, ScriptRepository, ScriptRepositoryError};
@@ -76,7 +76,7 @@ async fn seed_into(
routes
.create(NewRoute {
owner: ScriptOwner::App(default.id),
app_id: default.id,
script_id: script.id,
host_kind: HostKind::Any,
host: String::new(),
@@ -88,7 +88,7 @@ async fn seed_into(
method: None,
dispatch_mode: picloud_shared::DispatchMode::Sync,
enabled: true,
sealed: false,
from_template: None,
})
.await?;

View File

@@ -103,25 +103,6 @@ pub trait AppRepository: Send + Sync {
/// single transaction so a partial delete cannot be observed.
async fn delete_cascade(&self, id: AppId) -> Result<(), ScriptRepositoryError>;
async fn count_scripts_in_app(&self, id: AppId) -> Result<i64, ScriptRepositoryError>;
/// F-030 per-app CORS. The allow-list of origins each app permits on its
/// user routes, keyed by app_id — read once per domain-cache rebuild to
/// push into the orchestrator (which has no DB). Default: none (CORS off).
async fn cors_all(&self) -> Result<Vec<(AppId, Vec<String>)>, ScriptRepositoryError> {
Ok(Vec::new())
}
/// The CORS allow-list for one app (`[]` = CORS disabled).
async fn get_cors(&self, _id: AppId) -> Result<Vec<String>, ScriptRepositoryError> {
Ok(Vec::new())
}
/// Replace an app's CORS allow-list. `["*"]` allows any origin.
async fn set_cors(
&self,
_id: AppId,
_origins: Vec<String>,
) -> Result<(), ScriptRepositoryError> {
Ok(())
}
}
pub struct PostgresAppRepository {
@@ -462,32 +443,6 @@ impl AppRepository for PostgresAppRepository {
.await?;
Ok(count.0)
}
async fn cors_all(&self) -> Result<Vec<(AppId, Vec<String>)>, ScriptRepositoryError> {
let rows: Vec<(uuid::Uuid, sqlx::types::Json<Vec<String>>)> =
sqlx::query_as("SELECT id, cors_allowed_origins FROM apps")
.fetch_all(&self.pool)
.await?;
Ok(rows.into_iter().map(|(id, j)| (id.into(), j.0)).collect())
}
async fn get_cors(&self, id: AppId) -> Result<Vec<String>, ScriptRepositoryError> {
let row: Option<(sqlx::types::Json<Vec<String>>,)> =
sqlx::query_as("SELECT cors_allowed_origins FROM apps WHERE id = $1")
.bind(id.into_inner())
.fetch_optional(&self.pool)
.await?;
Ok(row.map(|(j,)| j.0).unwrap_or_default())
}
async fn set_cors(&self, id: AppId, origins: Vec<String>) -> Result<(), ScriptRepositoryError> {
sqlx::query("UPDATE apps SET cors_allowed_origins = $2, updated_at = NOW() WHERE id = $1")
.bind(id.into_inner())
.bind(sqlx::types::Json(origins))
.execute(&self.pool)
.await?;
Ok(())
}
}
#[derive(sqlx::FromRow)]

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -16,7 +16,7 @@ use axum::http::StatusCode;
use axum::response::{IntoResponse, Json, Response};
use axum::routing::{delete, get, post};
use axum::{Extension, Router};
use picloud_orchestrator_core::routing::{pattern, AppDomainTable, CompiledAppDomain, RouteTable};
use picloud_orchestrator_core::routing::{pattern, AppDomainTable, CompiledAppDomain};
use picloud_shared::{App, AppDomain, AppId, AppRole, InstanceRole, Principal};
use serde::{Deserialize, Serialize};
use serde_json::json;
@@ -43,19 +43,11 @@ pub struct AppsState {
/// Cached host → app_id lookup; replaced after every domain CRUD
/// operation so the orchestrator sees changes immediately.
pub domain_table: Arc<AppDomainTable>,
/// §11 tail: the route match snapshot. Creating/deleting an app changes
/// which group route TEMPLATES are inherited, so it is rebuilt here
/// (full-live invalidation) — mirroring the domain_table refresh.
pub route_table: Arc<RouteTable>,
/// Capability gate — Phase 3.5.
pub authz: Arc<dyn AuthzRepo>,
/// Group tree — resolves an app's parent group at create time
/// (defaults to the instance root).
pub groups: Arc<dyn GroupRepository>,
/// §4.5 M5: creating/deleting an app changes which ancestor-group stateful
/// templates it inherits, so its materialized cron/queue copies are
/// reconciled here (full-live, mirroring the route_table rebuild).
pub pool: sqlx::PgPool,
}
pub fn apps_router(state: AppsState) -> Router {
@@ -74,10 +66,6 @@ pub fn apps_router(state: AppsState) -> Router {
"/apps/{id_or_slug}/domains/{domain_id}",
delete(delete_domain),
)
.route(
"/apps/{id_or_slug}/cors",
get(get_cors_handler).put(set_cors_handler),
)
.with_state(state)
}
@@ -138,13 +126,6 @@ pub struct SlugCheckResponse {
pub reason: Option<String>,
}
/// F-030 per-app CORS config: the set of browser origins allowed to call this
/// app's user routes. `["*"]` = any origin. Empty = CORS off.
#[derive(Debug, Serialize, Deserialize)]
pub struct CorsConfig {
pub allowed_origins: Vec<String>,
}
#[derive(Debug, Deserialize)]
pub struct CreateDomainRequest {
pub pattern: String,
@@ -245,9 +226,6 @@ async fn create_app(
)
.await?
};
// The new app may sit under a group that owns route TEMPLATES — make them
// servable immediately (full-live invalidation).
refresh_route_cache(&s).await;
Ok((StatusCode::CREATED, Json(created)))
}
@@ -390,8 +368,6 @@ async fn delete_app(
s.apps.delete(app.id).await?;
}
refresh_domain_cache(&s).await?;
// Drop the deleted app's slice (and any routes it owned) from the snapshot.
refresh_route_cache(&s).await;
Ok(StatusCode::NO_CONTENT)
}
@@ -524,44 +500,6 @@ async fn delete_domain(
Ok(StatusCode::NO_CONTENT)
}
/// F-030: read an app's CORS allow-list. `AppRead` — same gate as viewing
/// domain claims (CORS is host-adjacent config).
async fn get_cors_handler(
State(s): State<AppsState>,
Extension(principal): Extension<Principal>,
Path(id_or_slug): Path<String>,
) -> Result<Json<CorsConfig>, AppsApiError> {
let app = resolve_app(&*s.apps, &id_or_slug).await?.app;
require(s.authz.as_ref(), &principal, Capability::AppRead(app.id)).await?;
let allowed_origins = s.apps.get_cors(app.id).await?;
Ok(Json(CorsConfig { allowed_origins }))
}
/// F-030: replace an app's CORS allow-list, then refresh the domain cache so
/// the orchestrator serves the new policy immediately. `AppManageDomains` —
/// the same gate as domain-claim CRUD.
async fn set_cors_handler(
State(s): State<AppsState>,
Extension(principal): Extension<Principal>,
Path(id_or_slug): Path<String>,
Json(input): Json<CorsConfig>,
) -> Result<Json<CorsConfig>, AppsApiError> {
let app = resolve_app(&*s.apps, &id_or_slug).await?.app;
require(
s.authz.as_ref(),
&principal,
Capability::AppManageDomains(app.id),
)
.await?;
s.apps
.set_cors(app.id, input.allowed_origins.clone())
.await?;
refresh_domain_cache(&s).await?;
Ok(Json(CorsConfig {
allowed_origins: input.allowed_origins,
}))
}
// ----------------------------------------------------------------------------
// Helpers
// ----------------------------------------------------------------------------
@@ -607,31 +545,6 @@ fn validate_slug(slug: &str) -> Result<(), AppsApiError> {
/// Rebuild the in-memory host → app_id cache used by the orchestrator.
/// Called after every domain CRUD operation.
/// §11 tail: rebuild the route match snapshot after a tree mutation that
/// changes route inheritance (an app gained/lost its place in the group tree,
/// so the set of ancestor-group route TEMPLATES it serves changed). Best-effort
/// to mirror the apply path — a failure leaves a stale-but-self-healing table
/// rather than failing the committed mutation.
async fn refresh_route_cache(state: &AppsState) {
if let Err(e) =
crate::route_admin::rebuild_route_table(state.routes.as_ref(), &state.route_table).await
{
tracing::warn!(error = %e, "apps: route table refresh failed; it will self-heal");
}
// §4.5 M5: reconcile materialized stateful-template copies for the changed
// app set (best-effort; self-heals on the next mutation).
match crate::materialize::rematerialize_stateful_templates(&state.pool).await {
Ok(warnings) => {
for w in warnings {
tracing::warn!(warning = %w, "apps: stateful-template materialization warning");
}
}
Err(e) => {
tracing::warn!(error = %e, "apps: stateful-template materialization failed; it will self-heal");
}
}
}
pub async fn refresh_domain_cache(state: &AppsState) -> Result<(), AppsApiError> {
let all = state.domains.list_all().await?;
let compiled = all
@@ -651,10 +564,6 @@ pub async fn refresh_domain_cache(state: &AppsState) -> Result<(), AppsApiError>
})
.collect();
state.domain_table.replace(compiled);
// F-030: push the per-app CORS allow-lists into the same in-memory cache so
// the orchestrator can answer preflights + tag responses without a DB hit.
let cors: std::collections::HashMap<_, _> = state.apps.cors_all().await?.into_iter().collect();
state.domain_table.replace_cors(cors);
Ok(())
}

File diff suppressed because it is too large Load Diff

View File

@@ -156,19 +156,12 @@ async fn login(
};
let token = generate_session_token();
let now = Utc::now();
// C1: an admin session dies at the EARLIER of its sliding window and its
// absolute cap. At create the sliding window is shorter, so `expires_at`
// starts as `now + ttl`; the absolute cap `now + absolute_ttl` is stored so
// the sliding `touch` can clamp to it as the session ages.
let expires_at =
now + ChronoDuration::from_std(state.ttl).unwrap_or_else(|_| ChronoDuration::hours(24));
let absolute_expires_at = now
+ ChronoDuration::from_std(state.absolute_ttl).unwrap_or_else(|_| ChronoDuration::days(30));
let expires_at = Utc::now()
+ ChronoDuration::from_std(state.ttl).unwrap_or_else(|_| ChronoDuration::hours(24));
if let Err(err) = state
.sessions
.create(user_id, &token.hash, expires_at, absolute_expires_at)
.create(user_id, &token.hash, expires_at)
.await
{
tracing::error!(?err, "admin_sessions insert failed");
@@ -181,7 +174,6 @@ async fn login(
(
StatusCode::OK,
no_store_headers(),
Json(LoginResponse {
user: AdminUserDto {
id: user_row.id,
@@ -196,15 +188,6 @@ async fn login(
.into_response()
}
/// `Cache-Control: no-store` for a response carrying a raw credential (a session
/// token, an API key). Prevents a browser / proxy / CDN cache from retaining the
/// secret where a later client could read it back. (C2)
pub(crate) fn no_store_headers() -> HeaderMap {
let mut headers = HeaderMap::new();
headers.insert(header::CACHE_CONTROL, HeaderValue::from_static("no-store"));
headers
}
async fn logout(State(state): State<AuthState>, req: Request<Body>) -> Response {
// Pull token without requiring a valid session (logout is idempotent).
let token = extract_token_for_logout(&req);

View File

@@ -137,11 +137,6 @@ pub struct AuthState {
pub sessions: Arc<dyn AdminSessionRepository>,
pub keys: Arc<dyn ApiKeyRepository>,
pub ttl: Duration,
/// C1: absolute hard cap on an admin session's lifetime. The sliding
/// `touch` bump is clamped at `session_start + absolute_ttl`, so even a
/// continuously-used token self-expires. Set at login via the session's
/// stored `absolute_expires_at`; this value seeds that at create time.
pub absolute_ttl: Duration,
/// F-P-009 — shared cache of resolved Principals. Constructed once
/// at startup and cloned (same `Arc`) into every router state that
/// resolves or revokes credentials, so a revocation-side eviction is
@@ -284,10 +279,8 @@ async fn verify_session(
};
// Sliding-window bump — inline so a DB blip surfaces as 500 rather
// than silent stale sessions. C1: clamp the bump at the session's absolute
// cap so a continuously-used token can't slide forever.
let sliding = Utc::now() + chrono::Duration::from_std(state.ttl).unwrap_or_default();
let new_expires_at = sliding.min(lookup.absolute_expires_at);
// than silent stale sessions. Same shape as Phase 3a.
let new_expires_at = Utc::now() + chrono::Duration::from_std(state.ttl).unwrap_or_default();
if let Err(err) = state.sessions.touch(&token_hash, new_expires_at).await {
tracing::error!(?err, "admin_sessions touch failed");
return Err(InternalError);
@@ -311,15 +304,7 @@ async fn verify_api_key(state: &AuthState, rest: &str) -> Result<Option<Principa
if rest.len() <= API_KEY_PREFIX_LEN {
return Ok(None);
}
// `rest` is the raw, attacker-controlled bearer value after `pic_` — it is
// NOT yet validated as the base32 body a real key has, so a byte-index slice
// (`&rest[..8]`) could split a multibyte UTF-8 codepoint straddling byte 8
// and panic, aborting the request task (an unauthenticated DoS). `get(..)`
// returns `None` at a non-char-boundary, so a malformed bearer falls through
// to "no match" instead of panicking.
let Some(prefix) = rest.get(..API_KEY_PREFIX_LEN) else {
return Ok(None);
};
let prefix = &rest[..API_KEY_PREFIX_LEN];
let mut candidates = match state.keys.find_active_by_prefix(prefix).await {
Ok(v) => v,
@@ -523,20 +508,6 @@ mod tests {
}
}
#[test]
fn api_key_prefix_slice_is_char_boundary_safe() {
// H2 regression (audit 2026-07-11): `rest` is the raw bearer body after
// `pic_`, sliced BEFORE any base32 validation. A byte-index slice
// `&rest[..API_KEY_PREFIX_LEN]` panics when a multibyte codepoint
// straddles byte 8. `€` is 3 bytes, so `aaaaaa€…` puts byte 8 mid-
// codepoint — the boundary-safe `get(..)` must return None, not panic.
let malformed = "aaaaaa\u{20ac}bbbb";
assert!(malformed.len() > API_KEY_PREFIX_LEN);
assert!(malformed.get(..API_KEY_PREFIX_LEN).is_none());
// A clean ASCII body slices to exactly the indexed prefix.
assert_eq!("abcdefghXYZ".get(..API_KEY_PREFIX_LEN), Some("abcdefgh"));
}
#[test]
fn evict_user_drops_only_that_users_entries() {
// Audit 2026-06-11 (PrincipalCache revocation-lag).

View File

@@ -142,38 +142,6 @@ pub enum Capability {
/// the group; maps to `script:write` — the same tier as `AppWriteScript`
/// for an app, lifted to the group owner.
GroupScriptsWrite(GroupId),
/// Read a group-owned shared KV collection (§11.6). Resolved via the group
/// ancestor walk; viewer+ on the owning group. Maps to `script:read`. The
/// reads-open trust model means a script with no principal (anonymous
/// public HTTP) bypasses this check — the structural subtree boundary
/// (the collection only resolves for apps under the owning group) is the
/// hard isolation; this cap refines access for authenticated callers.
GroupKvRead(GroupId),
/// Write a group-owned shared KV collection (§11.6). editor+ on the owning
/// group; maps to `script:write`. Unlike the read cap, a write FAILS CLOSED
/// for an anonymous principal — mutation of shared data always requires an
/// authenticated caller (enforced at the service layer, not by skipping).
GroupKvWrite(GroupId),
/// Read a group-owned shared DOCS collection (§11.6). Viewer+ on the owning
/// group; same reads-open trust model as `GroupKvRead`.
GroupDocsRead(GroupId),
/// Write a group-owned shared DOCS collection (§11.6). editor+ on the owning
/// group; fails closed for an anonymous principal, like `GroupKvWrite`.
GroupDocsWrite(GroupId),
/// Read a group-owned shared FILES collection (§11.6). Viewer+ on the owning
/// group; same reads-open trust model as `GroupKvRead`.
GroupFilesRead(GroupId),
/// Write a group-owned shared FILES collection (§11.6). editor+ on the owning
/// group; fails closed for an anonymous principal, like `GroupKvWrite`.
GroupFilesWrite(GroupId),
/// Publish to a group-owned shared TOPIC (§11.6 D2). editor+ on the owning
/// group; fails closed for an anonymous principal, like `GroupKvWrite` — a
/// publish is a write to shared state.
GroupPubsubPublish(GroupId),
/// Enqueue into a group-owned shared QUEUE (§11.6 D3). editor+ on the owning
/// group; fails closed for an anonymous principal, like `GroupKvWrite` — an
/// enqueue is a write to shared state.
GroupQueueEnqueue(GroupId),
/// Send an outbound email from a script in this app (v1.1.7). Maps
/// to `script:write` on API keys (sending mail is an outbound
/// side-effect like an HTTP request). Granted to `editor`+.
@@ -240,15 +208,7 @@ impl Capability {
| Self::GroupSecretsRead(_)
| Self::GroupSecretsWrite(_)
| Self::GroupScriptsRead(_)
| Self::GroupScriptsWrite(_)
| Self::GroupKvRead(_)
| Self::GroupKvWrite(_)
| Self::GroupDocsRead(_)
| Self::GroupDocsWrite(_)
| Self::GroupFilesRead(_)
| Self::GroupFilesWrite(_)
| Self::GroupPubsubPublish(_)
| Self::GroupQueueEnqueue(_) => None,
| Self::GroupScriptsWrite(_) => None,
Self::AppRead(id)
| Self::AppWriteScript(id)
| Self::AppWriteRoute(id)
@@ -300,10 +260,7 @@ impl Capability {
| Self::AppVarsRead(_)
| Self::GroupRead(_)
| Self::GroupVarsRead(_)
| Self::GroupScriptsRead(_)
| Self::GroupKvRead(_)
| Self::GroupDocsRead(_)
| Self::GroupFilesRead(_) => Scope::ScriptRead,
| Self::GroupScriptsRead(_) => Scope::ScriptRead,
Self::AppWriteScript(_)
| Self::AppKvWrite(_)
| Self::AppDocsWrite(_)
@@ -322,11 +279,6 @@ impl Capability {
// the admin tier below.
| Self::GroupSecretsWrite(_)
| Self::GroupScriptsWrite(_)
| Self::GroupKvWrite(_)
| Self::GroupDocsWrite(_)
| Self::GroupFilesWrite(_)
| Self::GroupPubsubPublish(_)
| Self::GroupQueueEnqueue(_)
| Self::AppInvoke(_) => Scope::ScriptWrite,
Self::AppWriteRoute(_) => Scope::RouteWrite,
Self::AppManageDomains(_) => Scope::DomainManage,
@@ -458,30 +410,6 @@ pub enum AuthzDenied {
Repo(#[from] AuthzError),
}
/// Run [`require`] for a known principal and map the `AuthzDenied` verdict onto
/// the caller's service-specific error via `forbidden`/`backend`. The single
/// home for the `Denied → forbidden() / Repo(e) → backend(e)` mapping that
/// [`script_gate`], [`script_gate_require_principal`], and the stateful services
/// share.
///
/// # Errors
///
/// `forbidden()` on `AuthzDenied::Denied`; `backend(repo_err.to_string())` on
/// `AuthzDenied::Repo`.
pub async fn require_mapped<E>(
repo: &dyn AuthzRepo,
principal: &Principal,
cap: Capability,
forbidden: impl FnOnce() -> E,
backend: impl FnOnce(String) -> E,
) -> Result<(), E> {
match require(repo, principal, cap).await {
Ok(()) => Ok(()),
Err(AuthzDenied::Denied) => Err(forbidden()),
Err(AuthzDenied::Repo(e)) => Err(backend(e.to_string())),
}
}
/// Script-as-gate authz: anonymous public-HTTP scripts skip the check
/// (`cx.principal` is `None`); authenticated callers must hold `cap`.
///
@@ -506,31 +434,11 @@ pub async fn script_gate<E>(
let Some(principal) = cx.principal.as_ref() else {
return Ok(());
};
require_mapped(repo, principal, cap, forbidden, backend).await
}
/// Like [`script_gate`], but **fails closed on an anonymous principal**: a
/// script with `cx.principal == None` is rejected with `forbidden()` rather
/// than skipped. Used for actions that must always be performed by an
/// authenticated caller even though the surrounding service skips authz for
/// public scripts — e.g. §11.6 group-collection WRITES (reads stay open via
/// `script_gate`, writes require an authenticated editor+ on the owning group).
///
/// # Errors
///
/// Returns `forbidden()` when the principal is absent or the capability is
/// denied, or `backend(repo_err.to_string())` on a repo error.
pub async fn script_gate_require_principal<E>(
repo: &dyn AuthzRepo,
cx: &picloud_shared::SdkCallCx,
cap: Capability,
forbidden: impl FnOnce() -> E,
backend: impl FnOnce(String) -> E,
) -> Result<(), E> {
let Some(principal) = cx.principal.as_ref() else {
return Err(forbidden());
};
require_mapped(repo, principal, cap, forbidden, backend).await
match require(repo, principal, cap).await {
Ok(()) => Ok(()),
Err(AuthzDenied::Denied) => Err(forbidden()),
Err(AuthzDenied::Repo(e)) => Err(backend(e.to_string())),
}
}
// ----------------------------------------------------------------------------
@@ -558,15 +466,7 @@ async fn role_grants(
| Capability::GroupSecretsRead(g)
| Capability::GroupSecretsWrite(g)
| Capability::GroupScriptsRead(g)
| Capability::GroupScriptsWrite(g)
| Capability::GroupKvRead(g)
| Capability::GroupKvWrite(g)
| Capability::GroupDocsRead(g)
| Capability::GroupDocsWrite(g)
| Capability::GroupFilesRead(g)
| Capability::GroupFilesWrite(g)
| Capability::GroupPubsubPublish(g)
| Capability::GroupQueueEnqueue(g) => {
| Capability::GroupScriptsWrite(g) => {
group_member_grants(repo, principal.user_id, cap, g).await
}
// Creating a root-level group is an instance act — members
@@ -626,23 +526,15 @@ async fn group_member_grants(
/// viewer→read, editor→write, group_admin(=AppAdmin)→admin.
const fn group_role_satisfies(role: AppRole, cap: Capability) -> bool {
match cap {
// viewer+ reads group metadata, config vars, scripts, and shared KV.
// viewer+ reads group metadata, config vars, and scripts.
Capability::GroupRead(_)
| Capability::GroupVarsRead(_)
| Capability::GroupScriptsRead(_)
| Capability::GroupKvRead(_)
| Capability::GroupDocsRead(_)
| Capability::GroupFilesRead(_) => true,
// editor+ writes config vars/secrets/scripts and shared KV.
| Capability::GroupScriptsRead(_) => true,
// editor+ writes config vars/secrets/scripts.
Capability::GroupWrite(_)
| Capability::GroupVarsWrite(_)
| Capability::GroupSecretsWrite(_)
| Capability::GroupScriptsWrite(_)
| Capability::GroupKvWrite(_)
| Capability::GroupDocsWrite(_)
| Capability::GroupFilesWrite(_)
| Capability::GroupPubsubPublish(_)
| Capability::GroupQueueEnqueue(_) => {
| Capability::GroupScriptsWrite(_) => {
matches!(role, AppRole::Editor | AppRole::AppAdmin)
}
// group_admin manages the group + reads secret VALUES (the
@@ -1323,205 +1215,6 @@ mod tests {
);
}
#[tokio::test]
async fn group_kv_caps_resolve_by_role_up_the_chain() {
// §11.6: shared-KV read is viewer+, write is editor+, both resolved via
// the group ancestor walk; an outsider gets nothing.
let repo = InMemoryAuthzRepo::default();
let root = GroupId::new();
let team = GroupId::new();
repo.add_group(root, None).await;
repo.add_group(team, Some(root)).await;
// A viewer at root can READ a descendant group's shared KV but not write.
let viewer = principal(InstanceRole::Member);
repo.grant_group(viewer.user_id, root, AppRole::Viewer)
.await;
assert!(can(&repo, &viewer, Capability::GroupKvRead(team))
.await
.unwrap()
.is_allow());
assert_eq!(
can(&repo, &viewer, Capability::GroupKvWrite(team))
.await
.unwrap(),
Decision::Deny
);
// An editor at root can WRITE it.
let editor = principal(InstanceRole::Member);
repo.grant_group(editor.user_id, root, AppRole::Editor)
.await;
assert!(can(&repo, &editor, Capability::GroupKvWrite(team))
.await
.unwrap()
.is_allow());
// An unrelated member gets neither.
let outsider = principal(InstanceRole::Member);
assert_eq!(
can(&repo, &outsider, Capability::GroupKvRead(team))
.await
.unwrap(),
Decision::Deny
);
// Group caps carry no app_id, so a bound key is denied at the binding
// layer regardless of role.
let bound = Principal {
user_id: AdminUserId::new(),
instance_role: InstanceRole::Owner,
scopes: Some(vec![Scope::ScriptWrite]),
app_binding: Some(AppId::new()),
};
assert_eq!(
can(&repo, &bound, Capability::GroupKvWrite(team))
.await
.unwrap(),
Decision::Deny
);
}
/// §11.6 D2/D3: shared-topic PUBLISH and shared-queue ENQUEUE are writes, so
/// they require editor+ on the owning group. The three older group data caps
/// (KV/docs/files) have this test; these two — the newest, and the ones whose
/// blast radius is an entire subtree's handlers — did not. A `role_satisfies`
/// regression that let a Viewer publish/enqueue would go uncaught otherwise.
#[tokio::test]
async fn group_pubsub_and_queue_write_caps_require_editor() {
let repo = InMemoryAuthzRepo::default();
let root = GroupId::new();
let team = GroupId::new();
repo.add_group(root, None).await;
repo.add_group(team, Some(root)).await;
// A Viewer at root is denied BOTH writes on a descendant group.
let viewer = principal(InstanceRole::Member);
repo.grant_group(viewer.user_id, root, AppRole::Viewer)
.await;
for cap in [
Capability::GroupPubsubPublish(team),
Capability::GroupQueueEnqueue(team),
] {
assert_eq!(
can(&repo, &viewer, cap).await.unwrap(),
Decision::Deny,
"a Viewer must not publish/enqueue into a shared group collection"
);
}
// An Editor at root is allowed both (resolved up the chain).
let editor = principal(InstanceRole::Member);
repo.grant_group(editor.user_id, root, AppRole::Editor)
.await;
for cap in [
Capability::GroupPubsubPublish(team),
Capability::GroupQueueEnqueue(team),
] {
assert!(
can(&repo, &editor, cap).await.unwrap().is_allow(),
"an Editor on the owning group's ancestor must be allowed"
);
}
// An unrelated member gets neither.
let outsider = principal(InstanceRole::Member);
assert_eq!(
can(&repo, &outsider, Capability::GroupPubsubPublish(team))
.await
.unwrap(),
Decision::Deny
);
assert_eq!(
can(&repo, &outsider, Capability::GroupQueueEnqueue(team))
.await
.unwrap(),
Decision::Deny
);
}
#[tokio::test]
async fn group_docs_caps_resolve_by_role_up_the_chain() {
// §11.6 docs: same trust shape as shared KV — read is viewer+, write is
// editor+, resolved via the group ancestor walk.
let repo = InMemoryAuthzRepo::default();
let root = GroupId::new();
let team = GroupId::new();
repo.add_group(root, None).await;
repo.add_group(team, Some(root)).await;
let viewer = principal(InstanceRole::Member);
repo.grant_group(viewer.user_id, root, AppRole::Viewer)
.await;
assert!(can(&repo, &viewer, Capability::GroupDocsRead(team))
.await
.unwrap()
.is_allow());
assert_eq!(
can(&repo, &viewer, Capability::GroupDocsWrite(team))
.await
.unwrap(),
Decision::Deny
);
let editor = principal(InstanceRole::Member);
repo.grant_group(editor.user_id, root, AppRole::Editor)
.await;
assert!(can(&repo, &editor, Capability::GroupDocsWrite(team))
.await
.unwrap()
.is_allow());
let outsider = principal(InstanceRole::Member);
assert_eq!(
can(&repo, &outsider, Capability::GroupDocsRead(team))
.await
.unwrap(),
Decision::Deny
);
}
#[tokio::test]
async fn group_files_caps_resolve_by_role_up_the_chain() {
// §11.6 files: same trust shape as shared KV/docs — read is viewer+,
// write is editor+, resolved via the group ancestor walk.
let repo = InMemoryAuthzRepo::default();
let root = GroupId::new();
let team = GroupId::new();
repo.add_group(root, None).await;
repo.add_group(team, Some(root)).await;
let viewer = principal(InstanceRole::Member);
repo.grant_group(viewer.user_id, root, AppRole::Viewer)
.await;
assert!(can(&repo, &viewer, Capability::GroupFilesRead(team))
.await
.unwrap()
.is_allow());
assert_eq!(
can(&repo, &viewer, Capability::GroupFilesWrite(team))
.await
.unwrap(),
Decision::Deny
);
let editor = principal(InstanceRole::Member);
repo.grant_group(editor.user_id, root, AppRole::Editor)
.await;
assert!(can(&repo, &editor, Capability::GroupFilesWrite(team))
.await
.unwrap()
.is_allow());
let outsider = principal(InstanceRole::Member);
assert_eq!(
can(&repo, &outsider, Capability::GroupFilesRead(team))
.await
.unwrap(),
Decision::Deny
);
}
#[tokio::test]
async fn admin_implicitly_manages_the_whole_group_tree() {
let repo = InMemoryAuthzRepo::default();

View File

@@ -226,17 +226,7 @@ pub async fn fetch_var_candidates(
pool: &PgPool,
app_id: AppId,
) -> Result<Vec<Candidate>, sqlx::Error> {
let sql = format!(
"{CHAIN_LEVELS_CTE} \
SELECT c.depth, \
CASE WHEN v.app_id IS NOT NULL THEN 'app' ELSE 'group' END AS owner_kind, \
COALESCE(v.app_id, v.group_id) AS owner_id, \
v.environment_scope, v.key, v.value, v.is_tombstone \
FROM chain c \
JOIN vars v ON (v.app_id = c.app_owner OR v.group_id = c.group_owner) \
WHERE v.environment_scope = '*' OR v.environment_scope = c.app_env \
ORDER BY c.depth ASC"
);
let sql = format!("{CHAIN_LEVELS_CTE} {VAR_CANDIDATES_TAIL}");
let rows = sqlx::query_as::<_, VarCandidateRow>(&sql)
.bind(app_id.into_inner())
.fetch_all(pool)
@@ -244,6 +234,39 @@ pub async fn fetch_var_candidates(
Ok(rows.into_iter().map(Into::into).collect())
}
/// Transaction-scoped twin of [`fetch_var_candidates`]: runs the identical
/// `CHAIN_LEVELS_CTE` query against an open transaction, so it sees vars
/// **written earlier in the same transaction** (not just committed state).
/// Used by template expansion so a `{var:NAME}` placeholder resolves against
/// a var set in the *same* `pic apply` rather than landing one apply late.
///
/// # Errors
/// Propagates sqlx errors.
pub async fn fetch_var_candidates_tx(
tx: &mut sqlx::Transaction<'_, sqlx::Postgres>,
app_id: AppId,
) -> Result<Vec<Candidate>, sqlx::Error> {
let sql = format!("{CHAIN_LEVELS_CTE} {VAR_CANDIDATES_TAIL}");
let rows = sqlx::query_as::<_, VarCandidateRow>(&sql)
.bind(app_id.into_inner())
.fetch_all(&mut **tx)
.await?;
Ok(rows.into_iter().map(Into::into).collect())
}
/// The `SELECT … FROM chain …` tail appended after [`CHAIN_LEVELS_CTE`] to
/// pull env-eligible `vars` candidates, nearest-first. Shared by the pool and
/// transaction variants so the two can't drift.
const VAR_CANDIDATES_TAIL: &str = "\
SELECT c.depth, \
CASE WHEN v.app_id IS NOT NULL THEN 'app' ELSE 'group' END AS owner_kind, \
COALESCE(v.app_id, v.group_id) AS owner_id, \
v.environment_scope, v.key, v.value, v.is_tombstone \
FROM chain c \
JOIN vars v ON (v.app_id = c.app_owner OR v.group_id = c.group_owner) \
WHERE v.environment_scope = '*' OR v.environment_scope = c.app_env \
ORDER BY c.depth ASC";
/// The masked, resolved view of one inherited secret for `config/effective`:
/// which owner/level/scope supplies it — **never** the value.
#[derive(Debug, Clone)]

View File

@@ -119,7 +119,7 @@ async fn tick(pool: &PgPool, now: DateTime<Utc>) -> Result<usize, sqlx::Error> {
d.schedule, d.timezone, d.last_fired_at \
FROM cron_trigger_details d \
JOIN triggers t ON t.id = d.trigger_id \
WHERE t.enabled = TRUE AND t.app_id IS NOT NULL \
WHERE t.enabled = TRUE \
FOR UPDATE OF d SKIP LOCKED",
)
.fetch_all(&mut *tx)

View File

@@ -31,8 +31,7 @@ use picloud_orchestrator_core::{ExecutionGate, ExecutorClient};
use picloud_shared::{
AppId, DeadLetterId, ExecResponseSummary, ExecutionId, ExecutionLogSink, ExecutionSource,
HttpDispatchPayload, InboxDeliveryOutcome, InboxFailureKind, InboxResolver, InboxResult,
QueueMessageId, RequestId, Script, ScriptId, ScriptOwner, ScriptSandbox, TriggerEvent,
TriggerId,
RequestId, Script, ScriptId, ScriptOwner, ScriptSandbox, TriggerEvent,
};
use rand::Rng;
use uuid::Uuid;
@@ -66,10 +65,6 @@ pub struct Dispatcher {
/// v1.1.9. Reads `queue_messages` for the queue arm + the reclaim
/// task. None in tests / harnesses that don't exercise queues.
pub queue: Arc<dyn QueueRepo>,
/// §11.6 D3. The group shared-queue store. A materialized consumer of a
/// SHARED group queue template (`consumer.shared_group.is_some()`) claims
/// from here instead of the per-app `queue`.
pub group_queue: Arc<dyn crate::group_queue_repo::GroupQueueRepo>,
pub config: TriggerConfig,
/// Stable id for this dispatcher instance — written into
/// `outbox.claimed_by` for forensics. In MVP this is the host's
@@ -230,9 +225,6 @@ impl Dispatcher {
// Reclaim task: independent cadence (default 30s) so it doesn't
// contend with the per-100ms dispatcher tick.
let reclaim_queue = self.queue.clone();
let reclaim_group_queue = self.group_queue.clone();
let reclaim_outbox = self.outbox.clone();
let outbox_claim_timeout = self.config.outbox_claim_timeout_secs;
let reclaim_interval =
Duration::from_millis(u64::from(self.config.queue_reclaim_interval_ms));
tokio::spawn(async move {
@@ -240,34 +232,11 @@ impl Dispatcher {
ticker.tick().await;
loop {
ticker.tick().await;
// A dispatcher that died mid-dispatch left its claimed rows
// stranded — `claim_due` only takes unclaimed rows, so without
// this they would never fire again.
match reclaim_outbox
.reclaim_stale_claims(outbox_claim_timeout)
.await
{
Ok(0) => {}
Ok(n) => tracing::warn!(
reclaimed = n,
timeout_secs = outbox_claim_timeout,
"reclaimed stale outbox claims — a dispatcher died mid-dispatch"
),
Err(e) => tracing::warn!(?e, "outbox reclaim task errored"),
}
match reclaim_queue.reclaim_visibility_timeouts().await {
Ok(0) => {}
Ok(n) => tracing::info!(reclaimed = n, "queue visibility-timeout reclaim"),
Err(e) => tracing::warn!(?e, "queue reclaim task errored"),
}
// §11.6 D3: the group shared-queue store has the same reclaim.
match reclaim_group_queue.reclaim_visibility_timeouts().await {
Ok(0) => {}
Ok(n) => {
tracing::info!(reclaimed = n, "group-queue visibility-timeout reclaim");
}
Err(e) => tracing::warn!(?e, "group-queue reclaim task errored"),
}
}
});
@@ -344,205 +313,6 @@ impl Dispatcher {
Ok(())
}
// §11.6 D3: route the four queue store operations to the per-app store or
// the group shared store, based on whether this consumer was materialized
// from a SHARED group queue template (`consumer.shared_group`). A group
// claim is normalized to a `ClaimedMessage` under the CONSUMING app so the
// rest of the queue arm (handler dispatch, event, logging) is unchanged.
async fn q_claim(&self, c: &ActiveQueueConsumer) -> Result<Option<ClaimedMessage>, String> {
match c.shared_group {
Some(g) => Ok(self
.group_queue
.claim(g, &c.queue_name)
.await
.map_err(|e| e.to_string())?
.map(|m| ClaimedMessage {
id: m.id,
app_id: c.app_id,
queue_name: m.collection,
payload: m.payload,
enqueued_at: m.enqueued_at,
attempt: m.attempt,
max_attempts: m.max_attempts,
claim_token: m.claim_token,
})),
None => self
.queue
.claim(c.app_id, &c.queue_name)
.await
.map_err(|e| e.to_string()),
}
}
async fn q_ack(
&self,
c: &ActiveQueueConsumer,
id: QueueMessageId,
token: uuid::Uuid,
) -> Result<bool, String> {
match c.shared_group {
Some(_) => self
.group_queue
.ack(id, token)
.await
.map_err(|e| e.to_string()),
None => self.queue.ack(id, token).await.map_err(|e| e.to_string()),
}
}
async fn q_nack(
&self,
c: &ActiveQueueConsumer,
id: QueueMessageId,
token: uuid::Uuid,
delay: chrono::Duration,
) -> Result<bool, String> {
match c.shared_group {
Some(_) => self
.group_queue
.nack(id, token, delay)
.await
.map_err(|e| e.to_string()),
None => self
.queue
.nack(id, token, delay)
.await
.map_err(|e| e.to_string()),
}
}
/// TRANSIENT release — the handler never ran (gate saturated, or the script
/// was disabled at fire time). Re-queue WITHOUT counting the claim's
/// pre-increment against `max_attempts`, so overload/disable windows can't
/// dead-letter a message that executed zero times.
async fn q_release(
&self,
c: &ActiveQueueConsumer,
id: QueueMessageId,
token: uuid::Uuid,
delay: chrono::Duration,
) -> Result<bool, String> {
match c.shared_group {
Some(_) => self
.group_queue
.release(id, token, delay)
.await
.map_err(|e| e.to_string()),
None => self
.queue
.release(id, token, delay)
.await
.map_err(|e| e.to_string()),
}
}
/// Terminal disposition of a message that can't be processed (script
/// missing / cross-app / exhausted). Per-app → dead-letter into `dead_letters`
/// (+ the caller fans out `dead_letter` triggers). Group shared queue →
/// dead-letter into `group_dead_letters` (Track A M2) and return `None` so the
/// per-app fan-out is skipped (a competing-consumer message has no single app
/// to fire per-app handlers under). Returns the dead-letter id only for the
/// per-app path (drives fan-out).
async fn q_terminal(
&self,
c: &ActiveQueueConsumer,
claimed: &ClaimedMessage,
trigger_id: Option<TriggerId>,
script_id: Option<ScriptId>,
reason: &str,
) -> Option<DeadLetterId> {
match c.shared_group {
Some(group_id) => {
// §11.6 D3: persist the exhausted message to the group dead-letter
// store instead of dropping it. We return None (not the dl id) so
// the per-app `fan_out_dead_letter` below is SKIPPED — firing the
// consuming app's *per-app* dead_letter handlers on a shared-queue
// message (competing consumers → nondeterministic app) would be
// wrong.
//
// §11.6 B2: instead, fan out to the group's *shared* dead_letter
// handlers (`shared = true` on the owning group). Each runs under
// the WRITER app (`claimed.app_id`, the consumer that exhausted
// the message) — the M2 shared-write model.
match self
.group_queue
.dead_letter(
claimed.id,
claimed.claim_token,
group_id,
&claimed.queue_name,
trigger_id,
script_id,
claimed.attempt,
claimed.enqueued_at,
reason,
)
.await
{
Ok(dl_id) => {
tracing::warn!(
reason,
queue = %claimed.queue_name,
dead_letter_id = %dl_id.into_inner(),
"shared-queue message dead-lettered"
);
let original = TriggerEvent::Queue {
queue_name: claimed.queue_name.clone(),
message: claimed.payload.clone(),
enqueued_at: claimed.enqueued_at,
attempt: claimed.attempt,
message_id: claimed.id.to_string(),
};
self.fan_out_shared_dead_letter(
group_id,
DeadLetterFanOutCtx {
// Writer app: the consumer that exhausted the msg.
app_id: claimed.app_id,
original,
source: "queue".to_string(),
dead_letter_id: dl_id,
attempts: claimed.attempt,
last_error: reason.to_string(),
trigger_id,
script_id,
first_attempt_at: claimed.enqueued_at,
last_attempt_at: Utc::now(),
// Shared-queue messages root a depth-1 chain (the
// queue is depth 0; a DL handler ticks up).
trigger_depth: 1,
root_execution_id: None,
},
)
.await;
}
Err(e) => tracing::error!(?e, "shared-queue dead-letter write failed"),
}
None
}
None => match self
.queue
.dead_letter(
claimed.id,
claimed.claim_token,
claimed.app_id,
&claimed.queue_name,
trigger_id,
script_id,
claimed.attempt,
claimed.enqueued_at,
reason,
)
.await
{
Ok(dl_id) => Some(dl_id),
Err(e) => {
tracing::error!(?e, "queue dead-letter write failed");
None
}
},
}
}
#[allow(clippy::too_many_lines)]
async fn dispatch_one_queue(
&self,
@@ -550,9 +320,10 @@ impl Dispatcher {
) -> Result<(), DispatcherError> {
// Atomic claim — None → nothing pending right now for this queue.
let Some(claimed) = self
.q_claim(consumer)
.queue
.claim(consumer.app_id, &consumer.queue_name)
.await
.map_err(DispatcherError::Outbox)?
.map_err(|e| DispatcherError::Outbox(e.to_string()))?
else {
return Ok(());
};
@@ -561,12 +332,9 @@ impl Dispatcher {
// immediate (which lets the next tick re-claim). Mirrors the
// outbox arm.
let Ok(permit) = self.gate.try_acquire() else {
// Transient: never executed → release without burning the retry
// budget (else sustained overload could dead-letter a message that
// ran zero times).
let _ = self
.q_release(
consumer,
.queue
.nack(
claimed.id,
claimed.claim_token,
chrono::Duration::milliseconds(100),
@@ -591,11 +359,16 @@ impl Dispatcher {
Ok(None) => {
tracing::warn!(script_id = %consumer.script_id, "queue trigger script missing; dead-lettering");
let _ = self
.q_terminal(
consumer,
&claimed,
.queue
.dead_letter(
claimed.id,
claimed.claim_token,
claimed.app_id,
&claimed.queue_name,
Some(consumer.trigger_id),
Some(consumer.script_id),
claimed.attempt,
claimed.enqueued_at,
"queue trigger script not found",
)
.await;
@@ -625,11 +398,16 @@ impl Dispatcher {
"queue consumer script belongs to a different app; dead-lettering"
);
let _ = self
.q_terminal(
consumer,
&claimed,
.queue
.dead_letter(
claimed.id,
claimed.claim_token,
claimed.app_id,
&claimed.queue_name,
Some(consumer.trigger_id),
Some(consumer.script_id),
claimed.attempt,
claimed.enqueued_at,
"queue consumer target belongs to a different app",
)
.await;
@@ -656,15 +434,15 @@ impl Dispatcher {
"queue consumer script disabled at fire time; releasing claim"
);
if let Err(e) = self
.q_release(
consumer,
.queue
.nack(
claimed.id,
claimed.claim_token,
chrono::Duration::seconds(1),
)
.await
{
tracing::warn!(?e, "queue release on disabled consumer failed");
tracing::warn!(?e, "queue nack on disabled consumer failed");
}
drop(permit);
return Ok(());
@@ -737,7 +515,7 @@ impl Dispatcher {
match outcome {
Ok(_) => {
// Auto-ack on success.
if let Err(e) = self.q_ack(consumer, claimed.id, claimed.claim_token).await {
if let Err(e) = self.queue.ack(claimed.id, claimed.claim_token).await {
tracing::warn!(?e, "queue ack failed");
}
}
@@ -764,7 +542,8 @@ impl Dispatcher {
);
let delay = chrono::Duration::milliseconds(i64::from(delay_ms));
if let Err(e) = self
.q_nack(consumer, claimed.id, claimed.claim_token, delay)
.queue
.nack(claimed.id, claimed.claim_token, delay)
.await
{
tracing::warn!(?e, "queue nack failed");
@@ -779,18 +558,27 @@ impl Dispatcher {
// same way here.
let now = Utc::now();
let last_error = err.to_string();
// Per-app → dead-letter (+ fan out below). Shared group queue →
// dead-lettered into `group_dead_letters` inside q_terminal (Track A M2),
// which returns None so the per-app fan-out is skipped.
let dl_id = self
.q_terminal(
consumer,
claimed,
let dl_id = match self
.queue
.dead_letter(
claimed.id,
claimed.claim_token,
claimed.app_id,
&claimed.queue_name,
Some(consumer.trigger_id),
Some(consumer.script_id),
claimed.attempt,
claimed.enqueued_at,
&last_error,
)
.await;
.await
{
Ok(dl_id) => Some(dl_id),
Err(e) => {
tracing::error!(?e, "queue dead-letter write failed");
None
}
};
if let Some(dead_letter_id) = dl_id {
let original = TriggerEvent::Queue {
queue_name: claimed.queue_name.clone(),
@@ -1533,83 +1321,6 @@ impl Dispatcher {
}
}
/// §11.6 B2: the shared analogue of `fan_out_dead_letter`. When a message in
/// a group's SHARED queue is exhausted, fire the group's `shared = true`
/// `dead_letter` handlers. Each outbox row is stamped `ctx.app_id` (the
/// writer/consumer that exhausted the message) so the handler runs under
/// that app's `SdkCallCx` — the M2 shared-write model. Best-effort, mirroring
/// the per-app path (the group dead-letter row is already durably written).
async fn fan_out_shared_dead_letter(
&self,
owning_group: picloud_shared::GroupId,
ctx: DeadLetterFanOutCtx,
) {
let DeadLetterFanOutCtx {
app_id,
original,
source,
dead_letter_id,
attempts,
last_error,
trigger_id,
script_id,
first_attempt_at,
last_attempt_at,
trigger_depth,
root_execution_id,
} = ctx;
let matches = match self
.triggers
.list_matching_shared_dead_letter(owning_group, &source, trigger_id, script_id)
.await
{
Ok(m) => m,
Err(e) => {
tracing::error!(?e, "shared dead-letter trigger lookup failed");
return;
}
};
for m in matches {
let event = TriggerEvent::DeadLetter {
dead_letter_id,
original: Box::new(original.clone()),
attempts,
last_error: last_error.clone(),
trigger_id,
script_id,
first_attempt_at,
last_attempt_at,
};
let payload = match serde_json::to_value(&event) {
Ok(p) => p,
Err(e) => {
tracing::error!(?e, "failed to serialize shared dead-letter event");
continue;
}
};
if let Err(e) = self
.outbox
.insert(NewOutboxRow {
// Writer app — the consumer that exhausted the message.
app_id,
source_kind: OutboxSourceKind::DeadLetter,
trigger_id: Some(m.trigger_id),
script_id: Some(m.script_id),
reply_to: None,
payload,
origin_principal: Some(m.registered_by_principal),
trigger_depth: trigger_depth.saturating_add(1),
root_execution_id,
})
.await
{
tracing::error!(?e, "failed to enqueue shared dead-letter handler delivery");
}
}
}
async fn deliver_inbox(&self, row: &OutboxRow, inbox_id: Uuid, result: InboxResult) {
match self.inbox.deliver(inbox_id, result.clone()).await {
InboxDeliveryOutcome::Delivered => {}
@@ -1681,7 +1392,6 @@ fn summarize(resp: &ExecResponse) -> ExecResponseSummary {
status_code: resp.status_code,
headers: resp.headers.clone(),
body: resp.body.clone(),
body_base64: resp.body_base64.clone(),
}
}
@@ -1802,89 +1512,6 @@ fn apply_jitter(raw: u32, pct: u32) -> u32 {
mod tests {
use super::*;
/// §11.6 D3: a do-nothing group-queue store for dispatcher tests that don't
/// exercise SHARED queues (every consumer has `shared_group: None`, so these
/// methods are never reached — they just satisfy the struct field).
struct NoopGroupQueue;
#[async_trait::async_trait]
impl crate::group_queue_repo::GroupQueueRepo for NoopGroupQueue {
async fn enqueue(
&self,
_msg: crate::group_queue_repo::NewGroupQueueMessage,
) -> Result<QueueMessageId, crate::group_queue_repo::GroupQueueRepoError> {
unreachable!("shared queue not exercised")
}
async fn claim(
&self,
_group_id: picloud_shared::GroupId,
_collection: &str,
) -> Result<
Option<crate::group_queue_repo::ClaimedGroupMessage>,
crate::group_queue_repo::GroupQueueRepoError,
> {
Ok(None)
}
async fn ack(
&self,
_id: QueueMessageId,
_token: Uuid,
) -> Result<bool, crate::group_queue_repo::GroupQueueRepoError> {
Ok(false)
}
async fn nack(
&self,
_id: QueueMessageId,
_token: Uuid,
_delay: chrono::Duration,
) -> Result<bool, crate::group_queue_repo::GroupQueueRepoError> {
Ok(false)
}
async fn release(
&self,
_id: QueueMessageId,
_token: Uuid,
_delay: chrono::Duration,
) -> Result<bool, crate::group_queue_repo::GroupQueueRepoError> {
Ok(false)
}
#[allow(clippy::too_many_arguments)]
async fn dead_letter(
&self,
_id: QueueMessageId,
_token: Uuid,
_group_id: picloud_shared::GroupId,
_collection: &str,
_trigger_id: Option<picloud_shared::TriggerId>,
_script_id: Option<ScriptId>,
_attempt: u32,
_first_attempt_at: chrono::DateTime<chrono::Utc>,
_last_error: &str,
) -> Result<picloud_shared::DeadLetterId, crate::group_queue_repo::GroupQueueRepoError>
{
unreachable!("shared queue not exercised")
}
async fn reclaim_visibility_timeouts(
&self,
) -> Result<u64, crate::group_queue_repo::GroupQueueRepoError> {
Ok(0)
}
async fn depth(
&self,
_group_id: picloud_shared::GroupId,
_collection: &str,
) -> Result<u64, crate::group_queue_repo::GroupQueueRepoError> {
Ok(0)
}
async fn depth_pending(
&self,
_group_id: picloud_shared::GroupId,
_collection: &str,
) -> Result<u64, crate::group_queue_repo::GroupQueueRepoError> {
Ok(0)
}
}
#[test]
fn exponential_backoff_doubles_per_attempt() {
// No jitter (pct=0) for a deterministic check.
@@ -2131,7 +1758,6 @@ mod tests {
struct ClaimNackQueue {
claimed: ClaimedMessage,
nacked: Arc<AtomicBool>,
released: Arc<AtomicBool>,
}
#[async_trait]
@@ -2152,29 +1778,20 @@ mod tests {
}
async fn ack(
&self,
_message_id: QueueMessageId,
_message_id: picloud_shared::QueueMessageId,
_claim_token: Uuid,
) -> Result<bool, crate::queue_repo::QueueRepoError> {
unimplemented!("not used by this test")
}
async fn nack(
&self,
_message_id: QueueMessageId,
_message_id: picloud_shared::QueueMessageId,
_claim_token: Uuid,
_retry_delay: chrono::Duration,
) -> Result<bool, crate::queue_repo::QueueRepoError> {
self.nacked.store(true, Ordering::SeqCst);
Ok(true)
}
async fn release(
&self,
_message_id: QueueMessageId,
_claim_token: Uuid,
_retry_delay: chrono::Duration,
) -> Result<bool, crate::queue_repo::QueueRepoError> {
self.released.store(true, Ordering::SeqCst);
Ok(true)
}
async fn reclaim_visibility_timeouts(
&self,
) -> Result<u64, crate::queue_repo::QueueRepoError> {
@@ -2203,7 +1820,7 @@ mod tests {
#[allow(clippy::too_many_arguments)]
async fn dead_letter(
&self,
_message_id: QueueMessageId,
_message_id: picloud_shared::QueueMessageId,
_claim_token: Uuid,
_app_id: AppId,
_queue_name: &str,
@@ -2473,12 +2090,6 @@ mod tests {
#[async_trait]
impl OutboxRepo for UnusedOutbox {
async fn reclaim_stale_claims(
&self,
_timeout_secs: u32,
) -> Result<u64, crate::outbox_repo::OutboxRepoError> {
Ok(0)
}
async fn insert(
&self,
_row: NewOutboxRow,
@@ -2594,11 +2205,9 @@ mod tests {
retry_backoff: BackoffShape::Exponential,
retry_base_ms: 1000,
registered_by_principal: AdminUserId::new(),
shared_group: None,
};
let nacked = Arc::new(AtomicBool::new(false));
let released = Arc::new(AtomicBool::new(false));
let executed = Arc::new(AtomicBool::new(false));
let dispatcher = Dispatcher {
@@ -2619,9 +2228,7 @@ mod tests {
queue: Arc::new(ClaimNackQueue {
claimed,
nacked: nacked.clone(),
released: released.clone(),
}),
group_queue: Arc::new(NoopGroupQueue),
config: TriggerConfig::from_env(),
instance_id: "test-instance".into(),
};
@@ -2630,16 +2237,10 @@ mod tests {
// 1. The disabled path returns Ok(()).
assert!(result.is_ok(), "dispatch_one_queue returned {result:?}");
// 2. The claim was RELEASED (transient — the handler never ran), not
// nacked: a disabled-at-fire release must not burn the retry
// budget (audit fix #4).
// 2. The claim was released via nack.
assert!(
released.load(Ordering::SeqCst),
"expected a transient release for a disabled consumer"
);
assert!(
!nacked.load(Ordering::SeqCst),
"a disabled-at-fire release must NOT count as a nack (retry budget)"
nacked.load(Ordering::SeqCst),
"expected nack to release the claim for a disabled consumer"
);
// 3. The executor was never reached. If the `if !script.enabled`
// gate is deleted, the flow falls through to resolve →
@@ -2709,22 +2310,14 @@ mod tests {
}
async fn ack(
&self,
_message_id: QueueMessageId,
_message_id: picloud_shared::QueueMessageId,
_claim_token: Uuid,
) -> Result<bool, crate::queue_repo::QueueRepoError> {
unimplemented!("not used by this test")
}
async fn nack(
&self,
_message_id: QueueMessageId,
_claim_token: Uuid,
_retry_delay: chrono::Duration,
) -> Result<bool, crate::queue_repo::QueueRepoError> {
unimplemented!("not used by this test")
}
async fn release(
&self,
_message_id: QueueMessageId,
_message_id: picloud_shared::QueueMessageId,
_claim_token: Uuid,
_retry_delay: chrono::Duration,
) -> Result<bool, crate::queue_repo::QueueRepoError> {
@@ -2761,7 +2354,7 @@ mod tests {
#[allow(clippy::too_many_arguments)]
async fn dead_letter(
&self,
_message_id: QueueMessageId,
_message_id: picloud_shared::QueueMessageId,
_claim_token: Uuid,
_app_id: AppId,
_queue_name: &str,
@@ -2784,12 +2377,6 @@ mod tests {
#[async_trait]
impl OutboxRepo for RecordingOutbox {
async fn reclaim_stale_claims(
&self,
_timeout_secs: u32,
) -> Result<u64, crate::outbox_repo::OutboxRepoError> {
Ok(0)
}
async fn insert(
&self,
_row: NewOutboxRow,
@@ -2874,7 +2461,6 @@ mod tests {
log_sink: Arc::new(UnusedLogSink),
inbox: Arc::new(UnusedInbox),
queue: Arc::new(UnusedQueue),
group_queue: Arc::new(NoopGroupQueue),
config: TriggerConfig::from_env(),
instance_id: "test-instance".into(),
};
@@ -2897,76 +2483,5 @@ mod tests {
"executor ran for a disabled HTTP outbox row; fire-time gate missing"
);
}
/// `trigger_depth` bounds chain DEPTH. The sync `invoke()` path is pinned
/// (sdk/invoke.rs), but the ASYNC dispatcher enforcement — `dispatch_one`
/// dropping a row past `max_trigger_depth` — was not. Without it, a trigger
/// whose handler writes back to the collection it watches is an infinite
/// outbox self-amplification loop (write→trigger→write→…) from one request.
#[tokio::test]
async fn outbox_row_past_max_depth_is_dropped_without_executing() {
let app_id = AppId::new();
// ENABLED script — so what stops execution is the DEPTH gate, not the
// disabled-drop path. (The depth check runs before script resolution,
// so the repo is never even consulted, but keep it honest.)
let mut script = disabled_script(app_id);
script.enabled = true;
let config = TriggerConfig::from_env();
let row_id = Uuid::new_v4();
let row = OutboxRow {
id: row_id,
app_id,
source_kind: OutboxSourceKind::Kv,
trigger_id: Some(TriggerId::new()),
script_id: Some(script.id),
reply_to: None,
payload: serde_json::json!({}),
origin_principal: None,
// One past the ceiling — must be dropped, not dispatched.
trigger_depth: config.max_trigger_depth + 1,
root_execution_id: None,
attempt_count: 0,
next_attempt_at: Utc::now(),
created_at: Utc::now(),
};
let deleted = Arc::new(Mutex::new(None));
let executed = Arc::new(AtomicBool::new(false));
let dispatcher = Dispatcher {
outbox: Arc::new(RecordingOutbox {
deleted: deleted.clone(),
}),
triggers: Arc::new(UnusedTriggers),
scripts: Arc::new(DisabledScriptRepo { script }),
dead_letters: Arc::new(UnusedDeadLetters),
abandoned: Arc::new(UnusedAbandoned),
principals: Arc::new(OkPrincipals),
executor: Arc::new(RecordingExecutor {
executed: executed.clone(),
}),
gate: Arc::new(ExecutionGate::new(1)),
log_sink: Arc::new(UnusedLogSink),
inbox: Arc::new(UnusedInbox),
queue: Arc::new(UnusedQueue),
group_queue: Arc::new(NoopGroupQueue),
config,
instance_id: "test-instance".into(),
};
let result = dispatcher.dispatch_one(row).await;
assert!(result.is_ok(), "dispatch_one returned {result:?}");
assert_eq!(
*deleted.lock().unwrap(),
Some(row_id),
"a depth-exceeded row must be deleted (not left to re-amplify)"
);
assert!(
!executed.load(Ordering::SeqCst),
"executor ran for a depth-exceeded row — the depth gate is missing, \
so a self-writing trigger would loop unbounded"
);
}
}
}

View File

@@ -1,186 +0,0 @@
//! `/api/v1/admin/apps/{id}/docs*` — read-only docs inspection.
//!
//! Mirrors `kv_api` / `files_api` so the `pic docs … --app` CLI (and a
//! future dashboard tab) can browse stored documents without a script.
//! The per-group equivalent shipped first (`group_blobs_api`); this is
//! its per-app counterpart, filling the one admin-read gap the other
//! data-plane services already covered. **Read-only by design** — docs
//! writes go through `docs::create/update/delete` in scripts, which emit
//! change events the trigger framework depends on; an admin write would
//! bypass that, so it is deliberately out of scope (matching kv/files).
//!
//! Two operations:
//! * `GET /apps/{id}/docs?collection=<c>&cursor=&limit=` — list docs in
//! a collection (cursor-paginated).
//! * `GET /apps/{id}/docs/{collection}/{doc_id}` — fetch one document.
//!
//! Capability: `AppDocsRead`, resolved against the app loaded from the
//! path (same tier the SDK read path uses).
use std::sync::Arc;
use axum::extract::{Path, Query, State};
use axum::response::{IntoResponse, Json, Response};
use axum::routing::get;
use axum::{Extension, Router};
use picloud_shared::{AppId, Principal};
use serde::{Deserialize, Serialize};
use serde_json::json;
use uuid::Uuid;
use crate::app_repo::AppRepository;
use crate::authz::{require, AuthzDenied, AuthzRepo, Capability};
use crate::docs_repo::DocsRepo;
#[derive(Clone)]
pub struct DocsAdminState {
pub docs: Arc<dyn DocsRepo>,
pub apps: Arc<dyn AppRepository>,
pub authz: Arc<dyn AuthzRepo>,
}
pub fn docs_admin_router(state: DocsAdminState) -> Router {
Router::new()
.route("/apps/{app_id}/docs", get(list_docs))
.route("/apps/{app_id}/docs/{collection}/{doc_id}", get(get_doc))
.with_state(state)
}
#[derive(Debug, Deserialize)]
pub struct ListDocsQuery {
pub collection: String,
#[serde(default)]
pub cursor: Option<String>,
#[serde(default)]
pub limit: Option<u32>,
}
/// Mirrors the `group_blobs_api` docs shape so the CLI can share one
/// deserialize across `--app` and `--group`.
#[derive(Debug, Serialize)]
struct DocEntry {
id: String,
data: serde_json::Value,
}
#[derive(Debug, Serialize)]
struct ListDocsResponse {
docs: Vec<DocEntry>,
next_cursor: Option<String>,
}
async fn list_docs(
State(s): State<DocsAdminState>,
Extension(principal): Extension<Principal>,
Path(id_or_slug): Path<String>,
Query(q): Query<ListDocsQuery>,
) -> Result<Json<ListDocsResponse>, DocsApiError> {
let app_id = resolve_app(&*s.apps, &id_or_slug).await?;
require(
s.authz.as_ref(),
&principal,
Capability::AppDocsRead(app_id),
)
.await?;
let page = s
.docs
.list(
app_id,
&q.collection,
q.cursor.as_deref(),
q.limit.unwrap_or(0),
)
.await
.map_err(|e| DocsApiError::Backend(e.to_string()))?;
Ok(Json(ListDocsResponse {
docs: page
.docs
.into_iter()
.map(|d| DocEntry {
id: d.id.to_string(),
data: d.data,
})
.collect(),
next_cursor: page.next_cursor,
}))
}
async fn get_doc(
State(s): State<DocsAdminState>,
Extension(principal): Extension<Principal>,
Path((id_or_slug, collection, doc_id)): Path<(String, String, String)>,
) -> Result<Json<serde_json::Value>, DocsApiError> {
let app_id = resolve_app(&*s.apps, &id_or_slug).await?;
require(
s.authz.as_ref(),
&principal,
Capability::AppDocsRead(app_id),
)
.await?;
let id = doc_id.parse::<Uuid>().map_err(|_| DocsApiError::NotFound)?;
let row = s
.docs
.get(app_id, &collection, id)
.await
.map_err(|e| DocsApiError::Backend(e.to_string()))?
.ok_or(DocsApiError::NotFound)?;
Ok(Json(json!({ "id": row.id.to_string(), "data": row.data })))
}
async fn resolve_app(apps: &dyn AppRepository, ident: &str) -> Result<AppId, DocsApiError> {
crate::app_repo::resolve_app(apps, ident)
.await
.map_err(|e| DocsApiError::Backend(e.to_string()))?
.map(|l| l.app.id)
.ok_or(DocsApiError::AppNotFound)
}
#[derive(Debug, thiserror::Error)]
pub enum DocsApiError {
#[error("app not found")]
AppNotFound,
#[error("document not found")]
NotFound,
#[error("forbidden")]
Forbidden,
#[error("authorization repo error: {0}")]
AuthzRepo(String),
#[error("docs backend: {0}")]
Backend(String),
}
impl From<AuthzDenied> for DocsApiError {
fn from(d: AuthzDenied) -> Self {
match d {
AuthzDenied::Denied => Self::Forbidden,
AuthzDenied::Repo(e) => Self::AuthzRepo(e.to_string()),
}
}
}
impl IntoResponse for DocsApiError {
fn into_response(self) -> Response {
use axum::http::StatusCode;
let (status, body) = match &self {
Self::AppNotFound | Self::NotFound => {
(StatusCode::NOT_FOUND, json!({ "error": self.to_string() }))
}
Self::Forbidden => (StatusCode::FORBIDDEN, json!({ "error": self.to_string() })),
Self::AuthzRepo(e) => {
tracing::error!(error = %e, "docs admin authz error");
(
StatusCode::INTERNAL_SERVER_ERROR,
json!({ "error": "internal error" }),
)
}
Self::Backend(e) => {
tracing::error!(error = %e, "docs admin backend error");
(
StatusCode::INTERNAL_SERVER_ERROR,
json!({ "error": "internal error" }),
)
}
};
(status, Json(body)).into_response()
}
}

View File

@@ -150,12 +150,6 @@ pub enum ComparisonOp {
/// `$in` — `= ANY($M::text[])` where the value list is bound as
/// a TEXT[].
In,
/// `$contains` — JSONB containment (`@>`) for array-membership:
/// matches a doc whose stored *array* field contains the given
/// scalar element (e.g. `{ tags: { "$contains": "rust" } }` finds
/// docs tagged `rust`). The `$in` operator is the inverse (field is
/// one of a list); this one is (list field contains a value).
Contains,
}
impl ComparisonOp {
@@ -175,7 +169,6 @@ impl ComparisonOp {
"$lt" => Ok(Self::Lt),
"$lte" => Ok(Self::Lte),
"$in" => Ok(Self::In),
"$contains" => Ok(Self::Contains),
other => Err(FilterParseError::UnsupportedOperator(format!(
"docs::find: operator '{other}' is not supported in v1.1.2; planned for v1.2 advanced query"
))),
@@ -328,7 +321,6 @@ const fn op_name(op: ComparisonOp) -> &'static str {
ComparisonOp::Lt => "$lt",
ComparisonOp::Lte => "$lte",
ComparisonOp::In => "$in",
ComparisonOp::Contains => "$contains",
}
}
@@ -473,17 +465,6 @@ mod tests {
// $in needs an array.
let f = parse(json!({ "tier": { "$in": ["gold", "platinum"] } })).unwrap();
assert_eq!(f.conditions[0].op, ComparisonOp::In);
// $contains takes a scalar (the element to find in an array field).
let f = parse(json!({ "tags": { "$contains": "rust" } })).unwrap();
assert_eq!(f.conditions[0].op, ComparisonOp::Contains);
}
#[test]
fn contains_with_object_value_rejected() {
// The element to find must be a scalar, not a nested object/array.
let err = parse(json!({ "tags": { "$contains": { "k": 1 } } })).unwrap_err();
assert!(err.to_string().contains("'$contains'"));
assert!(err.to_string().contains("scalar"));
}
#[test]

View File

@@ -118,7 +118,24 @@ impl DocsRepo for PostgresDocsRepo {
collection: &str,
data: Value,
) -> Result<DocRow, DocsRepoError> {
create_on(&self.pool, app_id, collection, data).await
let id = Uuid::new_v4();
let row: (DateTime<Utc>, DateTime<Utc>) = sqlx::query_as(
"INSERT INTO docs (app_id, collection, id, data) \
VALUES ($1, $2, $3, $4) \
RETURNING created_at, updated_at",
)
.bind(app_id.into_inner())
.bind(collection)
.bind(id)
.bind(&data)
.fetch_one(&self.pool)
.await?;
Ok(DocRow {
id,
data,
created_at: row.0,
updated_at: row.1,
})
}
async fn get(
@@ -150,7 +167,7 @@ impl DocsRepo for PostgresDocsRepo {
collection: &str,
filter: &DocsFilter,
) -> Result<Vec<DocRow>, DocsRepoError> {
let mut qb = build_find_query("docs", "app_id", app_id.into_inner(), collection, filter);
let mut qb = build_find_query(app_id, collection, filter);
let rows = qb.build().fetch_all(&self.pool).await?;
rows.into_iter().map(row_to_doc).collect()
}
@@ -162,7 +179,34 @@ impl DocsRepo for PostgresDocsRepo {
id: DocId,
data: Value,
) -> Result<Option<Value>, DocsRepoError> {
update_on(&self.pool, app_id, collection, id, data).await
// Same CTE shape as KV's set ([kv_repo.rs:101-132]): SELECT the
// previous data before the UPDATE so the service can emit
// `prev_data` in the update ServiceEvent. Single statement, no
// explicit transaction. Inherits KV's last-writer-wins race
// under concurrent writers; documented as a known limitation
// for v1.1.2.
let row: Option<(Option<Value>,)> = sqlx::query_as(
"WITH prev AS ( \
SELECT data FROM docs \
WHERE app_id = $1 AND collection = $2 AND id = $3 \
), \
updated AS ( \
UPDATE docs SET data = $4, updated_at = NOW() \
WHERE app_id = $1 AND collection = $2 AND id = $3 \
RETURNING 1 \
) \
SELECT (SELECT data FROM prev) FROM updated",
)
.bind(app_id.into_inner())
.bind(collection)
.bind(id)
.bind(&data)
.fetch_optional(&self.pool)
.await?;
// `row` is None when the UPDATE matched no rows (missing doc);
// Some((Some(prev),)) on success. `data` is JSONB NOT NULL so
// the inner Option is always Some when prev exists.
Ok(row.and_then(|(v,)| v))
}
async fn delete(
@@ -171,7 +215,17 @@ impl DocsRepo for PostgresDocsRepo {
collection: &str,
id: DocId,
) -> Result<Option<Value>, DocsRepoError> {
delete_on(&self.pool, app_id, collection, id).await
let row: Option<(Value,)> = sqlx::query_as(
"DELETE FROM docs \
WHERE app_id = $1 AND collection = $2 AND id = $3 \
RETURNING data",
)
.bind(app_id.into_inner())
.bind(collection)
.bind(id)
.fetch_optional(&self.pool)
.await?;
Ok(row.map(|(v,)| v))
}
async fn list(
@@ -227,7 +281,7 @@ impl DocsRepo for PostgresDocsRepo {
}
}
pub(crate) fn row_to_doc(row: PgRow) -> Result<DocRow, DocsRepoError> {
fn row_to_doc(row: PgRow) -> Result<DocRow, DocsRepoError> {
Ok(DocRow {
id: row.try_get("id")?,
data: row.try_get("data")?,
@@ -262,22 +316,14 @@ fn decode_cursor(cursor: &str) -> Result<Uuid, DocsRepoError> {
// **No user input ever lands in the SQL text unparameterized.**
// ----------------------------------------------------------------------------
/// Build the `find` query for a docs store. `table` and `owner_col` are
/// compile-time literals (`"docs"`/`"app_id"` for app docs, `"group_docs"`/
/// `"group_id"` for §11.6 group-shared docs) — never user input, so
/// interpolating them is injection-safe. `pub(crate)` so the group-docs repo
/// reuses this single source for the (security-sensitive) query SQL.
pub(crate) fn build_find_query<'a>(
table: &'static str,
owner_col: &'static str,
owner_id: uuid::Uuid,
fn build_find_query<'a>(
app_id: AppId,
collection: &'a str,
filter: &'a DocsFilter,
) -> QueryBuilder<'a, Postgres> {
let mut qb = QueryBuilder::new(format!(
"SELECT id, data, created_at, updated_at FROM {table} WHERE {owner_col} = "
));
qb.push_bind(owner_id);
let mut qb =
QueryBuilder::new("SELECT id, data, created_at, updated_at FROM docs WHERE app_id = ");
qb.push_bind(app_id.into_inner());
qb.push(" AND collection = ");
qb.push_bind(collection);
@@ -311,20 +357,6 @@ fn emit_condition<'a>(
qb: &mut QueryBuilder<'a, Postgres>,
cond: &'a crate::docs_filter::FieldCondition,
) {
// `$contains` uses JSONB containment (`@>`), which needs the field's
// *JSONB* value, not the text rendering the other operators compare
// against — so it builds its own path expression and short-circuits.
// Emits `jsonb_extract_path(data, $seg…) @> $val::jsonb`, matching a doc
// whose stored array field contains the given scalar element (F-015).
if cond.op == ComparisonOp::Contains {
push_jsonb_path_value(qb, cond.path.segments());
qb.push(" @> ");
// `Value::to_string()` is the compact JSON encoding (`"rust"`, `5`,
// `true`), i.e. a valid JSONB literal — bound, never interpolated.
qb.push_bind(cond.value.to_string());
qb.push("::jsonb");
return;
}
push_jsonb_path(qb, cond.path.segments());
match cond.op {
ComparisonOp::Eq => {
@@ -372,7 +404,6 @@ fn emit_condition<'a>(
qb.push_bind(texts);
qb.push(")");
}
ComparisonOp::Contains => unreachable!("handled before the match"),
}
}
@@ -388,18 +419,6 @@ fn push_jsonb_path<'a>(qb: &mut QueryBuilder<'a, Postgres>, segments: &'a [Strin
qb.push(")");
}
/// Like [`push_jsonb_path`] but returns the *JSONB* value rather than its
/// text rendering (`jsonb_extract_path`, no `_text`). Used by `$contains`,
/// whose `@>` operator needs a JSONB left operand. Segments bound as params.
fn push_jsonb_path_value<'a>(qb: &mut QueryBuilder<'a, Postgres>, segments: &'a [String]) {
qb.push("jsonb_extract_path(data");
for seg in segments {
qb.push(", ");
qb.push_bind(seg.as_str());
}
qb.push(")");
}
/// JSON scalar → TEXT for binding. `Value::Null` is preserved as
/// `None` so the binding lands as SQL NULL (handled specially above for
/// `Eq` / `Ne`). Arrays + objects serialize to compact JSON; the user
@@ -421,113 +440,6 @@ fn value_to_text(v: &Value) -> Option<String> {
// pin the cross-app isolation invariant at the SQL level.
// ----------------------------------------------------------------------------
// ----------------------------------------------------------------------------
// Connection-scoped mutations
//
// Generic over the executor so the same SQL serves the pooled trait methods
// above and `crate::atomic_write`, which runs the write and the trigger fan-out
// it produces on ONE connection inside ONE transaction.
// ----------------------------------------------------------------------------
pub(crate) async fn create_on<'c, E>(
exec: E,
app_id: AppId,
collection: &str,
data: Value,
) -> Result<DocRow, DocsRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let id = Uuid::new_v4();
let row: (DateTime<Utc>, DateTime<Utc>) = sqlx::query_as(
"INSERT INTO docs (app_id, collection, id, data) \
VALUES ($1, $2, $3, $4) \
RETURNING created_at, updated_at",
)
.bind(app_id.into_inner())
.bind(collection)
.bind(id)
.bind(&data)
.fetch_one(exec)
.await?;
Ok(DocRow {
id,
data,
created_at: row.0,
updated_at: row.1,
})
}
/// Returns the previous data (for the update event), `None` if the doc is
/// missing. The CTE captures the prior data alongside the UPDATE so the service
/// can emit `prev_data` without a second round trip.
pub(crate) async fn update_on<'c, E>(
exec: E,
app_id: AppId,
collection: &str,
id: DocId,
data: Value,
) -> Result<Option<Value>, DocsRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let row: Option<(Option<Value>,)> = sqlx::query_as(
"WITH prev AS ( \
SELECT data FROM docs \
WHERE app_id = $1 AND collection = $2 AND id = $3 \
), \
updated AS ( \
UPDATE docs SET data = $4, updated_at = NOW() \
WHERE app_id = $1 AND collection = $2 AND id = $3 \
RETURNING 1 \
) \
SELECT (SELECT data FROM prev) FROM updated",
)
.bind(app_id.into_inner())
.bind(collection)
.bind(id)
.bind(&data)
.fetch_optional(exec)
.await?;
Ok(row.and_then(|(v,)| v))
}
/// Returns the deleted doc's data if it existed, `None` if no such doc.
pub(crate) async fn delete_on<'c, E>(
exec: E,
app_id: AppId,
collection: &str,
id: DocId,
) -> Result<Option<Value>, DocsRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let row: Option<(Value,)> = sqlx::query_as(
"DELETE FROM docs \
WHERE app_id = $1 AND collection = $2 AND id = $3 \
RETURNING data",
)
.bind(app_id.into_inner())
.bind(collection)
.bind(id)
.fetch_optional(exec)
.await?;
Ok(row.map(|(v,)| v))
}
/// Total doc count for the app, across all its collections. Backs the per-app
/// row ceiling; run only on `create`.
pub(crate) async fn count_rows_on<'c, E>(exec: E, app_id: AppId) -> Result<u64, DocsRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let (n,): (i64,) = sqlx::query_as("SELECT COUNT(*) FROM docs WHERE app_id = $1")
.bind(app_id.into_inner())
.fetch_one(exec)
.await?;
Ok(u64::try_from(n).unwrap_or(0))
}
#[cfg(test)]
mod sql_shape_tests {
use super::*;
@@ -536,13 +448,7 @@ mod sql_shape_tests {
fn sql_for(filter_json: serde_json::Value) -> String {
let filter = parse_filter(&filter_json).unwrap();
let qb = build_find_query(
"docs",
"app_id",
AppId::new().into_inner(),
"users",
&filter,
);
let qb = build_find_query(AppId::new(), "users", &filter);
qb.sql().to_string()
}
@@ -561,7 +467,6 @@ mod sql_shape_tests {
json!({ "$sort": { "created_at": -1 }, "$limit": 5 }),
json!({ "tier": "gold", "$sort": { "created_at": 1 } }),
json!({ "deleted_at": { "$ne": null } }),
json!({ "tags": { "$contains": "rust" } }),
];
for case in cases {
let sql = sql_for(case.clone());
@@ -648,16 +553,4 @@ mod sql_shape_tests {
let sql = sql_for(json!({ "user.email": "a@b" }));
assert!(sql.contains("jsonb_extract_path_text(data"), "sql: {sql}");
}
#[test]
fn contains_emits_jsonb_containment() {
// `$contains` must use the JSONB (not text) extractor + `@>`, with the
// element bound as a `::jsonb` parameter — never interpolated (F-015).
let sql = sql_for(json!({ "tags": { "$contains": "rust" } }));
assert!(sql.contains("jsonb_extract_path(data"), "sql: {sql}");
assert!(!sql.contains("jsonb_extract_path_text(data"), "sql: {sql}");
assert!(sql.contains(" @> "), "sql: {sql}");
assert!(sql.contains("::jsonb"), "sql: {sql}");
assert!(!sql.contains("rust"), "value leaked into SQL string: {sql}");
}
}

View File

@@ -26,10 +26,10 @@ use std::sync::Arc;
use async_trait::async_trait;
use picloud_shared::{
DocId, DocRow, DocsError, DocsListPage, DocsService, SdkCallCx, ServiceEventEmitter,
DocId, DocRow, DocsError, DocsListPage, DocsService, SdkCallCx, ServiceEvent,
ServiceEventEmitter,
};
use crate::atomic_write::{BestEffortDocsWriter, DocsWriter, PostgresDocsWriter};
use crate::authz::{self, AuthzRepo, Capability};
use crate::docs_filter::{parse_filter, FilterParseError};
use crate::docs_repo::{DocsRepo, DocsRepoError};
@@ -55,11 +55,9 @@ pub fn docs_max_value_bytes_from_env() -> usize {
}
pub struct DocsServiceImpl {
/// Reads only. Mutations go through `writer`, which owns the write AND the
/// trigger fan-out so the two can share a transaction.
repo: Arc<dyn DocsRepo>,
authz: Arc<dyn AuthzRepo>,
writer: Arc<dyn DocsWriter>,
events: Arc<dyn ServiceEventEmitter>,
max_value_bytes: usize,
}
@@ -81,23 +79,13 @@ impl DocsServiceImpl {
max_value_bytes: usize,
) -> Self {
Self {
writer: Arc::new(BestEffortDocsWriter::new(repo.clone(), events)),
repo,
authz,
events,
max_value_bytes,
}
}
/// Swap the best-effort writer for the transactional one: the write and its
/// trigger fan-out then commit together, so an outbox failure rolls the
/// write back instead of silently losing the event. The host always calls
/// this; the in-memory unit tests do not.
#[must_use]
pub fn with_atomic_writes(mut self, pool: sqlx::PgPool, max_rows: u64) -> Self {
self.writer = Arc::new(PostgresDocsWriter::new(pool, max_rows));
self
}
fn check_data_size(&self, data: &serde_json::Value) -> Result<(), DocsError> {
let encoded_len = serde_json::to_vec(data)
.map(|v| v.len())
@@ -175,7 +163,30 @@ impl DocsService for DocsServiceImpl {
validate_data(&data)?;
self.check_data_size(&data)?;
self.check_write(cx).await?;
self.writer.create(cx, collection, data).await
let row = self
.repo
.create(cx.app_id, collection, data.clone())
.await?;
// Best-effort emit — a failed emit logs but does not roll back
// the write (mirrors KV's pattern).
if let Err(e) = self
.events
.emit(
cx,
ServiceEvent {
source: "docs",
op: "create",
collection: Some(collection.to_string()),
key: Some(row.id.to_string()),
payload: Some(data),
old_payload: None,
},
)
.await
{
tracing::error!(error = %e, source = "docs", op = "create", event_emit_failure = true, "event emit failed");
}
Ok(row.id)
}
async fn get(
@@ -230,17 +241,60 @@ impl DocsService for DocsServiceImpl {
validate_data(&data)?;
self.check_data_size(&data)?;
self.check_write(cx).await?;
if self.writer.update(cx, collection, id, data).await? {
Ok(())
} else {
Err(DocsError::NotFound)
let previous = self
.repo
.update(cx.app_id, collection, id, data.clone())
.await?;
match previous {
Some(prev) => {
if let Err(e) = self
.events
.emit(
cx,
ServiceEvent {
source: "docs",
op: "update",
collection: Some(collection.to_string()),
key: Some(id.to_string()),
payload: Some(data),
old_payload: Some(prev),
},
)
.await
{
tracing::error!(error = %e, source = "docs", op = "update", event_emit_failure = true, "event emit failed");
}
Ok(())
}
None => Err(DocsError::NotFound),
}
}
async fn delete(&self, cx: &SdkCallCx, collection: &str, id: DocId) -> Result<bool, DocsError> {
validate_collection(collection)?;
self.check_write(cx).await?;
self.writer.delete(cx, collection, id).await
let previous = self.repo.delete(cx.app_id, collection, id).await?;
let was_present = previous.is_some();
if let Some(prev) = previous {
if let Err(e) = self
.events
.emit(
cx,
ServiceEvent {
source: "docs",
op: "delete",
collection: Some(collection.to_string()),
key: Some(id.to_string()),
payload: None,
old_payload: Some(prev),
},
)
.await
{
tracing::error!(error = %e, source = "docs", op = "delete", event_emit_failure = true, "event emit failed");
}
}
Ok(was_present)
}
async fn list(
@@ -452,22 +506,6 @@ mod tests {
};
arr.iter().any(|v| actual == json_text(v).as_deref())
}
Contains => {
// `actual` is the text rendering of the stored field; for an
// array field that's compact JSON. Parse it back and test
// element membership, mirroring Postgres `@>` (a scalar field
// equal to the value also "contains" it).
let Some(text) = actual else {
return false;
};
match serde_json::from_str::<serde_json::Value>(text) {
Ok(serde_json::Value::Array(items)) => {
items.iter().any(|v| json_text(v).as_deref() == want_ref)
}
Ok(other) => json_text(&other).as_deref() == want_ref,
Err(_) => false,
}
}
}
}
@@ -845,35 +883,6 @@ mod tests {
assert_eq!(hits.len(), 2);
}
#[tokio::test]
async fn find_with_contains_matches_array_membership() {
// F-015: `$contains` finds docs whose ARRAY field holds the element —
// the "posts with tag X" query that `$in` cannot express.
let s = svc();
let cx = anon_cx(AppId::new());
s.create(&cx, "posts", json!({ "tags": ["intro", "rust"] }))
.await
.unwrap();
s.create(&cx, "posts", json!({ "tags": ["rust", "async"] }))
.await
.unwrap();
s.create(&cx, "posts", json!({ "tags": ["cooking"] }))
.await
.unwrap();
let hits = s
.find(&cx, "posts", json!({ "tags": { "$contains": "rust" } }))
.await
.unwrap();
assert_eq!(hits.len(), 2, "expected the two rust-tagged posts");
let none = s
.find(&cx, "posts", json!({ "tags": { "$contains": "python" } }))
.await
.unwrap();
assert!(none.is_empty());
}
#[tokio::test]
async fn find_one_explicit_limit_is_honoured() {
// The service injects limit=1 ONLY when caller didn't set
@@ -931,64 +940,4 @@ mod tests {
s.update(&cx, "users", id, json!({ "x": 2 })).await.unwrap();
let _ = s.delete(&cx, "users", id).await.unwrap();
}
/// `PICLOUD_DOCS_MAX_VALUE_BYTES` — the docs analogue of the KV anti-DoS rail.
/// Pins that an oversized document is rejected BEFORE authz, and covers create
/// AND update (both encode-and-check). The cx is an authenticated member with
/// no role (authz-denied) so the ordering is observable: size-first returns
/// `ValueTooLarge`, authz-first would return `Forbidden`. An anon cx couldn't
/// tell the two apart.
#[tokio::test]
async fn oversized_document_is_rejected_before_authz() {
let s = DocsServiceImpl::with_max_value_bytes(
Arc::new(InMemoryDocsRepo::default()),
Arc::new(DenyingAuthzRepo),
Arc::new(NoopEventEmitter),
16,
);
let cx = member_no_role_cx(AppId::new());
let err = s
.create(&cx, "users", json!({ "blob": "x".repeat(100) }))
.await
.unwrap_err();
assert!(
matches!(err, DocsError::ValueTooLarge { limit: 16, .. }),
"an oversized create must be ValueTooLarge before authz; got {err:?}"
);
// Control: an under-cap create with the same cx is Forbidden — the cx is
// genuinely authz-denied, so the case above bypassed authz. And it seeds a
// doc via an ALLOWING service so we can then test the update path.
assert!(
matches!(
s.create(&cx, "users", json!({ "x": 1 })).await.unwrap_err(),
DocsError::Forbidden
),
"the control confirms this cx is authz-denied"
);
// Update also encodes-and-checks before authz — an update that skipped the
// cap would be a free bypass. Seed with an allowing service, then update
// through the denied one.
let seeder = DocsServiceImpl::with_max_value_bytes(
Arc::new(InMemoryDocsRepo::default()),
Arc::new(AllowingAuthzRepo),
Arc::new(NoopEventEmitter),
16,
);
let owner = owner_cx(AppId::new());
let id = seeder
.create(&owner, "users", json!({ "x": 1 }))
.await
.unwrap();
let err = seeder
.update(&owner, "users", id, json!({ "blob": "x".repeat(100) }))
.await
.unwrap_err();
assert!(
matches!(err, DocsError::ValueTooLarge { limit: 16, .. }),
"an oversized update must be ValueTooLarge too; got {err:?}"
);
}
}

View File

@@ -36,7 +36,7 @@ use serde_json::json;
use sha2::{Digest, Sha256};
use crate::outbox_repo::{NewOutboxRow, OutboxRepo, OutboxSourceKind};
use crate::secrets_service::{open_email, SecretOwner};
use crate::secrets_service::open_legacy;
use crate::trigger_repo::TriggerRepo;
type HmacSha256 = Hmac<Sha256>;
@@ -244,20 +244,7 @@ async fn receive_inbound_email(
s.bad_sig_limiter.record_failure(app_id, trigger_id);
return Err(EmailInboundError::Unauthorized);
};
// §M5.5 / audit H-D1: recover the SEALING owner for a v1 (AAD-bound) secret —
// the source-template group for a materialized copy, else this app. v0 rows
// ignore the owner (legacy no-AAD path).
let sealing_owner = match target.sealing_group {
Some(group_id) => SecretOwner::Group(group_id),
None => SecretOwner::App(app_id),
};
let secret = decrypt_secret(
&s.master_key,
sealing_owner,
ct,
nonce,
target.inbound_secret_version,
)?;
let secret = decrypt_secret(&s.master_key, ct, nonce)?;
if let Err(err) = verify_signature(&headers, &body, secret.as_bytes(), &s.nonce_dedup) {
s.bad_sig_limiter.record_failure(app_id, trigger_id);
return Err(err);
@@ -299,18 +286,16 @@ async fn receive_inbound_email(
Ok(StatusCode::ACCEPTED)
}
/// Decrypt the stored inbound secret back to its raw string. Sealed as a JSON
/// string by the admin/apply layer; `open_email` dispatches on `version` (v0 =
/// legacy no-AAD, v1 = AAD bound to the sealing `owner`), yielding a
/// Decrypt the stored inbound secret back to its raw string. It was
/// sealed as a JSON string by the admin layer (v0, no AAD — see
/// secrets_service::seal_legacy), so `open_legacy` yields a
/// `Value::String`.
fn decrypt_secret(
master_key: &MasterKey,
owner: SecretOwner,
ciphertext: &[u8],
nonce: &[u8],
version: i16,
) -> Result<String, EmailInboundError> {
let value = open_email(master_key, owner, ciphertext, nonce, version).map_err(|_| {
let value = open_legacy(master_key, ciphertext, nonce).map_err(|_| {
// Corrupted secret means we can't verify — fail closed (401).
EmailInboundError::Unauthorized
})?;
@@ -436,9 +421,8 @@ mod tests {
//! Postgres in `crates/picloud/tests/email_inbound.rs`.
use super::*;
use crate::secrets_service::seal_legacy;
use crate::secrets_service::DEFAULT_SECRET_MAX_VALUE_BYTES;
use crate::secrets_service::{seal_email, seal_legacy};
use picloud_shared::{AppId, GroupId};
fn sign(secret: &[u8], ts: i64, body: &[u8]) -> String {
let mut mac = HmacSha256::new_from_slice(secret).unwrap();
@@ -536,36 +520,14 @@ mod tests {
#[test]
fn secret_round_trips_through_seal_open() {
let key = MasterKey::from_bytes([3u8; 32]);
// v0 (legacy no-AAD) still opens — backward compatibility for pre-M3 rows.
let (ct0, nonce0) = seal_legacy(
let (ct, nonce) = seal_legacy(
&key,
&serde_json::Value::String("provider-secret".into()),
DEFAULT_SECRET_MAX_VALUE_BYTES,
)
.unwrap();
let owner = SecretOwner::App(AppId::from(uuid::Uuid::from_u128(1)));
let recovered = decrypt_secret(&key, owner, &ct0, &nonce0, 0).unwrap();
let recovered = decrypt_secret(&key, &ct, &nonce).unwrap();
assert_eq!(recovered, "provider-secret");
// v1 (AAD bound to the sealing owner) round-trips under the SAME owner.
let (ct1, nonce1, ver) = seal_email(
&key,
owner,
&serde_json::Value::String("provider-secret".into()),
DEFAULT_SECRET_MAX_VALUE_BYTES,
)
.unwrap();
assert_eq!(ver, 1);
assert_eq!(
decrypt_secret(&key, owner, &ct1, &nonce1, ver).unwrap(),
"provider-secret"
);
// A DIFFERENT owner (cross-tenant relocation) fails the GCM tag → 401.
let other = SecretOwner::Group(GroupId::from(uuid::Uuid::from_u128(2)));
assert!(
decrypt_secret(&key, other, &ct1, &nonce1, ver).is_err(),
"a v1 secret must not open under a different sealing owner"
);
let body = br#"{"from":"x@y.com"}"#;
let ts = now_ts();
let sig = sign(recovered.as_bytes(), ts, body);

View File

@@ -1,117 +0,0 @@
//! Extension-point markers (§5.5) — the `extension_points` table (0051).
//!
//! An extension point is a pure marker `(owner, name)` declaring that a module
//! name is one descendants may provide/override (the import resolver then
//! resolves it dynamically against the inheriting app — see
//! [`crate::module_source`]). This module holds the read + transactional-write
//! helpers; it mirrors the var/secret tx-function style (free functions over a
//! `&PgPool` / `&mut Transaction`), keyed by the shared [`ScriptOwner`].
use picloud_shared::{AppId, ScriptOwner};
use sqlx::{PgPool, Postgres, Transaction};
use crate::config_resolver::CHAIN_LEVELS_CTE;
/// List the EP names declared **directly at** `owner` (not inherited),
/// case-insensitively sorted. Used by `load_current` (apply diff), the
/// no-provider check, and the read-only `extension-points ls`.
pub async fn list_for_owner(pool: &PgPool, owner: ScriptOwner) -> Result<Vec<String>, sqlx::Error> {
let rows: Vec<(String,)> = match owner {
ScriptOwner::App(a) => {
sqlx::query_as(
"SELECT name FROM extension_points WHERE app_id = $1 ORDER BY LOWER(name)",
)
.bind(a.into_inner())
.fetch_all(pool)
.await?
}
ScriptOwner::Group(g) => {
sqlx::query_as(
"SELECT name FROM extension_points WHERE group_id = $1 ORDER BY LOWER(name)",
)
.bind(g.into_inner())
.fetch_all(pool)
.await?
}
};
Ok(rows.into_iter().map(|(n,)| n).collect())
}
/// All distinct EP names **visible to an app** — declared at the app or any
/// ancestor group, walking `apps.group_id → groups.parent_id`. Used by the
/// no-provider plan check and the read-only `extension-points ls --app`.
pub async fn list_on_app_chain(pool: &PgPool, app_id: AppId) -> Result<Vec<String>, sqlx::Error> {
let rows: Vec<(String,)> = sqlx::query_as(&format!(
"{CHAIN_LEVELS_CTE} \
SELECT DISTINCT e.name FROM extension_points e \
JOIN chain c ON (e.app_id = c.app_owner OR e.group_id = c.group_owner) \
ORDER BY e.name",
))
.bind(app_id.into_inner())
.fetch_all(pool)
.await?;
Ok(rows.into_iter().map(|(n,)| n).collect())
}
/// Insert an EP marker at `owner`, in the apply transaction. Idempotent: a
/// re-apply of an already-declared name is a no-op (`ON CONFLICT DO NOTHING`),
/// so the marker survives without a spurious version bump.
pub async fn insert_extension_point_tx(
tx: &mut Transaction<'_, Postgres>,
owner: ScriptOwner,
name: &str,
) -> Result<(), sqlx::Error> {
match owner {
ScriptOwner::App(a) => {
sqlx::query(
"INSERT INTO extension_points (app_id, name) VALUES ($1, $2) \
ON CONFLICT (app_id, LOWER(name)) WHERE app_id IS NOT NULL DO NOTHING",
)
.bind(a.into_inner())
.bind(name)
.execute(&mut **tx)
.await?;
}
ScriptOwner::Group(g) => {
sqlx::query(
"INSERT INTO extension_points (group_id, name) VALUES ($1, $2) \
ON CONFLICT (group_id, LOWER(name)) WHERE group_id IS NOT NULL DO NOTHING",
)
.bind(g.into_inner())
.bind(name)
.execute(&mut **tx)
.await?;
}
}
Ok(())
}
/// Delete an EP marker at `owner` (case-insensitive), in the apply transaction.
/// Used by `--prune` when the manifest stops declaring a name.
pub async fn delete_extension_point_tx(
tx: &mut Transaction<'_, Postgres>,
owner: ScriptOwner,
name: &str,
) -> Result<(), sqlx::Error> {
match owner {
ScriptOwner::App(a) => {
sqlx::query(
"DELETE FROM extension_points WHERE app_id = $1 AND LOWER(name) = LOWER($2)",
)
.bind(a.into_inner())
.bind(name)
.execute(&mut **tx)
.await?;
}
ScriptOwner::Group(g) => {
sqlx::query(
"DELETE FROM extension_points WHERE group_id = $1 AND LOWER(name) = LOWER($2)",
)
.bind(g.into_inner())
.bind(name)
.execute(&mut **tx)
.await?;
}
}
Ok(())
}

View File

@@ -163,12 +163,9 @@ async fn get_file(
.await?
.ok_or(FilesApiError::NotFound)?;
let safe_ct = picloud_shared::sanitize_stored_content_type(&meta.content_type);
// The stored name is upload-supplied; sanitize to header-safe ASCII so the
// response builder can never error on a control byte (which `.expect()`
// below would turn into a per-request panic).
let disposition = format!(
"attachment; filename=\"{}\"",
picloud_shared::sanitize_stored_filename(&meta.name)
meta.name.replace('"', "").replace(['\r', '\n'], "")
);
let len = bytes.len();
Ok(Response::builder()

View File

@@ -209,6 +209,11 @@ impl FsFilesRepo {
}
Ok(())
}
fn final_path(&self, app_id: AppId, collection: &str, id: Uuid) -> PathBuf {
final_path_at(&self.config.root, app_id, collection, id)
}
fn write_atomic(
&self,
app_id: AppId,
@@ -216,53 +221,29 @@ impl FsFilesRepo {
id: Uuid,
bytes: &[u8],
) -> Result<String, FilesRepoError> {
write_atomic_at(
&self.config.root,
&app_owner_dir(app_id),
collection,
id,
bytes,
)
write_atomic_at(&self.config.root, app_id, collection, id, bytes)
}
}
/// The owner-relative subdirectory of an **app**'s blobs under `<root>/files/`:
/// just `<app_id>`. Group-shared blobs (§11.6) shard at `groups/<group_id>`
/// instead (see `group_files_repo::group_owner_dir`) — a disjoint subtree under
/// the same `files/` base, so the orphan sweeper covers both with one walk.
pub(crate) fn app_owner_dir(app_id: AppId) -> PathBuf {
PathBuf::from(app_id.into_inner().to_string())
}
/// `<root>/files/<owner_rel>/<collection>/<id[0:2]>/`. `owner_rel` is built
/// by the caller from a server-generated id (`<app_id>` or `groups/<group_id>`)
/// — never user input — and `collection` is path-guarded one layer up, so no
/// component can escape the `files/` base.
pub(crate) fn shard_dir_at(
root: &Path,
owner_rel: &Path,
collection: &str,
id_str: &str,
) -> PathBuf {
fn shard_dir_at(root: &Path, app_id: AppId, collection: &str, id_str: &str) -> PathBuf {
root.join("files")
.join(owner_rel)
.join(app_id.into_inner().to_string())
.join(collection)
.join(&id_str[..2])
}
pub(crate) fn final_path_at(root: &Path, owner_rel: &Path, collection: &str, id: Uuid) -> PathBuf {
fn final_path_at(root: &Path, app_id: AppId, collection: &str, id: Uuid) -> PathBuf {
let id_str = id.to_string();
shard_dir_at(root, owner_rel, collection, &id_str).join(&id_str)
shard_dir_at(root, app_id, collection, &id_str).join(&id_str)
}
/// Steps 26 of the atomic-write protocol. Returns the lowercase hex
/// SHA-256 of the bytes (computed in a single pass over the in-memory
/// buffer — the file is never re-read). `pub(crate)` + owner-relative so the
/// app and group-shared (§11.6) repos share this single source for the
/// security-sensitive write+checksum mechanics, unit-testable without a pool.
pub(crate) fn write_atomic_at(
/// buffer — the file is never re-read). Free function so the fs
/// mechanics are unit-testable without a Postgres pool.
fn write_atomic_at(
root: &Path,
owner_rel: &Path,
app_id: AppId,
collection: &str,
id: Uuid,
bytes: &[u8],
@@ -270,7 +251,7 @@ pub(crate) fn write_atomic_at(
use std::io::Write as _;
let id_str = id.to_string();
let dir = shard_dir_at(root, owner_rel, collection, &id_str);
let dir = shard_dir_at(root, app_id, collection, &id_str);
create_dir_all_secure(&dir)?;
// Single-pass checksum over the in-memory buffer.
@@ -297,16 +278,15 @@ pub(crate) fn write_atomic_at(
/// Read + checksum-verify the bytes at the given path-set. Free
/// function mirror of the `get` read path. Returns `Corrupted` when the
/// bytes are missing or don't match `expected_checksum`. `pub(crate)` +
/// owner-relative — shared with the group-files (§11.6) repo.
pub(crate) fn read_verify_at(
/// bytes are missing or don't match `expected_checksum`.
fn read_verify_at(
root: &Path,
owner_rel: &Path,
app_id: AppId,
collection: &str,
id: Uuid,
expected_checksum: &str,
) -> Result<Vec<u8>, FilesRepoError> {
let path = final_path_at(root, owner_rel, collection, id);
let path = final_path_at(root, app_id, collection, id);
let bytes = match std::fs::read(&path) {
Ok(b) => b,
Err(e) => {
@@ -403,13 +383,7 @@ impl FilesRepo for FsFilesRepo {
let Some((stored_checksum,)) = row else {
return Ok(None);
};
let bytes = read_verify_at(
&self.config.root,
&app_owner_dir(app_id),
collection,
id,
&stored_checksum,
)?;
let bytes = read_verify_at(&self.config.root, app_id, collection, id, &stored_checksum)?;
Ok(Some(bytes))
}
@@ -463,14 +437,42 @@ impl FilesRepo for FsFilesRepo {
id: Uuid,
) -> Result<Option<FileMeta>, FilesRepoError> {
Self::guard_collection(collection)?;
// `DELETE ... RETURNING` is one statement, so the old SELECT-FOR-UPDATE
// + DELETE pair is unnecessary. Unlink only AFTER the row is gone: the
// reverse order would destroy the bytes of a row that a failure keeps.
let meta = delete_meta_on(&self.pool, app_id, collection, id).await?;
if meta.is_some() {
unlink_blob(&self.config.root, &app_owner_dir(app_id), collection, id);
// SELECT + DELETE in one tx; unlink afterwards (outside the tx).
let mut tx = self.pool.begin().await?;
let row: Option<FileRow> = sqlx::query_as(
"SELECT id, collection, name, content_type, size_bytes, \
checksum_sha256, created_at, updated_at \
FROM files WHERE app_id = $1 AND collection = $2 AND id = $3 \
FOR UPDATE",
)
.bind(app_id.into_inner())
.bind(collection)
.bind(id)
.fetch_optional(&mut *tx)
.await?;
let Some(row) = row else {
tx.rollback().await?;
return Ok(None);
};
sqlx::query("DELETE FROM files WHERE app_id = $1 AND collection = $2 AND id = $3")
.bind(app_id.into_inner())
.bind(collection)
.bind(id)
.execute(&mut *tx)
.await?;
tx.commit().await?;
// Row is gone; unlink the bytes. A failure here leaves an orphan
// file (reclaimed by a future sweep) — not fatal.
let path = self.final_path(app_id, collection, id);
if let Err(e) = std::fs::remove_file(&path) {
if e.kind() != std::io::ErrorKind::NotFound {
tracing::warn!(path = %path.display(), error = %e, "files: unlink after delete failed (orphan)");
}
}
Ok(meta)
Ok(Some(row.into_meta()))
}
async fn list(
@@ -553,11 +555,11 @@ fn hex_lower(bytes: &[u8]) -> String {
s
}
pub(crate) fn encode_cursor(last_id: Uuid) -> String {
fn encode_cursor(last_id: Uuid) -> String {
URL_SAFE_NO_PAD.encode(last_id.to_string().as_bytes())
}
pub(crate) fn decode_cursor(cursor: &str) -> Result<Uuid, FilesRepoError> {
fn decode_cursor(cursor: &str) -> Result<Uuid, FilesRepoError> {
let bytes = URL_SAFE_NO_PAD
.decode(cursor)
.map_err(|_| FilesRepoError::InvalidCursor)?;
@@ -566,7 +568,7 @@ pub(crate) fn decode_cursor(cursor: &str) -> Result<Uuid, FilesRepoError> {
}
#[derive(sqlx::FromRow)]
pub(crate) struct FileRow {
struct FileRow {
id: Uuid,
collection: String,
name: String,
@@ -578,7 +580,7 @@ pub(crate) struct FileRow {
}
impl FileRow {
pub(crate) fn into_meta(self) -> FileMeta {
fn into_meta(self) -> FileMeta {
FileMeta {
id: self.id,
collection: self.collection,
@@ -592,194 +594,6 @@ impl FileRow {
}
}
// ----------------------------------------------------------------------------
// Connection-scoped metadata operations
//
// The BYTES are written to disk outside any transaction (they are not
// transactional and cannot be). What these give `crate::atomic_write` is a way
// to commit the metadata row and the trigger fan-out it produces TOGETHER, so a
// committed file row always has its event enqueued.
//
// A rollback after the bytes are on disk leaves an orphan blob. That hazard is
// not new — the pre-existing repo already wrote the blob, then inserted the row
// in a separate statement that could fail — and the writer now explicitly
// unlinks the blob on the rollback path, so the window is strictly smaller than
// it was.
// ----------------------------------------------------------------------------
/// The four metadata columns a blob write sets. Bundled so the `*_meta_on`
/// helpers take a target (owner, collection, id) plus one payload, rather than a
/// long positional tail it would be easy to transpose.
#[derive(Clone, Copy)]
pub(crate) struct MetaFields<'a> {
pub name: &'a str,
pub content_type: &'a str,
pub size: i64,
pub checksum: &'a str,
}
/// The same, for an update: `name`/`content_type` are `None` to keep the stored
/// value (`COALESCE`), while the bytes always change.
#[derive(Clone, Copy)]
pub(crate) struct MetaPatch<'a> {
pub name: Option<&'a str>,
pub content_type: Option<&'a str>,
pub size: i64,
pub checksum: &'a str,
}
const FILE_COLS: &str = "id, collection, name, content_type, size_bytes, \
checksum_sha256, created_at, updated_at";
pub(crate) async fn head_on<'c, E>(
exec: E,
app_id: AppId,
collection: &str,
id: Uuid,
) -> Result<Option<FileMeta>, FilesRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let row: Option<FileRow> = sqlx::query_as(&format!(
"SELECT {FILE_COLS} FROM files \
WHERE app_id = $1 AND collection = $2 AND id = $3"
))
.bind(app_id.into_inner())
.bind(collection)
.bind(id)
.fetch_optional(exec)
.await?;
Ok(row.map(FileRow::into_meta))
}
pub(crate) async fn insert_meta_on<'c, E>(
exec: E,
app_id: AppId,
collection: &str,
id: Uuid,
meta: MetaFields<'_>,
) -> Result<FileMeta, FilesRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let row: FileRow = sqlx::query_as(&format!(
"INSERT INTO files \
(app_id, collection, id, name, content_type, size_bytes, checksum_sha256) \
VALUES ($1, $2, $3, $4, $5, $6, $7) \
RETURNING {FILE_COLS}"
))
.bind(app_id.into_inner())
.bind(collection)
.bind(id)
.bind(meta.name)
.bind(meta.content_type)
.bind(meta.size)
.bind(meta.checksum)
.fetch_one(exec)
.await?;
Ok(row.into_meta())
}
/// `None` when the file row is gone (the caller maps that to `NotFound`).
pub(crate) async fn update_meta_on<'c, E>(
exec: E,
app_id: AppId,
collection: &str,
id: Uuid,
meta: MetaPatch<'_>,
) -> Result<Option<FileMeta>, FilesRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let row: Option<FileRow> = sqlx::query_as(&format!(
"UPDATE files SET \
name = COALESCE($4, name), \
content_type = COALESCE($5, content_type), \
size_bytes = $6, \
checksum_sha256 = $7, \
updated_at = NOW() \
WHERE app_id = $1 AND collection = $2 AND id = $3 \
RETURNING {FILE_COLS}"
))
.bind(app_id.into_inner())
.bind(collection)
.bind(id)
.bind(meta.name)
.bind(meta.content_type)
.bind(meta.size)
.bind(meta.checksum)
.fetch_optional(exec)
.await?;
Ok(row.map(FileRow::into_meta))
}
/// `DELETE ... RETURNING` — one statement, so no `SELECT ... FOR UPDATE` dance.
/// The caller unlinks the bytes AFTER the transaction commits: unlinking first
/// would destroy the blob of a row that a rollback then keeps.
pub(crate) async fn delete_meta_on<'c, E>(
exec: E,
app_id: AppId,
collection: &str,
id: Uuid,
) -> Result<Option<FileMeta>, FilesRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let row: Option<FileRow> = sqlx::query_as(&format!(
"DELETE FROM files \
WHERE app_id = $1 AND collection = $2 AND id = $3 \
RETURNING {FILE_COLS}"
))
.bind(app_id.into_inner())
.bind(collection)
.bind(id)
.fetch_optional(exec)
.await?;
Ok(row.map(FileRow::into_meta))
}
/// Best-effort unlink of a blob whose metadata write was rolled back (or whose
/// row was just deleted). A failure leaves an orphan — logged, never fatal.
pub(crate) fn unlink_blob(root: &Path, owner_rel: &Path, collection: &str, id: Uuid) {
let path = final_path_at(root, owner_rel, collection, id);
if let Err(e) = std::fs::remove_file(&path) {
if e.kind() != std::io::ErrorKind::NotFound {
tracing::warn!(
path = %path.display(), error = %e,
"files: unlink failed (orphan blob left on disk)"
);
}
}
}
/// The PROJECTED total stored bytes for the app AFTER this write — the current
/// SUM, minus the bytes of the file being replaced (`replacing = Some(id)` on
/// update, `None` on create), plus the incoming blob's. Backs the per-app disk
/// ceiling; the subtraction is what lets an update near the cap still go through.
pub(crate) async fn projected_total_bytes_on<'c, E>(
exec: E,
app_id: AppId,
replacing: Option<Uuid>,
incoming: i64,
) -> Result<u64, FilesRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let (n,): (i64,) = sqlx::query_as(
"SELECT ( \
COALESCE((SELECT SUM(size_bytes) FROM files WHERE app_id = $1), 0) \
- COALESCE((SELECT size_bytes FROM files WHERE app_id = $1 AND id = $2), 0) \
+ $3 \
)::BIGINT",
)
.bind(app_id.into_inner())
.bind(replacing)
.bind(incoming)
.fetch_one(exec)
.await?;
Ok(u64::try_from(n).unwrap_or(0))
}
#[cfg(test)]
mod tests {
use super::*;
@@ -858,13 +672,13 @@ mod tests {
let id = Uuid::new_v4();
let bytes = b"hello picloud files".to_vec();
let checksum = write_atomic_at(&root, &app_owner_dir(app), "avatars", id, &bytes).unwrap();
let checksum = write_atomic_at(&root, app, "avatars", id, &bytes).unwrap();
// Single-pass checksum matches an independent hash of the bytes.
let mut h = Sha256::new();
h.update(&bytes);
assert_eq!(checksum, hex_lower(&h.finalize()));
let read = read_verify_at(&root, &app_owner_dir(app), "avatars", id, &checksum).unwrap();
let read = read_verify_at(&root, app, "avatars", id, &checksum).unwrap();
assert_eq!(read, bytes);
std::fs::remove_dir_all(&root).ok();
@@ -875,13 +689,13 @@ mod tests {
let root = unique_tmp_root();
let app = AppId::new();
let id = Uuid::new_v4();
let checksum = write_atomic_at(&root, &app_owner_dir(app), "c", id, b"original").unwrap();
let checksum = write_atomic_at(&root, app, "c", id, b"original").unwrap();
// Mutate the bytes behind the repo's back.
let path = final_path_at(&root, &app_owner_dir(app), "c", id);
let path = final_path_at(&root, app, "c", id);
std::fs::write(&path, b"tampered").unwrap();
let err = read_verify_at(&root, &app_owner_dir(app), "c", id, &checksum).unwrap_err();
let err = read_verify_at(&root, app, "c", id, &checksum).unwrap_err();
assert!(matches!(err, FilesRepoError::Corrupted));
std::fs::remove_dir_all(&root).ok();
@@ -893,7 +707,7 @@ mod tests {
let app = AppId::new();
let id = Uuid::new_v4();
// No write — the file never existed.
let err = read_verify_at(&root, &app_owner_dir(app), "c", id, "deadbeef").unwrap_err();
let err = read_verify_at(&root, app, "c", id, "deadbeef").unwrap_err();
assert!(matches!(err, FilesRepoError::Corrupted));
std::fs::remove_dir_all(&root).ok();
}
@@ -903,10 +717,10 @@ mod tests {
let root = unique_tmp_root();
let app = AppId::new();
let id = Uuid::new_v4();
write_atomic_at(&root, &app_owner_dir(app), "c", id, b"data").unwrap();
write_atomic_at(&root, app, "c", id, b"data").unwrap();
let id_str = id.to_string();
let dir = shard_dir_at(&root, &app_owner_dir(app), "c", &id_str);
let dir = shard_dir_at(&root, app, "c", &id_str);
let entries: Vec<_> = std::fs::read_dir(&dir)
.unwrap()
.filter_map(Result::ok)
@@ -925,7 +739,7 @@ mod tests {
let app = AppId::new();
let id = Uuid::new_v4();
let id_str = id.to_string();
let path = final_path_at(&root, &app_owner_dir(app), "col", id);
let path = final_path_at(&root, app, "col", id);
let shard = &id_str[..2];
assert!(path
.to_string_lossy()
@@ -939,9 +753,9 @@ mod tests {
let root = unique_tmp_root();
let app = AppId::new();
let id = Uuid::new_v4();
write_atomic_at(&root, &app_owner_dir(app), "c", id, b"data").unwrap();
write_atomic_at(&root, app, "c", id, b"data").unwrap();
let id_str = id.to_string();
let dir = shard_dir_at(&root, &app_owner_dir(app), "c", &id_str);
let dir = shard_dir_at(&root, app, "c", &id_str);
let mode = std::fs::metadata(&dir).unwrap().permissions().mode();
assert_eq!(mode & 0o777, 0o700, "shard dir should be 0o700");
std::fs::remove_dir_all(&root).ok();

View File

@@ -18,18 +18,17 @@ use std::sync::Arc;
use async_trait::async_trait;
use picloud_shared::{
sanitize_stored_content_type, validate_files_collection, FileMeta, FileUpdate, FilesError,
FilesListPage, FilesService, NewFile, SdkCallCx, ServiceEventEmitter,
FilesListPage, FilesService, NewFile, SdkCallCx, ServiceEvent, ServiceEventEmitter,
};
use uuid::Uuid;
use crate::atomic_write::{BestEffortFilesWriter, FilesWriter, PostgresFilesWriter};
use crate::authz::{self, AuthzRepo, Capability};
use crate::files_repo::{FilesRepo, FilesRepoError};
use crate::files_repo::{FileUpdated, FilesRepo, FilesRepoError};
pub struct FilesServiceImpl {
repo: Arc<dyn FilesRepo>,
authz: Arc<dyn AuthzRepo>,
writer: Arc<dyn FilesWriter>,
events: Arc<dyn ServiceEventEmitter>,
max_file_size_bytes: usize,
}
@@ -42,28 +41,13 @@ impl FilesServiceImpl {
max_file_size_bytes: usize,
) -> Self {
Self {
writer: Arc::new(BestEffortFilesWriter::new(repo.clone(), events)),
repo,
authz,
events,
max_file_size_bytes,
}
}
/// Swap the best-effort writer for the transactional one: the metadata row
/// and the trigger fan-out then commit together, so an outbox failure rolls
/// the metadata back (and unlinks the blob) instead of silently losing the
/// event. The host always calls this; the in-memory unit tests do not.
#[must_use]
pub fn with_atomic_writes(
mut self,
pool: sqlx::PgPool,
root: std::path::PathBuf,
max_total_bytes: u64,
) -> Self {
self.writer = Arc::new(PostgresFilesWriter::new(pool, root, max_total_bytes));
self
}
async fn check_read(&self, cx: &SdkCallCx) -> Result<(), FilesError> {
authz::script_gate(
&*self.authz,
@@ -85,6 +69,37 @@ impl FilesServiceImpl {
)
.await
}
/// Best-effort `ServiceEvent` emission. A failed emit is logged but
/// never rolls back the (already-durable) file write.
async fn emit(
&self,
cx: &SdkCallCx,
op: &'static str,
collection: &str,
meta: &FileMeta,
old: Option<&FileMeta>,
) {
let payload = serde_json::to_value(meta).ok();
let old_payload = old.and_then(|m| serde_json::to_value(m).ok());
if let Err(e) = self
.events
.emit(
cx,
ServiceEvent {
source: "files",
op,
collection: Some(collection.to_string()),
key: Some(meta.id.to_string()),
payload,
old_payload,
},
)
.await
{
tracing::error!(error = %e, source = "files", op, event_emit_failure = true, "event emit failed");
}
}
}
/// Parse a script-supplied id. Invalid UUIDs aren't an error shape the
@@ -117,7 +132,9 @@ impl FilesService for FilesServiceImpl {
// Audit 2026-06-11 C-2 — coerce dangerous render types to
// application/octet-stream after the shape checks pass.
new.content_type = sanitize_stored_content_type(&new.content_type);
self.writer.create(cx, collection, new).await
let meta = self.repo.create(cx.app_id, collection, new).await?;
self.emit(cx, "create", collection, &meta, None).await;
Ok(meta.id)
}
async fn head(
@@ -165,10 +182,12 @@ impl FilesService for FilesServiceImpl {
let Some(uuid) = parse_id(id) else {
return Err(FilesError::NotFound);
};
if self.writer.update(cx, collection, uuid, upd).await? {
Ok(())
} else {
Err(FilesError::NotFound)
match self.repo.update(cx.app_id, collection, uuid, upd).await? {
Some(FileUpdated { new, prev }) => {
self.emit(cx, "update", collection, &new, Some(&prev)).await;
Ok(())
}
None => Err(FilesError::NotFound),
}
}
@@ -178,7 +197,16 @@ impl FilesService for FilesServiceImpl {
let Some(uuid) = parse_id(id) else {
return Ok(false);
};
self.writer.delete(cx, collection, uuid).await
match self.repo.delete(cx.app_id, collection, uuid).await? {
Some(meta) => {
// On delete, the top-level metadata AND `prev` both carry
// the deleted row (per docs/v1.1.x design + the brief).
self.emit(cx, "delete", collection, &meta, Some(&meta))
.await;
Ok(true)
}
None => Ok(false),
}
}
async fn list(
@@ -204,7 +232,6 @@ impl FilesService for FilesServiceImpl {
mod tests {
use super::*;
use crate::authz::{AuthzError, AuthzRepo};
use crate::files_repo::FileUpdated;
use async_trait::async_trait;
use chrono::Utc;
use picloud_shared::{

View File

@@ -165,23 +165,6 @@ mod tests {
assert!(tmp.exists());
}
#[test]
fn sweeps_group_shared_tmp_files() {
// §11.6 group-shared files shard under `files/groups/<group_id>/...`,
// a disjoint subtree from the per-app `files/<app_id>/...`. The walk is
// recursive from `<root>/files/`, so an orphan there is reaped without
// any group-specific sweeper change — this pins that "covered for free"
// guarantee against a future refactor.
let root = tmp_root();
let group_shard = root.join("files/groups/6f693a33-9ae5-485b-804e-191e9fd33524/assets/ab");
std::fs::create_dir_all(&group_shard).unwrap();
let tmp = group_shard.join("uuid.tmp.123-0");
touch(&tmp);
let stats = sweep_orphan_tmp_files(&root, Duration::ZERO);
assert_eq!(stats.files_deleted, 1, "group-shared orphan must be reaped");
assert!(!tmp.exists());
}
#[test]
fn keeps_non_tmp_files() {
let root = tmp_root();

View File

@@ -1,403 +0,0 @@
//! `/api/v1/admin/groups/{id}/{kv,docs,files}*` — read-only operator inspection
//! of a group's §11.6 SHARED collections (M4). Mirrors the per-app `kv_api` /
//! `files_api` admin surface for groups so `pic {kv,docs,files} ls --group` (and
//! a future dashboard tab) can browse shared data without a script.
//!
//! **Read-only by design** — shared writes go through the SDK
//! (`kv::shared_collection(...)` etc.), which run the reads-open / writes-authed
//! authz + fire `shared = true` triggers; an admin write would bypass both. The
//! deferral (operator write/delete on shared blobs) stands.
//!
//! Capabilities: `GroupKvRead` / `GroupDocsRead` / `GroupFilesRead`, resolved
//! against the group loaded from the path (the same tier the SDK read path uses).
use std::sync::Arc;
use axum::extract::{Path, Query, State};
use axum::http::header::{CONTENT_DISPOSITION, CONTENT_LENGTH, CONTENT_TYPE};
use axum::http::StatusCode;
use axum::response::{IntoResponse, Json, Response};
use axum::routing::get;
use axum::{Extension, Router};
use picloud_shared::{GroupId, Principal};
use serde::{Deserialize, Serialize};
use serde_json::json;
use uuid::Uuid;
use crate::authz::{require, AuthzDenied, AuthzRepo, Capability};
use crate::group_dead_letter_repo::GroupDeadLetterRepo;
use crate::group_docs_repo::GroupDocsRepo;
use crate::group_files_repo::GroupFilesRepo;
use crate::group_kv_repo::GroupKvRepo;
use crate::group_repo::GroupRepository;
/// Default/max page size for the dead-letters listing (an operator view, not a
/// bulk export).
const DEAD_LETTERS_LIMIT_DEFAULT: i64 = 100;
const DEAD_LETTERS_LIMIT_MAX: i64 = 500;
#[derive(Clone)]
pub struct GroupBlobsState {
pub kv: Arc<dyn GroupKvRepo>,
pub docs: Arc<dyn GroupDocsRepo>,
pub files: Arc<dyn GroupFilesRepo>,
pub dead_letters: Arc<dyn GroupDeadLetterRepo>,
pub groups: Arc<dyn GroupRepository>,
pub authz: Arc<dyn AuthzRepo>,
}
pub fn group_blobs_router(state: GroupBlobsState) -> Router {
Router::new()
.route("/groups/{id}/kv", get(list_kv))
.route("/groups/{id}/kv/{collection}/{key}", get(get_kv))
.route("/groups/{id}/docs", get(list_docs))
.route("/groups/{id}/docs/{collection}/{doc_id}", get(get_doc))
.route("/groups/{id}/files", get(list_files))
.route("/groups/{id}/files/{collection}/{file_id}", get(get_file))
.route("/groups/{id}/dead-letters", get(list_dead_letters))
.with_state(state)
}
#[derive(Debug, Deserialize)]
pub struct ListQuery {
pub collection: String,
#[serde(default)]
pub cursor: Option<String>,
#[serde(default)]
pub limit: Option<u32>,
}
#[derive(Debug, Serialize)]
struct ListKeysResponse {
keys: Vec<String>,
next_cursor: Option<String>,
}
#[derive(Debug, Deserialize)]
pub struct DeadLettersQuery {
#[serde(default)]
pub unresolved: bool,
#[serde(default)]
pub limit: Option<u32>,
}
/// One group dead-letter, as returned by the operator listing.
#[derive(Debug, Serialize)]
struct DeadLetterDto {
id: Uuid,
collection: String,
source: String,
op: String,
attempt_count: u32,
last_error: String,
created_at: String,
resolved_at: Option<String>,
payload: serde_json::Value,
}
impl From<crate::group_dead_letter_repo::GroupDeadLetterRow> for DeadLetterDto {
fn from(r: crate::group_dead_letter_repo::GroupDeadLetterRow) -> Self {
Self {
id: r.id.into_inner(),
collection: r.collection,
source: r.source,
op: r.op,
attempt_count: r.attempt_count,
last_error: r.last_error,
created_at: r.created_at.to_rfc3339(),
resolved_at: r.resolved_at.map(|t| t.to_rfc3339()),
payload: r.payload,
}
}
}
async fn list_kv(
State(s): State<GroupBlobsState>,
Extension(principal): Extension<Principal>,
Path(ident): Path<String>,
Query(q): Query<ListQuery>,
) -> Result<Json<ListKeysResponse>, GroupBlobsError> {
let group_id = resolve_group(&*s.groups, &ident).await?;
require(
s.authz.as_ref(),
&principal,
Capability::GroupKvRead(group_id),
)
.await?;
let page =
s.kv.list(
group_id,
&q.collection,
q.cursor.as_deref(),
q.limit.unwrap_or(0),
)
.await
.map_err(|e| GroupBlobsError::Backend(e.to_string()))?;
Ok(Json(ListKeysResponse {
keys: page.keys,
next_cursor: page.next_cursor,
}))
}
async fn get_kv(
State(s): State<GroupBlobsState>,
Extension(principal): Extension<Principal>,
Path((ident, collection, key)): Path<(String, String, String)>,
) -> Result<Json<serde_json::Value>, GroupBlobsError> {
let group_id = resolve_group(&*s.groups, &ident).await?;
require(
s.authz.as_ref(),
&principal,
Capability::GroupKvRead(group_id),
)
.await?;
let value =
s.kv.get(group_id, &collection, &key)
.await
.map_err(|e| GroupBlobsError::Backend(e.to_string()))?
.ok_or(GroupBlobsError::NotFound)?;
Ok(Json(json!({ "value": value })))
}
#[derive(Debug, Serialize)]
struct DocEntry {
id: String,
data: serde_json::Value,
}
#[derive(Debug, Serialize)]
struct ListDocsResponse {
docs: Vec<DocEntry>,
next_cursor: Option<String>,
}
async fn list_docs(
State(s): State<GroupBlobsState>,
Extension(principal): Extension<Principal>,
Path(ident): Path<String>,
Query(q): Query<ListQuery>,
) -> Result<Json<ListDocsResponse>, GroupBlobsError> {
let group_id = resolve_group(&*s.groups, &ident).await?;
require(
s.authz.as_ref(),
&principal,
Capability::GroupDocsRead(group_id),
)
.await?;
let page = s
.docs
.list(
group_id,
&q.collection,
q.cursor.as_deref(),
q.limit.unwrap_or(0),
)
.await
.map_err(|e| GroupBlobsError::Backend(e.to_string()))?;
Ok(Json(ListDocsResponse {
docs: page
.docs
.into_iter()
.map(|d| DocEntry {
id: d.id.to_string(),
data: d.data,
})
.collect(),
next_cursor: page.next_cursor,
}))
}
async fn get_doc(
State(s): State<GroupBlobsState>,
Extension(principal): Extension<Principal>,
Path((ident, collection, doc_id)): Path<(String, String, String)>,
) -> Result<Json<serde_json::Value>, GroupBlobsError> {
let group_id = resolve_group(&*s.groups, &ident).await?;
require(
s.authz.as_ref(),
&principal,
Capability::GroupDocsRead(group_id),
)
.await?;
let id = doc_id
.parse::<Uuid>()
.map_err(|_| GroupBlobsError::NotFound)?;
let row = s
.docs
.get(group_id, &collection, id)
.await
.map_err(|e| GroupBlobsError::Backend(e.to_string()))?
.ok_or(GroupBlobsError::NotFound)?;
Ok(Json(json!({ "id": row.id.to_string(), "data": row.data })))
}
async fn list_files(
State(s): State<GroupBlobsState>,
Extension(principal): Extension<Principal>,
Path(ident): Path<String>,
Query(q): Query<ListQuery>,
) -> Result<Json<serde_json::Value>, GroupBlobsError> {
let group_id = resolve_group(&*s.groups, &ident).await?;
require(
s.authz.as_ref(),
&principal,
Capability::GroupFilesRead(group_id),
)
.await?;
let page = s
.files
.list(
group_id,
&q.collection,
q.cursor.as_deref(),
q.limit.unwrap_or(0),
)
.await
.map_err(|e| GroupBlobsError::Backend(e.to_string()))?;
// FileMeta is Serialize; return the metadata list + cursor.
Ok(Json(json!({
"files": page.files,
"next_cursor": page.next_cursor,
})))
}
async fn get_file(
State(s): State<GroupBlobsState>,
Extension(principal): Extension<Principal>,
Path((ident, collection, file_id)): Path<(String, String, String)>,
) -> Result<Response, GroupBlobsError> {
let group_id = resolve_group(&*s.groups, &ident).await?;
require(
s.authz.as_ref(),
&principal,
Capability::GroupFilesRead(group_id),
)
.await?;
let id = file_id
.parse::<Uuid>()
.map_err(|_| GroupBlobsError::NotFound)?;
let meta = s
.files
.head(group_id, &collection, id)
.await
.map_err(|e| GroupBlobsError::Backend(e.to_string()))?
.ok_or(GroupBlobsError::NotFound)?;
let bytes = s
.files
.get(group_id, &collection, id)
.await
.map_err(|e| GroupBlobsError::Backend(e.to_string()))?
.ok_or(GroupBlobsError::NotFound)?;
// Same download hardening as the per-app path (`files_api::get_file`): a
// sanitized content-type, forced `attachment` disposition with a header-safe
// filename, plus nosniff/CSP so a stored HTML/SVG shared file can't render
// inline (stored-XSS) and no attacker-influenced byte can panic the builder.
let safe_ct = picloud_shared::sanitize_stored_content_type(&meta.content_type);
let disposition = format!(
"attachment; filename=\"{}\"",
picloud_shared::sanitize_stored_filename(&meta.name)
);
let len = bytes.len();
Response::builder()
.status(StatusCode::OK)
.header(CONTENT_TYPE, safe_ct)
.header(CONTENT_DISPOSITION, disposition)
.header(CONTENT_LENGTH, len)
.header("X-Content-Type-Options", "nosniff")
.header(
"Content-Security-Policy",
"default-src 'none'; sandbox; frame-ancestors 'none'",
)
.header("Referrer-Policy", "no-referrer")
.body(axum::body::Body::from(bytes))
.map_err(|e| GroupBlobsError::Backend(e.to_string()))
}
/// §11.6 D3: list a group's shared-queue dead-letters (newest-first). Guarded by
/// `GroupKvRead` (the same read tier as the shared collections above; no new
/// capability). `?unresolved=true` filters to still-open rows; `?limit=` caps
/// the page (default 100, max 500).
async fn list_dead_letters(
State(s): State<GroupBlobsState>,
Extension(principal): Extension<Principal>,
Path(ident): Path<String>,
Query(q): Query<DeadLettersQuery>,
) -> Result<Json<Vec<DeadLetterDto>>, GroupBlobsError> {
let group_id = resolve_group(&*s.groups, &ident).await?;
require(
s.authz.as_ref(),
&principal,
Capability::GroupKvRead(group_id),
)
.await?;
let limit = q.limit.map_or(DEAD_LETTERS_LIMIT_DEFAULT, |n| {
i64::from(n).clamp(1, DEAD_LETTERS_LIMIT_MAX)
});
let rows = s
.dead_letters
.list_for_group(group_id, q.unresolved, limit)
.await
.map_err(|e| GroupBlobsError::Backend(e.to_string()))?;
Ok(Json(rows.into_iter().map(DeadLetterDto::from).collect()))
}
async fn resolve_group(
groups: &dyn GroupRepository,
ident: &str,
) -> Result<GroupId, GroupBlobsError> {
let found = if let Ok(uuid) = ident.parse::<Uuid>() {
groups
.get_by_id(uuid.into())
.await
.map_err(|e| GroupBlobsError::Backend(e.to_string()))?
} else {
groups
.get_by_slug(ident)
.await
.map_err(|e| GroupBlobsError::Backend(e.to_string()))?
};
found.map(|g| g.id).ok_or(GroupBlobsError::GroupNotFound)
}
#[derive(Debug, thiserror::Error)]
pub enum GroupBlobsError {
#[error("group not found")]
GroupNotFound,
#[error("not found")]
NotFound,
#[error("forbidden")]
Forbidden,
#[error("authorization repo error: {0}")]
AuthzRepo(String),
#[error("backend: {0}")]
Backend(String),
}
impl From<AuthzDenied> for GroupBlobsError {
fn from(d: AuthzDenied) -> Self {
match d {
AuthzDenied::Denied => Self::Forbidden,
AuthzDenied::Repo(e) => Self::AuthzRepo(e.to_string()),
}
}
}
impl IntoResponse for GroupBlobsError {
fn into_response(self) -> Response {
use axum::http::StatusCode;
let (status, body) = match &self {
Self::GroupNotFound | Self::NotFound => {
(StatusCode::NOT_FOUND, json!({ "error": self.to_string() }))
}
Self::Forbidden => (StatusCode::FORBIDDEN, json!({ "error": self.to_string() })),
Self::AuthzRepo(e) | Self::Backend(e) => {
tracing::error!(error = %e, "group blobs admin error");
(
StatusCode::INTERNAL_SERVER_ERROR,
json!({ "error": "internal error" }),
)
}
};
(status, Json(body)).into_response()
}
}

View File

@@ -1,188 +0,0 @@
//! Group-collection markers (§11.6) — the `group_collections` table (0052).
//!
//! A marker `(owner, name, kind)` declares that a collection name is
//! group-shared. `kind` selects the storage backend: `'kv'` →
//! `group_kv_entries` (0053), `'docs'` → `group_docs` (0054). This module holds
//! the read + transactional-write helpers, the runtime resolver, and the
//! injectable [`GroupCollectionResolver`] trait the per-kind services consume.
//!
//! The load-bearing function is [`resolve_owning_group`]: it walks the reading
//! app's ancestor chain ([`CHAIN_LEVELS_CTE`]) for the nearest group declaring
//! the collection of the requested `kind`. That walk **is** the isolation
//! boundary — a foreign app's chain never contains the owning group, so the
//! name does not resolve.
use async_trait::async_trait;
use picloud_shared::{AppId, GroupId, ScriptOwner};
use sqlx::{PgPool, Postgres, Transaction};
use crate::config_resolver::CHAIN_LEVELS_CTE;
/// List **all** shared-collection declarations at `owner` as `(name, kind)`
/// pairs, sorted. Used by the kind-aware apply diff and `collections ls`.
pub async fn list_all_for_owner(
pool: &PgPool,
owner: ScriptOwner,
) -> Result<Vec<(String, String)>, sqlx::Error> {
let rows: Vec<(String, String)> = match owner {
ScriptOwner::App(a) => {
sqlx::query_as(
"SELECT name, kind FROM group_collections \
WHERE app_id = $1 ORDER BY kind, LOWER(name)",
)
.bind(a.into_inner())
.fetch_all(pool)
.await?
}
ScriptOwner::Group(g) => {
sqlx::query_as(
"SELECT name, kind FROM group_collections \
WHERE group_id = $1 ORDER BY kind, LOWER(name)",
)
.bind(g.into_inner())
.fetch_all(pool)
.await?
}
};
Ok(rows)
}
/// Resolve the group that OWNS the shared collection `(name, kind)` for a
/// reading app: the nearest ancestor group on the app's chain that declares it.
/// Returns `None` when no group on the chain shares that `(name, kind)` — the
/// structural "not shared with you" boundary. Nearest-wins (CoW shadowing) is
/// enforced by `ORDER BY depth ASC LIMIT 1` and is security-relevant.
///
/// The join is on `group_owner` only: an app-declared marker (the degenerate
/// case) never makes a collection visible to *other* apps — sharing is a group
/// property.
pub async fn resolve_owning_group(
pool: &PgPool,
app_id: AppId,
name: &str,
kind: &str,
) -> Result<Option<GroupId>, sqlx::Error> {
let row: Option<(uuid::Uuid,)> = sqlx::query_as(&format!(
"{CHAIN_LEVELS_CTE} \
SELECT gc.group_id FROM group_collections gc \
JOIN chain c ON gc.group_id = c.group_owner \
WHERE LOWER(gc.name) = LOWER($2) AND gc.kind = $3 \
ORDER BY c.depth ASC LIMIT 1",
))
.bind(app_id.into_inner())
.bind(name)
.bind(kind)
.fetch_optional(pool)
.await?;
Ok(row.map(|(id,)| GroupId::from(id)))
}
/// Insert a `(name, kind)` marker at `owner`, in the apply transaction.
/// Idempotent: a re-apply of an already-declared marker is a no-op
/// (`ON CONFLICT DO NOTHING`), so it survives without a spurious version bump.
pub async fn insert_collection_tx(
tx: &mut Transaction<'_, Postgres>,
owner: ScriptOwner,
name: &str,
kind: &str,
) -> Result<(), sqlx::Error> {
match owner {
ScriptOwner::App(a) => {
sqlx::query(
"INSERT INTO group_collections (app_id, name, kind) VALUES ($1, $2, $3) \
ON CONFLICT (app_id, LOWER(name), kind) WHERE app_id IS NOT NULL DO NOTHING",
)
.bind(a.into_inner())
.bind(name)
.bind(kind)
.execute(&mut **tx)
.await?;
}
ScriptOwner::Group(g) => {
sqlx::query(
"INSERT INTO group_collections (group_id, name, kind) VALUES ($1, $2, $3) \
ON CONFLICT (group_id, LOWER(name), kind) WHERE group_id IS NOT NULL DO NOTHING",
)
.bind(g.into_inner())
.bind(name)
.bind(kind)
.execute(&mut **tx)
.await?;
}
}
Ok(())
}
/// Delete a `(name, kind)` marker at `owner` (case-insensitive), in the apply
/// transaction. Used by `--prune`. The storage data is NOT dropped here —
/// pruning a marker hides the store but leaves the data until the owning group
/// is deleted.
pub async fn delete_collection_tx(
tx: &mut Transaction<'_, Postgres>,
owner: ScriptOwner,
name: &str,
kind: &str,
) -> Result<(), sqlx::Error> {
match owner {
ScriptOwner::App(a) => {
sqlx::query(
"DELETE FROM group_collections \
WHERE app_id = $1 AND LOWER(name) = LOWER($2) AND kind = $3",
)
.bind(a.into_inner())
.bind(name)
.bind(kind)
.execute(&mut **tx)
.await?;
}
ScriptOwner::Group(g) => {
sqlx::query(
"DELETE FROM group_collections \
WHERE group_id = $1 AND LOWER(name) = LOWER($2) AND kind = $3",
)
.bind(g.into_inner())
.bind(name)
.bind(kind)
.execute(&mut **tx)
.await?;
}
}
Ok(())
}
/// Resolves a shared-collection name+kind to its owning group for a calling app
/// (nearest ancestor group on the app's chain that declares it). Behind a trait
/// so the per-kind services (`GroupKvServiceImpl`, `GroupDocsServiceImpl`) can
/// inject a fake in unit tests without Postgres.
#[async_trait]
pub trait GroupCollectionResolver: Send + Sync {
async fn resolve_owning_group(
&self,
app_id: AppId,
name: &str,
kind: &str,
) -> Result<Option<GroupId>, sqlx::Error>;
}
/// Postgres-backed resolver — delegates to [`resolve_owning_group`].
pub struct PostgresGroupCollectionResolver {
pool: PgPool,
}
impl PostgresGroupCollectionResolver {
#[must_use]
pub fn new(pool: PgPool) -> Self {
Self { pool }
}
}
#[async_trait]
impl GroupCollectionResolver for PostgresGroupCollectionResolver {
async fn resolve_owning_group(
&self,
app_id: AppId,
name: &str,
kind: &str,
) -> Result<Option<GroupId>, sqlx::Error> {
resolve_owning_group(&self.pool, app_id, name, kind).await
}
}

View File

@@ -1,136 +0,0 @@
//! `GroupDeadLetterRepo` — READ side over `group_dead_letters` (§11.6 D3).
//!
//! The group counterpart to the per-app [`crate::dead_letter_repo::DeadLetterRepo`]
//! read surface, keyed by `(group_id, collection)`. The WRITE side lives in
//! [`crate::group_queue_repo::GroupQueueRepo::dead_letter`] (co-located with the
//! queue-message delete for one-transaction atomicity, exactly as the per-app
//! `queue_repo::dead_letter` inlines its own INSERT). This repo backs the
//! read-only operator surface GET /api/v1/admin/groups/{id}/dead-letters, so an
//! operator can see shared-queue messages that exhausted their retries rather
//! than having them vanish.
use async_trait::async_trait;
use chrono::{DateTime, Utc};
use picloud_shared::{DeadLetterId, GroupId, ScriptId, TriggerId};
use sqlx::PgPool;
use uuid::Uuid;
#[derive(Debug, thiserror::Error)]
pub enum GroupDeadLetterRepoError {
#[error("database error: {0}")]
Db(#[from] sqlx::Error),
}
/// One group dead-letter row (mirror of `dead_letter_repo::DeadLetterRow`,
/// group-keyed).
#[derive(Debug, Clone)]
pub struct GroupDeadLetterRow {
pub id: DeadLetterId,
pub group_id: GroupId,
pub collection: String,
pub original_event_id: Uuid,
pub source: String,
pub op: String,
pub trigger_id: Option<TriggerId>,
pub script_id: Option<ScriptId>,
pub payload: serde_json::Value,
pub attempt_count: u32,
pub first_attempt_at: DateTime<Utc>,
pub last_attempt_at: DateTime<Utc>,
pub last_error: String,
pub created_at: DateTime<Utc>,
pub resolved_at: Option<DateTime<Utc>>,
pub resolution: Option<String>,
}
#[async_trait]
pub trait GroupDeadLetterRepo: Send + Sync {
/// A group's dead-letters, newest-first, capped at `limit` (bounded by the
/// caller). `unresolved_only` filters to `resolved_at IS NULL`.
async fn list_for_group(
&self,
group_id: GroupId,
unresolved_only: bool,
limit: i64,
) -> Result<Vec<GroupDeadLetterRow>, GroupDeadLetterRepoError>;
}
pub struct PostgresGroupDeadLetterRepo {
pool: PgPool,
}
impl PostgresGroupDeadLetterRepo {
#[must_use]
pub fn new(pool: PgPool) -> Self {
Self { pool }
}
}
const COLS: &str = "id, group_id, collection, original_event_id, source, op, \
trigger_id, script_id, payload, attempt_count, first_attempt_at, \
last_attempt_at, last_error, created_at, resolved_at, resolution";
#[async_trait]
impl GroupDeadLetterRepo for PostgresGroupDeadLetterRepo {
async fn list_for_group(
&self,
group_id: GroupId,
unresolved_only: bool,
limit: i64,
) -> Result<Vec<GroupDeadLetterRow>, GroupDeadLetterRepoError> {
let rows: Vec<RawRow> = sqlx::query_as(&format!(
"SELECT {COLS} FROM group_dead_letters \
WHERE group_id = $1 AND ($2 = FALSE OR resolved_at IS NULL) \
ORDER BY created_at DESC LIMIT $3"
))
.bind(group_id.into_inner())
.bind(unresolved_only)
.bind(limit)
.fetch_all(&self.pool)
.await?;
Ok(rows.into_iter().map(Into::into).collect())
}
}
#[derive(sqlx::FromRow)]
struct RawRow {
id: Uuid,
group_id: Uuid,
collection: String,
original_event_id: Uuid,
source: String,
op: String,
trigger_id: Option<Uuid>,
script_id: Option<Uuid>,
payload: serde_json::Value,
attempt_count: i32,
first_attempt_at: DateTime<Utc>,
last_attempt_at: DateTime<Utc>,
last_error: String,
created_at: DateTime<Utc>,
resolved_at: Option<DateTime<Utc>>,
resolution: Option<String>,
}
impl From<RawRow> for GroupDeadLetterRow {
fn from(r: RawRow) -> Self {
Self {
id: DeadLetterId(r.id),
group_id: r.group_id.into(),
collection: r.collection,
original_event_id: r.original_event_id,
source: r.source,
op: r.op,
trigger_id: r.trigger_id.map(Into::into),
script_id: r.script_id.map(Into::into),
payload: r.payload,
attempt_count: u32::try_from(r.attempt_count).unwrap_or(0),
first_attempt_at: r.first_attempt_at,
last_attempt_at: r.last_attempt_at,
last_error: r.last_error,
created_at: r.created_at,
resolved_at: r.resolved_at,
resolution: r.resolution,
}
}
}

View File

@@ -1,436 +0,0 @@
//! Low-level Postgres CRUD over `group_docs` (§11.6). A near-clone of
//! [`crate::docs_repo`] keyed by the owning `group_id` instead of `app_id`.
//! The `find` query is built by the shared [`crate::docs_repo::build_find_query`]
//! (parameterized on table + owner column), so the security-sensitive filter
//! SQL has a single source. Authorization, group resolution, value validation,
//! and event policy live one layer up in `GroupDocsServiceImpl`.
use async_trait::async_trait;
use base64::engine::general_purpose::URL_SAFE_NO_PAD;
use base64::Engine as _;
use chrono::{DateTime, Utc};
use picloud_shared::{DocId, DocRow, DocsListPage, GroupId};
use serde_json::Value;
use sqlx::PgPool;
use uuid::Uuid;
use crate::docs_filter::DocsFilter;
use crate::docs_repo::{build_find_query, row_to_doc, DocsRepoError};
#[derive(Debug, thiserror::Error)]
pub enum GroupDocsRepoError {
#[error("database error: {0}")]
Db(#[from] sqlx::Error),
#[error("invalid pagination cursor")]
InvalidCursor,
}
impl From<DocsRepoError> for GroupDocsRepoError {
fn from(e: DocsRepoError) -> Self {
match e {
DocsRepoError::Db(e) => Self::Db(e),
DocsRepoError::InvalidCursor => Self::InvalidCursor,
}
}
}
#[async_trait]
pub trait GroupDocsRepo: Send + Sync {
async fn create(
&self,
group_id: GroupId,
collection: &str,
data: Value,
) -> Result<DocRow, GroupDocsRepoError>;
async fn get(
&self,
group_id: GroupId,
collection: &str,
id: DocId,
) -> Result<Option<DocRow>, GroupDocsRepoError>;
async fn find(
&self,
group_id: GroupId,
collection: &str,
filter: &DocsFilter,
) -> Result<Vec<DocRow>, GroupDocsRepoError>;
/// Returns the previous data (for the would-be event), `None` if missing.
async fn update(
&self,
group_id: GroupId,
collection: &str,
id: DocId,
data: Value,
) -> Result<Option<Value>, GroupDocsRepoError>;
async fn delete(
&self,
group_id: GroupId,
collection: &str,
id: DocId,
) -> Result<Option<Value>, GroupDocsRepoError>;
async fn list(
&self,
group_id: GroupId,
collection: &str,
cursor: Option<&str>,
limit: u32,
) -> Result<DocsListPage, GroupDocsRepoError>;
/// §11.6 quota: total doc count across the group's shared-docs collections.
/// Default `Ok(0)` so non-Postgres impls skip the quota check.
async fn count_rows(&self, group_id: GroupId) -> Result<u64, GroupDocsRepoError> {
let _ = group_id;
Ok(0)
}
/// §11.6 M4 quota: total stored JSON bytes across the group's shared-docs
/// collections (`octet_length(data::text)`). Default `Ok(0)`.
async fn total_bytes(&self, group_id: GroupId) -> Result<u64, GroupDocsRepoError> {
let _ = group_id;
Ok(0)
}
/// §11.6 M4 quota: the PROJECTED total stored bytes for the group AFTER
/// writing `new_data` — current SUM, minus the existing doc's bytes
/// (`replacing = Some(id)` on update; `None` on create), plus the new data's
/// bytes. Computed entirely in SQL so ALL three terms use the SAME canonical
/// metric (`octet_length(data::text)`); the Rust-side subtract/add previously
/// mixed the DB canonical SUM with a `serde_json` compact length, drifting the
/// ceiling permissive. Default `Ok(0)`.
async fn projected_total_bytes(
&self,
group_id: GroupId,
replacing: Option<DocId>,
new_data: &serde_json::Value,
) -> Result<u64, GroupDocsRepoError> {
let _ = (group_id, replacing, new_data);
Ok(0)
}
}
pub struct PostgresGroupDocsRepo {
pool: PgPool,
}
impl PostgresGroupDocsRepo {
#[must_use]
pub fn new(pool: PgPool) -> Self {
Self { pool }
}
}
const DOCS_LIST_MAX_LIMIT: u32 = 1_000;
const DOCS_LIST_DEFAULT_LIMIT: u32 = 100;
#[async_trait]
impl GroupDocsRepo for PostgresGroupDocsRepo {
async fn create(
&self,
group_id: GroupId,
collection: &str,
data: Value,
) -> Result<DocRow, GroupDocsRepoError> {
create_on(&self.pool, group_id, collection, data).await
}
async fn get(
&self,
group_id: GroupId,
collection: &str,
id: DocId,
) -> Result<Option<DocRow>, GroupDocsRepoError> {
let row: Option<(Value, DateTime<Utc>, DateTime<Utc>)> = sqlx::query_as(
"SELECT data, created_at, updated_at FROM group_docs \
WHERE group_id = $1 AND collection = $2 AND id = $3",
)
.bind(group_id.into_inner())
.bind(collection)
.bind(id)
.fetch_optional(&self.pool)
.await?;
Ok(row.map(|(data, created_at, updated_at)| DocRow {
id,
data,
created_at,
updated_at,
}))
}
async fn find(
&self,
group_id: GroupId,
collection: &str,
filter: &DocsFilter,
) -> Result<Vec<DocRow>, GroupDocsRepoError> {
let mut qb = build_find_query(
"group_docs",
"group_id",
group_id.into_inner(),
collection,
filter,
);
let rows = qb.build().fetch_all(&self.pool).await?;
rows.into_iter()
.map(row_to_doc)
.collect::<Result<Vec<_>, _>>()
.map_err(Into::into)
}
async fn update(
&self,
group_id: GroupId,
collection: &str,
id: DocId,
data: Value,
) -> Result<Option<Value>, GroupDocsRepoError> {
update_on(&self.pool, group_id, collection, id, data).await
}
async fn delete(
&self,
group_id: GroupId,
collection: &str,
id: DocId,
) -> Result<Option<Value>, GroupDocsRepoError> {
delete_on(&self.pool, group_id, collection, id).await
}
async fn list(
&self,
group_id: GroupId,
collection: &str,
cursor: Option<&str>,
limit: u32,
) -> Result<DocsListPage, GroupDocsRepoError> {
let limit = if limit == 0 {
DOCS_LIST_DEFAULT_LIMIT
} else {
limit.min(DOCS_LIST_MAX_LIMIT)
};
let last_id = match cursor {
Some(c) => Some(decode_cursor(c)?),
None => None,
};
let take = i64::from(limit) + 1;
let rows: Vec<(Uuid, Value, DateTime<Utc>, DateTime<Utc>)> = sqlx::query_as(
"SELECT id, data, created_at, updated_at FROM group_docs \
WHERE group_id = $1 AND collection = $2 \
AND ($3::uuid IS NULL OR id > $3) \
ORDER BY id ASC \
LIMIT $4",
)
.bind(group_id.into_inner())
.bind(collection)
.bind(last_id)
.bind(take)
.fetch_all(&self.pool)
.await?;
let mut docs: Vec<DocRow> = rows
.into_iter()
.map(|(id, data, created_at, updated_at)| DocRow {
id,
data,
created_at,
updated_at,
})
.collect();
let next_cursor = if docs.len() > limit as usize {
docs.truncate(limit as usize);
docs.last().map(|d| encode_cursor(&d.id))
} else {
None
};
Ok(DocsListPage { docs, next_cursor })
}
async fn count_rows(&self, group_id: GroupId) -> Result<u64, GroupDocsRepoError> {
count_rows_on(&self.pool, group_id).await
}
async fn total_bytes(&self, group_id: GroupId) -> Result<u64, GroupDocsRepoError> {
let (n,): (i64,) = sqlx::query_as(
"SELECT COALESCE(SUM(octet_length(data::text)), 0)::BIGINT \
FROM group_docs WHERE group_id = $1",
)
.bind(group_id.into_inner())
.fetch_one(&self.pool)
.await?;
Ok(u64::try_from(n).unwrap_or(0))
}
async fn projected_total_bytes(
&self,
group_id: GroupId,
replacing: Option<DocId>,
new_data: &serde_json::Value,
) -> Result<u64, GroupDocsRepoError> {
projected_total_bytes_on(&self.pool, group_id, replacing, new_data).await
}
}
// ----------------------------------------------------------------------------
// Connection-scoped operations
//
// Generic over the executor so the same SQL serves the pooled trait methods
// above and `crate::atomic_write`, which runs the per-group advisory lock, the
// quota reads, the write, and the shared-trigger fan-out on ONE connection
// inside ONE transaction (audit #6/#8).
// ----------------------------------------------------------------------------
pub(crate) async fn create_on<'c, E>(
exec: E,
group_id: GroupId,
collection: &str,
data: Value,
) -> Result<DocRow, GroupDocsRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let id = Uuid::new_v4();
let row: (DateTime<Utc>, DateTime<Utc>) = sqlx::query_as(
"INSERT INTO group_docs (group_id, collection, id, data) \
VALUES ($1, $2, $3, $4) \
RETURNING created_at, updated_at",
)
.bind(group_id.into_inner())
.bind(collection)
.bind(id)
.bind(&data)
.fetch_one(exec)
.await?;
Ok(DocRow {
id,
data,
created_at: row.0,
updated_at: row.1,
})
}
/// Returns the previous data (for the update event), `None` if the doc is absent.
pub(crate) async fn update_on<'c, E>(
exec: E,
group_id: GroupId,
collection: &str,
id: DocId,
data: Value,
) -> Result<Option<Value>, GroupDocsRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let row: Option<(Option<Value>,)> = sqlx::query_as(
"WITH prev AS ( \
SELECT data FROM group_docs \
WHERE group_id = $1 AND collection = $2 AND id = $3 \
), \
updated AS ( \
UPDATE group_docs SET data = $4, updated_at = NOW() \
WHERE group_id = $1 AND collection = $2 AND id = $3 \
RETURNING 1 \
) \
SELECT (SELECT data FROM prev) FROM updated",
)
.bind(group_id.into_inner())
.bind(collection)
.bind(id)
.bind(&data)
.fetch_optional(exec)
.await?;
Ok(row.and_then(|(v,)| v))
}
pub(crate) async fn delete_on<'c, E>(
exec: E,
group_id: GroupId,
collection: &str,
id: DocId,
) -> Result<Option<Value>, GroupDocsRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let row: Option<(Value,)> = sqlx::query_as(
"DELETE FROM group_docs \
WHERE group_id = $1 AND collection = $2 AND id = $3 \
RETURNING data",
)
.bind(group_id.into_inner())
.bind(collection)
.bind(id)
.fetch_optional(exec)
.await?;
Ok(row.map(|(v,)| v))
}
/// Total doc count across the group's shared-docs collections (§11.6 row quota).
pub(crate) async fn count_rows_on<'c, E>(
exec: E,
group_id: GroupId,
) -> Result<u64, GroupDocsRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let (n,): (i64,) = sqlx::query_as("SELECT COUNT(*) FROM group_docs WHERE group_id = $1")
.bind(group_id.into_inner())
.fetch_one(exec)
.await?;
Ok(u64::try_from(n).unwrap_or(0))
}
/// §11.6 M4 quota: the PROJECTED total stored bytes for the group AFTER writing
/// `new_data` — current SUM, minus the replaced doc's bytes (`replacing =
/// Some(id)` on update, `None` on create), plus the new data's. Computed
/// entirely in SQL so all three terms use the SAME canonical metric
/// (`octet_length(data::text)`).
pub(crate) async fn projected_total_bytes_on<'c, E>(
exec: E,
group_id: GroupId,
replacing: Option<DocId>,
new_data: &serde_json::Value,
) -> Result<u64, GroupDocsRepoError>
where
E: sqlx::PgExecutor<'c>,
{
// The new data is bound as compact TEXT and re-parsed through `::jsonb::text`
// so PG measures it in the same canonical form it stores. `replacing = None`
// (create) → the subtrahend row never matches → 0.
let new_text = serde_json::to_string(new_data).unwrap_or_else(|_| "null".to_string());
let (n,): (i64,) = sqlx::query_as(
"SELECT ( \
COALESCE((SELECT SUM(octet_length(data::text)) \
FROM group_docs WHERE group_id = $1), 0) \
- COALESCE((SELECT octet_length(data::text) \
FROM group_docs WHERE group_id = $1 AND id = $2), 0) \
+ octet_length($3::jsonb::text) \
)::BIGINT",
)
.bind(group_id.into_inner())
.bind(replacing)
.bind(new_text)
.fetch_one(exec)
.await?;
Ok(u64::try_from(n).unwrap_or(0))
}
fn encode_cursor(last_id: &Uuid) -> String {
URL_SAFE_NO_PAD.encode(last_id.as_bytes())
}
fn decode_cursor(cursor: &str) -> Result<Uuid, GroupDocsRepoError> {
let bytes = URL_SAFE_NO_PAD
.decode(cursor)
.map_err(|_| GroupDocsRepoError::InvalidCursor)?;
let arr: [u8; 16] = bytes
.as_slice()
.try_into()
.map_err(|_| GroupDocsRepoError::InvalidCursor)?;
Ok(Uuid::from_bytes(arr))
}

View File

@@ -1,663 +0,0 @@
//! `GroupDocsServiceImpl` — wires `GroupDocsRepo` + the group-collection
//! registry underneath the `picloud_shared::GroupDocsService` trait that scripts
//! reach via the `docs::shared_collection("name")` Rhai handle (§11.6).
//!
//! Combines the group-KV service pattern (owner resolution from `cx.app_id`,
//! reads-open / writes-authed authz) with the docs surface (filter parsing,
//! JSON-object validation, value-size cap). No event emission in the MVP (the
//! "group trigger has no app to watch" deferral).
use std::sync::Arc;
use async_trait::async_trait;
use picloud_shared::{
DocId, DocRow, DocsListPage, GroupDocsError, GroupDocsService, GroupId, NoopEventEmitter,
SdkCallCx, ServiceEventEmitter,
};
use crate::atomic_write::{
BestEffortGroupDocsWriter, GroupDocsTarget, GroupDocsWriter, PostgresGroupDocsWriter,
};
use crate::authz::{self, AuthzRepo, Capability};
use crate::docs_filter::{parse_filter, FilterParseError};
use crate::docs_service::docs_max_value_bytes_from_env;
use crate::group_collection_repo::GroupCollectionResolver;
use crate::group_docs_repo::{GroupDocsRepo, GroupDocsRepoError};
use crate::quota::GroupWriteQuota;
/// The registry `kind` this service resolves.
const KIND_DOCS: &str = "docs";
pub struct GroupDocsServiceImpl {
repo: Arc<dyn GroupDocsRepo>,
resolver: Arc<dyn GroupCollectionResolver>,
authz: Arc<dyn AuthzRepo>,
max_value_bytes: usize,
/// §11.6 per-group quota: max total docs across the group's shared-docs
/// collections (`PICLOUD_GROUP_DOCS_MAX_ROWS`).
max_rows: u64,
/// §11.6 M4 per-group quota: max total stored bytes across the group's
/// shared-docs collections (`PICLOUD_GROUP_DOCS_MAX_TOTAL_BYTES`). Checked on
/// the projected total so a same/smaller update near the cap is still allowed.
max_total_bytes: u64,
/// Mutations: the quota decision, the row write, and the shared-trigger
/// fan-out, which for the host all commit in one advisory-locked
/// transaction. Reads stay on `repo`.
writer: Arc<dyn GroupDocsWriter>,
/// Set once [`Self::with_atomic_writes`] has installed the transactional
/// writer. `with_events` refuses to overwrite it — silently downgrading a
/// service back to non-transactional writes (and dropping its quota
/// enforcement) because a builder call was appended in the wrong order is
/// exactly the kind of regression the type system will not catch.
atomic: bool,
}
impl GroupDocsServiceImpl {
#[must_use]
pub fn new(
repo: Arc<dyn GroupDocsRepo>,
resolver: Arc<dyn GroupCollectionResolver>,
authz: Arc<dyn AuthzRepo>,
) -> Self {
Self::with_max_value_bytes(repo, resolver, authz, docs_max_value_bytes_from_env())
}
/// Wire the event emitter (§11.6 shared-collection triggers) on the
/// best-effort writer. Superseded by [`Self::with_atomic_writes`].
#[must_use]
pub fn with_events(mut self, events: Arc<dyn ServiceEventEmitter>) -> Self {
if self.atomic {
tracing::warn!(
"with_events() called after with_atomic_writes() — ignored; the \
transactional writer already resolves and writes the fan-out itself"
);
return self;
}
self.writer = Arc::new(BestEffortGroupDocsWriter::new(self.repo.clone(), events));
self
}
/// Use the transactional writer: the quota reads, the row write, and the
/// shared-trigger fan-out all run on ONE connection under a per-group
/// advisory lock, so the quota is a real bound and an emit failure rolls the
/// write back. Supersedes [`Self::with_events`].
#[must_use]
pub fn with_atomic_writes(mut self, pool: sqlx::PgPool) -> Self {
self.writer = Arc::new(PostgresGroupDocsWriter::new(pool));
self.atomic = true;
self
}
#[must_use]
pub fn with_max_value_bytes(
repo: Arc<dyn GroupDocsRepo>,
resolver: Arc<dyn GroupCollectionResolver>,
authz: Arc<dyn AuthzRepo>,
max_value_bytes: usize,
) -> Self {
Self {
writer: Arc::new(BestEffortGroupDocsWriter::new(
repo.clone(),
Arc::new(NoopEventEmitter),
)),
atomic: false,
max_rows: crate::quota::group_docs_max_rows_from_env(),
max_total_bytes: crate::quota::group_docs_max_total_bytes_from_env(),
repo,
resolver,
authz,
max_value_bytes,
}
}
/// Override the per-group total-bytes quota (§11.6 M4). Mainly for tests.
#[must_use]
pub fn with_max_total_bytes(mut self, max_total_bytes: u64) -> Self {
self.max_total_bytes = max_total_bytes;
self
}
/// The ceilings a shared-collection write must fit under. The row/byte
/// policy itself lives in `group_quota::check_group_write`, shared with KV.
fn quota(&self) -> GroupWriteQuota {
GroupWriteQuota {
max_rows: self.max_rows,
max_total_bytes: self.max_total_bytes,
max_value_bytes: self.max_value_bytes,
}
}
async fn owning_group(
&self,
cx: &SdkCallCx,
collection: &str,
) -> Result<GroupId, GroupDocsError> {
if collection.is_empty() {
return Err(GroupDocsError::InvalidCollection);
}
self.resolver
.resolve_owning_group(cx.app_id, collection, KIND_DOCS)
.await
.map_err(|e| GroupDocsError::Backend(e.to_string()))?
.ok_or_else(|| GroupDocsError::CollectionNotShared(collection.to_string()))
}
fn check_data_size(&self, data: &serde_json::Value) -> Result<(), GroupDocsError> {
let encoded_len = serde_json::to_vec(data)
.map(|v| v.len())
.map_err(|e| GroupDocsError::Backend(format!("encode doc data: {e}")))?;
if encoded_len > self.max_value_bytes {
return Err(GroupDocsError::ValueTooLarge {
limit: self.max_value_bytes,
actual: encoded_len,
});
}
Ok(())
}
async fn check_read(&self, cx: &SdkCallCx, group_id: GroupId) -> Result<(), GroupDocsError> {
authz::script_gate(
&*self.authz,
cx,
Capability::GroupDocsRead(group_id),
|| GroupDocsError::Forbidden,
GroupDocsError::Backend,
)
.await
}
async fn check_write(&self, cx: &SdkCallCx, group_id: GroupId) -> Result<(), GroupDocsError> {
// Fails closed for an anonymous principal — shared mutation always needs
// an authenticated editor+ on the owning group.
authz::script_gate_require_principal(
&*self.authz,
cx,
Capability::GroupDocsWrite(group_id),
|| GroupDocsError::Forbidden,
GroupDocsError::Backend,
)
.await
}
}
fn validate_data(data: &serde_json::Value) -> Result<(), GroupDocsError> {
if !data.is_object() {
return Err(GroupDocsError::InvalidData);
}
Ok(())
}
impl From<GroupDocsRepoError> for GroupDocsError {
fn from(e: GroupDocsRepoError) -> Self {
Self::Backend(e.to_string())
}
}
impl From<FilterParseError> for GroupDocsError {
fn from(e: FilterParseError) -> Self {
match e {
FilterParseError::InvalidFilter(s) => Self::InvalidFilter(s),
FilterParseError::UnsupportedOperator(s) => Self::UnsupportedOperator(s),
}
}
}
#[async_trait]
impl GroupDocsService for GroupDocsServiceImpl {
async fn create(
&self,
cx: &SdkCallCx,
collection: &str,
data: serde_json::Value,
) -> Result<DocId, GroupDocsError> {
let group_id = self.owning_group(cx, collection).await?;
validate_data(&data)?;
self.check_data_size(&data)?;
self.check_write(cx, group_id).await?;
self.writer
.create(
cx,
GroupDocsTarget {
group_id,
collection,
},
data,
self.quota(),
)
.await
}
async fn get(
&self,
cx: &SdkCallCx,
collection: &str,
id: DocId,
) -> Result<Option<DocRow>, GroupDocsError> {
let group_id = self.owning_group(cx, collection).await?;
self.check_read(cx, group_id).await?;
Ok(self.repo.get(group_id, collection, id).await?)
}
async fn find(
&self,
cx: &SdkCallCx,
collection: &str,
filter: serde_json::Value,
) -> Result<Vec<DocRow>, GroupDocsError> {
let group_id = self.owning_group(cx, collection).await?;
self.check_read(cx, group_id).await?;
let parsed = parse_filter(&filter)?;
Ok(self.repo.find(group_id, collection, &parsed).await?)
}
async fn find_one(
&self,
cx: &SdkCallCx,
collection: &str,
filter: serde_json::Value,
) -> Result<Option<DocRow>, GroupDocsError> {
let group_id = self.owning_group(cx, collection).await?;
self.check_read(cx, group_id).await?;
let mut parsed = parse_filter(&filter)?;
if parsed.limit.is_none() {
parsed.limit = Some(1);
}
let rows = self.repo.find(group_id, collection, &parsed).await?;
Ok(rows.into_iter().next())
}
async fn update(
&self,
cx: &SdkCallCx,
collection: &str,
id: DocId,
data: serde_json::Value,
) -> Result<(), GroupDocsError> {
let group_id = self.owning_group(cx, collection).await?;
validate_data(&data)?;
self.check_data_size(&data)?;
self.check_write(cx, group_id).await?;
if self
.writer
.update(
cx,
GroupDocsTarget {
group_id,
collection,
},
id,
data,
self.quota(),
)
.await?
{
Ok(())
} else {
Err(GroupDocsError::NotFound)
}
}
async fn delete(
&self,
cx: &SdkCallCx,
collection: &str,
id: DocId,
) -> Result<bool, GroupDocsError> {
let group_id = self.owning_group(cx, collection).await?;
self.check_write(cx, group_id).await?;
self.writer
.delete(
cx,
GroupDocsTarget {
group_id,
collection,
},
id,
)
.await
}
async fn list(
&self,
cx: &SdkCallCx,
collection: &str,
cursor: Option<&str>,
limit: u32,
) -> Result<DocsListPage, GroupDocsError> {
let group_id = self.owning_group(cx, collection).await?;
self.check_read(cx, group_id).await?;
Ok(self.repo.list(group_id, collection, cursor, limit).await?)
}
}
// ----------------------------------------------------------------------------
// Tests — in-memory repo + a fake resolver so unit tests don't need Postgres.
// ----------------------------------------------------------------------------
#[cfg(test)]
mod tests {
use super::*;
use crate::authz::{AuthzError, AuthzRepo};
use crate::docs_filter::DocsFilter;
use chrono::Utc;
use picloud_shared::{
AdminUserId, AppId, AppRole, ExecutionId, InstanceRole, Principal, RequestId, ScriptId,
UserId,
};
use std::collections::HashMap;
use tokio::sync::Mutex;
use uuid::Uuid;
#[derive(Default)]
struct InMemoryGroupDocsRepo {
// (group, collection) -> id -> data
data: Mutex<HashMap<(GroupId, String), HashMap<Uuid, serde_json::Value>>>,
}
fn row(id: Uuid, data: serde_json::Value) -> DocRow {
DocRow {
id,
data,
created_at: Utc::now(),
updated_at: Utc::now(),
}
}
#[async_trait]
impl GroupDocsRepo for InMemoryGroupDocsRepo {
async fn create(
&self,
group_id: GroupId,
collection: &str,
data: serde_json::Value,
) -> Result<DocRow, GroupDocsRepoError> {
let id = Uuid::new_v4();
self.data
.lock()
.await
.entry((group_id, collection.to_string()))
.or_default()
.insert(id, data.clone());
Ok(row(id, data))
}
async fn get(
&self,
group_id: GroupId,
collection: &str,
id: DocId,
) -> Result<Option<DocRow>, GroupDocsRepoError> {
Ok(self
.data
.lock()
.await
.get(&(group_id, collection.to_string()))
.and_then(|m| m.get(&id).cloned())
.map(|d| row(id, d)))
}
// The fake ignores the filter (filter SQL is covered by docs_repo tests
// + the journey); returns all docs in the collection.
async fn find(
&self,
group_id: GroupId,
collection: &str,
_filter: &DocsFilter,
) -> Result<Vec<DocRow>, GroupDocsRepoError> {
Ok(self
.data
.lock()
.await
.get(&(group_id, collection.to_string()))
.map(|m| m.iter().map(|(id, d)| row(*id, d.clone())).collect())
.unwrap_or_default())
}
async fn update(
&self,
group_id: GroupId,
collection: &str,
id: DocId,
data: serde_json::Value,
) -> Result<Option<serde_json::Value>, GroupDocsRepoError> {
let mut guard = self.data.lock().await;
let coll = guard.entry((group_id, collection.to_string())).or_default();
Ok(coll.insert(id, data))
}
async fn delete(
&self,
group_id: GroupId,
collection: &str,
id: DocId,
) -> Result<Option<serde_json::Value>, GroupDocsRepoError> {
Ok(self
.data
.lock()
.await
.get_mut(&(group_id, collection.to_string()))
.and_then(|m| m.remove(&id)))
}
async fn list(
&self,
_group_id: GroupId,
_collection: &str,
_cursor: Option<&str>,
_limit: u32,
) -> Result<DocsListPage, GroupDocsRepoError> {
Ok(DocsListPage {
docs: vec![],
next_cursor: None,
})
}
async fn total_bytes(&self, group_id: GroupId) -> Result<u64, GroupDocsRepoError> {
let data = self.data.lock().await;
Ok(data
.iter()
.filter(|((g, _), _)| *g == group_id)
.flat_map(|(_, docs)| docs.values())
.map(|v| serde_json::to_vec(v).map_or(0, |b| b.len() as u64))
.sum())
}
/// Same projection the Postgres impl computes in SQL, but with this
/// mock's own (compact) metric: every doc in the group EXCEPT the one
/// being replaced, plus the new data — i.e. `used - old + new`.
async fn projected_total_bytes(
&self,
group_id: GroupId,
replacing: Option<DocId>,
new_data: &serde_json::Value,
) -> Result<u64, GroupDocsRepoError> {
let data = self.data.lock().await;
let others: u64 = data
.iter()
.filter(|((g, _), _)| *g == group_id)
.flat_map(|(_, docs)| docs.iter())
.filter(|(id, _)| Some(**id) != replacing)
.map(|(_, v)| serde_json::to_vec(v).map_or(0, |b| b.len() as u64))
.sum();
let new_len = serde_json::to_vec(new_data).map_or(0, |b| b.len() as u64);
Ok(others + new_len)
}
}
#[derive(Default)]
struct FakeResolver {
map: HashMap<(AppId, String), GroupId>,
}
#[async_trait]
impl GroupCollectionResolver for FakeResolver {
async fn resolve_owning_group(
&self,
app_id: AppId,
name: &str,
_kind: &str,
) -> Result<Option<GroupId>, sqlx::Error> {
Ok(self.map.get(&(app_id, name.to_lowercase())).copied())
}
}
#[derive(Default)]
struct DenyingAuthzRepo;
#[async_trait]
impl AuthzRepo for DenyingAuthzRepo {
async fn membership(
&self,
_user_id: UserId,
_app_id: AppId,
) -> Result<Option<AppRole>, AuthzError> {
Ok(None)
}
}
fn cx_with(app_id: AppId, principal: Option<Principal>) -> SdkCallCx {
SdkCallCx {
app_id,
script_id: ScriptId::new(),
principal,
execution_id: ExecutionId::new(),
request_id: RequestId::new(),
trigger_depth: 0,
root_execution_id: ExecutionId::new(),
is_dead_letter_handler: false,
event: None,
}
}
fn owner() -> Principal {
Principal {
user_id: AdminUserId::new(),
instance_role: InstanceRole::Owner,
scopes: None,
app_binding: None,
}
}
fn svc(resolver: FakeResolver) -> GroupDocsServiceImpl {
GroupDocsServiceImpl::new(
Arc::new(InMemoryGroupDocsRepo::default()),
Arc::new(resolver),
Arc::new(DenyingAuthzRepo),
)
}
#[tokio::test]
async fn unrelated_app_gets_collection_not_shared() {
let app_a = AppId::new();
let app_b = AppId::new();
let group = GroupId::new();
let mut resolver = FakeResolver::default();
resolver.map.insert((app_a, "articles".into()), group);
let docs = svc(resolver);
let cx_a = cx_with(app_a, Some(owner()));
let id = docs
.create(&cx_a, "articles", serde_json::json!({"t": "hi"}))
.await
.unwrap();
assert!(docs.get(&cx_a, "articles", id).await.unwrap().is_some());
let cx_b = cx_with(app_b, Some(owner()));
let err = docs
.find(&cx_b, "articles", serde_json::json!({}))
.await
.unwrap_err();
assert!(matches!(err, GroupDocsError::CollectionNotShared(c) if c == "articles"));
}
#[tokio::test]
async fn reads_open_writes_require_auth() {
let app = AppId::new();
let group = GroupId::new();
let mut resolver = FakeResolver::default();
resolver.map.insert((app, "articles".into()), group);
let docs = svc(resolver);
let owner_cx = cx_with(app, Some(owner()));
docs.create(&owner_cx, "articles", serde_json::json!({"t": "hi"}))
.await
.unwrap();
// Anonymous READ (find) is allowed.
let anon = cx_with(app, None);
let hits = docs
.find(&anon, "articles", serde_json::json!({}))
.await
.unwrap();
assert_eq!(hits.len(), 1);
// Anonymous WRITE (create) fails closed.
let err = docs
.create(&anon, "articles", serde_json::json!({"t": "x"}))
.await
.unwrap_err();
assert!(matches!(err, GroupDocsError::Forbidden));
}
#[tokio::test]
async fn non_object_data_rejected() {
let app = AppId::new();
let group = GroupId::new();
let mut resolver = FakeResolver::default();
resolver.map.insert((app, "articles".into()), group);
let docs = svc(resolver);
let cx = cx_with(app, Some(owner()));
let err = docs
.create(&cx, "articles", serde_json::json!("not-an-object"))
.await
.unwrap_err();
assert!(matches!(err, GroupDocsError::InvalidData));
}
#[tokio::test]
async fn per_group_byte_quota_uses_the_projected_total() {
// §11.6 M4: the shared-docs byte ceiling is checked on the PROJECTED
// total — a create over the cap is rejected, and a shrinking update near
// the cap is allowed (the delta rule).
let app = AppId::new();
let group = GroupId::new();
let mut resolver = FakeResolver::default();
resolver.map.insert((app, "articles".into()), group);
let docs = GroupDocsServiceImpl::new(
Arc::new(InMemoryGroupDocsRepo::default()),
Arc::new(resolver),
Arc::new(DenyingAuthzRepo),
)
.with_max_total_bytes(40);
let cx = cx_with(app, Some(owner()));
// {"t":"xxxxx"} encodes to 13 bytes.
let id = docs
.create(&cx, "articles", serde_json::json!({"t": "xxxxx"}))
.await
.unwrap(); // used = 13
docs.create(&cx, "articles", serde_json::json!({"t": "xxxxx"}))
.await
.unwrap(); // used = 26
// A third 13-byte doc → projected 39 ≤ 40 → ok.
docs.create(&cx, "articles", serde_json::json!({"t": "xxxxx"}))
.await
.unwrap(); // used = 39
// A fourth → projected 52 > 40 → rejected.
let err = docs
.create(&cx, "articles", serde_json::json!({"t": "xxxxx"}))
.await
.unwrap_err();
assert!(
matches!(
err,
GroupDocsError::TotalBytesQuotaExceeded { limit: 40, .. }
),
"got {err:?}"
);
// Shrinking an existing doc frees budget (projected 39 - 13 + 9 = 35).
docs.update(&cx, "articles", id, serde_json::json!({"t": "y"}))
.await
.expect("a shrinking update near the cap must be allowed");
}
}

View File

@@ -1,559 +0,0 @@
//! Low-level metadata (Postgres `group_files`) + blob bytes (filesystem) storage
//! for §11.6 group-shared FILES collections. A near-clone of [`crate::files_repo`]
//! keyed by the owning `group_id` instead of `app_id`.
//!
//! The security-sensitive disk mechanics — the atomic write+checksum protocol and
//! checksum-on-read — are **not** duplicated: they come from the owner-relative
//! free functions in [`crate::files_repo`] (`write_atomic_at` / `read_verify_at` /
//! `final_path_at`), called with this repo's owner sub-path. Group blobs shard at
//! `<root>/files/groups/<group_id>/<collection>/<id[0:2]>/<id>` — a `groups/` infix
//! disjoint from the per-app `files/<app_id>/...` subtree, so the orphan sweeper
//! ([crate::files_sweep]) covers both with one walk. Authorization, group
//! resolution, value validation, and content-type sanitization live one layer up
//! in `GroupFilesServiceImpl`.
use std::path::{Path, PathBuf};
use async_trait::async_trait;
use chrono::{DateTime, Utc};
use picloud_shared::{FileMeta, FileUpdate, FilesListPage, GroupId, NewFile};
use sqlx::PgPool;
use uuid::Uuid;
use crate::files_repo::{
decode_cursor, encode_cursor, final_path_at, read_verify_at, write_atomic_at, FileUpdated,
FilesRepoError,
};
const FILES_LIST_MAX_LIMIT: u32 = 1_000;
const FILES_LIST_DEFAULT_LIMIT: u32 = 100;
#[derive(Debug, thiserror::Error)]
pub enum GroupFilesRepoError {
#[error("database error: {0}")]
Db(#[from] sqlx::Error),
#[error("filesystem error: {0}")]
Io(String),
#[error("invalid collection name: {0}")]
InvalidCollection(String),
/// Bytes on disk no longer match the stored checksum (or are missing).
#[error("file content corrupted (checksum mismatch)")]
Corrupted,
#[error("invalid pagination cursor")]
InvalidCursor,
}
impl From<FilesRepoError> for GroupFilesRepoError {
fn from(e: FilesRepoError) -> Self {
match e {
FilesRepoError::Db(e) => Self::Db(e),
FilesRepoError::Io(s) => Self::Io(s),
FilesRepoError::InvalidCollection(c) => Self::InvalidCollection(c),
FilesRepoError::Corrupted => Self::Corrupted,
FilesRepoError::InvalidCursor => Self::InvalidCursor,
}
}
}
/// The owner-relative subdirectory of a **group**'s shared blobs under
/// `<root>/files/`: `groups/<group_id>`. A UUID app-dir can never equal the
/// literal `groups`, so app and group blob trees can't collide.
pub(crate) fn group_owner_dir(group_id: GroupId) -> PathBuf {
Path::new("groups").join(group_id.into_inner().to_string())
}
#[async_trait]
pub trait GroupFilesRepo: Send + Sync {
async fn create(
&self,
group_id: GroupId,
collection: &str,
new: NewFile,
) -> Result<FileMeta, GroupFilesRepoError>;
async fn head(
&self,
group_id: GroupId,
collection: &str,
id: Uuid,
) -> Result<Option<FileMeta>, GroupFilesRepoError>;
/// Reads + checksum-verifies the bytes. `Ok(None)` when no row exists;
/// `Err(Corrupted)` when the row exists but the bytes are missing/mismatched.
async fn get(
&self,
group_id: GroupId,
collection: &str,
id: Uuid,
) -> Result<Option<Vec<u8>>, GroupFilesRepoError>;
/// `Ok(None)` when no row exists (the service maps that to `NotFound`).
async fn update(
&self,
group_id: GroupId,
collection: &str,
id: Uuid,
upd: FileUpdate,
) -> Result<Option<FileUpdated>, GroupFilesRepoError>;
async fn delete(
&self,
group_id: GroupId,
collection: &str,
id: Uuid,
) -> Result<Option<FileMeta>, GroupFilesRepoError>;
async fn list(
&self,
group_id: GroupId,
collection: &str,
cursor: Option<&str>,
limit: u32,
) -> Result<FilesListPage, GroupFilesRepoError>;
/// §11.6 quota: total stored bytes across the group's shared-files
/// collections. Default `Ok(0)` so non-Postgres impls skip the check.
async fn total_bytes(&self, group_id: GroupId) -> Result<u64, GroupFilesRepoError> {
let _ = group_id;
Ok(0)
}
}
/// Filesystem-bytes + Postgres-metadata repo for group-shared files.
pub struct FsGroupFilesRepo {
pool: PgPool,
root: PathBuf,
}
impl FsGroupFilesRepo {
#[must_use]
pub fn new(pool: PgPool, root: PathBuf) -> Self {
Self { pool, root }
}
/// Belt-and-suspenders path guard (the service validates at the SDK
/// boundary). Mirrors `FsFilesRepo::guard_collection`.
fn guard_collection(collection: &str) -> Result<(), GroupFilesRepoError> {
if collection.is_empty()
|| collection.contains('/')
|| collection.contains('\\')
|| collection.contains("..")
|| collection.contains('\0')
{
return Err(GroupFilesRepoError::InvalidCollection(
collection.to_string(),
));
}
Ok(())
}
fn write_atomic(
&self,
group_id: GroupId,
collection: &str,
id: Uuid,
bytes: &[u8],
) -> Result<String, GroupFilesRepoError> {
Ok(write_atomic_at(
&self.root,
&group_owner_dir(group_id),
collection,
id,
bytes,
)?)
}
}
#[async_trait]
impl GroupFilesRepo for FsGroupFilesRepo {
async fn create(
&self,
group_id: GroupId,
collection: &str,
new: NewFile,
) -> Result<FileMeta, GroupFilesRepoError> {
Self::guard_collection(collection)?;
let id = Uuid::new_v4();
let size = i64::try_from(new.data.len()).unwrap_or(i64::MAX);
let checksum = self.write_atomic(group_id, collection, id, &new.data)?;
let row: GroupFileRow = sqlx::query_as(
"INSERT INTO group_files \
(group_id, collection, id, name, content_type, size_bytes, checksum_sha256) \
VALUES ($1, $2, $3, $4, $5, $6, $7) \
RETURNING id, collection, name, content_type, size_bytes, \
checksum_sha256, created_at, updated_at",
)
.bind(group_id.into_inner())
.bind(collection)
.bind(id)
.bind(&new.name)
.bind(&new.content_type)
.bind(size)
.bind(&checksum)
.fetch_one(&self.pool)
.await?;
Ok(row.into_meta())
}
async fn head(
&self,
group_id: GroupId,
collection: &str,
id: Uuid,
) -> Result<Option<FileMeta>, GroupFilesRepoError> {
Self::guard_collection(collection)?;
let row: Option<GroupFileRow> = sqlx::query_as(
"SELECT id, collection, name, content_type, size_bytes, \
checksum_sha256, created_at, updated_at \
FROM group_files WHERE group_id = $1 AND collection = $2 AND id = $3",
)
.bind(group_id.into_inner())
.bind(collection)
.bind(id)
.fetch_optional(&self.pool)
.await?;
Ok(row.map(GroupFileRow::into_meta))
}
async fn get(
&self,
group_id: GroupId,
collection: &str,
id: Uuid,
) -> Result<Option<Vec<u8>>, GroupFilesRepoError> {
Self::guard_collection(collection)?;
let row: Option<(String,)> = sqlx::query_as(
"SELECT checksum_sha256 FROM group_files \
WHERE group_id = $1 AND collection = $2 AND id = $3",
)
.bind(group_id.into_inner())
.bind(collection)
.bind(id)
.fetch_optional(&self.pool)
.await?;
let Some((stored_checksum,)) = row else {
return Ok(None);
};
let bytes = read_verify_at(
&self.root,
&group_owner_dir(group_id),
collection,
id,
&stored_checksum,
)?;
Ok(Some(bytes))
}
async fn update(
&self,
group_id: GroupId,
collection: &str,
id: Uuid,
upd: FileUpdate,
) -> Result<Option<FileUpdated>, GroupFilesRepoError> {
Self::guard_collection(collection)?;
let Some(prev) = self.head(group_id, collection, id).await? else {
return Ok(None);
};
let size = i64::try_from(upd.data.len()).unwrap_or(i64::MAX);
let checksum = self.write_atomic(group_id, collection, id, &upd.data)?;
let row: GroupFileRow = sqlx::query_as(
"UPDATE group_files SET \
name = COALESCE($4, name), \
content_type = COALESCE($5, content_type), \
size_bytes = $6, \
checksum_sha256 = $7, \
updated_at = NOW() \
WHERE group_id = $1 AND collection = $2 AND id = $3 \
RETURNING id, collection, name, content_type, size_bytes, \
checksum_sha256, created_at, updated_at",
)
.bind(group_id.into_inner())
.bind(collection)
.bind(id)
.bind(upd.name.as_deref())
.bind(upd.content_type.as_deref())
.bind(size)
.bind(&checksum)
.fetch_one(&self.pool)
.await?;
Ok(Some(FileUpdated {
new: row.into_meta(),
prev,
}))
}
async fn delete(
&self,
group_id: GroupId,
collection: &str,
id: Uuid,
) -> Result<Option<FileMeta>, GroupFilesRepoError> {
Self::guard_collection(collection)?;
let mut tx = self.pool.begin().await?;
let row: Option<GroupFileRow> = sqlx::query_as(
"SELECT id, collection, name, content_type, size_bytes, \
checksum_sha256, created_at, updated_at \
FROM group_files WHERE group_id = $1 AND collection = $2 AND id = $3 \
FOR UPDATE",
)
.bind(group_id.into_inner())
.bind(collection)
.bind(id)
.fetch_optional(&mut *tx)
.await?;
let Some(row) = row else {
tx.rollback().await?;
return Ok(None);
};
sqlx::query("DELETE FROM group_files WHERE group_id = $1 AND collection = $2 AND id = $3")
.bind(group_id.into_inner())
.bind(collection)
.bind(id)
.execute(&mut *tx)
.await?;
tx.commit().await?;
// Row gone; unlink the bytes. A failure here leaves an orphan file
// (reclaimed by the sweep) — not fatal.
let path = final_path_at(&self.root, &group_owner_dir(group_id), collection, id);
if let Err(e) = std::fs::remove_file(&path) {
if e.kind() != std::io::ErrorKind::NotFound {
tracing::warn!(path = %path.display(), error = %e, "group files: unlink after delete failed (orphan)");
}
}
Ok(Some(row.into_meta()))
}
async fn list(
&self,
group_id: GroupId,
collection: &str,
cursor: Option<&str>,
limit: u32,
) -> Result<FilesListPage, GroupFilesRepoError> {
Self::guard_collection(collection)?;
let limit = if limit == 0 {
FILES_LIST_DEFAULT_LIMIT
} else {
limit.min(FILES_LIST_MAX_LIMIT)
};
let last_id = match cursor {
Some(c) => Some(decode_cursor(c)?),
None => None,
};
let take = i64::from(limit) + 1;
let rows: Vec<GroupFileRow> = sqlx::query_as(
"SELECT id, collection, name, content_type, size_bytes, \
checksum_sha256, created_at, updated_at \
FROM group_files \
WHERE group_id = $1 AND collection = $2 \
AND ($3::uuid IS NULL OR id > $3) \
ORDER BY id ASC \
LIMIT $4",
)
.bind(group_id.into_inner())
.bind(collection)
.bind(last_id)
.bind(take)
.fetch_all(&self.pool)
.await?;
let mut files: Vec<FileMeta> = rows.into_iter().map(GroupFileRow::into_meta).collect();
let next_cursor = if files.len() > limit as usize {
files.truncate(limit as usize);
files.last().map(|m| encode_cursor(m.id))
} else {
None
};
Ok(FilesListPage { files, next_cursor })
}
async fn total_bytes(&self, group_id: GroupId) -> Result<u64, GroupFilesRepoError> {
let (n,): (i64,) = sqlx::query_as(
"SELECT COALESCE(SUM(size_bytes), 0)::bigint FROM group_files WHERE group_id = $1",
)
.bind(group_id.into_inner())
.fetch_one(&self.pool)
.await?;
Ok(u64::try_from(n).unwrap_or(0))
}
}
#[derive(sqlx::FromRow)]
pub(crate) struct GroupFileRow {
id: Uuid,
collection: String,
name: String,
content_type: String,
size_bytes: i64,
checksum_sha256: String,
created_at: DateTime<Utc>,
updated_at: DateTime<Utc>,
}
impl GroupFileRow {
pub(crate) fn into_meta(self) -> FileMeta {
FileMeta {
id: self.id,
collection: self.collection,
name: self.name,
content_type: self.content_type,
size: u64::try_from(self.size_bytes).unwrap_or(0),
checksum: self.checksum_sha256,
created_at: self.created_at,
updated_at: self.updated_at,
}
}
}
// ----------------------------------------------------------------------------
// Connection-scoped metadata operations (see `files_repo` for the rationale)
// ----------------------------------------------------------------------------
use crate::files_repo::{MetaFields, MetaPatch};
const GROUP_FILE_COLS: &str = "id, collection, name, content_type, size_bytes, \
checksum_sha256, created_at, updated_at";
pub(crate) async fn head_on<'c, E>(
exec: E,
group_id: GroupId,
collection: &str,
id: Uuid,
) -> Result<Option<FileMeta>, GroupFilesRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let row: Option<GroupFileRow> = sqlx::query_as(&format!(
"SELECT {GROUP_FILE_COLS} FROM group_files \
WHERE group_id = $1 AND collection = $2 AND id = $3"
))
.bind(group_id.into_inner())
.bind(collection)
.bind(id)
.fetch_optional(exec)
.await?;
Ok(row.map(GroupFileRow::into_meta))
}
pub(crate) async fn insert_meta_on<'c, E>(
exec: E,
group_id: GroupId,
collection: &str,
id: Uuid,
meta: MetaFields<'_>,
) -> Result<FileMeta, GroupFilesRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let row: GroupFileRow = sqlx::query_as(&format!(
"INSERT INTO group_files \
(group_id, collection, id, name, content_type, size_bytes, checksum_sha256) \
VALUES ($1, $2, $3, $4, $5, $6, $7) \
RETURNING {GROUP_FILE_COLS}"
))
.bind(group_id.into_inner())
.bind(collection)
.bind(id)
.bind(meta.name)
.bind(meta.content_type)
.bind(meta.size)
.bind(meta.checksum)
.fetch_one(exec)
.await?;
Ok(row.into_meta())
}
pub(crate) async fn update_meta_on<'c, E>(
exec: E,
group_id: GroupId,
collection: &str,
id: Uuid,
meta: MetaPatch<'_>,
) -> Result<Option<FileMeta>, GroupFilesRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let row: Option<GroupFileRow> = sqlx::query_as(&format!(
"UPDATE group_files SET \
name = COALESCE($4, name), \
content_type = COALESCE($5, content_type), \
size_bytes = $6, \
checksum_sha256 = $7, \
updated_at = NOW() \
WHERE group_id = $1 AND collection = $2 AND id = $3 \
RETURNING {GROUP_FILE_COLS}"
))
.bind(group_id.into_inner())
.bind(collection)
.bind(id)
.bind(meta.name)
.bind(meta.content_type)
.bind(meta.size)
.bind(meta.checksum)
.fetch_optional(exec)
.await?;
Ok(row.map(GroupFileRow::into_meta))
}
pub(crate) async fn delete_meta_on<'c, E>(
exec: E,
group_id: GroupId,
collection: &str,
id: Uuid,
) -> Result<Option<FileMeta>, GroupFilesRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let row: Option<GroupFileRow> = sqlx::query_as(&format!(
"DELETE FROM group_files \
WHERE group_id = $1 AND collection = $2 AND id = $3 \
RETURNING {GROUP_FILE_COLS}"
))
.bind(group_id.into_inner())
.bind(collection)
.bind(id)
.fetch_optional(exec)
.await?;
Ok(row.map(GroupFileRow::into_meta))
}
/// §11.6 quota: the PROJECTED total stored bytes for the group AFTER this write
/// — the current SUM, minus the bytes of the file being replaced (`replacing =
/// Some(id)` on update; `None` on create), plus the incoming file's bytes.
///
/// The subtraction is what `GroupFilesService::update` was missing entirely: it
/// checked no quota at all, so a 1-byte file could be updated to a 100 MB one
/// without ever consulting the ceiling.
pub(crate) async fn projected_total_bytes_on<'c, E>(
exec: E,
group_id: GroupId,
replacing: Option<Uuid>,
incoming: i64,
) -> Result<u64, GroupFilesRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let (n,): (i64,) = sqlx::query_as(
"SELECT ( \
COALESCE((SELECT SUM(size_bytes) FROM group_files WHERE group_id = $1), 0) \
- COALESCE((SELECT size_bytes FROM group_files \
WHERE group_id = $1 AND id = $2), 0) \
+ $3 \
)::BIGINT",
)
.bind(group_id.into_inner())
.bind(replacing)
.bind(incoming)
.fetch_one(exec)
.await?;
Ok(u64::try_from(n).unwrap_or(0))
}

View File

@@ -1,550 +0,0 @@
//! `GroupFilesServiceImpl` — wires `GroupFilesRepo` + the group-collection
//! registry underneath the `picloud_shared::GroupFilesService` trait that scripts
//! reach via the `files::shared_collection("name")` Rhai handle (§11.6).
//!
//! Combines the group-KV/docs service pattern (owner resolution from `cx.app_id`,
//! reads-open / writes-authed authz) with the files surface (collection
//! path-validation, field + size-cap validation, content-type sanitization). No
//! event emission in the MVP (the "group trigger has no app to watch" deferral).
use std::sync::Arc;
use async_trait::async_trait;
use picloud_shared::{
sanitize_stored_content_type, validate_files_collection, FileMeta, FileUpdate, FilesListPage,
GroupFilesError, GroupFilesService, GroupId, NewFile, NoopEventEmitter, SdkCallCx,
ServiceEventEmitter,
};
use uuid::Uuid;
use crate::atomic_write::{
BestEffortGroupFilesWriter, GroupFilesQuota, GroupFilesWriter, PostgresGroupFilesWriter,
};
use crate::authz::{self, AuthzRepo, Capability};
use crate::group_collection_repo::GroupCollectionResolver;
use crate::group_files_repo::{GroupFilesRepo, GroupFilesRepoError};
/// The registry `kind` this service resolves.
const KIND_FILES: &str = "files";
pub struct GroupFilesServiceImpl {
repo: Arc<dyn GroupFilesRepo>,
resolver: Arc<dyn GroupCollectionResolver>,
authz: Arc<dyn AuthzRepo>,
max_file_size_bytes: usize,
/// §11.6 per-group quota: max total stored bytes across the group's
/// shared-files collections (`PICLOUD_GROUP_FILES_MAX_TOTAL_BYTES`).
max_total_bytes: u64,
/// §11.6: fires `shared = true` files triggers on a shared-collection write.
/// Mutations: the byte-quota decision, the blob + metadata write, and the
/// shared-trigger fan-out — for the host, all under one advisory-locked
/// transaction. Reads stay on `repo`.
writer: Arc<dyn GroupFilesWriter>,
/// Set once [`Self::with_atomic_writes`] has installed the transactional
/// writer. `with_events` refuses to overwrite it — silently downgrading a
/// service back to non-transactional writes (and dropping its quota
/// enforcement) because a builder call was appended in the wrong order is
/// exactly the kind of regression the type system will not catch.
atomic: bool,
}
impl GroupFilesServiceImpl {
#[must_use]
pub fn new(
repo: Arc<dyn GroupFilesRepo>,
resolver: Arc<dyn GroupCollectionResolver>,
authz: Arc<dyn AuthzRepo>,
max_file_size_bytes: usize,
) -> Self {
Self {
max_total_bytes: crate::quota::group_files_max_total_bytes_from_env(),
writer: Arc::new(BestEffortGroupFilesWriter::new(
repo.clone(),
Arc::new(NoopEventEmitter),
)),
atomic: false,
repo,
resolver,
authz,
max_file_size_bytes,
}
}
/// The ceiling a shared-files write must fit under.
fn quota(&self) -> GroupFilesQuota {
GroupFilesQuota {
max_total_bytes: self.max_total_bytes,
}
}
/// Wire the event emitter (§11.6 shared-collection triggers) on the
/// best-effort writer. Superseded by [`Self::with_atomic_writes`].
#[must_use]
pub fn with_events(mut self, events: Arc<dyn ServiceEventEmitter>) -> Self {
if self.atomic {
tracing::warn!(
"with_events() called after with_atomic_writes() — ignored; the \
transactional writer already resolves and writes the fan-out itself"
);
return self;
}
self.writer = Arc::new(BestEffortGroupFilesWriter::new(self.repo.clone(), events));
self
}
/// Use the transactional writer: the byte-quota check, the metadata row, and
/// the shared-trigger fan-out run on ONE connection under a per-group
/// advisory lock, so concurrent uploads can't each see the same pre-write
/// total and together overshoot the ceiling. Supersedes
/// [`Self::with_events`].
#[must_use]
pub fn with_atomic_writes(mut self, pool: sqlx::PgPool, root: std::path::PathBuf) -> Self {
self.writer = Arc::new(PostgresGroupFilesWriter::new(pool, root));
self.atomic = true;
self
}
async fn owning_group(
&self,
cx: &SdkCallCx,
collection: &str,
) -> Result<GroupId, GroupFilesError> {
self.resolver
.resolve_owning_group(cx.app_id, collection, KIND_FILES)
.await
.map_err(|e| GroupFilesError::Backend(e.to_string()))?
.ok_or_else(|| GroupFilesError::CollectionNotShared(collection.to_string()))
}
async fn check_read(&self, cx: &SdkCallCx, group_id: GroupId) -> Result<(), GroupFilesError> {
authz::script_gate(
&*self.authz,
cx,
Capability::GroupFilesRead(group_id),
|| GroupFilesError::Forbidden,
GroupFilesError::Backend,
)
.await
}
async fn check_write(&self, cx: &SdkCallCx, group_id: GroupId) -> Result<(), GroupFilesError> {
// Fails closed for an anonymous principal — shared mutation always needs
// an authenticated editor+ on the owning group.
authz::script_gate_require_principal(
&*self.authz,
cx,
Capability::GroupFilesWrite(group_id),
|| GroupFilesError::Forbidden,
GroupFilesError::Backend,
)
.await
}
}
/// Invalid UUIDs aren't an error shape the SDK exposes — for reads/deletes they
/// simply mean "no such file" (mirrors `files_service::parse_id`).
fn parse_id(id: &str) -> Option<Uuid> {
Uuid::parse_str(id).ok()
}
impl From<GroupFilesRepoError> for GroupFilesError {
fn from(e: GroupFilesRepoError) -> Self {
match e {
GroupFilesRepoError::Corrupted => Self::Corrupted,
GroupFilesRepoError::InvalidCollection(c) => Self::InvalidCollection(c),
other => Self::Backend(other.to_string()),
}
}
}
#[async_trait]
impl GroupFilesService for GroupFilesServiceImpl {
async fn create(
&self,
cx: &SdkCallCx,
collection: &str,
mut new: NewFile,
) -> Result<Uuid, GroupFilesError> {
validate_files_collection(collection)?;
let group_id = self.owning_group(cx, collection).await?;
new.validate(self.max_file_size_bytes)?;
// Coerce dangerous render types to application/octet-stream after the
// shape checks pass (same as app files, audit 2026-06-11 C-2).
new.content_type = sanitize_stored_content_type(&new.content_type);
self.check_write(cx, group_id).await?;
self.writer
.create(cx, group_id, collection, new, self.quota())
.await
}
async fn head(
&self,
cx: &SdkCallCx,
collection: &str,
id: &str,
) -> Result<Option<FileMeta>, GroupFilesError> {
validate_files_collection(collection)?;
let group_id = self.owning_group(cx, collection).await?;
self.check_read(cx, group_id).await?;
let Some(uuid) = parse_id(id) else {
return Ok(None);
};
Ok(self.repo.head(group_id, collection, uuid).await?)
}
async fn get(
&self,
cx: &SdkCallCx,
collection: &str,
id: &str,
) -> Result<Option<Vec<u8>>, GroupFilesError> {
validate_files_collection(collection)?;
let group_id = self.owning_group(cx, collection).await?;
self.check_read(cx, group_id).await?;
let Some(uuid) = parse_id(id) else {
return Ok(None);
};
Ok(self.repo.get(group_id, collection, uuid).await?)
}
async fn update(
&self,
cx: &SdkCallCx,
collection: &str,
id: &str,
mut upd: FileUpdate,
) -> Result<(), GroupFilesError> {
validate_files_collection(collection)?;
let group_id = self.owning_group(cx, collection).await?;
upd.validate(self.max_file_size_bytes)?;
if let Some(ct) = upd.content_type.as_deref() {
upd.content_type = Some(sanitize_stored_content_type(ct));
}
self.check_write(cx, group_id).await?;
let Some(uuid) = parse_id(id) else {
return Err(GroupFilesError::NotFound);
};
if self
.writer
.update(cx, group_id, collection, uuid, upd, self.quota())
.await?
{
Ok(())
} else {
Err(GroupFilesError::NotFound)
}
}
async fn delete(
&self,
cx: &SdkCallCx,
collection: &str,
id: &str,
) -> Result<bool, GroupFilesError> {
validate_files_collection(collection)?;
let group_id = self.owning_group(cx, collection).await?;
self.check_write(cx, group_id).await?;
let Some(uuid) = parse_id(id) else {
return Ok(false);
};
self.writer.delete(cx, group_id, collection, uuid).await
}
async fn list(
&self,
cx: &SdkCallCx,
collection: &str,
cursor: Option<&str>,
limit: u32,
) -> Result<FilesListPage, GroupFilesError> {
validate_files_collection(collection)?;
let group_id = self.owning_group(cx, collection).await?;
self.check_read(cx, group_id).await?;
Ok(self.repo.list(group_id, collection, cursor, limit).await?)
}
}
// ----------------------------------------------------------------------------
// Tests — in-memory repo + a fake resolver so unit tests don't need Postgres or
// a filesystem. The on-disk atomic-write/checksum mechanics are covered by the
// tempdir tests in `files_repo.rs`.
// ----------------------------------------------------------------------------
#[cfg(test)]
mod tests {
use super::*;
use crate::authz::{AuthzError, AuthzRepo};
use crate::files_repo::FileUpdated;
use crate::group_files_repo::GroupFilesRepoError;
use chrono::Utc;
use picloud_shared::{
AdminUserId, AppId, AppRole, ExecutionId, InstanceRole, Principal, RequestId, ScriptId,
UserId,
};
use std::collections::HashMap;
use tokio::sync::Mutex;
#[derive(Default)]
struct InMemoryGroupFilesRepo {
#[allow(clippy::type_complexity)]
data: Mutex<HashMap<(GroupId, String, Uuid), (FileMeta, Vec<u8>)>>,
}
fn meta(id: Uuid, collection: &str, new: &NewFile) -> FileMeta {
FileMeta {
id,
collection: collection.to_string(),
name: new.name.clone(),
content_type: new.content_type.clone(),
size: new.data.len() as u64,
checksum: String::new(),
created_at: Utc::now(),
updated_at: Utc::now(),
}
}
#[async_trait]
impl GroupFilesRepo for InMemoryGroupFilesRepo {
async fn create(
&self,
group_id: GroupId,
collection: &str,
new: NewFile,
) -> Result<FileMeta, GroupFilesRepoError> {
let id = Uuid::new_v4();
let m = meta(id, collection, &new);
self.data.lock().await.insert(
(group_id, collection.to_string(), id),
(m.clone(), new.data),
);
Ok(m)
}
async fn head(
&self,
group_id: GroupId,
collection: &str,
id: Uuid,
) -> Result<Option<FileMeta>, GroupFilesRepoError> {
Ok(self
.data
.lock()
.await
.get(&(group_id, collection.to_string(), id))
.map(|(m, _)| m.clone()))
}
async fn get(
&self,
group_id: GroupId,
collection: &str,
id: Uuid,
) -> Result<Option<Vec<u8>>, GroupFilesRepoError> {
Ok(self
.data
.lock()
.await
.get(&(group_id, collection.to_string(), id))
.map(|(_, b)| b.clone()))
}
async fn update(
&self,
group_id: GroupId,
collection: &str,
id: Uuid,
upd: FileUpdate,
) -> Result<Option<FileUpdated>, GroupFilesRepoError> {
let mut data = self.data.lock().await;
let key = (group_id, collection.to_string(), id);
let Some((prev, _)) = data.get(&key).cloned() else {
return Ok(None);
};
let mut m = prev.clone();
m.size = upd.data.len() as u64;
if let Some(n) = &upd.name {
m.name = n.clone();
}
data.insert(key, (m.clone(), upd.data));
Ok(Some(FileUpdated { new: m, prev }))
}
async fn delete(
&self,
group_id: GroupId,
collection: &str,
id: Uuid,
) -> Result<Option<FileMeta>, GroupFilesRepoError> {
Ok(self
.data
.lock()
.await
.remove(&(group_id, collection.to_string(), id))
.map(|(m, _)| m))
}
async fn list(
&self,
group_id: GroupId,
collection: &str,
_cursor: Option<&str>,
_limit: u32,
) -> Result<FilesListPage, GroupFilesRepoError> {
let files = self
.data
.lock()
.await
.iter()
.filter(|((g, c, _), _)| *g == group_id && c == collection)
.map(|(_, (m, _))| m.clone())
.collect();
Ok(FilesListPage {
files,
next_cursor: None,
})
}
}
#[derive(Default)]
struct FakeResolver {
map: HashMap<(AppId, String), GroupId>,
}
#[async_trait]
impl GroupCollectionResolver for FakeResolver {
async fn resolve_owning_group(
&self,
app_id: AppId,
name: &str,
_kind: &str,
) -> Result<Option<GroupId>, sqlx::Error> {
Ok(self.map.get(&(app_id, name.to_lowercase())).copied())
}
}
#[derive(Default)]
struct DenyingAuthzRepo;
#[async_trait]
impl AuthzRepo for DenyingAuthzRepo {
async fn membership(
&self,
_user_id: UserId,
_app_id: AppId,
) -> Result<Option<AppRole>, AuthzError> {
Ok(None)
}
}
fn cx_with(app_id: AppId, principal: Option<Principal>) -> SdkCallCx {
SdkCallCx {
app_id,
script_id: ScriptId::new(),
principal,
execution_id: ExecutionId::new(),
request_id: RequestId::new(),
trigger_depth: 0,
root_execution_id: ExecutionId::new(),
is_dead_letter_handler: false,
event: None,
}
}
fn owner() -> Principal {
Principal {
user_id: AdminUserId::new(),
instance_role: InstanceRole::Owner,
scopes: None,
app_binding: None,
}
}
fn svc(resolver: FakeResolver) -> GroupFilesServiceImpl {
GroupFilesServiceImpl::new(
Arc::new(InMemoryGroupFilesRepo::default()),
Arc::new(resolver),
Arc::new(DenyingAuthzRepo),
10 * 1024 * 1024,
)
}
fn new_file(name: &str, data: &[u8]) -> NewFile {
NewFile {
name: name.to_string(),
content_type: "application/octet-stream".to_string(),
data: data.to_vec(),
}
}
#[tokio::test]
async fn unrelated_app_gets_collection_not_shared() {
let app_a = AppId::new();
let app_b = AppId::new();
let group = GroupId::new();
let mut resolver = FakeResolver::default();
resolver.map.insert((app_a, "assets".into()), group);
let files = svc(resolver);
let cx_a = cx_with(app_a, Some(owner()));
let id = files
.create(&cx_a, "assets", new_file("a.txt", b"hi"))
.await
.unwrap();
assert!(files
.get(&cx_a, "assets", &id.to_string())
.await
.unwrap()
.is_some());
let cx_b = cx_with(app_b, Some(owner()));
let err = files.list(&cx_b, "assets", None, 100).await.unwrap_err();
assert!(matches!(err, GroupFilesError::CollectionNotShared(c) if c == "assets"));
}
#[tokio::test]
async fn reads_open_writes_require_auth() {
let app = AppId::new();
let group = GroupId::new();
let mut resolver = FakeResolver::default();
resolver.map.insert((app, "assets".into()), group);
let files = svc(resolver);
let owner_cx = cx_with(app, Some(owner()));
let id = files
.create(&owner_cx, "assets", new_file("a.txt", b"hi"))
.await
.unwrap();
// Anonymous READ (get) is allowed.
let anon = cx_with(app, None);
let bytes = files.get(&anon, "assets", &id.to_string()).await.unwrap();
assert_eq!(bytes, Some(b"hi".to_vec()));
// Anonymous WRITE (create) fails closed.
let err = files
.create(&anon, "assets", new_file("x.txt", b"x"))
.await
.unwrap_err();
assert!(matches!(err, GroupFilesError::Forbidden));
}
#[tokio::test]
async fn oversize_blob_rejected() {
let app = AppId::new();
let group = GroupId::new();
let mut resolver = FakeResolver::default();
resolver.map.insert((app, "assets".into()), group);
let files = GroupFilesServiceImpl::new(
Arc::new(InMemoryGroupFilesRepo::default()),
Arc::new(resolver),
Arc::new(DenyingAuthzRepo),
8, // tiny cap
);
let cx = cx_with(app, Some(owner()));
let err = files
.create(&cx, "assets", new_file("big", b"123456789"))
.await
.unwrap_err();
assert!(matches!(err, GroupFilesError::TooLarge { limit: 8, .. }));
}
}

View File

@@ -1,450 +0,0 @@
//! Low-level Postgres CRUD over `group_kv_entries` (§11.6). A near-clone of
//! [`crate::kv_repo`] keyed by the owning `group_id` instead of `app_id` —
//! authorization, group resolution, event policy, and empty-collection
//! validation live one layer up in `GroupKvServiceImpl`.
use async_trait::async_trait;
use base64::engine::general_purpose::URL_SAFE_NO_PAD;
use base64::Engine as _;
use picloud_shared::{GroupId, KvListPage};
use sqlx::PgPool;
#[derive(Debug, thiserror::Error)]
pub enum GroupKvRepoError {
#[error("database error: {0}")]
Db(#[from] sqlx::Error),
#[error("invalid pagination cursor")]
InvalidCursor,
}
/// Repo surface. The trait is exposed so tests can substitute an in-memory
/// backing without spinning up Postgres.
#[async_trait]
pub trait GroupKvRepo: Send + Sync {
async fn get(
&self,
group_id: GroupId,
collection: &str,
key: &str,
) -> Result<Option<serde_json::Value>, GroupKvRepoError>;
/// Upserts the row. Returns the previous value (if any) so callers can
/// determine whether this was an `insert` or an `update`.
async fn set(
&self,
group_id: GroupId,
collection: &str,
key: &str,
value: serde_json::Value,
) -> Result<Option<serde_json::Value>, GroupKvRepoError>;
/// Compare-and-swap. Atomically writes `new` iff the current value equals
/// `expected` (`expected = None` → iff the key is ABSENT). Returns `true`
/// when the write happened. One SQL statement — atomic without a tx.
async fn set_if(
&self,
group_id: GroupId,
collection: &str,
key: &str,
expected: Option<serde_json::Value>,
new: serde_json::Value,
) -> Result<bool, GroupKvRepoError>;
/// Returns the deleted value if present, `None` if the row didn't exist.
async fn delete(
&self,
group_id: GroupId,
collection: &str,
key: &str,
) -> Result<Option<serde_json::Value>, GroupKvRepoError>;
async fn has(
&self,
group_id: GroupId,
collection: &str,
key: &str,
) -> Result<bool, GroupKvRepoError>;
async fn list(
&self,
group_id: GroupId,
collection: &str,
cursor: Option<&str>,
limit: u32,
) -> Result<KvListPage, GroupKvRepoError>;
/// §11.6 quota: total key count across all of the group's shared-KV
/// collections. Default `Ok(0)` so non-Postgres impls skip the quota check.
async fn count_rows(&self, group_id: GroupId) -> Result<u64, GroupKvRepoError> {
let _ = group_id;
Ok(0)
}
/// §11.6 M4 quota: total stored JSON bytes across all of the group's shared-KV
/// collections (`octet_length(value::text)`). Default `Ok(0)` so non-Postgres
/// impls skip the byte-quota check.
async fn total_bytes(&self, group_id: GroupId) -> Result<u64, GroupKvRepoError> {
let _ = group_id;
Ok(0)
}
/// §11.6 M4 quota: the PROJECTED total stored bytes for the group AFTER
/// writing `new_value` at `(collection, key)` — i.e. current SUM minus the
/// existing row's bytes (if any) plus the new value's bytes. Computed
/// entirely in SQL so ALL three terms use the SAME canonical metric
/// (`octet_length(value::text)`); the Rust-side subtract/add previously
/// mixed the DB canonical SUM with a `serde_json` compact length, which
/// drifted the ceiling permissive. Default `Ok(0)` (non-Postgres impls skip
/// the byte quota).
async fn projected_total_bytes(
&self,
group_id: GroupId,
collection: &str,
key: &str,
new_value: &serde_json::Value,
) -> Result<u64, GroupKvRepoError> {
let _ = (group_id, collection, key, new_value);
Ok(0)
}
}
pub struct PostgresGroupKvRepo {
pool: PgPool,
}
impl PostgresGroupKvRepo {
#[must_use]
pub fn new(pool: PgPool) -> Self {
Self { pool }
}
}
/// Hard ceiling on `list` page size (mirrors `kv_repo`).
const KV_LIST_MAX_LIMIT: u32 = 1_000;
const KV_LIST_DEFAULT_LIMIT: u32 = 100;
#[async_trait]
impl GroupKvRepo for PostgresGroupKvRepo {
async fn get(
&self,
group_id: GroupId,
collection: &str,
key: &str,
) -> Result<Option<serde_json::Value>, GroupKvRepoError> {
get_on(&self.pool, group_id, collection, key).await
}
async fn set(
&self,
group_id: GroupId,
collection: &str,
key: &str,
value: serde_json::Value,
) -> Result<Option<serde_json::Value>, GroupKvRepoError> {
set_on(&self.pool, group_id, collection, key, value).await
}
async fn set_if(
&self,
group_id: GroupId,
collection: &str,
key: &str,
expected: Option<serde_json::Value>,
new: serde_json::Value,
) -> Result<bool, GroupKvRepoError> {
set_if_on(&self.pool, group_id, collection, key, expected, new).await
}
async fn delete(
&self,
group_id: GroupId,
collection: &str,
key: &str,
) -> Result<Option<serde_json::Value>, GroupKvRepoError> {
delete_on(&self.pool, group_id, collection, key).await
}
async fn has(
&self,
group_id: GroupId,
collection: &str,
key: &str,
) -> Result<bool, GroupKvRepoError> {
let row: Option<(i64,)> = sqlx::query_as(
"SELECT 1 FROM group_kv_entries \
WHERE group_id = $1 AND collection = $2 AND key = $3",
)
.bind(group_id.into_inner())
.bind(collection)
.bind(key)
.fetch_optional(&self.pool)
.await?;
Ok(row.is_some())
}
async fn list(
&self,
group_id: GroupId,
collection: &str,
cursor: Option<&str>,
limit: u32,
) -> Result<KvListPage, GroupKvRepoError> {
let limit = if limit == 0 {
KV_LIST_DEFAULT_LIMIT
} else {
limit.min(KV_LIST_MAX_LIMIT)
};
let last_key = match cursor {
Some(c) => Some(decode_cursor(c)?),
None => None,
};
let take = i64::from(limit) + 1;
let rows: Vec<(String,)> = sqlx::query_as(
"SELECT key FROM group_kv_entries \
WHERE group_id = $1 AND collection = $2 \
AND ($3::text IS NULL OR key > $3) \
ORDER BY key ASC \
LIMIT $4",
)
.bind(group_id.into_inner())
.bind(collection)
.bind(last_key.as_deref())
.bind(take)
.fetch_all(&self.pool)
.await?;
let mut keys: Vec<String> = rows.into_iter().map(|(k,)| k).collect();
let next_cursor = if keys.len() > limit as usize {
keys.truncate(limit as usize);
keys.last().map(|k| encode_cursor(k))
} else {
None
};
Ok(KvListPage { keys, next_cursor })
}
async fn count_rows(&self, group_id: GroupId) -> Result<u64, GroupKvRepoError> {
count_rows_on(&self.pool, group_id).await
}
async fn total_bytes(&self, group_id: GroupId) -> Result<u64, GroupKvRepoError> {
let (n,): (i64,) = sqlx::query_as(
"SELECT COALESCE(SUM(octet_length(value::text)), 0)::BIGINT \
FROM group_kv_entries WHERE group_id = $1",
)
.bind(group_id.into_inner())
.fetch_one(&self.pool)
.await?;
Ok(u64::try_from(n).unwrap_or(0))
}
async fn projected_total_bytes(
&self,
group_id: GroupId,
collection: &str,
key: &str,
new_value: &serde_json::Value,
) -> Result<u64, GroupKvRepoError> {
projected_total_bytes_on(&self.pool, group_id, collection, key, new_value).await
}
}
// ----------------------------------------------------------------------------
// Connection-scoped operations
//
// Generic over the executor so the same SQL serves the pooled trait methods
// above and `crate::atomic_write`, which runs the per-group advisory lock, the
// quota reads, the write, and the shared-trigger fan-out on ONE connection
// inside ONE transaction. The quota reads in particular MUST run on the writing
// transaction's connection — that is what makes check-then-write atomic rather
// than a race (audit #8).
// ----------------------------------------------------------------------------
pub(crate) async fn get_on<'c, E>(
exec: E,
group_id: GroupId,
collection: &str,
key: &str,
) -> Result<Option<serde_json::Value>, GroupKvRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let row: Option<(serde_json::Value,)> = sqlx::query_as(
"SELECT value FROM group_kv_entries \
WHERE group_id = $1 AND collection = $2 AND key = $3",
)
.bind(group_id.into_inner())
.bind(collection)
.bind(key)
.fetch_optional(exec)
.await?;
Ok(row.map(|(v,)| v))
}
/// Upsert. Returns the previous value (`None` = this was an insert).
pub(crate) async fn set_on<'c, E>(
exec: E,
group_id: GroupId,
collection: &str,
key: &str,
value: serde_json::Value,
) -> Result<Option<serde_json::Value>, GroupKvRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let row: Option<(Option<serde_json::Value>,)> = sqlx::query_as(
"WITH prev AS (\
SELECT value FROM group_kv_entries \
WHERE group_id = $1 AND collection = $2 AND key = $3\
), \
upserted AS (\
INSERT INTO group_kv_entries (group_id, collection, key, value) \
VALUES ($1, $2, $3, $4) \
ON CONFLICT (group_id, collection, key) DO UPDATE \
SET value = EXCLUDED.value, updated_at = NOW() \
RETURNING 1\
) \
SELECT (SELECT value FROM prev) FROM upserted",
)
.bind(group_id.into_inner())
.bind(collection)
.bind(key)
.bind(value)
.fetch_optional(exec)
.await?;
Ok(row.and_then(|(v,)| v))
}
/// Compare-and-swap. Writes `new` iff the current value equals `expected`
/// (`expected = None` → iff the key is ABSENT). Returns whether it wrote.
pub(crate) async fn set_if_on<'c, E>(
exec: E,
group_id: GroupId,
collection: &str,
key: &str,
expected: Option<serde_json::Value>,
new: serde_json::Value,
) -> Result<bool, GroupKvRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let affected = match expected {
Some(exp) => sqlx::query(
"UPDATE group_kv_entries SET value = $4, updated_at = NOW() \
WHERE group_id = $1 AND collection = $2 AND key = $3 AND value = $5",
)
.bind(group_id.into_inner())
.bind(collection)
.bind(key)
.bind(new)
.bind(exp)
.execute(exec)
.await?
.rows_affected(),
None => sqlx::query(
"INSERT INTO group_kv_entries (group_id, collection, key, value) \
VALUES ($1, $2, $3, $4) \
ON CONFLICT (group_id, collection, key) DO NOTHING",
)
.bind(group_id.into_inner())
.bind(collection)
.bind(key)
.bind(new)
.execute(exec)
.await?
.rows_affected(),
};
Ok(affected == 1)
}
/// Returns the deleted value if present, `None` if the row didn't exist.
pub(crate) async fn delete_on<'c, E>(
exec: E,
group_id: GroupId,
collection: &str,
key: &str,
) -> Result<Option<serde_json::Value>, GroupKvRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let row: Option<(serde_json::Value,)> = sqlx::query_as(
"DELETE FROM group_kv_entries \
WHERE group_id = $1 AND collection = $2 AND key = $3 \
RETURNING value",
)
.bind(group_id.into_inner())
.bind(collection)
.bind(key)
.fetch_optional(exec)
.await?;
Ok(row.map(|(v,)| v))
}
/// Total key count across the group's shared-KV collections (§11.6 row quota).
pub(crate) async fn count_rows_on<'c, E>(
exec: E,
group_id: GroupId,
) -> Result<u64, GroupKvRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let (n,): (i64,) = sqlx::query_as("SELECT COUNT(*) FROM group_kv_entries WHERE group_id = $1")
.bind(group_id.into_inner())
.fetch_one(exec)
.await?;
Ok(u64::try_from(n).unwrap_or(0))
}
/// §11.6 M4 quota: the PROJECTED total stored bytes for the group AFTER writing
/// `new_value` at `(collection, key)` — current SUM, minus the existing row's
/// bytes (0 if the key is new), plus the new value's bytes. Computed entirely in
/// SQL so all three terms use the SAME canonical metric
/// (`octet_length(value::text)`); mixing the DB's canonical SUM with a
/// `serde_json` compact length drifts the ceiling permissive.
pub(crate) async fn projected_total_bytes_on<'c, E>(
exec: E,
group_id: GroupId,
collection: &str,
key: &str,
new_value: &serde_json::Value,
) -> Result<u64, GroupKvRepoError>
where
E: sqlx::PgExecutor<'c>,
{
// The new value is bound as compact TEXT and re-parsed through `::jsonb::text`
// so PG measures it in the same canonical form it stores.
// A `serde_json::Value` always serializes; fall back defensively.
let new_text = serde_json::to_string(new_value).unwrap_or_else(|_| "null".to_string());
let (n,): (i64,) = sqlx::query_as(
"SELECT ( \
COALESCE((SELECT SUM(octet_length(value::text)) \
FROM group_kv_entries WHERE group_id = $1), 0) \
- COALESCE((SELECT octet_length(value::text) \
FROM group_kv_entries \
WHERE group_id = $1 AND collection = $2 AND key = $3), 0) \
+ octet_length($4::jsonb::text) \
)::BIGINT",
)
.bind(group_id.into_inner())
.bind(collection)
.bind(key)
.bind(new_text)
.fetch_one(exec)
.await?;
Ok(u64::try_from(n).unwrap_or(0))
}
fn encode_cursor(last_key: &str) -> String {
URL_SAFE_NO_PAD.encode(last_key.as_bytes())
}
fn decode_cursor(cursor: &str) -> Result<String, GroupKvRepoError> {
let bytes = URL_SAFE_NO_PAD
.decode(cursor)
.map_err(|_| GroupKvRepoError::InvalidCursor)?;
String::from_utf8(bytes).map_err(|_| GroupKvRepoError::InvalidCursor)
}

View File

@@ -1,760 +0,0 @@
//! `GroupKvServiceImpl` — wires `GroupKvRepo` + the group-collection registry
//! underneath the `picloud_shared::GroupKvService` trait that scripts reach via
//! the `kv::shared_collection("name")` Rhai handle (§11.6).
//!
//! Layers added over the raw repo:
//!
//! 1. Empty-collection rejection.
//! 2. **Owner resolution** (the structural isolation boundary): the collection
//! name is resolved to the nearest ancestor group that declares it shared,
//! walking the calling app's chain. No declaration on the chain → a clean
//! `CollectionNotShared`, BEFORE any authz or storage access.
//! 3. **Reads-open / writes-authed authz**: reads use `script_gate`
//! (anonymous public scripts skip); writes use `script_gate_require_principal`
//! (anonymous fails closed — shared mutation always needs an authenticated
//! editor+ on the owning group).
//! 4. Per-value size cap (reuses `PICLOUD_KV_MAX_VALUE_BYTES`) + a per-group
//! row quota (§11.6 M3, `PICLOUD_GROUP_KV_MAX_ROWS`).
//! 5. §11.6 M2 shared-collection triggers: a write fires `shared = true`
//! triggers on the owning group, under the writer app (`emit_shared`).
use std::sync::Arc;
use async_trait::async_trait;
use picloud_shared::{
GroupId, GroupKvError, GroupKvService, KvListPage, NoopEventEmitter, SdkCallCx,
ServiceEventEmitter,
};
use crate::atomic_write::{
BestEffortGroupKvWriter, GroupKvTarget, GroupKvWriter, PostgresGroupKvWriter,
};
use crate::authz::{self, AuthzRepo, Capability};
use crate::group_collection_repo::GroupCollectionResolver;
use crate::group_kv_repo::{GroupKvRepo, GroupKvRepoError};
use crate::kv_service::kv_max_value_bytes_from_env;
use crate::quota::GroupWriteQuota;
/// The registry `kind` this service resolves.
const KIND_KV: &str = "kv";
pub struct GroupKvServiceImpl {
repo: Arc<dyn GroupKvRepo>,
resolver: Arc<dyn GroupCollectionResolver>,
authz: Arc<dyn AuthzRepo>,
max_value_bytes: usize,
/// §11.6 per-group quota: max total keys across the group's shared-KV
/// collections (`PICLOUD_GROUP_KV_MAX_ROWS`). Checked only when inserting a
/// NEW key.
max_rows: u64,
/// §11.6 M4 per-group quota: max total stored bytes across the group's
/// shared-KV collections (`PICLOUD_GROUP_KV_MAX_TOTAL_BYTES`). Checked on the
/// projected total (old value subtracted, new added) so a same/smaller update
/// near the cap is still allowed.
max_total_bytes: u64,
/// Mutations: the quota decision, the row write, and the shared-trigger
/// fan-out, which for the host all commit in one advisory-locked
/// transaction. Reads stay on `repo`.
writer: Arc<dyn GroupKvWriter>,
/// Set once [`Self::with_atomic_writes`] has installed the transactional
/// writer. `with_events` refuses to overwrite it — silently downgrading a
/// service back to non-transactional writes (and dropping its quota
/// enforcement) because a builder call was appended in the wrong order is
/// exactly the kind of regression the type system will not catch.
atomic: bool,
}
impl GroupKvServiceImpl {
#[must_use]
pub fn new(
repo: Arc<dyn GroupKvRepo>,
resolver: Arc<dyn GroupCollectionResolver>,
authz: Arc<dyn AuthzRepo>,
) -> Self {
Self::with_max_value_bytes(repo, resolver, authz, kv_max_value_bytes_from_env())
}
/// Wire the event emitter (§11.6 shared-collection triggers) on the
/// best-effort writer. Without this — or [`Self::with_atomic_writes`], which
/// supersedes it — shared writes fire nothing.
#[must_use]
pub fn with_events(mut self, events: Arc<dyn ServiceEventEmitter>) -> Self {
if self.atomic {
tracing::warn!(
"with_events() called after with_atomic_writes() — ignored; the \
transactional writer already resolves and writes the fan-out itself"
);
return self;
}
self.writer = Arc::new(BestEffortGroupKvWriter::new(self.repo.clone(), events));
self
}
/// Use the transactional writer: the quota reads, the row write, and the
/// shared-trigger fan-out all run on ONE connection under a per-group
/// advisory lock. That makes the quota an actual bound (concurrent writers
/// to a group serialize, so one sees the other's row) and makes an emit
/// failure roll the write back instead of silently losing the event.
///
/// Supersedes [`Self::with_events`] — the transactional writer resolves and
/// writes the fan-out itself and needs no injected emitter.
#[must_use]
pub fn with_atomic_writes(mut self, pool: sqlx::PgPool) -> Self {
self.writer = Arc::new(PostgresGroupKvWriter::new(pool));
self.atomic = true;
self
}
/// Override the per-group row quota (§11.6). Mainly for tests; the host uses
/// the env-var default.
#[must_use]
pub fn with_max_rows(mut self, max_rows: u64) -> Self {
self.max_rows = max_rows;
self
}
#[must_use]
pub fn with_max_value_bytes(
repo: Arc<dyn GroupKvRepo>,
resolver: Arc<dyn GroupCollectionResolver>,
authz: Arc<dyn AuthzRepo>,
max_value_bytes: usize,
) -> Self {
Self {
writer: Arc::new(BestEffortGroupKvWriter::new(
repo.clone(),
Arc::new(NoopEventEmitter),
)),
atomic: false,
max_rows: crate::quota::group_kv_max_rows_from_env(),
max_total_bytes: crate::quota::group_kv_max_total_bytes_from_env(),
repo,
resolver,
authz,
max_value_bytes,
}
}
/// Override the per-group total-bytes quota (§11.6 M4). Mainly for tests.
#[must_use]
pub fn with_max_total_bytes(mut self, max_total_bytes: u64) -> Self {
self.max_total_bytes = max_total_bytes;
self
}
/// The structural boundary: resolve the collection to its owning group on
/// the calling app's chain. `CollectionNotShared` when no ancestor group
/// declares it — the same outcome a foreign app gets, by construction.
async fn owning_group(
&self,
cx: &SdkCallCx,
collection: &str,
) -> Result<GroupId, GroupKvError> {
if collection.is_empty() {
return Err(GroupKvError::InvalidCollection);
}
self.resolver
.resolve_owning_group(cx.app_id, collection, KIND_KV)
.await
.map_err(|e| GroupKvError::Backend(e.to_string()))?
.ok_or_else(|| GroupKvError::CollectionNotShared(collection.to_string()))
}
/// The ceilings a shared-collection write must fit under.
fn quota(&self) -> GroupWriteQuota {
GroupWriteQuota {
max_rows: self.max_rows,
max_total_bytes: self.max_total_bytes,
max_value_bytes: self.max_value_bytes,
}
}
/// Enforce the per-value byte cap. The per-GROUP byte quota is measured
/// canonically in SQL inside the writing transaction — see
/// `group_quota::check_group_write`.
fn check_value_size(&self, value: &serde_json::Value) -> Result<(), GroupKvError> {
let encoded_len = serde_json::to_vec(value)
.map(|v| v.len())
.map_err(|e| GroupKvError::Backend(format!("encode value: {e}")))?;
if encoded_len > self.max_value_bytes {
return Err(GroupKvError::ValueTooLarge {
limit: self.max_value_bytes,
actual: encoded_len,
});
}
Ok(())
}
async fn check_read(&self, cx: &SdkCallCx, group_id: GroupId) -> Result<(), GroupKvError> {
authz::script_gate(
&*self.authz,
cx,
Capability::GroupKvRead(group_id),
|| GroupKvError::Forbidden,
GroupKvError::Backend,
)
.await
}
async fn check_write(&self, cx: &SdkCallCx, group_id: GroupId) -> Result<(), GroupKvError> {
// Fails closed for an anonymous principal: shared mutation always needs
// an authenticated editor+ on the owning group.
authz::script_gate_require_principal(
&*self.authz,
cx,
Capability::GroupKvWrite(group_id),
|| GroupKvError::Forbidden,
GroupKvError::Backend,
)
.await
}
}
impl From<GroupKvRepoError> for GroupKvError {
fn from(e: GroupKvRepoError) -> Self {
Self::Backend(e.to_string())
}
}
#[async_trait]
impl GroupKvService for GroupKvServiceImpl {
async fn get(
&self,
cx: &SdkCallCx,
collection: &str,
key: &str,
) -> Result<Option<serde_json::Value>, GroupKvError> {
let group_id = self.owning_group(cx, collection).await?;
self.check_read(cx, group_id).await?;
Ok(self.repo.get(group_id, collection, key).await?)
}
async fn set(
&self,
cx: &SdkCallCx,
collection: &str,
key: &str,
value: serde_json::Value,
) -> Result<(), GroupKvError> {
let group_id = self.owning_group(cx, collection).await?;
self.check_value_size(&value)?;
self.check_write(cx, group_id).await?;
self.writer
.set(
cx,
GroupKvTarget {
group_id,
collection,
key,
},
value,
self.quota(),
)
.await
}
async fn set_if(
&self,
cx: &SdkCallCx,
collection: &str,
key: &str,
expected: Option<serde_json::Value>,
new: serde_json::Value,
) -> Result<bool, GroupKvError> {
let group_id = self.owning_group(cx, collection).await?;
self.check_value_size(&new)?;
self.check_write(cx, group_id).await?;
self.writer
.set_if(
cx,
GroupKvTarget {
group_id,
collection,
key,
},
expected,
new,
self.quota(),
)
.await
}
async fn delete(
&self,
cx: &SdkCallCx,
collection: &str,
key: &str,
) -> Result<bool, GroupKvError> {
let group_id = self.owning_group(cx, collection).await?;
self.check_write(cx, group_id).await?;
self.writer
.delete(
cx,
GroupKvTarget {
group_id,
collection,
key,
},
)
.await
}
async fn has(&self, cx: &SdkCallCx, collection: &str, key: &str) -> Result<bool, GroupKvError> {
let group_id = self.owning_group(cx, collection).await?;
self.check_read(cx, group_id).await?;
Ok(self.repo.has(group_id, collection, key).await?)
}
async fn list(
&self,
cx: &SdkCallCx,
collection: &str,
cursor: Option<&str>,
limit: u32,
) -> Result<KvListPage, GroupKvError> {
let group_id = self.owning_group(cx, collection).await?;
self.check_read(cx, group_id).await?;
Ok(self.repo.list(group_id, collection, cursor, limit).await?)
}
}
// ----------------------------------------------------------------------------
// Tests — in-memory repo + a fake resolver so unit tests don't need Postgres.
// ----------------------------------------------------------------------------
#[cfg(test)]
mod tests {
use super::*;
use crate::authz::{AuthzError, AuthzRepo};
use picloud_shared::{
AdminUserId, AppId, AppRole, ExecutionId, InstanceRole, Principal, RequestId, ScriptId,
UserId,
};
use std::collections::{BTreeMap, HashMap};
use tokio::sync::Mutex;
#[derive(Default)]
struct InMemoryGroupKvRepo {
data: Mutex<BTreeMap<(GroupId, String, String), serde_json::Value>>,
}
#[async_trait]
impl GroupKvRepo for InMemoryGroupKvRepo {
async fn get(
&self,
group_id: GroupId,
collection: &str,
key: &str,
) -> Result<Option<serde_json::Value>, GroupKvRepoError> {
Ok(self
.data
.lock()
.await
.get(&(group_id, collection.to_string(), key.to_string()))
.cloned())
}
async fn set(
&self,
group_id: GroupId,
collection: &str,
key: &str,
value: serde_json::Value,
) -> Result<Option<serde_json::Value>, GroupKvRepoError> {
Ok(self
.data
.lock()
.await
.insert((group_id, collection.to_string(), key.to_string()), value))
}
async fn set_if(
&self,
group_id: GroupId,
collection: &str,
key: &str,
expected: Option<serde_json::Value>,
new: serde_json::Value,
) -> Result<bool, GroupKvRepoError> {
let mut data = self.data.lock().await;
let k = (group_id, collection.to_string(), key.to_string());
let matches = match (&expected, data.get(&k)) {
(None, None) => true,
(Some(exp), Some(cur)) => exp == cur,
_ => false,
};
if matches {
data.insert(k, new);
}
Ok(matches)
}
async fn delete(
&self,
group_id: GroupId,
collection: &str,
key: &str,
) -> Result<Option<serde_json::Value>, GroupKvRepoError> {
Ok(self
.data
.lock()
.await
.remove(&(group_id, collection.to_string(), key.to_string())))
}
async fn has(
&self,
group_id: GroupId,
collection: &str,
key: &str,
) -> Result<bool, GroupKvRepoError> {
Ok(self.data.lock().await.contains_key(&(
group_id,
collection.to_string(),
key.to_string(),
)))
}
async fn list(
&self,
group_id: GroupId,
collection: &str,
_cursor: Option<&str>,
limit: u32,
) -> Result<KvListPage, GroupKvRepoError> {
let data = self.data.lock().await;
let mut keys: Vec<String> = data
.iter()
.filter(|((g, c, _), _)| *g == group_id && c == collection)
.map(|((_, _, k), _)| k.clone())
.collect();
keys.sort();
keys.truncate((limit as usize).max(1));
Ok(KvListPage {
keys,
next_cursor: None,
})
}
async fn count_rows(&self, group_id: GroupId) -> Result<u64, GroupKvRepoError> {
let data = self.data.lock().await;
Ok(data.keys().filter(|(g, _, _)| *g == group_id).count() as u64)
}
async fn total_bytes(&self, group_id: GroupId) -> Result<u64, GroupKvRepoError> {
let data = self.data.lock().await;
Ok(data
.iter()
.filter(|((g, _, _), _)| *g == group_id)
.map(|(_, v)| serde_json::to_vec(v).map_or(0, |b| b.len() as u64))
.sum())
}
/// Same projection the Postgres impl computes in SQL, but with this
/// mock's own (compact) metric: every OTHER row in the group, plus the
/// new value — i.e. `used - old + new`, measured consistently.
async fn projected_total_bytes(
&self,
group_id: GroupId,
collection: &str,
key: &str,
new_value: &serde_json::Value,
) -> Result<u64, GroupKvRepoError> {
let data = self.data.lock().await;
let target = (group_id, collection.to_string(), key.to_string());
let others: u64 = data
.iter()
.filter(|(k, _)| k.0 == group_id && **k != target)
.map(|(_, v)| serde_json::to_vec(v).map_or(0, |b| b.len() as u64))
.sum();
let new_len = serde_json::to_vec(new_value).map_or(0, |b| b.len() as u64);
Ok(others + new_len)
}
}
/// Maps `(app_id, lowercased name) -> owning group`. Anything absent is
/// "not shared with you".
#[derive(Default)]
struct FakeResolver {
map: HashMap<(AppId, String), GroupId>,
}
#[async_trait]
impl GroupCollectionResolver for FakeResolver {
async fn resolve_owning_group(
&self,
app_id: AppId,
name: &str,
_kind: &str,
) -> Result<Option<GroupId>, sqlx::Error> {
Ok(self.map.get(&(app_id, name.to_lowercase())).copied())
}
}
#[derive(Default)]
struct DenyingAuthzRepo;
#[async_trait]
impl AuthzRepo for DenyingAuthzRepo {
async fn membership(
&self,
_user_id: UserId,
_app_id: AppId,
) -> Result<Option<AppRole>, AuthzError> {
Ok(None)
}
}
fn cx_with(app_id: AppId, principal: Option<Principal>) -> SdkCallCx {
SdkCallCx {
app_id,
script_id: ScriptId::new(),
principal,
execution_id: ExecutionId::new(),
request_id: RequestId::new(),
trigger_depth: 0,
root_execution_id: ExecutionId::new(),
is_dead_letter_handler: false,
event: None,
}
}
fn owner() -> Principal {
Principal {
user_id: AdminUserId::new(),
instance_role: InstanceRole::Owner,
scopes: None,
app_binding: None,
}
}
fn member_no_role() -> Principal {
Principal {
user_id: AdminUserId::new(),
instance_role: InstanceRole::Member,
scopes: None,
app_binding: None,
}
}
fn svc(resolver: FakeResolver) -> GroupKvServiceImpl {
GroupKvServiceImpl::new(
Arc::new(InMemoryGroupKvRepo::default()),
Arc::new(resolver),
Arc::new(DenyingAuthzRepo),
)
}
#[tokio::test]
async fn per_group_row_quota_rejects_new_keys_but_allows_updates() {
// §11.6: a group's shared-KV store has a row ceiling; a NEW key over the
// cap is rejected, but updating an existing key (net-zero) still works.
let app = AppId::new();
let group = GroupId::new();
let mut resolver = FakeResolver::default();
resolver.map.insert((app, "catalog".into()), group);
let kv = GroupKvServiceImpl::new(
Arc::new(InMemoryGroupKvRepo::default()),
Arc::new(resolver),
Arc::new(DenyingAuthzRepo),
)
.with_max_rows(2);
let cx = cx_with(app, Some(owner()));
kv.set(&cx, "catalog", "a", serde_json::json!(1))
.await
.unwrap();
kv.set(&cx, "catalog", "b", serde_json::json!(2))
.await
.unwrap();
// A third distinct key exceeds the cap.
let err = kv
.set(&cx, "catalog", "c", serde_json::json!(3))
.await
.unwrap_err();
assert!(
matches!(
err,
GroupKvError::QuotaExceeded {
limit: 2,
actual: 2
}
),
"got {err:?}"
);
// Updating an existing key is exempt (net-zero).
kv.set(&cx, "catalog", "a", serde_json::json!(11))
.await
.expect("update of an existing key must be allowed at the cap");
}
#[tokio::test]
async fn per_group_byte_quota_uses_the_projected_total() {
// §11.6 M4: the byte ceiling is checked on the PROJECTED total, so a
// write that would push the group over is rejected, but a same/smaller
// update near the cap is still allowed (unlike a naive used+new check).
let app = AppId::new();
let group = GroupId::new();
let mut resolver = FakeResolver::default();
resolver.map.insert((app, "catalog".into()), group);
// A 20-byte ceiling; each `"xxxxx"` value encodes to 7 bytes ("\"xxxxx\"").
let kv = GroupKvServiceImpl::new(
Arc::new(InMemoryGroupKvRepo::default()),
Arc::new(resolver),
Arc::new(DenyingAuthzRepo),
)
.with_max_rows(1000)
.with_max_total_bytes(20);
let cx = cx_with(app, Some(owner()));
kv.set(&cx, "catalog", "a", serde_json::json!("xxxxx"))
.await
.unwrap(); // used = 7
kv.set(&cx, "catalog", "b", serde_json::json!("xxxxx"))
.await
.unwrap(); // used = 14
// A third 7-byte value → projected 21 > 20 → rejected.
let err = kv
.set(&cx, "catalog", "c", serde_json::json!("xxxxx"))
.await
.unwrap_err();
assert!(
matches!(err, GroupKvError::TotalBytesQuotaExceeded { limit: 20, .. }),
"got {err:?}"
);
// A same-size update of an existing key stays within budget (projected
// = 14 - 7 + 7 = 14 ≤ 20) — the delta rule, not used+new.
kv.set(&cx, "catalog", "a", serde_json::json!("yyyyy"))
.await
.expect("a same-size update near the cap must be allowed");
// Growing an existing value past the cap is still rejected.
let err = kv
.set(&cx, "catalog", "a", serde_json::json!("zzzzzzzzzzzzzzz"))
.await
.unwrap_err();
assert!(
matches!(err, GroupKvError::TotalBytesQuotaExceeded { .. }),
"got {err:?}"
);
}
#[tokio::test]
async fn unrelated_app_gets_collection_not_shared() {
// app_a's chain declares "catalog"; app_b's does not.
let app_a = AppId::new();
let app_b = AppId::new();
let group = GroupId::new();
let mut resolver = FakeResolver::default();
resolver.map.insert((app_a, "catalog".into()), group);
let kv = svc(resolver);
// app_a (owner principal) can write+read.
let cx_a = cx_with(app_a, Some(owner()));
kv.set(&cx_a, "catalog", "k", serde_json::json!(1))
.await
.unwrap();
assert_eq!(
kv.get(&cx_a, "catalog", "k").await.unwrap(),
Some(serde_json::json!(1))
);
// app_b is off-chain — the name does not resolve.
let cx_b = cx_with(app_b, Some(owner()));
let err = kv.get(&cx_b, "catalog", "k").await.unwrap_err();
assert!(matches!(err, GroupKvError::CollectionNotShared(c) if c == "catalog"));
}
#[tokio::test]
async fn reads_open_writes_require_auth() {
let app = AppId::new();
let group = GroupId::new();
let mut resolver = FakeResolver::default();
resolver.map.insert((app, "catalog".into()), group);
let kv = svc(resolver);
// Seed a value as owner.
let owner_cx = cx_with(app, Some(owner()));
kv.set(&owner_cx, "catalog", "k", serde_json::json!("v"))
.await
.unwrap();
// Anonymous (principal=None) READ is allowed (reads-open).
let anon = cx_with(app, None);
assert_eq!(
kv.get(&anon, "catalog", "k").await.unwrap(),
Some(serde_json::json!("v"))
);
// Anonymous WRITE fails closed.
let err = kv
.set(&anon, "catalog", "k", serde_json::json!("x"))
.await
.unwrap_err();
assert!(matches!(err, GroupKvError::Forbidden));
// Authenticated member with no group role: write forbidden.
let member = cx_with(app, Some(member_no_role()));
let err = kv.delete(&member, "catalog", "k").await.unwrap_err();
assert!(matches!(err, GroupKvError::Forbidden));
}
#[tokio::test]
async fn empty_collection_rejected_before_resolve() {
let kv = svc(FakeResolver::default());
let cx = cx_with(AppId::new(), None);
let err = kv.get(&cx, "", "k").await.unwrap_err();
assert!(matches!(err, GroupKvError::InvalidCollection));
}
#[tokio::test]
async fn set_if_is_compare_and_swap_and_writes_fail_closed() {
let app = AppId::new();
let group = GroupId::new();
let mut resolver = FakeResolver::default();
resolver.map.insert((app, "catalog".into()), group);
let kv = svc(resolver);
let cx = cx_with(app, Some(owner()));
// Insert-if-absent, then swap-if-equal (as the per-app path).
assert!(kv
.set_if(&cx, "catalog", "k", None, serde_json::json!(1))
.await
.unwrap());
assert!(!kv
.set_if(&cx, "catalog", "k", None, serde_json::json!(2))
.await
.unwrap());
assert!(!kv
.set_if(
&cx,
"catalog",
"k",
Some(serde_json::json!(9)),
serde_json::json!(3)
)
.await
.unwrap());
assert!(kv
.set_if(
&cx,
"catalog",
"k",
Some(serde_json::json!(1)),
serde_json::json!(3)
)
.await
.unwrap());
assert_eq!(
kv.get(&cx, "catalog", "k").await.unwrap(),
Some(serde_json::json!(3))
);
// A shared-collection CAS still fails closed for an anonymous writer.
let anon = cx_with(app, None);
let err = kv
.set_if(&anon, "catalog", "k", None, serde_json::json!(4))
.await
.unwrap_err();
assert!(matches!(err, GroupKvError::Forbidden));
}
}

View File

@@ -1,453 +0,0 @@
//! `GroupPubsubServiceImpl` — wires the group-collection registry + the pubsub
//! fan-out repo underneath the `picloud_shared::GroupPubsubService` trait that
//! scripts reach via the `pubsub::shared_topic("name")` Rhai handle (§11.6 D2).
//!
//! A shared topic is storeless — a publish resolves the OWNING GROUP (the
//! nearest ancestor group declaring the topic shared, kind `topic`), then fans
//! out to that group's `shared = true` pubsub triggers, each delivery stamped
//! with the WRITER `cx.app_id` so the handler runs under the publishing app
//! (matching the M2 shared-collection trigger model). Owner resolution is the
//! isolation boundary — a foreign app's chain never reaches the owning group.
//!
//! Trust model: a publish is a WRITE, so it uses `script_gate_require_principal`
//! (anonymous fails closed — shared publish always needs an authenticated
//! editor+ on the owning group).
use std::sync::Arc;
use async_trait::async_trait;
use picloud_shared::{
GroupId, GroupPubsubError, GroupPubsubService, RealtimeBroadcaster, RealtimeEvent, SdkCallCx,
TriggerEvent,
};
use crate::authz::{self, AuthzRepo, Capability};
use crate::group_collection_repo::GroupCollectionResolver;
use crate::pubsub_repo::{PublishCtx, PubsubRepo};
use crate::pubsub_service::pubsub_max_message_bytes_from_env;
/// The registry `kind` this service resolves.
const KIND_TOPIC: &str = "topic";
pub struct GroupPubsubServiceImpl {
repo: Arc<dyn PubsubRepo>,
resolver: Arc<dyn GroupCollectionResolver>,
authz: Arc<dyn AuthzRepo>,
max_message_bytes: usize,
/// §11.6 D2: the realtime broadcaster, attached by the host via
/// [`Self::with_realtime`]. `None` in test bundles → no external SSE fan-out
/// (durable trigger fan-out is unaffected).
realtime: Option<Arc<dyn RealtimeBroadcaster>>,
}
impl GroupPubsubServiceImpl {
#[must_use]
pub fn new(
repo: Arc<dyn PubsubRepo>,
resolver: Arc<dyn GroupCollectionResolver>,
authz: Arc<dyn AuthzRepo>,
) -> Self {
Self {
repo,
resolver,
authz,
max_message_bytes: pubsub_max_message_bytes_from_env(),
realtime: None,
}
}
/// §11.6 D2: attach the realtime broadcaster so a shared-topic publish also
/// fans out to external SSE subscribers (in addition to the durable
/// `shared = true` trigger fan-out). Mirrors `PubsubServiceImpl::with_realtime`.
#[must_use]
pub fn with_realtime(mut self, broadcaster: Arc<dyn RealtimeBroadcaster>) -> Self {
self.realtime = Some(broadcaster);
self
}
/// The structural boundary: resolve the topic namespace to its owning group
/// on the calling app's chain. `CollectionNotShared` when no ancestor group
/// declares it — the same outcome a foreign app gets, by construction.
async fn owning_group(
&self,
cx: &SdkCallCx,
namespace: &str,
) -> Result<GroupId, GroupPubsubError> {
self.resolver
.resolve_owning_group(cx.app_id, namespace, KIND_TOPIC)
.await
.map_err(|e| GroupPubsubError::Backend(e.to_string()))?
.ok_or_else(|| GroupPubsubError::CollectionNotShared(namespace.to_string()))
}
async fn check_write(&self, cx: &SdkCallCx, group_id: GroupId) -> Result<(), GroupPubsubError> {
authz::script_gate_require_principal(
&*self.authz,
cx,
Capability::GroupPubsubPublish(group_id),
|| GroupPubsubError::Forbidden,
GroupPubsubError::Backend,
)
.await
}
}
#[async_trait]
impl GroupPubsubService for GroupPubsubServiceImpl {
async fn publish(
&self,
cx: &SdkCallCx,
namespace: &str,
subtopic: &str,
message: serde_json::Value,
) -> Result<u32, GroupPubsubError> {
if namespace.trim().is_empty() {
return Err(GroupPubsubError::InvalidTopic);
}
// Size cap first (no DB) — a cheap reject before the resolve/authz work.
let encoded_len = serde_json::to_vec(&message)
.map(|v| v.len())
.map_err(|e| GroupPubsubError::Backend(format!("encode message: {e}")))?;
if encoded_len > self.max_message_bytes {
return Err(GroupPubsubError::MessageTooLarge {
limit: self.max_message_bytes,
actual: encoded_len,
});
}
let group_id = self.owning_group(cx, namespace).await?;
self.check_write(cx, group_id).await?;
// The full topic is `namespace` or `namespace.subtopic`; shared triggers
// match it with the same `topic_matches` semantics as the per-app path.
let topic = if subtopic.trim().is_empty() {
namespace.to_string()
} else {
format!("{namespace}.{subtopic}")
};
let published_at = chrono::Utc::now();
let event = TriggerEvent::Pubsub {
topic: topic.clone(),
message: message.clone(),
published_at,
};
let payload = serde_json::to_value(&event)
.map_err(|e| GroupPubsubError::Backend(format!("event serialize: {e}")))?;
let publish_ctx = PublishCtx {
app_id: cx.app_id,
origin_principal: cx.principal.as_ref().map(|p| p.user_id),
trigger_depth: cx.trigger_depth,
root_execution_id: cx.root_execution_id,
};
// Durable trigger fan-out FIRST (the authoritative delivery path).
let count = self
.repo
.fan_out_shared_publish(publish_ctx, group_id, &topic, payload)
.await
.map_err(|e| GroupPubsubError::Backend(e.to_string()))?;
// Then the best-effort realtime broadcast to external SSE subscribers,
// keyed by the OWNING GROUP so every subtree subscriber shares one
// channel. A broadcast failure never fails the publish.
if let Some(realtime) = &self.realtime {
realtime
.publish_group(
group_id,
&topic,
RealtimeEvent {
topic: topic.clone(),
message,
published_at,
},
)
.await;
}
Ok(count)
}
}
#[cfg(test)]
mod tests {
use super::*;
use picloud_shared::{
AdminUserId, AppId, BroadcasterError, ExecutionId, InstanceRole, Principal,
RealtimeBroadcaster, RealtimeEvent, RequestId, ScriptId,
};
use std::sync::Mutex;
use tokio::sync::broadcast;
use crate::authz::AuthzError;
use crate::pubsub_repo::PubsubRepoError;
/// Records `publish_group` calls so the bridge can be asserted.
#[derive(Default)]
struct RecordingBroadcaster {
group_publishes: Mutex<Vec<(GroupId, String, serde_json::Value)>>,
}
#[async_trait]
impl RealtimeBroadcaster for RecordingBroadcaster {
async fn subscribe(
&self,
_: picloud_shared::AppId,
_: &str,
) -> Result<broadcast::Receiver<RealtimeEvent>, BroadcasterError> {
Ok(broadcast::channel(1).1)
}
async fn publish(&self, _: picloud_shared::AppId, _: &str, _: RealtimeEvent) {}
async fn drop_topic(&self, _: picloud_shared::AppId, _: &str) {}
async fn publish_group(&self, group_id: GroupId, topic: &str, event: RealtimeEvent) {
self.group_publishes
.lock()
.unwrap()
.push((group_id, topic.to_string(), event.message));
}
}
struct FakeRepo;
#[async_trait]
impl PubsubRepo for FakeRepo {
async fn fan_out_publish(
&self,
_: PublishCtx,
_: &str,
_: serde_json::Value,
) -> Result<u32, PubsubRepoError> {
Ok(0)
}
async fn fan_out_shared_publish(
&self,
_: PublishCtx,
_: GroupId,
_: &str,
_: serde_json::Value,
) -> Result<u32, PubsubRepoError> {
Ok(2) // pretend two shared triggers matched
}
}
struct FixedResolver(GroupId);
#[async_trait]
impl GroupCollectionResolver for FixedResolver {
async fn resolve_owning_group(
&self,
_: AppId,
_: &str,
kind: &str,
) -> Result<Option<GroupId>, sqlx::Error> {
Ok((kind == KIND_TOPIC).then_some(self.0))
}
}
struct DenyAuthz;
#[async_trait]
impl AuthzRepo for DenyAuthz {
async fn membership(
&self,
_: picloud_shared::UserId,
_: AppId,
) -> Result<Option<picloud_shared::AppRole>, AuthzError> {
Ok(None)
}
}
fn owner_cx(app_id: AppId) -> SdkCallCx {
SdkCallCx {
app_id,
script_id: ScriptId::new(),
// Instance Owner bypasses the capability check (as in the group_kv
// tests), so DenyAuthz is fine — we're testing the broadcast bridge.
principal: Some(Principal {
user_id: AdminUserId::new(),
instance_role: InstanceRole::Owner,
scopes: None,
app_binding: None,
}),
execution_id: ExecutionId::new(),
request_id: RequestId::new(),
trigger_depth: 0,
root_execution_id: ExecutionId::new(),
is_dead_letter_handler: false,
event: None,
}
}
#[tokio::test]
async fn publish_bridges_to_the_group_broadcaster() {
// §11.6 D2: a shared-topic publish fans out to the durable triggers AND,
// when a broadcaster is attached, to the OWNING GROUP's realtime channel
// with the full topic (`namespace.subtopic`).
let group = GroupId::new();
let app = AppId::new();
let recorder = Arc::new(RecordingBroadcaster::default());
let svc = GroupPubsubServiceImpl::new(
Arc::new(FakeRepo),
Arc::new(FixedResolver(group)),
Arc::new(DenyAuthz),
)
.with_realtime(recorder.clone());
let count = svc
.publish(
&owner_cx(app),
"events",
"created",
serde_json::json!({ "id": 1 }),
)
.await
.unwrap();
assert_eq!(count, 2, "durable fan-out count is returned");
let calls = recorder.group_publishes.lock().unwrap();
assert_eq!(calls.len(), 1, "one realtime broadcast");
assert_eq!(calls[0].0, group, "keyed by the owning group");
assert_eq!(calls[0].1, "events.created", "full topic");
assert_eq!(calls[0].2, serde_json::json!({ "id": 1 }));
}
#[tokio::test]
async fn publish_without_broadcaster_still_fans_out() {
// No broadcaster attached → durable fan-out only, no panic.
let group = GroupId::new();
let svc = GroupPubsubServiceImpl::new(
Arc::new(FakeRepo),
Arc::new(FixedResolver(group)),
Arc::new(DenyAuthz),
);
let count = svc
.publish(&owner_cx(AppId::new()), "events", "", serde_json::json!({}))
.await
.unwrap();
assert_eq!(count, 2);
}
// ------------------------------------------------------------------
// The two invariants the three older sibling services pin, and this one
// did not. Both existing tests above use `FixedResolver` (always resolves)
// and `owner_cx` (an instance Owner, who bypasses the capability check) —
// so neither the isolation boundary nor the anon gate was exercised at all.
// ------------------------------------------------------------------
/// Resolves a topic only for apps whose chain declares it — the fake stand-in
/// for the ancestor walk. The real walk is pinned against Postgres in
/// `tests/group_collection_isolation.rs`.
#[derive(Default)]
struct ChainResolver {
map: std::collections::HashMap<(AppId, String), GroupId>,
}
#[async_trait]
impl GroupCollectionResolver for ChainResolver {
async fn resolve_owning_group(
&self,
app_id: AppId,
name: &str,
kind: &str,
) -> Result<Option<GroupId>, sqlx::Error> {
if kind != KIND_TOPIC {
return Ok(None);
}
Ok(self.map.get(&(app_id, name.to_lowercase())).copied())
}
}
fn cx(app_id: AppId, principal: Option<Principal>) -> SdkCallCx {
SdkCallCx {
app_id,
script_id: ScriptId::new(),
principal,
execution_id: ExecutionId::new(),
request_id: RequestId::new(),
trigger_depth: 0,
root_execution_id: ExecutionId::new(),
is_dead_letter_handler: false,
event: None,
}
}
fn member_no_role() -> Principal {
Principal {
user_id: AdminUserId::new(),
instance_role: InstanceRole::Member,
scopes: None,
app_binding: None,
}
}
fn owner_principal() -> Principal {
Principal {
user_id: AdminUserId::new(),
instance_role: InstanceRole::Owner,
scopes: None,
app_binding: None,
}
}
#[tokio::test]
async fn unrelated_app_gets_collection_not_shared() {
let (app_a, app_b, group) = (AppId::new(), AppId::new(), GroupId::new());
let mut resolver = ChainResolver::default();
resolver.map.insert((app_a, "events".into()), group);
let svc = GroupPubsubServiceImpl::new(
Arc::new(FakeRepo),
Arc::new(resolver),
Arc::new(DenyAuthz),
);
// Positive control — app_a, on the chain, publishes fine.
svc.publish(&owner_cx(app_a), "events", "created", serde_json::json!({}))
.await
.expect("an app on the chain can publish");
// THE BOUNDARY — app_b is in another subtree.
let err = svc
.publish(
&cx(app_b, Some(owner_principal())),
"events",
"created",
serde_json::json!({}),
)
.await
.unwrap_err();
assert!(
matches!(&err, GroupPubsubError::CollectionNotShared(t) if t == "events"),
"a foreign app must not publish into another subtree's shared topic, got {err:?}"
);
}
#[tokio::test]
async fn publish_fails_closed_for_anon() {
let (app, group) = (AppId::new(), GroupId::new());
let mut resolver = ChainResolver::default();
resolver.map.insert((app, "events".into()), group);
let svc = GroupPubsubServiceImpl::new(
Arc::new(FakeRepo),
Arc::new(resolver),
Arc::new(DenyAuthz),
);
// A shared publish fans out to every `shared = true` trigger on the group,
// each handler running under the writer's app_id. An anonymous public route
// must not be able to drive that. This is the assertion that fires if
// `script_gate_require_principal` is swapped for `script_gate`.
let err = svc
.publish(&cx(app, None), "events", "created", serde_json::json!({}))
.await
.unwrap_err();
assert!(
matches!(err, GroupPubsubError::Forbidden),
"an ANONYMOUS shared publish must fail closed, got {err:?}"
);
// Authenticated but with no editor+ on the owning group: also denied.
let err = svc
.publish(
&cx(app, Some(member_no_role())),
"events",
"created",
serde_json::json!({}),
)
.await
.unwrap_err();
assert!(
matches!(err, GroupPubsubError::Forbidden),
"authentication alone is not authorization — editor+ on the owning group is required"
);
}
}

View File

@@ -1,399 +0,0 @@
//! `GroupQueueRepo` — CRUD over `group_queue_messages`, the §11.6 D3 group-
//! shared durable queue store.
//!
//! The group counterpart to [`crate::queue_repo::QueueRepo`], keyed by
//! `(group_id, collection)` instead of `(app_id, queue_name)`. Producers enqueue
//! via `GroupQueueServiceImpl` (`queue::shared_collection(name).enqueue(...)`);
//! the dispatcher's queue arm claims from here for a materialized shared-queue
//! consumer (competing consumers — every descendant app runs a consumer copy,
//! all claiming this one store with `FOR UPDATE SKIP LOCKED`, so each message is
//! delivered at-most-once across the subtree).
//!
//! §11.6 D3 (dead-lettering): an exhausted shared-queue message moves to the
//! group dead-letter store (`group_dead_letters`, 0068) via [`GroupQueueRepo::dead_letter`]
//! instead of being dropped — operator-visible at GET .../groups/{id}/dead-letters.
//! Deferred: fan-out to a *shared* dead-letter TRIGGER (needs a new trigger kind).
use async_trait::async_trait;
use chrono::{DateTime, Utc};
use picloud_shared::{AdminUserId, GroupId, QueueMessageId};
use sqlx::PgPool;
use uuid::Uuid;
/// Cap on `depth`/`depth_pending` scans (mirrors `queue_repo`).
const QUEUE_DEPTH_SCAN_CAP: i64 = 10_000;
#[derive(Debug, thiserror::Error)]
pub enum GroupQueueRepoError {
#[error("database error: {0}")]
Db(#[from] sqlx::Error),
}
/// Insert payload — what `GroupQueueService::enqueue` hands the repo.
#[derive(Debug, Clone)]
pub struct NewGroupQueueMessage {
pub group_id: GroupId,
pub collection: String,
pub payload: serde_json::Value,
pub deliver_after: Option<DateTime<Utc>>,
pub max_attempts: u32,
pub enqueued_by_principal: Option<AdminUserId>,
}
/// One claimed message ready for handler dispatch.
#[derive(Debug, Clone)]
pub struct ClaimedGroupMessage {
pub id: QueueMessageId,
pub group_id: GroupId,
pub collection: String,
pub payload: serde_json::Value,
pub enqueued_at: DateTime<Utc>,
pub attempt: u32,
pub max_attempts: u32,
pub claim_token: Uuid,
}
#[async_trait]
pub trait GroupQueueRepo: Send + Sync {
async fn enqueue(
&self,
msg: NewGroupQueueMessage,
) -> Result<QueueMessageId, GroupQueueRepoError>;
/// Atomic claim of one ready message from `(group_id, collection)` with
/// `FOR UPDATE SKIP LOCKED` — the competing-consumer primitive. `Ok(None)`
/// when nothing is claimable.
async fn claim(
&self,
group_id: GroupId,
collection: &str,
) -> Result<Option<ClaimedGroupMessage>, GroupQueueRepoError>;
/// Handler succeeded: delete the row iff `claim_token` matches.
async fn ack(
&self,
message_id: QueueMessageId,
claim_token: Uuid,
) -> Result<bool, GroupQueueRepoError>;
/// Handler threw: clear the claim, defer redelivery by `retry_delay`.
async fn nack(
&self,
message_id: QueueMessageId,
claim_token: Uuid,
retry_delay: chrono::Duration,
) -> Result<bool, GroupQueueRepoError>;
/// TRANSIENT release (handler never ran): re-queue AND undo `claim`'s
/// pre-increment of `attempt`, so a non-execution doesn't burn the retry
/// budget. Mirrors `queue_repo::release`. Floors `attempt` at 0.
async fn release(
&self,
message_id: QueueMessageId,
claim_token: Uuid,
retry_delay: chrono::Duration,
) -> Result<bool, GroupQueueRepoError>;
/// §11.6 D3: a message exhausted its attempts — move it to the group dead-
/// letter store (`group_dead_letters`) and delete it from the live queue, in
/// one transaction (mirrors `queue_repo::dead_letter`). Filtered by
/// `claim_token` so a lost lease can't dead-letter a re-claimed message.
/// Returns the new dead-letter id.
#[allow(clippy::too_many_arguments)]
async fn dead_letter(
&self,
message_id: QueueMessageId,
claim_token: Uuid,
group_id: GroupId,
collection: &str,
trigger_id: Option<picloud_shared::TriggerId>,
script_id: Option<picloud_shared::ScriptId>,
attempt: u32,
first_attempt_at: DateTime<Utc>,
last_error: &str,
) -> Result<picloud_shared::DeadLetterId, GroupQueueRepoError>;
/// Periodic safety net: clear claims older than a shared-queue consumer's
/// `visibility_timeout_secs`. Returns the number of rows reclaimed.
async fn reclaim_visibility_timeouts(&self) -> Result<u64, GroupQueueRepoError>;
async fn depth(&self, group_id: GroupId, collection: &str) -> Result<u64, GroupQueueRepoError>;
async fn depth_pending(
&self,
group_id: GroupId,
collection: &str,
) -> Result<u64, GroupQueueRepoError>;
}
pub struct PostgresGroupQueueRepo {
pool: PgPool,
}
impl PostgresGroupQueueRepo {
#[must_use]
pub fn new(pool: PgPool) -> Self {
Self { pool }
}
}
#[async_trait]
impl GroupQueueRepo for PostgresGroupQueueRepo {
async fn enqueue(
&self,
msg: NewGroupQueueMessage,
) -> Result<QueueMessageId, GroupQueueRepoError> {
let (id,): (Uuid,) = sqlx::query_as(
"INSERT INTO group_queue_messages ( \
group_id, collection, payload, deliver_after, \
max_attempts, enqueued_by_principal \
) VALUES ($1, $2, $3, $4, $5, $6) RETURNING id",
)
.bind(msg.group_id.into_inner())
.bind(&msg.collection)
.bind(&msg.payload)
.bind(msg.deliver_after)
.bind(i32::try_from(msg.max_attempts).unwrap_or(3))
.bind(msg.enqueued_by_principal.map(AdminUserId::into_inner))
.fetch_one(&self.pool)
.await?;
Ok(id.into())
}
async fn claim(
&self,
group_id: GroupId,
collection: &str,
) -> Result<Option<ClaimedGroupMessage>, GroupQueueRepoError> {
let token = Uuid::new_v4();
let row: Option<ClaimedRow> = sqlx::query_as(
"UPDATE group_queue_messages \
SET claim_token = $3, claimed_at = NOW(), attempt = attempt + 1 \
WHERE id = ( \
SELECT id FROM group_queue_messages \
WHERE group_id = $1 AND collection = $2 \
AND claim_token IS NULL \
AND (deliver_after IS NULL OR deliver_after <= NOW()) \
ORDER BY enqueued_at \
FOR UPDATE SKIP LOCKED \
LIMIT 1 \
) \
RETURNING id, group_id, collection, payload, enqueued_at, attempt, max_attempts",
)
.bind(group_id.into_inner())
.bind(collection)
.bind(token)
.fetch_optional(&self.pool)
.await?;
Ok(row.map(|r| ClaimedGroupMessage {
id: r.id.into(),
group_id: r.group_id.into(),
collection: r.collection,
payload: r.payload,
enqueued_at: r.enqueued_at,
attempt: u32::try_from(r.attempt).unwrap_or(1),
max_attempts: u32::try_from(r.max_attempts).unwrap_or(3),
claim_token: token,
}))
}
async fn ack(
&self,
message_id: QueueMessageId,
claim_token: Uuid,
) -> Result<bool, GroupQueueRepoError> {
let res =
sqlx::query("DELETE FROM group_queue_messages WHERE id = $1 AND claim_token = $2")
.bind(message_id.into_inner())
.bind(claim_token)
.execute(&self.pool)
.await?;
Ok(res.rows_affected() == 1)
}
async fn nack(
&self,
message_id: QueueMessageId,
claim_token: Uuid,
retry_delay: chrono::Duration,
) -> Result<bool, GroupQueueRepoError> {
let next = Utc::now() + retry_delay;
let res = sqlx::query(
"UPDATE group_queue_messages \
SET claim_token = NULL, claimed_at = NULL, deliver_after = $3 \
WHERE id = $1 AND claim_token = $2",
)
.bind(message_id.into_inner())
.bind(claim_token)
.bind(next)
.execute(&self.pool)
.await?;
Ok(res.rows_affected() == 1)
}
async fn release(
&self,
message_id: QueueMessageId,
claim_token: Uuid,
retry_delay: chrono::Duration,
) -> Result<bool, GroupQueueRepoError> {
// TRANSIENT release (handler never ran): re-queue AND undo `claim`'s
// pre-increment of `attempt` so a non-execution doesn't burn the retry
// budget. Mirrors `queue_repo::release`. Floors at 0.
let next = Utc::now() + retry_delay;
let res = sqlx::query(
"UPDATE group_queue_messages \
SET claim_token = NULL, claimed_at = NULL, deliver_after = $3, \
attempt = GREATEST(attempt - 1, 0) \
WHERE id = $1 AND claim_token = $2",
)
.bind(message_id.into_inner())
.bind(claim_token)
.bind(next)
.execute(&self.pool)
.await?;
Ok(res.rows_affected() == 1)
}
#[allow(clippy::too_many_arguments)]
async fn dead_letter(
&self,
message_id: QueueMessageId,
claim_token: Uuid,
group_id: GroupId,
collection: &str,
trigger_id: Option<picloud_shared::TriggerId>,
script_id: Option<picloud_shared::ScriptId>,
attempt: u32,
first_attempt_at: DateTime<Utc>,
last_error: &str,
) -> Result<picloud_shared::DeadLetterId, GroupQueueRepoError> {
let mut tx = self.pool.begin().await?;
// Pull the row inside the tx for its payload; filtered by claim_token so
// a lost lease can't dead-letter a re-claimed message.
let row: DeadLetterSourceRow = sqlx::query_as(
"SELECT id, payload, enqueued_at FROM group_queue_messages \
WHERE id = $1 AND claim_token = $2",
)
.bind(message_id.into_inner())
.bind(claim_token)
.fetch_one(&mut *tx)
.await?;
let payload = serde_json::json!({
"source": "queue",
"queue_name": collection,
"message": row.payload,
"enqueued_at": row.enqueued_at,
"attempt": attempt,
"message_id": row.id.to_string(),
});
let dl_id = picloud_shared::DeadLetterId::new();
sqlx::query(
"INSERT INTO group_dead_letters ( \
id, group_id, collection, original_event_id, source, op, \
trigger_id, script_id, payload, attempt_count, \
first_attempt_at, last_attempt_at, last_error \
) VALUES ($1, $2, $3, $4, 'queue', 'receive', $5, $6, $7, $8, $9, NOW(), $10)",
)
.bind(dl_id.into_inner())
.bind(group_id.into_inner())
.bind(collection)
.bind(row.id)
.bind(trigger_id.map(picloud_shared::TriggerId::into_inner))
.bind(script_id.map(picloud_shared::ScriptId::into_inner))
.bind(&payload)
.bind(i32::try_from(attempt).unwrap_or(0))
.bind(first_attempt_at)
.bind(last_error)
.execute(&mut *tx)
.await?;
sqlx::query("DELETE FROM group_queue_messages WHERE id = $1 AND claim_token = $2")
.bind(message_id.into_inner())
.bind(claim_token)
.execute(&mut *tx)
.await?;
tx.commit().await?;
Ok(dl_id)
}
async fn reclaim_visibility_timeouts(&self) -> Result<u64, GroupQueueRepoError> {
// A shared-queue consumer is a materialized app-owned trigger
// (`shared = TRUE`, `group_id` on the SOURCE template) whose
// queue_trigger_details.queue_name IS the shared collection name. Join
// the group template to recover the owning group + visibility timeout.
let res = sqlx::query(
"UPDATE group_queue_messages m \
SET claim_token = NULL, claimed_at = NULL \
FROM triggers copy \
JOIN triggers tmpl ON tmpl.id = copy.materialized_from \
JOIN queue_trigger_details d ON d.trigger_id = copy.id \
WHERE tmpl.group_id = m.group_id \
AND d.queue_name = m.collection \
AND tmpl.shared = TRUE \
AND m.claim_token IS NOT NULL \
AND m.claimed_at < NOW() - (d.visibility_timeout_secs || ' seconds')::INTERVAL",
)
.execute(&self.pool)
.await?;
Ok(res.rows_affected())
}
async fn depth(&self, group_id: GroupId, collection: &str) -> Result<u64, GroupQueueRepoError> {
let (n,): (i64,) = sqlx::query_as(
"SELECT COUNT(*) FROM ( \
SELECT 1 FROM group_queue_messages \
WHERE group_id = $1 AND collection = $2 LIMIT $3 \
) sub",
)
.bind(group_id.into_inner())
.bind(collection)
.bind(QUEUE_DEPTH_SCAN_CAP)
.fetch_one(&self.pool)
.await?;
Ok(u64::try_from(n).unwrap_or(0))
}
async fn depth_pending(
&self,
group_id: GroupId,
collection: &str,
) -> Result<u64, GroupQueueRepoError> {
let (n,): (i64,) = sqlx::query_as(
"SELECT COUNT(*) FROM ( \
SELECT 1 FROM group_queue_messages \
WHERE group_id = $1 AND collection = $2 \
AND claim_token IS NULL \
AND (deliver_after IS NULL OR deliver_after <= NOW()) LIMIT $3 \
) sub",
)
.bind(group_id.into_inner())
.bind(collection)
.bind(QUEUE_DEPTH_SCAN_CAP)
.fetch_one(&self.pool)
.await?;
Ok(u64::try_from(n).unwrap_or(0))
}
}
#[derive(sqlx::FromRow)]
struct ClaimedRow {
id: Uuid,
group_id: Uuid,
collection: String,
payload: serde_json::Value,
enqueued_at: DateTime<Utc>,
attempt: i32,
max_attempts: i32,
}
/// Minimal projection of a to-be-dead-lettered message: its payload + timing.
#[derive(sqlx::FromRow)]
struct DeadLetterSourceRow {
id: Uuid,
payload: serde_json::Value,
enqueued_at: DateTime<Utc>,
}

Some files were not shown because too many files have changed in this diff Show More