`with_events()` and `with_atomic_writes()` both install a writer, last call wins.
Appending them in the wrong order in the host would swap the transactional writer
back out for the best-effort one — silently reverting the write-path invariant AND
dropping the per-group quota enforcement that lives in it, with nothing in the type
system to catch it.
The services now carry an `atomic` flag: once `with_atomic_writes` has run,
`with_events` warns and no-ops instead of clobbering it. Also enforce the group
byte ceiling in `BestEffortGroupFilesWriter::update`, so the non-transactional
writer can't be used as an unmetered bypass and a unit test can catch its removal.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
An app's OWN store had no ceiling of any kind — only the group-SHARED
collections did, which is the inversion of what you'd expect. And
`authz::script_gate` returns `Ok(())` when there is no principal, so a
PUBLIC UNAUTHENTICATED route could reach `files::collection(..).create(..)`
and write blobs in a loop until the disk filled. `PICLOUD_FILES_MAX_FILE_SIZE_BYTES`
caps ONE file at 100 MB; nothing capped the count. Same for KV keys and docs
against Postgres.
Ceilings: `PICLOUD_APP_KV_MAX_ROWS` / `PICLOUD_APP_DOCS_MAX_ROWS` (100k) and
`PICLOUD_APP_FILES_MAX_TOTAL_BYTES` (10 GiB).
They are enforced DIFFERENTLY from the group ones, on purpose — porting the
group design as-is would have been a serious regression:
* KV/docs check a ROW count, on INSERT only, with NO lock. `kv::set` is the
hottest write path in the system; taking a per-app advisory lock on every
set would serialize an app's entire data plane, and a `SUM(...)` scan would
make write cost grow with the app's stored size. What the lock buys is
small — unlocked overshoot is bounded by write concurrency (32) against a
ceiling of 100k, ~0.03%. These are anti-DoS rails, not billing. An update
adds no row and is bounded by the per-value cap, so it pays nothing at all.
* No per-app BYTE ceiling for KV/docs: every value is already capped at
`PICLOUD_KV_MAX_VALUE_BYTES`, so `max_rows x max_value_bytes` is ALREADY a
finite bound. A second scan would buy nothing.
* FILES are the exception and DO lock + sum: one blob may be 100 MB, so 32
racing uploads could overshoot by gigabytes of real disk, and a file write
is heavy enough that the lock and the SUM are lost in the noise. Checked on
create AND update — an update that skipped the ceiling would be a free
bypass (the same bug just fixed for group files: grow a 1-byte file to
100 MB, repeat).
`group_quota` is renamed `quota`: it now serves both owners, and the module
doc is where the two enforcement strategies are contrasted.
Pinned by tests/atomic_write.rs: the key ceiling refuses a new key while
still allowing an update at the ceiling (rejecting updates would brick an app
the moment it filled up — worse than the DoS the rail exists to stop), and 10
concurrent uploads cannot exceed the disk ceiling, nor can an update grow past it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Audit #8. `GroupKvServiceImpl` read the group's usage (`count_rows` /
`projected_total_bytes`) on one pooled connection and wrote on another. N
concurrent writers therefore each observed the same pre-write total, each
concluded it had room, and together sailed past the ceiling. The code
called this out as "best-effort ... quotas are safety rails, not exact
accounting" — but the overshoot is not small. With a ceiling of 5 and 20
concurrent writers, 19 rows land.
Route group-KV mutations through `atomic_write::GroupKvWriter`, whose
Postgres impl runs the quota reads, the row write, and the shared-trigger
fan-out on ONE connection in ONE transaction — and, crucially, takes a
per-group `pg_advisory_xact_lock` first.
The lock is the fix, not the transaction. Putting the check inside the
writing tx is necessary but NOT sufficient: under READ COMMITTED each
transaction's `COUNT(*)`/`SUM(...)` still sees a snapshot without the other
writers' uncommitted rows, so they all still pass. Serializing the
check-then-write per group is what makes a writer see its predecessor's row.
The lock key is namespaced per kind (kv/docs/…) so writes drawing on
different ceilings don't contend. A delete takes no lock — it only frees
space.
The quota POLICY (row ceiling, projected-bytes ceiling, and the
upper-bound fast path that skips the O(n) SUM scan for a group nowhere near
its cap) moves to one place, `group_quota::check_group_write`, over a small
`GroupUsage` trait. Both backends — the transactional one reading through
`&mut *tx`, and the in-memory one the unit tests use — share it, so there is
exactly one copy of the decision.
`set_if` gains a correctness nicety on the way: the row ceiling now keys on
whether the write ADDS a row, so a compare-and-swap against an absent key
with a `Some` precondition (which cannot swap, and so consumes nothing) is
no longer charged for one.
tests/atomic_write.rs pins both ceilings against 20/16 concurrent writers.
Both tests fail without the lock (19 rows stored against a ceiling of 5).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The §11.6 M4 byte quota mixed two incompatible byte measures: `used` came from
Postgres as `octet_length(value::text)` (jsonb CANONICAL text — whitespace after
`:` and `,`, keys normalized), while `old_len`/`new_len` were computed in Rust
with `serde_json::to_vec` (COMPACT, no spaces). The projection
`used - old_len + new_len` therefore subtracted and added a *smaller* measure
than what is actually stored, so a group could be admitted over its ceiling —
the cap drifted permissive. The two forms are not reconcilable in Rust (jsonb
also reorders/dedups keys), so the projection has to be measured by Postgres.
Add `projected_total_bytes` to `GroupKvRepo` (keyed by collection+key) and
`GroupDocsRepo` (keyed by the replaced doc id, `None` on create). Each computes
`SUM(canonical) - existing_row(canonical) + new_value(canonical)` in ONE query,
binding the new value as compact TEXT and re-parsing it through `::jsonb::text`
so PG measures it in exactly the form it stores. All three terms now share one
metric. The services call it instead of the mixed Rust math; the near-cap
fast-path (`rows_after * max_value_bytes <= ceiling` → skip the SUM) is kept, so
the common case still does no extra work. The docs `update` path no longer needs
its old-doc fetch (the subtraction happens in SQL), removing a read.
The in-memory test repos implement the same projection with their own (compact)
metric, so the quota unit tests keep their exact byte arithmetic. No schema
change.
Note: the check-then-write TOCTOU (concurrent writers can each pass and
collectively overshoot) is NOT addressed here — it needs a transaction spanning
the check and the write, the same missing infrastructure as the transactional
outbox. Tracked separately.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two clean, local dedups in the group shared-collection services:
- Extract `GroupKvServiceImpl::check_value_size` — `set` and `set_if` inlined
the identical encode-and-cap block; now one method (mirrors the sibling
`group_docs_service::check_data_size`).
- Add `group_collection_repo::best_effort_emit_shared` — the KV/docs/files
services each copied the same emit-and-log-on-error tail for shared-collection
triggers. Centralize the tail (logging the event's own `source`/`op`); each
service still builds its own `ServiceEvent` (payload shapes differ).
Pure refactor, pinned by the existing group-service unit tests. (The
`owning_group` resolver was evaluated for dedup too but left as-is: the five
error enums diverge — `GroupFilesError::InvalidCollection` carries a `String`,
`GroupPubsubError` lacks the variant, and files/pubsub skip the empty-check kv/
docs/queue do — so a shared conversion would be a behavior change, not a
refactor.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Remediate the MEDIUM backend correctness/perf findings from the 2026-07-11 audit.
B1 — a group cron template toggled disabled→enabled no longer re-fires every
descendant's cron. Materialization now KEEPS a disabled cron template's copies
(syncing `enabled` in place) instead of delete+recreate, preserving
`last_fired_at`; the scheduler already skips disabled copies, so keeping them is
inert. Queue/email arms still delete-on-disable (freeing the one-consumer slot).
B2 — `rematerialize_stateful_templates` warnings are now logged at the
app-create/delete and group-reparent chokepoints (were silently dropped); only
the `Err` arm was handled before.
B3 — the group KV/docs byte-quota check skips the O(rows) `SUM(octet_length(..))`
scan when a cheap upper bound (`rows_after * max_value_bytes`) is already under
the cap, so the full aggregate only runs near-cap. Every row is validated
≤ max_value_bytes, so the bound is sound (never lets an over-quota write through).
B6 — the in-process SSE broadcaster caps live channels per map
(`PICLOUD_REALTIME_MAX_CHANNELS`, default 100k); a subscribe that would open a
NEW channel past the cap is refused with 503 + `Retry-After: 1` rather than
growing the (app,topic)/(group,topic) maps unboundedly. Check+insert is atomic
under the map mutex (no TOCTOU); an existing-channel subscribe is exempt.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
No atomic CAS existed, so concurrent writers to the same KV key raced. Add
set_if(key, expected, new): writes only when the current value equals expected,
or (expected absent) only when the key is missing — the primitive for lock-free
counters / optimistic concurrency.
- KvService + GroupKvService traits gain set_if with a defaulted "unsupported"
impl (Noop + other impls untouched); the real services override it.
- kv_repo/group_kv_repo: atomic set_if — a single UPDATE ... WHERE value =
$expected (JSONB semantic equality) or INSERT ... ON CONFLICT DO NOTHING.
Atomic without a transaction; returns whether the write happened.
- Services enforce the same value-size cap + (group) writes-authed + row/byte
quotas as set; the event fires only on a successful swap.
- Executor Rhai SDK: kv::collection(c).set_if(key, expected, new) and the
shared_collection variant; Rhai () for expected means "only if absent".
Pinned by kv_service/group_kv_service set_if unit tests (CAS + fail-closed) +
sdk_kv::kv_set_if_compare_and_swap (through a real script). No migration.
docs/sdk-shape.md documents the primitive.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
M3 quotas capped a group's shared KV/docs by ROW count only; a group could still
blow past storage with few-but-huge values. Add the symmetric total-bytes ceiling
(files already had one).
- group_quota: PICLOUD_GROUP_{KV,DOCS}_MAX_TOTAL_BYTES env vars (default 256 MiB)
+ accessors.
- GroupKvRepo/GroupDocsRepo: total_bytes(group) = SUM(octet_length(value::text))
(default Ok(0) so non-Postgres impls skip the check).
- GroupKv/DocsServiceImpl: check the PROJECTED total (old value's bytes
subtracted, new added) on every set/create/update, so a same-or-smaller update
near the cap is still allowed (unlike a naive used+new check). New
TotalBytesQuotaExceeded errors; +with_max_total_bytes for tests.
- KV set switched has->get to obtain the old value's size for the delta.
Pinned by group_kv_service + group_docs_service per_group_byte_quota_uses_the_
projected_total unit tests (reject over-cap, allow same/smaller update near cap).
No migration. CLAUDE.md env-var table updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Global env-var ceilings enforced in the group write path (mirrors the
per-value caps): PICLOUD_GROUP_KV_MAX_ROWS / _DOCS_MAX_ROWS (per-group row
count, checked only on a NEW key/doc — updates exempt) and
_FILES_MAX_TOTAL_BYTES (per-group total blob bytes). New group_quota env
helpers; count_rows/total_bytes repo methods (trait-default 0 for non-Postgres);
QuotaExceeded error variants on the three group error enums; a with_max_rows
test builder. Pinned by a group_kv_service unit test (3rd key rejected, update
of an existing key at the cap allowed).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ServiceEventEmitter gains emit_shared(cx, owning_group, event) (default no-op);
OutboxEventEmitter implements it — matches shared triggers on the owning group
and writes outbox rows stamped with the WRITER app_id (dispatch runs under the
writer). GroupKv/Docs/FilesServiceImpl gain an events field (Noop default +
with_events builder) and emit on set/create/update/delete; the host wires the
outbox emitter via with_events. Closes the "group trigger has no app to watch"
gap. Authoring + validation land in M2.4.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Data layer for extending shared group collections from KV to docs. KV behavior
unchanged (callers pass kind="kv").
- 0054_group_docs.sql: widen the group_collections kind CHECK to ('kv','docs')
and add the group-keyed group_docs store (clone of docs/0013, keyed by
group_id, CASCADE on group delete) + its (group_id,collection) and data GIN
indexes.
- group_collection_repo: thread a `kind` parameter through list_for_owner,
resolve_owning_group, insert/delete_collection_tx; add list_all_for_owner ->
(name,kind) for the kind-aware diff (D4). Relocate the injectable
GroupCollectionResolver trait + Postgres impl here (now shared by kv+docs;
resolve takes kind) — was in group_kv_service.
- group_docs_repo: a near-clone of docs_repo keyed by group_id; find reuses the
generalized build_find_query.
- docs_repo: generalize build_find_query(table, owner_col, owner_id, …) — both
are compile-time literals (no injection); app docs passes ("docs","app_id"),
group docs ("group_docs","group_id"). row_to_doc/build_find_query are now
pub(crate) for reuse.
Schema snapshot re-blessed (55 migrations).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Finish the §11.6 KV slice: declarative authoring, read-only inspection, the
runtime journey, and docs — plus a forced SDK-name fix.
- SDK name: `shared` is a Rhai RESERVED keyword, so `kv::shared(...)` is a parse
error. Renamed the handle constructor to `kv::shared_collection(...)`
(keeps the user's "shared" intent; unambiguous). Updated everywhere.
- CLI manifest: `collections = [...]` on `[group]` only (ManifestApp has no such
field + deny_unknown_fields → an app manifest carrying it is a hard error);
Manifest::collections() accessor; build_bundle emits it.
- plan/apply: a `collection` row group in `pic plan`; created/deleted counts in
the apply summary; collections threaded through PlanDto/NodePlanDto/
ApplyReportDto.
- read-only `pic collections ls --group` → GET /admin/groups/{id}/collections
(viewer+), backed by ApplyService::collection_report + CollectionInfo.
- journey: a group declares `catalog`; app A writes, app B (same subtree) reads
it back; a sibling-subtree app gets CollectionNotShared; `collections ls` +
re-apply NoOp. Full suite 114/114.
- docs: groups-and-project-tool §11.6 (KV-only MVP shipped + deferrals),
sdk-shape.md (the shared-collection handle + trust model), CLAUDE.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The runtime for shared group collections. A script reaches a shared store via
the explicit kv::shared("name") handle (distinct GroupKvHandle Rhai type, so
private-vs-shared scope is a deliberate choice). The service resolves the
owning group from cx.app_id's ancestor chain — the script never names a group,
and that walk is the isolation boundary (a foreign app's chain never contains
the owning group → CollectionNotShared).
- shared: GroupKvService trait + GroupKvError + NoopGroupKvService; threaded
into the Services bundle as a field defaulted to noop, opted into via a new
with_group_kv() (keeps the ~14 Services::new call sites unchanged).
- manager-core: GroupKvServiceImpl with the reads-open/writes-authed split —
reads use script_gate (anonymous public scripts skip), writes use the new
script_gate_require_principal (anonymous fails closed). Owner resolution
behind a GroupCollectionResolver trait (Postgres impl + injectable fake);
unit tests cover off-chain CollectionNotShared, anonymous-read-OK/
anonymous-write-Forbidden, authed-no-role-Forbidden.
- executor-core: GroupKvHandle + kv::shared constructor + get/set/has/delete/
list (Rhai dispatches by receiver type, reusing the block_on bridge).
- picloud: wire PostgresGroupKvRepo + resolver + GroupKvServiceImpl.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
