fix(quota): make the per-group ceiling a bound, not a hint
Audit #8. `GroupKvServiceImpl` read the group's usage (`count_rows` / `projected_total_bytes`) on one pooled connection and wrote on another. N concurrent writers therefore each observed the same pre-write total, each concluded it had room, and together sailed past the ceiling. The code called this out as "best-effort ... quotas are safety rails, not exact accounting" — but the overshoot is not small. With a ceiling of 5 and 20 concurrent writers, 19 rows land. Route group-KV mutations through `atomic_write::GroupKvWriter`, whose Postgres impl runs the quota reads, the row write, and the shared-trigger fan-out on ONE connection in ONE transaction — and, crucially, takes a per-group `pg_advisory_xact_lock` first. The lock is the fix, not the transaction. Putting the check inside the writing tx is necessary but NOT sufficient: under READ COMMITTED each transaction's `COUNT(*)`/`SUM(...)` still sees a snapshot without the other writers' uncommitted rows, so they all still pass. Serializing the check-then-write per group is what makes a writer see its predecessor's row. The lock key is namespaced per kind (kv/docs/…) so writes drawing on different ceilings don't contend. A delete takes no lock — it only frees space. The quota POLICY (row ceiling, projected-bytes ceiling, and the upper-bound fast path that skips the O(n) SUM scan for a group nowhere near its cap) moves to one place, `group_quota::check_group_write`, over a small `GroupUsage` trait. Both backends — the transactional one reading through `&mut *tx`, and the in-memory one the unit tests use — share it, so there is exactly one copy of the decision. `set_if` gains a correctness nicety on the way: the row ceiling now keys on whether the write ADDS a row, so a compare-and-swap against an absent key with a `Some` precondition (which cannot swap, and so consumes nothing) is no longer charged for one. tests/atomic_write.rs pins both ceilings against 20/16 concurrent writers. Both tests fail without the lock (19 rows stored against a ceiling of 5). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -195,8 +195,11 @@ pub async fn build_app(
|
||||
authz.clone(),
|
||||
picloud_manager_core::kv_service::kv_max_value_bytes_from_env(),
|
||||
)
|
||||
// §11.6 shared-collection triggers: fire `shared = true` triggers.
|
||||
.with_events(events.clone()),
|
||||
// The quota reads, the write, and the `shared = true` trigger fan-out all
|
||||
// run on one connection under a per-group advisory lock: the quota is a
|
||||
// real bound (concurrent writers to a group serialize), and an emit
|
||||
// failure rolls the write back. Supersedes `.with_events(...)`.
|
||||
.with_atomic_writes(pool.clone()),
|
||||
);
|
||||
// §11.6 shared group collections (docs): group-keyed group_docs store.
|
||||
let group_docs: Arc<dyn picloud_shared::GroupDocsService> = Arc::new(
|
||||
|
||||
Reference in New Issue
Block a user