fabi/PiCloud
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(materialize): take the per-(app,queue) lock so a consumer slot can't double-fill

Browse Source 
The one-consumer-per-(app_id, queue_name) invariant has no DB unique index (the
parent's app_id and the detail's queue_name live in different tables), so it is
enforced by a pg_advisory_xact_lock on hashtext(app_id||queue_name) around a
SELECT-then-INSERT. But `materialize` guarded its own check-then-insert only
with the GLOBAL rematerialize lock, not that per-(app,queue) key — so a
tree-apply / app-create rematerialize could interleave with an interactive
`create_queue_trigger` for the same (app, queue): each SELECT sees no consumer
(the other's INSERT uncommitted), both INSERT, and the app ends up with TWO
enabled consumers draining the same queue, splitting messages unpredictably.

Fix: `materialize_one` now resolves the template's queue_name and takes the SAME
`advisory_lock_key(app_id, queue_name)` (exposed `pub(crate)`) before its slot
check + insert, so the two paths mutually exclude. No deadlock: the interactive
path never acquires the global rematerialize lock, so there is no lock cycle,
and the rematerialize lock already serializes reconcilers against each other.
No schema change.

stateful_templates DB + journey coverage stays green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
MechaCat02
2026-07-13 22:31:42 +02:00
parent 8b62b137c0
commit fc70f5e224
2 changed files with 41 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
crates/manager-core/src/trigger_repo.rs
View File

@@ -1878,7 +1878,13 @@ impl TriggerRepo for PostgresTriggerRepo {
/// Stable per-(app_id, queue_name) i64 derived from the UUID bytes +
/// the queue name. Used as the key for `pg_advisory_xact_lock` so
/// concurrent `create_queue_trigger` calls for the same queue serialize.
fn advisory_lock_key(app_id: AppId, queue_name: &str) -> i64 {
/// Stable pg-advisory-lock key for the one-consumer-per-`(app_id, queue_name)`
/// invariant. `pub(crate)` so `materialize` can take the SAME lock the
/// interactive `create_queue_trigger` uses — otherwise a tree-apply
/// rematerialize could race an interactive consumer-create and double-fill the
/// slot (there is no DB unique index: the parent's `app_id` and the detail's
/// `queue_name` live in different tables).
pub(crate) fn advisory_lock_key(app_id: AppId, queue_name: &str) -> i64 {
use std::collections::hash_map::DefaultHasher;
use std::hash::{Hash, Hasher};
let mut h = DefaultHasher::new();