fabi/PiCloud
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

refactor(outbox): make the trigger fan-out connection-scoped

Browse Source 
`OutboxEventEmitter` resolved matching triggers and inserted outbox rows
through `Arc<dyn TriggerRepo>` / `Arc<dyn OutboxRepo>`, i.e. against the
pool — each query on whatever connection it happened to get. That makes a
transactional outbox impossible: the data write and the outbox rows can
never share a transaction, so a crash (or an outbox error) between them
silently loses the trigger event.

Take the emitter down to a connection instead of a pool:

  * `outbox_repo::insert_on(exec, row)` and `trigger_repo::list_matching_on`
    / `list_matching_shared_on(exec, ...)` are generic over `PgExecutor`, so
    the same SQL serves a pooled connection or a `&mut *tx`. The repo trait
    methods delegate to them — the SQL keeps exactly one home.
  * `emit_on` / `emit_shared_on` take a `&mut PgConnection` and run the whole
    fan-out on it. The `ServiceEventEmitter` impl acquires one pooled
    connection and calls them, so behaviour is unchanged today; a caller
    holding a transaction can now pass `&mut *tx` and have the outbox rows
    commit with the write.
  * `OutboxEventEmitter::new` takes the `PgPool` directly (it was only ever
    constructed once, in the host wiring).

Also collapses the three copy-pasted per-app match queries (kv/docs/files
differ only in the `kind` discriminator and detail table) and the three
`emit_*` bodies into one `plan()` + one match fn, so the suppression
anti-join, the chain walk, and the empty-ops-means-any-op semantic each
exist once rather than three times.

No behaviour change — pure refactor. It is the seam the transactional
write lands on next.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
MechaCat02
2026-07-14 19:20:41 +02:00
parent 155c5471b7
commit a2360a9464
4 changed files with 351 additions and 505 deletions
Download Patch File Download Diff File Expand all files Collapse all files

473
crates/manager-core/src/outbox_event_emitter.rs
View File

@@ -1,139 +1,135 @@
//! `OutboxEventEmitter` — the real `ServiceEventEmitter` that replaces //! `OutboxEventEmitter` — the real `ServiceEventEmitter` behind every stateful
//! v1.1.0's `NoopEventEmitter` once the triggers framework lands. //! service. On each collection mutation (KV / docs / files) it looks up the
//! triggers the event matches and writes one outbox row per match; the
//! dispatcher reads them back out-of-band.
//! //!
//! On each `emit` (a KV mutation, future doc/file/pubsub event, etc.): //! **The fan-out is connection-scoped, not pool-scoped.** [`emit_on`] and
//! 1. Look up matching triggers for the event's (app_id, source, op, //! [`emit_shared_on`] take a `&mut PgConnection`, so a caller that already holds
//! collection) tuple via `TriggerRepo::list_matching_*`. //! a transaction can pass `&mut *tx` and have the outbox rows commit *with* the
//! 2. For each match, write one outbox row carrying the event payload //! data write that produced them — the transactional-outbox pattern. That closes
//! serialized as a `TriggerEvent`. //! the window where a row committed but its trigger never fired because the
//! outbox insert failed (or the process died) immediately afterwards. Services
//! that write through `crate::atomic_write` take that path; the
//! `ServiceEventEmitter` trait impl below is the non-transactional entry point,
//! kept for the callers (and the tests) that only need best-effort emission.
//! //!
//! Defaults applied at write time so `OutboxRow.payload` carries //! Non-collection `ServiceEvent` sources are silently dropped: the SDK calls
//! everything the dispatcher needs to reconstruct the executor //! `events.emit(...)` unconditionally for forward compat, and every source the
//! invocation without joining back to the trigger row. //! dispatcher has an arm for is handled here.
//!
//! Non-KV `ServiceEvent` sources are silently dropped in v1.1.1 — the
//! dispatcher only knows how to fire KV triggers this release. Future
//! sources (docs/files/pubsub) add their own dispatch arm.
use std::sync::Arc;
use async_trait::async_trait; use async_trait::async_trait;
use picloud_shared::{ use picloud_shared::{
DocsEventOp, EmitError, FileMeta, FilesEventOp, GroupId, KvEventOp, SdkCallCx, ServiceEvent, DocsEventOp, EmitError, FileMeta, FilesEventOp, GroupId, KvEventOp, SdkCallCx, ServiceEvent,
ServiceEventEmitter, TriggerEvent, ServiceEventEmitter, TriggerEvent,
}; };
use sqlx::{PgConnection, PgPool};
use crate::outbox_repo::{NewOutboxRow, OutboxRepo, OutboxSourceKind}; use crate::outbox_repo::{insert_on, NewOutboxRow, OutboxSourceKind};
use crate::trigger_repo::TriggerRepo; use crate::trigger_repo::{list_matching_on, list_matching_shared_on, KvMatchRow};
pub struct OutboxEventEmitter { pub struct OutboxEventEmitter {
triggers: Arc<dyn TriggerRepo>, pool: PgPool,
outbox: Arc<dyn OutboxRepo>,
} }
impl OutboxEventEmitter { impl OutboxEventEmitter {
#[must_use] #[must_use]
pub fn new(triggers: Arc<dyn TriggerRepo>, outbox: Arc<dyn OutboxRepo>) -> Self { pub fn new(pool: PgPool) -> Self {
Self { triggers, outbox } Self { pool }
} }
} }
#[async_trait] #[async_trait]
impl ServiceEventEmitter for OutboxEventEmitter { impl ServiceEventEmitter for OutboxEventEmitter {
async fn emit(&self, cx: &SdkCallCx, event: ServiceEvent) -> Result<(), EmitError> { async fn emit(&self, cx: &SdkCallCx, event: ServiceEvent) -> Result<(), EmitError> {
match event.source { let mut conn = self
"kv" => self.emit_kv(cx, event).await, .pool
"docs" => self.emit_docs(cx, event).await, .acquire()
"files" => self.emit_files(cx, event).await, .await
// Future sources land here. For now, silently drop — the .map_err(|e| EmitError::Unavailable(format!("acquire connection: {e}")))?;
// SDK calls `events.emit(...)` unconditionally for forward emit_on(&mut conn, cx, &event).await
// compat, so swallowing without an error is correct.
_ => Ok(()),
}
} }
#[allow(clippy::too_many_lines)] // one match arm per source kind (kv/docs/files)
async fn emit_shared( async fn emit_shared(
&self, &self,
cx: &SdkCallCx, cx: &SdkCallCx,
owning_group: GroupId, owning_group: GroupId,
event: ServiceEvent, event: ServiceEvent,
) -> Result<(), EmitError> { ) -> Result<(), EmitError> {
// §11.6: a shared-collection write. Match `shared = true` triggers on let mut conn = self
// the OWNING group; each fires under the WRITER app (`cx.app_id`), the .pool
// same "group template runs under the firing app" model. .acquire()
let source_kind = match event.source { .await
"kv" => OutboxSourceKind::Kv, .map_err(|e| EmitError::Unavailable(format!("acquire connection: {e}")))?;
"docs" => OutboxSourceKind::Docs, emit_shared_on(&mut conn, cx, owning_group, &event).await
"files" => OutboxSourceKind::Files, }
_ => return Ok(()), }
};
let Some(collection) = event.collection.clone() else { /// Everything the fan-out needs, derived from a `ServiceEvent` with no I/O: the
return Ok(()); /// trigger-table coordinates to match on, and the `TriggerEvent` the handler
}; /// will see as `ctx.event`. `None` for a source or op the dispatcher has no arm
let (matches, trigger_event) = match event.source { /// for — the caller drops the event quietly.
"kv" => { struct EventPlan {
let Some(op) = KvEventOp::from_wire(event.op) else { source_kind: OutboxSourceKind,
return Ok(()); /// `triggers.kind` discriminator. A literal — never user data, since it is
}; /// interpolated into the match SQL.
let m = self kind: &'static str,
.triggers /// Per-kind detail table. A literal, for the same reason.
.list_matching_shared_kv(owning_group, &collection, op) detail_table: &'static str,
.await collection: String,
.map_err(|e| EmitError::Unavailable(format!("trigger lookup: {e}")))?; op_str: &'static str,
let ev = TriggerEvent::Kv { trigger_event: TriggerEvent,
}
fn plan(event: &ServiceEvent) -> Option<EventPlan> {
// Collection events always carry a collection; defensively skip if not.
let collection = event.collection.clone()?;
let key = event.key.clone().unwrap_or_default();
match event.source {
"kv" => {
let op = KvEventOp::from_wire(event.op)?;
Some(EventPlan {
source_kind: OutboxSourceKind::Kv,
kind: "kv",
detail_table: "kv_trigger_details",
collection: collection.clone(),
op_str: op.as_str(),
trigger_event: TriggerEvent::Kv {
op, op,
collection, collection,
key: event.key.clone().unwrap_or_default(), key,
value: event.payload.clone(), value: event.payload.clone(),
}; },
( })
m.into_iter() }
.map(|x| (x.trigger_id, x.script_id)) "docs" => {
.collect::<Vec<_>>(), let op = DocsEventOp::from_wire(event.op)?;
ev, Some(EventPlan {
) source_kind: OutboxSourceKind::Docs,
} kind: "docs",
"docs" => { detail_table: "docs_trigger_details",
let Some(op) = DocsEventOp::from_wire(event.op) else { collection: collection.clone(),
return Ok(()); op_str: op.as_str(),
}; trigger_event: TriggerEvent::Docs {
let m = self
.triggers
.list_matching_shared_docs(owning_group, &collection, op)
.await
.map_err(|e| EmitError::Unavailable(format!("trigger lookup: {e}")))?;
let ev = TriggerEvent::Docs {
op, op,
collection, collection,
id: event.key.clone().unwrap_or_default(), id: key,
data: event.payload.clone(), data: event.payload.clone(),
prev_data: event.old_payload.clone(), prev_data: event.old_payload.clone(),
}; },
( })
m.into_iter() }
.map(|x| (x.trigger_id, x.script_id)) "files" => {
.collect::<Vec<_>>(), let op = FilesEventOp::from_wire(event.op)?;
ev, // The payload is the `FileMeta` JSON the files service emitted —
) // never the blob bytes.
} let meta: FileMeta = serde_json::from_value(event.payload.clone()?).ok()?;
"files" => { Some(EventPlan {
let Some(op) = FilesEventOp::from_wire(event.op) else { source_kind: OutboxSourceKind::Files,
return Ok(()); kind: "files",
}; detail_table: "files_trigger_details",
let Some(meta) = event collection: collection.clone(),
.payload op_str: op.as_str(),
.clone() trigger_event: TriggerEvent::Files {
.and_then(|v| serde_json::from_value::<FileMeta>(v).ok())
else {
return Ok(());
};
let m = self
.triggers
.list_matching_shared_files(owning_group, &collection, op)
.await
.map_err(|e| EmitError::Unavailable(format!("trigger lookup: {e}")))?;
let ev = TriggerEvent::Files {
op, op,
collection, collection,
id: meta.id.to_string(), id: meta.id.to_string(),
@@ -142,206 +138,87 @@ impl ServiceEventEmitter for OutboxEventEmitter {
size: meta.size, size: meta.size,
checksum: meta.checksum, checksum: meta.checksum,
prev: event.old_payload.clone(), prev: event.old_payload.clone(),
}; },
( })
m.into_iter()
.map(|x| (x.trigger_id, x.script_id))
.collect::<Vec<_>>(),
ev,
)
}
_ => return Ok(()),
};
if matches.is_empty() {
return Ok(());
} }
let payload = serde_json::to_value(&trigger_event) _ => None,
.map_err(|e| EmitError::Rejected(format!("event serialize: {e}")))?;
for (trigger_id, script_id) in matches {
self.outbox
.insert(NewOutboxRow {
app_id: cx.app_id,
source_kind,
trigger_id: Some(trigger_id),
script_id: Some(script_id),
reply_to: None,
payload: payload.clone(),
origin_principal: cx.principal.as_ref().map(|p| p.user_id),
trigger_depth: cx.trigger_depth.saturating_add(1),
root_execution_id: Some(cx.root_execution_id),
})
.await
.map_err(|e| EmitError::Unavailable(format!("outbox insert: {e}")))?;
}
Ok(())
} }
} }
impl OutboxEventEmitter { /// Fan a collection mutation out over the writing app's own + inherited
async fn emit_kv(&self, cx: &SdkCallCx, event: ServiceEvent) -> Result<(), EmitError> { /// triggers. Pass `&mut *tx` to have the outbox rows commit with the write.
let Some(op) = KvEventOp::from_wire(event.op) else { pub(crate) async fn emit_on(
return Ok(()); // unknown op — drop quietly conn: &mut PgConnection,
}; cx: &SdkCallCx,
let Some(collection) = event.collection.clone() else { event: &ServiceEvent,
return Ok(()); // KV events always carry a collection — defensively skip ) -> Result<(), EmitError> {
}; let Some(p) = plan(event) else { return Ok(()) };
let key = event.key.clone().unwrap_or_default(); let matches = list_matching_on(
&mut *conn,
let matches = self p.kind,
.triggers p.detail_table,
.list_matching_kv(cx.app_id, &collection, op) cx.app_id,
.await &p.collection,
.map_err(|e| EmitError::Unavailable(format!("trigger lookup: {e}")))?; p.op_str,
)
if matches.is_empty() { .await
return Ok(()); .map_err(|e| EmitError::Unavailable(format!("trigger lookup: {e}")))?;
} write_outbox_rows(conn, cx, &p, matches).await
}
// Serialize the originating event as a TriggerEvent so the
// dispatcher can hand it to the script as `ctx.event` without /// §11.6: fan a SHARED-collection write out over the `shared = true` triggers on
// round-tripping back to the trigger row. /// the OWNING group. Each fires under the WRITER app (`cx.app_id`) — the same
let trigger_event = TriggerEvent::Kv { /// "a group template runs under the firing app" model as an inherited trigger.
op, pub(crate) async fn emit_shared_on(
collection, conn: &mut PgConnection,
key, cx: &SdkCallCx,
value: event.payload.clone(), owning_group: GroupId,
}; event: &ServiceEvent,
let payload = serde_json::to_value(&trigger_event) ) -> Result<(), EmitError> {
.map_err(|e| EmitError::Rejected(format!("event serialize: {e}")))?; let Some(p) = plan(event) else { return Ok(()) };
let matches = list_matching_shared_on(
for m in matches { &mut *conn,
self.outbox p.kind,
.insert(NewOutboxRow { p.detail_table,
app_id: cx.app_id, owning_group,
source_kind: OutboxSourceKind::Kv, &p.collection,
trigger_id: Some(m.trigger_id), p.op_str,
script_id: Some(m.script_id), )
reply_to: None, .await
payload: payload.clone(), .map_err(|e| EmitError::Unavailable(format!("trigger lookup: {e}")))?;
origin_principal: cx.principal.as_ref().map(|p| p.user_id), write_outbox_rows(conn, cx, &p, matches).await
trigger_depth: cx.trigger_depth.saturating_add(1), }
root_execution_id: Some(cx.root_execution_id),
}) async fn write_outbox_rows(
.await conn: &mut PgConnection,
.map_err(|e| EmitError::Unavailable(format!("outbox insert: {e}")))?; cx: &SdkCallCx,
} p: &EventPlan,
Ok(()) matches: Vec<KvMatchRow>,
} ) -> Result<(), EmitError> {
if matches.is_empty() {
/// v1.1.2. Mirrors `emit_kv` — fan out a docs mutation across return Ok(());
/// matching docs triggers + write one outbox row each. The }
/// `prev_data` change-data-capture surface is preserved from the // Serialize the originating event once so the dispatcher can hand it to the
/// `ServiceEvent.old_payload` field (set by `DocsServiceImpl` on // handler as `ctx.event` without joining back to the trigger row.
/// update and delete; `None` for create). let payload = serde_json::to_value(&p.trigger_event)
async fn emit_docs(&self, cx: &SdkCallCx, event: ServiceEvent) -> Result<(), EmitError> { .map_err(|e| EmitError::Rejected(format!("event serialize: {e}")))?;
let Some(op) = DocsEventOp::from_wire(event.op) else { for m in matches {
return Ok(()); insert_on(
}; &mut *conn,
let Some(collection) = event.collection.clone() else { NewOutboxRow {
return Ok(()); app_id: cx.app_id,
}; source_kind: p.source_kind,
let id = event.key.clone().unwrap_or_default(); trigger_id: Some(m.id.into()),
script_id: Some(m.script_id.into()),
let matches = self reply_to: None,
.triggers payload: payload.clone(),
.list_matching_docs(cx.app_id, &collection, op) origin_principal: cx.principal.as_ref().map(|pr| pr.user_id),
.await trigger_depth: cx.trigger_depth.saturating_add(1),
.map_err(|e| EmitError::Unavailable(format!("trigger lookup: {e}")))?; root_execution_id: Some(cx.root_execution_id),
},
if matches.is_empty() { )
return Ok(()); .await
} .map_err(|e| EmitError::Unavailable(format!("outbox insert: {e}")))?;
}
let trigger_event = TriggerEvent::Docs { Ok(())
op,
collection,
id,
data: event.payload.clone(),
prev_data: event.old_payload.clone(),
};
let payload = serde_json::to_value(&trigger_event)
.map_err(|e| EmitError::Rejected(format!("event serialize: {e}")))?;
for m in matches {
self.outbox
.insert(NewOutboxRow {
app_id: cx.app_id,
source_kind: OutboxSourceKind::Docs,
trigger_id: Some(m.trigger_id),
script_id: Some(m.script_id),
reply_to: None,
payload: payload.clone(),
origin_principal: cx.principal.as_ref().map(|p| p.user_id),
trigger_depth: cx.trigger_depth.saturating_add(1),
root_execution_id: Some(cx.root_execution_id),
})
.await
.map_err(|e| EmitError::Unavailable(format!("outbox insert: {e}")))?;
}
Ok(())
}
/// v1.1.5. Fan out a files mutation across matching files triggers.
/// The `ServiceEvent.payload` is the file **metadata** (never the
/// blob bytes); `old_payload` is the prior metadata (the deleted
/// row's metadata on delete). The `TriggerEvent::Files` carries the
/// metadata fields explicitly + `prev` for the change-data-capture
/// surface.
async fn emit_files(&self, cx: &SdkCallCx, event: ServiceEvent) -> Result<(), EmitError> {
let Some(op) = FilesEventOp::from_wire(event.op) else {
return Ok(());
};
let Some(collection) = event.collection.clone() else {
return Ok(());
};
// The payload is the FileMeta JSON the FilesServiceImpl emitted.
let Some(meta) = event
.payload
.clone()
.and_then(|v| serde_json::from_value::<FileMeta>(v).ok())
else {
return Ok(());
};
let matches = self
.triggers
.list_matching_files(cx.app_id, &collection, op)
.await
.map_err(|e| EmitError::Unavailable(format!("trigger lookup: {e}")))?;
if matches.is_empty() {
return Ok(());
}
let trigger_event = TriggerEvent::Files {
op,
collection,
id: meta.id.to_string(),
name: meta.name,
content_type: meta.content_type,
size: meta.size,
checksum: meta.checksum,
prev: event.old_payload.clone(),
};
let payload = serde_json::to_value(&trigger_event)
.map_err(|e| EmitError::Rejected(format!("event serialize: {e}")))?;
for m in matches {
self.outbox
.insert(NewOutboxRow {
app_id: cx.app_id,
source_kind: OutboxSourceKind::Files,
trigger_id: Some(m.trigger_id),
script_id: Some(m.script_id),
reply_to: None,
payload: payload.clone(),
origin_principal: cx.principal.as_ref().map(|p| p.user_id),
trigger_depth: cx.trigger_depth.saturating_add(1),
root_execution_id: Some(cx.root_execution_id),
})
.await
.map_err(|e| EmitError::Unavailable(format!("outbox insert: {e}")))?;
}
Ok(())
}
} }

49
crates/manager-core/src/outbox_repo.rs
View File

@@ -145,28 +145,39 @@ impl PostgresOutboxRepo {
} }
} }
/// Insert one outbox row on an arbitrary executor — a pooled connection, or a
/// `&mut *tx` so the row commits atomically with the data write that produced
/// it (the transactional outbox; see `outbox_event_emitter`). The `insert`
/// trait method delegates here, so the INSERT has exactly one home.
pub(crate) async fn insert_on<'c, E>(exec: E, row: NewOutboxRow) -> Result<Uuid, OutboxRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let (id,): (Uuid,) = sqlx::query_as(
"INSERT INTO outbox ( \
app_id, source_kind, trigger_id, script_id, reply_to, \
payload, origin_principal, trigger_depth, root_execution_id \
) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9) \
RETURNING id",
)
.bind(row.app_id.into_inner())
.bind(row.source_kind.as_str())
.bind(row.trigger_id.map(TriggerId::into_inner))
.bind(row.script_id.map(ScriptId::into_inner))
.bind(row.reply_to)
.bind(row.payload)
.bind(row.origin_principal.map(AdminUserId::into_inner))
.bind(i32::try_from(row.trigger_depth).unwrap_or(0))
.bind(row.root_execution_id.map(ExecutionId::into_inner))
.fetch_one(exec)
.await?;
Ok(id)
}
#[async_trait] #[async_trait]
impl OutboxRepo for PostgresOutboxRepo { impl OutboxRepo for PostgresOutboxRepo {
async fn insert(&self, row: NewOutboxRow) -> Result<Uuid, OutboxRepoError> { async fn insert(&self, row: NewOutboxRow) -> Result<Uuid, OutboxRepoError> {
let (id,): (Uuid,) = sqlx::query_as( insert_on(&self.pool, row).await
"INSERT INTO outbox ( \
app_id, source_kind, trigger_id, script_id, reply_to, \
payload, origin_principal, trigger_depth, root_execution_id \
) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9) \
RETURNING id",
)
.bind(row.app_id.into_inner())
.bind(row.source_kind.as_str())
.bind(row.trigger_id.map(TriggerId::into_inner))
.bind(row.script_id.map(ScriptId::into_inner))
.bind(row.reply_to)
.bind(row.payload)
.bind(row.origin_principal.map(AdminUserId::into_inner))
.bind(i32::try_from(row.trigger_depth).unwrap_or(0))
.bind(row.root_execution_id.map(ExecutionId::into_inner))
.fetch_one(&self.pool)
.await?;
Ok(id)
} }
async fn claim_due( async fn claim_due(

329
crates/manager-core/src/trigger_repo.rs
View File

@@ -605,39 +605,95 @@ impl PostgresTriggerRepo {
pub fn new(pool: PgPool) -> Self { pub fn new(pool: PgPool) -> Self {
Self { pool } Self { pool }
} }
}
/// §11.6 shared-collection match: enabled `shared = true` triggers on the // ----------------------------------------------------------------------------
/// OWNING group, glob + op filtered in Rust (like the per-app path). `kind` // Connection-scoped event matching
/// and `detail_table` are hard-coded literals from the three call sites (no //
/// injection). No chain / no suppression anti-join — a shared trigger is // The three collection-event kinds (kv / docs / files) match identically apart
/// declared on the group that owns the collection and fires under the writer. // from the `triggers.kind` discriminator and their detail table, so both shapes
async fn shared_match_rows( // — the per-app CHAIN walk and the §11.6 SHARED-collection lookup — live here
&self, // once, parameterized on an executor.
kind: &str, //
detail_table: &str, // Taking an executor rather than `&self.pool` is what lets the transactional
owning_group: GroupId, // outbox work: `outbox_event_emitter` runs the match on the SAME connection
collection: &str, // (`&mut *tx`) as the data write it is fanning out, so the write and its outbox
op_str: &str, // rows commit together or not at all. The `list_matching_*` trait methods below
) -> Result<Vec<KvMatchRow>, TriggerRepoError> { // pass `&self.pool` and behave exactly as before.
let rows: Vec<KvMatchRow> = sqlx::query_as(&format!( //
"SELECT t.id, t.script_id, t.dispatch_mode, \ // `kind` and `detail_table` are interpolated into the SQL, so they must stay
t.retry_max_attempts, t.retry_backoff, t.retry_base_ms, \ // hard-coded literals from the call sites below — never user data.
t.registered_by_principal, \ // ----------------------------------------------------------------------------
d.collection_glob, d.ops \
FROM triggers t \ /// Glob + op filtering, in Rust rather than SQL. **Critical**: an empty `ops`
JOIN {detail_table} d ON d.trigger_id = t.id \ /// array means "any op", which a SQL `$op = ANY(ops)` predicate would silently
WHERE t.kind = '{kind}' AND t.enabled = TRUE \ /// exclude.
AND t.shared = TRUE AND t.group_id = $1" fn filter_match_rows(rows: Vec<KvMatchRow>, collection: &str, op_str: &str) -> Vec<KvMatchRow> {
)) rows.into_iter()
.bind(owning_group.into_inner()) .filter(|r| collection_matches(&r.collection_glob, collection))
.fetch_all(&self.pool) .filter(|r| r.ops.is_empty() || r.ops.iter().any(|o| o == op_str))
.await?; .collect()
Ok(rows }
.into_iter()
.filter(|r| collection_matches(&r.collection_glob, collection)) /// Per-app match: the firing app's OWN triggers plus any inherited from an
.filter(|r| r.ops.is_empty() || r.ops.iter().any(|o| o == op_str)) /// ancestor group, minus the ones a suppression on its chain declines.
.collect()) pub(crate) async fn list_matching_on<'c, E>(
} exec: E,
kind: &str,
detail_table: &str,
app_id: AppId,
collection: &str,
op_str: &str,
) -> Result<Vec<KvMatchRow>, TriggerRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let rows: Vec<KvMatchRow> = sqlx::query_as(&format!(
"{CHAIN_LEVELS_CTE} \
SELECT t.id, t.script_id, t.dispatch_mode, \
t.retry_max_attempts, t.retry_backoff, t.retry_base_ms, \
t.registered_by_principal, \
d.collection_glob, d.ops \
FROM triggers t \
JOIN {detail_table} d ON d.trigger_id = t.id \
JOIN chain c ON (t.app_id = c.app_owner OR t.group_id = c.group_owner) \
WHERE t.kind = '{kind}' AND t.enabled = TRUE \
AND t.shared = FALSE{TRIGGER_SUPPRESSION_ANTIJOIN}"
))
.bind(app_id.into_inner())
.fetch_all(exec)
.await?;
Ok(filter_match_rows(rows, collection, op_str))
}
/// §11.6 shared-collection match: enabled `shared = true` triggers on the
/// OWNING group. No chain and no suppression anti-join — a shared trigger is
/// declared on the group that owns the collection and fires under the writer.
pub(crate) async fn list_matching_shared_on<'c, E>(
exec: E,
kind: &str,
detail_table: &str,
owning_group: GroupId,
collection: &str,
op_str: &str,
) -> Result<Vec<KvMatchRow>, TriggerRepoError>
where
E: sqlx::PgExecutor<'c>,
{
let rows: Vec<KvMatchRow> = sqlx::query_as(&format!(
"SELECT t.id, t.script_id, t.dispatch_mode, \
t.retry_max_attempts, t.retry_backoff, t.retry_base_ms, \
t.registered_by_principal, \
d.collection_glob, d.ops \
FROM triggers t \
JOIN {detail_table} d ON d.trigger_id = t.id \
WHERE t.kind = '{kind}' AND t.enabled = TRUE \
AND t.shared = TRUE AND t.group_id = $1"
))
.bind(owning_group.into_inner())
.fetch_all(exec)
.await?;
Ok(filter_match_rows(rows, collection, op_str))
} }
/// Insert a trigger (parent row + per-kind detail) within an existing /// Insert a trigger (parent row + per-kind detail) within an existing
@@ -1483,48 +1539,16 @@ impl TriggerRepo for PostgresTriggerRepo {
collection: &str, collection: &str,
op: KvEventOp, op: KvEventOp,
) -> Result<Vec<KvTriggerMatch>, TriggerRepoError> { ) -> Result<Vec<KvTriggerMatch>, TriggerRepoError> {
// Fetch all enabled KV triggers for the app — glob matching let rows = list_matching_on(
// happens in Rust so we don't have to teach the query about &self.pool,
// `*` and `prefix:*`. Sets are tiny in practice (one app's "kv",
// worth of triggers, usually a handful). "kv_trigger_details",
let rows: Vec<KvMatchRow> = sqlx::query_as(&format!( app_id,
"{CHAIN_LEVELS_CTE} \ collection,
SELECT t.id, t.script_id, t.dispatch_mode, \ op.as_str(),
t.retry_max_attempts, t.retry_backoff, t.retry_base_ms, \ )
t.registered_by_principal, \
d.collection_glob, d.ops \
FROM triggers t \
JOIN kv_trigger_details d ON d.trigger_id = t.id \
JOIN chain c ON (t.app_id = c.app_owner OR t.group_id = c.group_owner) \
WHERE t.kind = 'kv' AND t.enabled = TRUE \
AND t.shared = FALSE{TRIGGER_SUPPRESSION_ANTIJOIN}"
))
.bind(app_id.into_inner())
.fetch_all(&self.pool)
.await?; .await?;
Ok(rows.into_iter().map(KvMatchRow::into_kv).collect())
let op_str = op.as_str();
let mut out = Vec::new();
for r in rows {
if !collection_matches(&r.collection_glob, collection) {
continue;
}
let any_op = r.ops.is_empty();
if !any_op && !r.ops.iter().any(|o| o == op_str) {
continue;
}
out.push(KvTriggerMatch {
trigger_id: r.id.into(),
script_id: r.script_id.into(),
dispatch_mode: dispatch_from_str(&r.dispatch_mode),
retry_max_attempts: u32::try_from(r.retry_max_attempts).unwrap_or(3),
retry_backoff: BackoffShape::from_wire(&r.retry_backoff)
.unwrap_or(BackoffShape::Exponential),
retry_base_ms: u32::try_from(r.retry_base_ms).unwrap_or(1000),
registered_by_principal: r.registered_by_principal.into(),
});
}
Ok(out)
} }
async fn list_matching_docs( async fn list_matching_docs(
@@ -1533,49 +1557,16 @@ impl TriggerRepo for PostgresTriggerRepo {
collection: &str, collection: &str,
op: DocsEventOp, op: DocsEventOp,
) -> Result<Vec<DocsTriggerMatch>, TriggerRepoError> { ) -> Result<Vec<DocsTriggerMatch>, TriggerRepoError> {
// Mirrors list_matching_kv: pull every enabled docs trigger, let rows = list_matching_on(
// filter glob + ops in Rust. **Critical**: do NOT push the &self.pool,
// ops check into SQL (`WHERE $op = ANY(ops)`) — that would "docs",
// exclude rows with `ops = '{}'` from the results, breaking "docs_trigger_details",
// the empty-array-means-any-op semantic. app_id,
let rows: Vec<KvMatchRow> = sqlx::query_as(&format!( collection,
"{CHAIN_LEVELS_CTE} \ op.as_str(),
SELECT t.id, t.script_id, t.dispatch_mode, \ )
t.retry_max_attempts, t.retry_backoff, t.retry_base_ms, \
t.registered_by_principal, \
d.collection_glob, d.ops \
FROM triggers t \
JOIN docs_trigger_details d ON d.trigger_id = t.id \
JOIN chain c ON (t.app_id = c.app_owner OR t.group_id = c.group_owner) \
WHERE t.kind = 'docs' AND t.enabled = TRUE \
AND t.shared = FALSE{TRIGGER_SUPPRESSION_ANTIJOIN}"
))
.bind(app_id.into_inner())
.fetch_all(&self.pool)
.await?; .await?;
Ok(rows.into_iter().map(KvMatchRow::into_docs).collect())
let op_str = op.as_str();
let mut out = Vec::new();
for r in rows {
if !collection_matches(&r.collection_glob, collection) {
continue;
}
let any_op = r.ops.is_empty();
if !any_op && !r.ops.iter().any(|o| o == op_str) {
continue;
}
out.push(DocsTriggerMatch {
trigger_id: r.id.into(),
script_id: r.script_id.into(),
dispatch_mode: dispatch_from_str(&r.dispatch_mode),
retry_max_attempts: u32::try_from(r.retry_max_attempts).unwrap_or(3),
retry_backoff: BackoffShape::from_wire(&r.retry_backoff)
.unwrap_or(BackoffShape::Exponential),
retry_base_ms: u32::try_from(r.retry_base_ms).unwrap_or(1000),
registered_by_principal: r.registered_by_principal.into(),
});
}
Ok(out)
} }
async fn list_matching_files( async fn list_matching_files(
@@ -1584,46 +1575,16 @@ impl TriggerRepo for PostgresTriggerRepo {
collection: &str, collection: &str,
op: FilesEventOp, op: FilesEventOp,
) -> Result<Vec<FilesTriggerMatch>, TriggerRepoError> { ) -> Result<Vec<FilesTriggerMatch>, TriggerRepoError> {
// Mirrors list_matching_kv: pull every enabled files trigger, let rows = list_matching_on(
// filter glob + ops in Rust (empty ops array means "any op"). &self.pool,
let rows: Vec<KvMatchRow> = sqlx::query_as(&format!( "files",
"{CHAIN_LEVELS_CTE} \ "files_trigger_details",
SELECT t.id, t.script_id, t.dispatch_mode, \ app_id,
t.retry_max_attempts, t.retry_backoff, t.retry_base_ms, \ collection,
t.registered_by_principal, \ op.as_str(),
d.collection_glob, d.ops \ )
FROM triggers t \
JOIN files_trigger_details d ON d.trigger_id = t.id \
JOIN chain c ON (t.app_id = c.app_owner OR t.group_id = c.group_owner) \
WHERE t.kind = 'files' AND t.enabled = TRUE \
AND t.shared = FALSE{TRIGGER_SUPPRESSION_ANTIJOIN}"
))
.bind(app_id.into_inner())
.fetch_all(&self.pool)
.await?; .await?;
Ok(rows.into_iter().map(KvMatchRow::into_files).collect())
let op_str = op.as_str();
let mut out = Vec::new();
for r in rows {
if !collection_matches(&r.collection_glob, collection) {
continue;
}
let any_op = r.ops.is_empty();
if !any_op && !r.ops.iter().any(|o| o == op_str) {
continue;
}
out.push(FilesTriggerMatch {
trigger_id: r.id.into(),
script_id: r.script_id.into(),
dispatch_mode: dispatch_from_str(&r.dispatch_mode),
retry_max_attempts: u32::try_from(r.retry_max_attempts).unwrap_or(3),
retry_backoff: BackoffShape::from_wire(&r.retry_backoff)
.unwrap_or(BackoffShape::Exponential),
retry_base_ms: u32::try_from(r.retry_base_ms).unwrap_or(1000),
registered_by_principal: r.registered_by_principal.into(),
});
}
Ok(out)
} }
async fn list_matching_shared_kv( async fn list_matching_shared_kv(
@@ -1632,15 +1593,15 @@ impl TriggerRepo for PostgresTriggerRepo {
collection: &str, collection: &str,
op: KvEventOp, op: KvEventOp,
) -> Result<Vec<KvTriggerMatch>, TriggerRepoError> { ) -> Result<Vec<KvTriggerMatch>, TriggerRepoError> {
let rows = self let rows = list_matching_shared_on(
.shared_match_rows( &self.pool,
"kv", "kv",
"kv_trigger_details", "kv_trigger_details",
owning_group, owning_group,
collection, collection,
op.as_str(), op.as_str(),
) )
.await?; .await?;
Ok(rows.into_iter().map(KvMatchRow::into_kv).collect()) Ok(rows.into_iter().map(KvMatchRow::into_kv).collect())
} }
@@ -1650,15 +1611,15 @@ impl TriggerRepo for PostgresTriggerRepo {
collection: &str, collection: &str,
op: DocsEventOp, op: DocsEventOp,
) -> Result<Vec<DocsTriggerMatch>, TriggerRepoError> { ) -> Result<Vec<DocsTriggerMatch>, TriggerRepoError> {
let rows = self let rows = list_matching_shared_on(
.shared_match_rows( &self.pool,
"docs", "docs",
"docs_trigger_details", "docs_trigger_details",
owning_group, owning_group,
collection, collection,
op.as_str(), op.as_str(),
) )
.await?; .await?;
Ok(rows.into_iter().map(KvMatchRow::into_docs).collect()) Ok(rows.into_iter().map(KvMatchRow::into_docs).collect())
} }
@@ -1668,15 +1629,15 @@ impl TriggerRepo for PostgresTriggerRepo {
collection: &str, collection: &str,
op: FilesEventOp, op: FilesEventOp,
) -> Result<Vec<FilesTriggerMatch>, TriggerRepoError> { ) -> Result<Vec<FilesTriggerMatch>, TriggerRepoError> {
let rows = self let rows = list_matching_shared_on(
.shared_match_rows( &self.pool,
"files", "files",
"files_trigger_details", "files_trigger_details",
owning_group, owning_group,
collection, collection,
op.as_str(), op.as_str(),
) )
.await?; .await?;
Ok(rows.into_iter().map(KvMatchRow::into_files).collect()) Ok(rows.into_iter().map(KvMatchRow::into_files).collect())
} }
@@ -2169,9 +2130,9 @@ struct DlDetailRow {
} }
#[derive(sqlx::FromRow)] #[derive(sqlx::FromRow)]
struct KvMatchRow { pub(crate) struct KvMatchRow {
id: Uuid, pub(crate) id: Uuid,
script_id: Uuid, pub(crate) script_id: Uuid,
dispatch_mode: String, dispatch_mode: String,
retry_max_attempts: i32, retry_max_attempts: i32,
retry_backoff: String, retry_backoff: String,

5
crates/picloud/src/lib.rs
View File

@@ -169,10 +169,7 @@ pub async fn build_app(
// dispatcher. // dispatcher.
let kv_repo = Arc::new(PostgresKvRepo::new(pool.clone())); let kv_repo = Arc::new(PostgresKvRepo::new(pool.clone()));
let docs_repo = Arc::new(PostgresDocsRepo::new(pool.clone())); let docs_repo = Arc::new(PostgresDocsRepo::new(pool.clone()));
let events: Arc<dyn ServiceEventEmitter> = Arc::new(OutboxEventEmitter::new( let events: Arc<dyn ServiceEventEmitter> = Arc::new(OutboxEventEmitter::new(pool.clone()));
trigger_repo.clone(),
outbox_repo.clone(),
));
let kv: Arc<dyn KvService> = Arc::new(KvServiceImpl::with_max_value_bytes( let kv: Arc<dyn KvService> = Arc::new(KvServiceImpl::with_max_value_bytes(
kv_repo.clone(), kv_repo.clone(),
authz.clone(), authz.clone(),