feat(v1.1.9): dispatcher queue arm + visibility-timeout reclaim task

Extends Dispatcher with the v1.1.9 queue path. The queue table IS the
outbox for queue semantics, so the dispatcher polls queue_messages
directly via FOR UPDATE SKIP LOCKED — no outbox indirection.

Per tick (every 100ms):
  1) outbox arm — unchanged
  2) queue arm: list_active_queue_consumers() → per (app_id, queue_name)
     attempt one claim. Bounded by registered-consumer count so one busy
     queue can't starve others.

Per claimed message:
  - Build TriggerEvent::Queue + ExecRequest (executes as the trigger's
    registering principal, matching design notes §4)
  - dispatch through executor.execute_with_identity (reuses AST cache)
  - success → queue.ack(id, claim_token) — DELETE WHERE id AND token
  - throw + attempt < max_attempts → queue.nack(...) — clear claim, set
    deliver_after = NOW() + compute_backoff(attempt, backoff, base_ms, jitter)
  - throw + exhausted → queue.dead_letter(...) atomic move to dead_letters
    + DELETE in one transaction. fan_out_dead_letter on the outbox arm
    fires registered dead_letter handlers off the new row without changes.

Visibility-timeout reclaim task: separate tokio::spawn ticking on
queue_reclaim_interval_ms (default 30000). UPDATE clears claim_token /
claimed_at on rows whose claimed_at exceeds the per-queue
visibility_timeout_secs (joined from queue_trigger_details). A crashed
consumer thus loses its lease and the message becomes claimable again.

Dispatcher gains queue: Arc<dyn QueueRepo>; picloud/lib.rs threads
queue_repo.clone() into the construction.

ActiveQueueConsumer extended with app_id so the queue claim can be
performed without a follow-up trigger lookup; list_active_queue_consumers
SQL extended to SELECT t.app_id.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
MechaCat02
2026-06-06 19:37:36 +02:00
parent cae2269932
commit 6891eda66c
3 changed files with 224 additions and 11 deletions

View File

@@ -281,10 +281,11 @@ pub struct CreateQueueTrigger {
}
/// One consumer the dispatcher's queue arm needs to scan messages for.
/// Pulled by `list_active_queue_consumers(app_id)` per tick.
/// Pulled by `list_active_queue_consumers()` per tick.
#[derive(Debug, Clone)]
pub struct ActiveQueueConsumer {
pub trigger_id: TriggerId,
pub app_id: AppId,
pub script_id: ScriptId,
pub queue_name: String,
pub visibility_timeout_secs: u32,
@@ -1290,7 +1291,7 @@ impl TriggerRepo for PostgresTriggerRepo {
&self,
) -> Result<Vec<ActiveQueueConsumer>, TriggerRepoError> {
let rows: Vec<QueueConsumerRow> = sqlx::query_as(
"SELECT t.id AS trigger_id, t.script_id, d.queue_name, \
"SELECT t.id AS trigger_id, t.app_id, t.script_id, d.queue_name, \
d.visibility_timeout_secs, \
t.retry_max_attempts, t.retry_backoff, t.retry_base_ms, \
t.registered_by_principal \
@@ -1304,6 +1305,7 @@ impl TriggerRepo for PostgresTriggerRepo {
.into_iter()
.map(|r| ActiveQueueConsumer {
trigger_id: r.trigger_id.into(),
app_id: r.app_id.into(),
script_id: r.script_id.into(),
queue_name: r.queue_name,
visibility_timeout_secs: u32::try_from(r.visibility_timeout_secs).unwrap_or(30),
@@ -1561,6 +1563,7 @@ struct QueueDetailRow {
#[derive(sqlx::FromRow)]
struct QueueConsumerRow {
trigger_id: Uuid,
app_id: Uuid,
script_id: Uuid,
queue_name: String,
visibility_timeout_secs: i32,