fix(stage-3): queue DL fan-out + emit-failure visibility
Closes the queue DL fan-out gap (audit Medium) and surfaces the KV/docs/files non-transactional emit gap to operators. - handle_queue_failure previously called queue.dead_letter to write the DL row but never invoked fan_out_dead_letter. The comment claimed "the outbox arm fires registered dead_letter handlers off the new row" — but queue DL rows are written via a separate path that bypasses the outbox entirely, so handlers filtered on source="queue" sat idle forever. Refactor fan_out_dead_letter to take a DeadLetterFanOutCtx struct so both the outbox arm and the queue arm can call it; the queue arm constructs a TriggerEvent::Queue from the claimed message and passes it through. - New integration test queue_dead_letter_fans_out_to_dead_letter_handler registers a dead_letter trigger filtered on "queue", forces queue exhaustion, asserts the handler fires with the correctly-shaped event. - KV/docs/files services committed the data write then ran events.emit as a separate operation, logging-and-swallowing on failure. Bump the six call sites from tracing::warn to tracing::error with an event_emit_failure=true marker so operators can grep them. The full single-tx repo refactor (extending ServiceEventEmitter with emit_in_tx + tx-aware *_repo methods) is documented in kv_service.rs as a v1.2 follow-up — it's a meaningful redesign that deserves its own pass (pubsub_service::fan_out_publish is the reference shape). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -155,8 +155,17 @@ impl KvService for KvServiceImpl {
|
||||
"insert"
|
||||
};
|
||||
// Emit unconditionally; the noop emitter drops it, the outbox
|
||||
// emitter persists it. Best-effort: a failed emit is logged
|
||||
// but does not roll back the write.
|
||||
// emitter persists it.
|
||||
//
|
||||
// Audit finding (Medium): this is non-transactional with the
|
||||
// data write — the row above has already committed by the time
|
||||
// emit() runs, so an emit failure means triggers silently
|
||||
// don't fire. Logged at `error` level (with
|
||||
// `event_emit_failure = true` for grepability); operators see
|
||||
// the gap. The full transactional refactor — extending the
|
||||
// ServiceEventEmitter trait with `emit_in_tx` and the KvRepo
|
||||
// surface with `set_with_tx` — is deferred to a v1.2 design
|
||||
// pass (pubsub_repo::fan_out_publish is the reference shape).
|
||||
if let Err(e) = self
|
||||
.events
|
||||
.emit(
|
||||
@@ -172,7 +181,7 @@ impl KvService for KvServiceImpl {
|
||||
)
|
||||
.await
|
||||
{
|
||||
tracing::warn!(error = %e, source = "kv", op, "event emit failed");
|
||||
tracing::error!(error = %e, source = "kv", op, event_emit_failure = true, "event emit failed");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
@@ -198,7 +207,7 @@ impl KvService for KvServiceImpl {
|
||||
)
|
||||
.await
|
||||
{
|
||||
tracing::warn!(error = %e, source = "kv", op = "delete", "event emit failed");
|
||||
tracing::error!(error = %e, source = "kv", op = "delete", event_emit_failure = true, "event emit failed");
|
||||
}
|
||||
}
|
||||
Ok(was_present)
|
||||
|
||||
Reference in New Issue
Block a user