docs: record the transactional-outbox write-path invariant

The three rules a future change to the data-plane write path must not
break, and the two that are counter-intuitive enough to be re-broken:

  * everything inside a tx runs on that tx's connection (reaching for a
    second pooled connection while holding one can deadlock the pool), and
    `shared` must stay sqlx-free — the transactional path is manager-core
    internal, the `ServiceEventEmitter` trait stays connection-free;
  * a per-group quota needs a LOCK, not just a transaction — under READ
    COMMITTED each tx's COUNT/SUM sees a snapshot without the others'
    uncommitted rows, so concurrent writers all pass the check anyway
    (measured: 19 rows stored against a ceiling of 5);
  * files order around the disk write, which cannot join a transaction.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
MechaCat02
2026-07-14 20:14:45 +02:00
parent 80a0d31cd2
commit 3de75fe4a8

View File

@@ -14,6 +14,11 @@ Authoritative design: [serverless_cloud_blueprint.md](serverless_cloud_blueprint
**Track A (v1.2 deferred-gap closeout, migrations 00670069) shipped to local main:** M1 hermetic approval gate · M2 shared-queue dead-letter store · M3 email-secret AAD v0→v1 · M4 per-group KV/docs byte quotas · M5 `set_if` compare-and-swap for KV (per-app + shared + Rhai SDK) · M6 shared-topic external SSE. **Audit 2026-07-11 remediation** (migration 0070, admin-session absolute cap) also shipped. **With that, v1.2 _Hierarchies_ is complete.** The **Workflows** track then shipped too (M1M6: DAG execution + conditional `when`, nested sub-workflows, durable orchestrator, `workflow::start` SDK + admin run API + `pic workflows`, dashboard DAG + run-history; migrations `0071`/`0072`). A **§9.4 service-interceptor** thin slice then shipped (migration `0073_interceptors.sql`): a `[[interceptors]]` block (app or group) binds a script to run before `kv::set`/`delete` and allow/deny it, resolved nearest-owner-wins on the app chain and run via the `invoke()` re-entry path; the rest of §9.4 (data transform, non-kv services, after-hooks, chaining) is deferred. Next: the rest of §9.4 and multi-node cluster mode (the deferred multi-node route/broadcast propagation lives there). **Track A (v1.2 deferred-gap closeout, migrations 00670069) shipped to local main:** M1 hermetic approval gate · M2 shared-queue dead-letter store · M3 email-secret AAD v0→v1 · M4 per-group KV/docs byte quotas · M5 `set_if` compare-and-swap for KV (per-app + shared + Rhai SDK) · M6 shared-topic external SSE. **Audit 2026-07-11 remediation** (migration 0070, admin-session absolute cap) also shipped. **With that, v1.2 _Hierarchies_ is complete.** The **Workflows** track then shipped too (M1M6: DAG execution + conditional `when`, nested sub-workflows, durable orchestrator, `workflow::start` SDK + admin run API + `pic workflows`, dashboard DAG + run-history; migrations `0071`/`0072`). A **§9.4 service-interceptor** thin slice then shipped (migration `0073_interceptors.sql`): a `[[interceptors]]` block (app or group) binds a script to run before `kv::set`/`delete` and allow/deny it, resolved nearest-owner-wins on the app chain and run via the `invoke()` re-entry path; the rest of §9.4 (data transform, non-kv services, after-hooks, chaining) is deferred. Next: the rest of §9.4 and multi-node cluster mode (the deferred multi-node route/broadcast propagation lives there).
**Write-path invariant — the transactional outbox (`manager-core::atomic_write`).** A data-plane mutation (KV / docs / files, per-app and group-shared) and the trigger fan-out it produces commit in **ONE transaction on ONE connection**, via a `*Writer` injected into each service (`with_atomic_writes(pool)` in the host). The services previously wrote the row, waited for it to commit, and *then* asked the emitter to resolve triggers and insert outbox rows — a second transaction on a second connection, so an outbox failure left a committed row whose trigger never fired, invisible to the caller and unrecoverable by any retry. Now an emit failure **rolls the write back** and surfaces as an error. Three rules make it work:
- **Everything inside a transaction runs on that transaction's connection.** A writer that held a `tx` and then reached for a *second* pooled connection could deadlock — the pool is sized to the execution-concurrency cap, so N executions each wanting 2 connections starve. Hence `outbox_event_emitter::emit_on(&mut *tx, …)` and the `*_on(exec, …)` repo free fns (the trait methods delegate to them, so each query has one home). **`shared` has no sqlx and must not gain it** — the `ServiceEventEmitter` trait stays connection-free; the transactional path is entirely manager-core-internal.
- **Per-group quotas need a lock, not just a transaction.** Under READ COMMITTED each transaction's `COUNT(*)`/`SUM(...)` sees a snapshot *without* the other writers' uncommitted rows, so concurrent writers all pass the check anyway (measured: 19 rows stored against a ceiling of 5). Every group write therefore takes `pg_advisory_xact_lock` on a per-(group, kind) key first, serializing check-then-write. The row/byte policy itself is one shared fn, `group_quota::check_group_write`.
- **Files order around the disk write**, which cannot join a transaction: create/update write the blob *first* and unlink it if the tx rolls back; delete commits the metadata removal *first* and unlinks after (the reverse would destroy the bytes of a row a rollback keeps).
**Data-model invariant:** app-owned data-plane tables (KV, docs, files, …) start with `app_id UUID NOT NULL REFERENCES apps(id) ON DELETE CASCADE`; the group-inheritable tables — _config_ (`vars`, `secrets`) and now group-owned _code_ (`scripts`, `0050`) — instead carry a **polymorphic owner**: nullable `group_id` and `app_id` with an exactly-one CHECK and per-owner partial-unique indexes (config is `ON DELETE CASCADE`, scripts `RESTRICT` — code is not data). Inheritance resolves **live** down `apps.group_id → groups.parent_id` via `CHAIN_LEVELS_CTE` (no materialized view); nearest-owner-wins with an app's own row shadowing the inherited one (CoW). Every Rhai SDK call resolves its app from `cx.app_id`, never a script-passed arg, and a group script always runs under the *inheriting* app's `cx.app_id` (the cross-app isolation boundary). **Data-model invariant:** app-owned data-plane tables (KV, docs, files, …) start with `app_id UUID NOT NULL REFERENCES apps(id) ON DELETE CASCADE`; the group-inheritable tables — _config_ (`vars`, `secrets`) and now group-owned _code_ (`scripts`, `0050`) — instead carry a **polymorphic owner**: nullable `group_id` and `app_id` with an exactly-one CHECK and per-owner partial-unique indexes (config is `ON DELETE CASCADE`, scripts `RESTRICT` — code is not data). Inheritance resolves **live** down `apps.group_id → groups.parent_id` via `CHAIN_LEVELS_CTE` (no materialized view); nearest-owner-wins with an app's own row shadowing the inherited one (CoW). Every Rhai SDK call resolves its app from `cx.app_id`, never a script-passed arg, and a group script always runs under the *inheriting* app's `cx.app_id` (the cross-app isolation boundary).
## Three-Service Architecture ## Three-Service Architecture