Audit #6 and #8 for the last of the three stores, plus a bypass found on
the way.
**#6.** create/update/delete wrote the metadata row, then emitted
best-effort, so an outbox failure left a committed file whose trigger never
fired. `atomic_write::FilesWriter` commits the metadata row and the fan-out
together.
Files are the one store where the ordering is subtle, because the BYTES live
on disk and cannot join a transaction:
* create/update — blob first, then commit metadata + fan-out. A rollback
unlinks the blob. (A crash at that exact point still orphans it; that
hazard predates this change — the repo already wrote the blob and then
inserted the row in a separate, failable statement — and the orphan is
inert, referenced by nothing.)
* delete — commit the metadata removal + fan-out FIRST, then unlink. The
reverse order would destroy the bytes of a row that a rollback keeps,
leaving a file that can never be read.
**#8.** `GroupFilesService::create` read `total_bytes` on one connection and
wrote on another, so concurrent uploads each saw the same pre-write total and
together overshot the ceiling. This is the worst instance of the race in the
codebase: the ceiling is DISK (10 GiB by default) and one file may be 100 MB,
so a racing fleet overshoots by gigabytes. `PostgresGroupFilesWriter` takes
the per-group advisory lock (on its own `files` key) across the check and the
write.
**The bypass.** `GroupFilesService::update` checked NO quota at all — so a
1-byte file could be updated to a 100 MB one without the ceiling ever being
consulted, repeatedly, for unbounded disk. It now checks the projected total
(the replaced file's bytes subtracted in SQL, so a same-size-or-smaller
update near the cap still goes through).
Also drive-by: `queue_e2e` asserted the ack the instant the marker appeared,
but the marker is written DURING the handler and the ack happens after it
returns — a zero-tolerance race. It polls now. (This does not fix the
suite's flakiness, which reproduces on the pre-pass commit too.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Global env-var ceilings enforced in the group write path (mirrors the
per-value caps): PICLOUD_GROUP_KV_MAX_ROWS / _DOCS_MAX_ROWS (per-group row
count, checked only on a NEW key/doc — updates exempt) and
_FILES_MAX_TOTAL_BYTES (per-group total blob bytes). New group_quota env
helpers; count_rows/total_bytes repo methods (trait-default 0 for non-Postgres);
QuotaExceeded error variants on the three group error enums; a with_max_rows
test builder. Pinned by a group_kv_service unit test (3rd key rejected, update
of an existing key at the cap allowed).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extends the §11.6 shared-collection machinery to the filesystem-backed
`files` blob store (after KV/0053 and docs/0054).
- 0055_group_files.sql: widen the group_collections kind CHECK to admit
'files'; add a group-keyed `group_files` metadata table mirroring
`files` (0018), CASCADE on group delete.
- files_repo: generalize the four path/IO free functions
(shard_dir_at / final_path_at / write_atomic_at / read_verify_at) to
take an owner-relative dir instead of `app_id`, and make them (plus the
cursor helpers) pub(crate). The security-sensitive atomic-write +
checksum-on-read mechanics now have a single source shared by the app
and group repos. App behavior is unchanged (apps shard at `<app_id>/`).
- group_files_repo: FsGroupFilesRepo keyed by group_id, blobs at
`<root>/files/groups/<group_id>/<collection>/<id[0:2]>/<id>` — a
`groups/` infix disjoint from the per-app subtree, so the existing
recursive orphan sweeper covers it with zero change.
Schema snapshot blessed (55 migrations); app-files fs tests still green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
