An app's OWN store had no ceiling of any kind — only the group-SHARED
collections did, which is the inversion of what you'd expect. And
`authz::script_gate` returns `Ok(())` when there is no principal, so a
PUBLIC UNAUTHENTICATED route could reach `files::collection(..).create(..)`
and write blobs in a loop until the disk filled. `PICLOUD_FILES_MAX_FILE_SIZE_BYTES`
caps ONE file at 100 MB; nothing capped the count. Same for KV keys and docs
against Postgres.
Ceilings: `PICLOUD_APP_KV_MAX_ROWS` / `PICLOUD_APP_DOCS_MAX_ROWS` (100k) and
`PICLOUD_APP_FILES_MAX_TOTAL_BYTES` (10 GiB).
They are enforced DIFFERENTLY from the group ones, on purpose — porting the
group design as-is would have been a serious regression:
* KV/docs check a ROW count, on INSERT only, with NO lock. `kv::set` is the
hottest write path in the system; taking a per-app advisory lock on every
set would serialize an app's entire data plane, and a `SUM(...)` scan would
make write cost grow with the app's stored size. What the lock buys is
small — unlocked overshoot is bounded by write concurrency (32) against a
ceiling of 100k, ~0.03%. These are anti-DoS rails, not billing. An update
adds no row and is bounded by the per-value cap, so it pays nothing at all.
* No per-app BYTE ceiling for KV/docs: every value is already capped at
`PICLOUD_KV_MAX_VALUE_BYTES`, so `max_rows x max_value_bytes` is ALREADY a
finite bound. A second scan would buy nothing.
* FILES are the exception and DO lock + sum: one blob may be 100 MB, so 32
racing uploads could overshoot by gigabytes of real disk, and a file write
is heavy enough that the lock and the SUM are lost in the noise. Checked on
create AND update — an update that skipped the ceiling would be a free
bypass (the same bug just fixed for group files: grow a 1-byte file to
100 MB, repeat).
`group_quota` is renamed `quota`: it now serves both owners, and the module
doc is where the two enforcement strategies are contrasted.
Pinned by tests/atomic_write.rs: the key ceiling refuses a new key while
still allowing an update at the ceiling (rejecting updates would brick an app
the moment it filled up — worse than the DoS the rail exists to stop), and 10
concurrent uploads cannot exceed the disk ceiling, nor can an update grow past it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
270 lines
9.5 KiB
Rust
270 lines
9.5 KiB
Rust
//! `DocsService` — the v1.1.2 schemaless document store contract.
|
|
//!
|
|
//! Lives in `picloud-shared` (not `executor-core`) for the same reason
|
|
//! `KvService` does: the Rhai bridge, the manager-core Postgres impl,
|
|
//! and any future in-memory test impl all depend on the same trait
|
|
//! without dragging `executor-core` into `manager-core`'s dep graph.
|
|
//!
|
|
//! Implementations MUST derive every storage `app_id` from `cx.app_id`
|
|
//! — never from a script-passed argument. That is the cross-app
|
|
//! isolation boundary; see `docs/sdk-shape.md`.
|
|
//!
|
|
//! Filter shape (per `docs::find` / `find_one`) is an opaque
|
|
//! `serde_json::Value` at this layer; the manager-core implementation
|
|
//! parses it into a structured DSL with explicit operator allowlist
|
|
//! before touching SQL. Parser errors surface as
|
|
//! `DocsError::InvalidFilter` / `DocsError::UnsupportedOperator` so
|
|
//! scripts get a clear message naming the offending key.
|
|
|
|
use async_trait::async_trait;
|
|
use chrono::{DateTime, Utc};
|
|
use thiserror::Error;
|
|
use uuid::Uuid;
|
|
|
|
use crate::SdkCallCx;
|
|
|
|
/// Server-generated document identifier. Scripts see the `to_string()`
|
|
/// form as a Rhai string; the trait surface keeps the typed `Uuid` so
|
|
/// no implementation accidentally accepts a string-shaped path
|
|
/// parameter from a script.
|
|
pub type DocId = Uuid;
|
|
|
|
/// One document as returned by `get` / `find` / `find_one`. The
|
|
/// envelope shape (decision D from the v1.1.2 plan): explicit
|
|
/// `id`+`data`+timestamps so user fields and platform metadata can't
|
|
/// alias. Scripts read user fields via `doc.data.<field>`; timestamps
|
|
/// + id are direct children.
|
|
#[derive(Debug, Clone, PartialEq)]
|
|
pub struct DocRow {
|
|
pub id: DocId,
|
|
pub data: serde_json::Value,
|
|
pub created_at: DateTime<Utc>,
|
|
pub updated_at: DateTime<Utc>,
|
|
}
|
|
|
|
/// One page of `list`. `next_cursor` is `Some` when more pages exist,
|
|
/// `None` when exhausted. Mirrors `KvListPage`'s shape; the cursor
|
|
/// encoding is implementation-defined (the Postgres impl base64-encodes
|
|
/// the last id).
|
|
#[derive(Debug, Clone)]
|
|
pub struct DocsListPage {
|
|
pub docs: Vec<DocRow>,
|
|
pub next_cursor: Option<String>,
|
|
}
|
|
|
|
/// Collection-scoped CRUD + cursor list + filter-based find.
|
|
///
|
|
/// Method shapes mirror `KvService`'s signature style (each takes
|
|
/// `&SdkCallCx` first non-self). The collection name is passed by
|
|
/// reference; the implementation rejects empty/whitespace-only
|
|
/// collections at the SDK boundary per `docs/sdk-shape.md`.
|
|
///
|
|
/// `find` and `find_one` take the filter as `serde_json::Value` — the
|
|
/// service implementation parses it into a structured AST. Keeping the
|
|
/// trait signature untyped here lets the bridge convert
|
|
/// `Rhai Map → serde_json::Value` and hand it off without dragging the
|
|
/// parser into the shared crate.
|
|
#[async_trait]
|
|
pub trait DocsService: Send + Sync {
|
|
/// Create a new document with a server-generated UUID. Returns the
|
|
/// new id so the script can read/update/delete it later. The
|
|
/// document `data` must be a JSON object.
|
|
async fn create(
|
|
&self,
|
|
cx: &SdkCallCx,
|
|
collection: &str,
|
|
data: serde_json::Value,
|
|
) -> Result<DocId, DocsError>;
|
|
|
|
/// Fetch one document by id. Returns `None` for missing — the
|
|
/// bridge maps that to Rhai's `()`.
|
|
async fn get(
|
|
&self,
|
|
cx: &SdkCallCx,
|
|
collection: &str,
|
|
id: DocId,
|
|
) -> Result<Option<DocRow>, DocsError>;
|
|
|
|
/// Filter-based query. Returns every matching document as a
|
|
/// `Vec<DocRow>` (empty when no matches). The filter is the
|
|
/// v1.1.2 query DSL shape — see `manager-core::docs_filter` for
|
|
/// the parser. Throws `InvalidFilter` / `UnsupportedOperator` on
|
|
/// parse errors.
|
|
async fn find(
|
|
&self,
|
|
cx: &SdkCallCx,
|
|
collection: &str,
|
|
filter: serde_json::Value,
|
|
) -> Result<Vec<DocRow>, DocsError>;
|
|
|
|
/// Single-result variant — equivalent to `find` with `$limit: 1`
|
|
/// then take-first. Returns `None` when no document matches.
|
|
async fn find_one(
|
|
&self,
|
|
cx: &SdkCallCx,
|
|
collection: &str,
|
|
filter: serde_json::Value,
|
|
) -> Result<Option<DocRow>, DocsError>;
|
|
|
|
/// Full document replace. v1.1.2 has no partial-update DSL —
|
|
/// scripts that want partial update do `get + modify + update`.
|
|
/// Returns `DocsError::NotFound` if no such doc; otherwise emits
|
|
/// an `update` ServiceEvent with `prev_data` and `data`.
|
|
async fn update(
|
|
&self,
|
|
cx: &SdkCallCx,
|
|
collection: &str,
|
|
id: DocId,
|
|
data: serde_json::Value,
|
|
) -> Result<(), DocsError>;
|
|
|
|
/// Delete by id. Returns `bool was-present` (matches the `delete`
|
|
/// shape of every v1.1.x service). Emits a `delete` ServiceEvent
|
|
/// with `prev_data: Some(deleted_doc.data)` when the doc existed.
|
|
async fn delete(&self, cx: &SdkCallCx, collection: &str, id: DocId) -> Result<bool, DocsError>;
|
|
|
|
/// Cursor-paginated listing of every doc in the collection,
|
|
/// ordered by `id ASC` for stable cursor encoding. `None` cursor
|
|
/// starts from the beginning. Implementations cap `limit` at a
|
|
/// reasonable ceiling internally.
|
|
async fn list(
|
|
&self,
|
|
cx: &SdkCallCx,
|
|
collection: &str,
|
|
cursor: Option<&str>,
|
|
limit: u32,
|
|
) -> Result<DocsListPage, DocsError>;
|
|
}
|
|
|
|
/// Stub for tests that build a `Services` bundle without spinning up
|
|
/// Postgres. Every call returns `DocsError::Backend("...")` so
|
|
/// accidental docs use surfaces clearly. Mirrors `NoopKvService`.
|
|
#[derive(Debug, Default, Clone, Copy)]
|
|
pub struct NoopDocsService;
|
|
|
|
#[async_trait]
|
|
impl DocsService for NoopDocsService {
|
|
async fn create(
|
|
&self,
|
|
_cx: &SdkCallCx,
|
|
_collection: &str,
|
|
_data: serde_json::Value,
|
|
) -> Result<DocId, DocsError> {
|
|
Err(DocsError::Backend("docs is not wired in".into()))
|
|
}
|
|
|
|
async fn get(
|
|
&self,
|
|
_cx: &SdkCallCx,
|
|
_collection: &str,
|
|
_id: DocId,
|
|
) -> Result<Option<DocRow>, DocsError> {
|
|
Err(DocsError::Backend("docs is not wired in".into()))
|
|
}
|
|
|
|
async fn find(
|
|
&self,
|
|
_cx: &SdkCallCx,
|
|
_collection: &str,
|
|
_filter: serde_json::Value,
|
|
) -> Result<Vec<DocRow>, DocsError> {
|
|
Err(DocsError::Backend("docs is not wired in".into()))
|
|
}
|
|
|
|
async fn find_one(
|
|
&self,
|
|
_cx: &SdkCallCx,
|
|
_collection: &str,
|
|
_filter: serde_json::Value,
|
|
) -> Result<Option<DocRow>, DocsError> {
|
|
Err(DocsError::Backend("docs is not wired in".into()))
|
|
}
|
|
|
|
async fn update(
|
|
&self,
|
|
_cx: &SdkCallCx,
|
|
_collection: &str,
|
|
_id: DocId,
|
|
_data: serde_json::Value,
|
|
) -> Result<(), DocsError> {
|
|
Err(DocsError::Backend("docs is not wired in".into()))
|
|
}
|
|
|
|
async fn delete(
|
|
&self,
|
|
_cx: &SdkCallCx,
|
|
_collection: &str,
|
|
_id: DocId,
|
|
) -> Result<bool, DocsError> {
|
|
Err(DocsError::Backend("docs is not wired in".into()))
|
|
}
|
|
|
|
async fn list(
|
|
&self,
|
|
_cx: &SdkCallCx,
|
|
_collection: &str,
|
|
_cursor: Option<&str>,
|
|
_limit: u32,
|
|
) -> Result<DocsListPage, DocsError> {
|
|
Err(DocsError::Backend("docs is not wired in".into()))
|
|
}
|
|
}
|
|
|
|
/// Failure modes surfaced to the Rhai bridge. The bridge converts each
|
|
/// to a Rhai runtime error string; the discriminants exist so internal
|
|
/// callers (admin endpoints, tests) can react more precisely.
|
|
#[derive(Debug, Error)]
|
|
pub enum DocsError {
|
|
/// Empty collection name; rejected at the SDK boundary per
|
|
/// `docs/sdk-shape.md`.
|
|
#[error("collection name must not be empty")]
|
|
InvalidCollection,
|
|
|
|
/// The app is at its per-app doc ceiling (`PICLOUD_APP_DOCS_MAX_ROWS`) and
|
|
/// this write would add a doc. An update is exempt (net-zero rows).
|
|
#[error("docs: app doc quota exceeded ({actual} docs; limit {limit})")]
|
|
QuotaExceeded { limit: u64, actual: u64 },
|
|
|
|
/// `create`/`update` was handed a non-object JSON value (data must
|
|
/// be a JSON object so it can be navigated by field paths in
|
|
/// queries).
|
|
#[error("document data must be a JSON object")]
|
|
InvalidData,
|
|
|
|
/// Parser rejected the filter — bad path syntax, malformed
|
|
/// operator value, multi-field `$sort`, etc. The string is the
|
|
/// script-visible message; it becomes part of the SDK contract
|
|
/// once a script depends on it.
|
|
#[error("invalid filter: {0}")]
|
|
InvalidFilter(String),
|
|
|
|
/// Filter used an operator that's not in the v1.1.2 allowlist
|
|
/// (`$or`, `$regex`, `$exists`, …). String includes the offending
|
|
/// operator name + v1.2 pointer.
|
|
#[error("unsupported operator: {0}")]
|
|
UnsupportedOperator(String),
|
|
|
|
/// `update` / `delete` target id does not exist. (`delete` returns
|
|
/// `Ok(false)` for "missing"; this variant is for `update` and any
|
|
/// future delete-must-exist callers.)
|
|
#[error("document not found")]
|
|
NotFound,
|
|
|
|
/// Caller principal lacked the required capability. Only raised
|
|
/// when `cx.principal.is_some()` — scripts running with
|
|
/// `principal: None` (public HTTP) operate under script-as-gate
|
|
/// semantics and skip the capability check.
|
|
#[error("forbidden")]
|
|
Forbidden,
|
|
|
|
/// JSON-encoded document data exceeded the per-app `max_value_bytes`
|
|
/// cap. Configured via `PICLOUD_DOCS_MAX_VALUE_BYTES`.
|
|
#[error("docs: data too large ({actual} bytes; limit {limit})")]
|
|
ValueTooLarge { limit: usize, actual: usize },
|
|
|
|
/// Anything else — Postgres unavailable, serialization failure,
|
|
/// etc. The string is safe to surface to a script.
|
|
#[error("docs backend error: {0}")]
|
|
Backend(String),
|
|
}
|