fabi/PiCloud
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(executor): bound Rhai→JSON materialization to prevent anonymous OOM

Browse Source 
The per-element Rhai sandbox caps (max_string_size 64 KiB, max_array_size /
max_map_size 10 000) do NOT bound the *materialized* size: Rhai shares strings
and arrays by Rc, so a 10 000-element array of one aliased 64 KiB string is
cheap to build (~10 000 ops, far under the 1 M op budget) yet dealiases to
~640 MiB of distinct JSON. Every Dynamic→JSON conversion deep-copies the
aliases; the HTTP response body path had NO size cap at all, and the KV/docs
value caps run only AFTER full materialization — so a handful of anonymous
requests could OOM the node (32 concurrent × ~640 MiB ≈ 20 GiB) on the stated
consumer-hardware target.

Add a byte-budgeted materializer that bails the moment the budget is exceeded,
so the transient allocation is bounded regardless of aliasing:
- bridge.rs: `dynamic_to_json_capped(value, max) -> Result<Json, JsonSizeError>`
  (charges a running budget) + `MAX_JSON_MATERIALIZE_BYTES` (16 MiB hard rail,
  far above the 256 KiB business caps). `dynamic_to_json` stays infallible for
  best-effort paths (logging) but is now internally bounded — it collapses an
  over-limit value to a marker string instead of OOMing.
- Route every user-value boundary through the fail-closed capped form: KV
  set/set_if (+ the CAS `expected`), docs create/update/find, secrets set,
  http request body, workflow input, `json::stringify`, and the HTTP RESPONSE
  body (previously the one fully-uncapped exit → now a 500). `invoke` args and
  the pubsub/queue message materializers get the same byte budget in place.
- `impl From<JsonSizeError> for Box<EvalAltResult>` so SDK sites protect
  themselves with a bare `?`.

Pinned by bridge unit tests: an aliased 10 000×64 KiB array errors on the
budget rather than materializing, and the infallible wrapper yields a marker
instead of OOMing. Workspace 914 + journeys 157/157 green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
MechaCat02
2026-07-13 21:50:16 +02:00
parent d08df88df5
commit bdcd3dc7f4
12 changed files with 363 additions and 80 deletions
Download Patch File Download Diff File Expand all files Collapse all files

20
crates/executor-core/src/sdk/docs.rs
View File

@@ -27,7 +27,9 @@ use picloud_shared::{DocId, DocRow, DocsService, GroupDocsService, SdkCallCx, Se
use rhai::{Array, Dynamic, Engine as RhaiEngine, EvalAltResult, Map, Module};
use uuid::Uuid;
use super::bridge::{block_on, dynamic_to_json, json_to_dynamic};
use super::bridge::{
block_on, dynamic_to_json_capped, json_to_dynamic, MAX_JSON_MATERIALIZE_BYTES,
};
/// Per-call handle captured by the Rhai SDK. Cheap to clone (two Arcs
/// plus an owned string).
@@ -117,7 +119,7 @@ fn register_create(engine: &mut RhaiEngine) {
"create",
|handle: &mut DocsHandle, data: Map| -> Result<String, Box<EvalAltResult>> {
let h = handle.clone();
let json = dynamic_to_json(&Dynamic::from(data));
let json = dynamic_to_json_capped(&Dynamic::from(data), MAX_JSON_MATERIALIZE_BYTES)?;
let id = block_on("docs", async move {
h.service.create(&h.cx, &h.collection, json).await
})?;
@@ -145,7 +147,7 @@ fn register_find(engine: &mut RhaiEngine) {
"find",
|handle: &mut DocsHandle, filter: Map| -> Result<Array, Box<EvalAltResult>> {
let h = handle.clone();
let json = dynamic_to_json(&Dynamic::from(filter));
let json = dynamic_to_json_capped(&Dynamic::from(filter), MAX_JSON_MATERIALIZE_BYTES)?;
let rows = block_on("docs", async move {
h.service.find(&h.cx, &h.collection, json).await
})?;
@@ -162,7 +164,7 @@ fn register_find_one(engine: &mut RhaiEngine) {
"find_one",
|handle: &mut DocsHandle, filter: Map| -> Result<Dynamic, Box<EvalAltResult>> {
let h = handle.clone();
let json = dynamic_to_json(&Dynamic::from(filter));
let json = dynamic_to_json_capped(&Dynamic::from(filter), MAX_JSON_MATERIALIZE_BYTES)?;
let row = block_on("docs", async move {
h.service.find_one(&h.cx, &h.collection, json).await
})?;
@@ -177,7 +179,7 @@ fn register_update(engine: &mut RhaiEngine) {
|handle: &mut DocsHandle, id: &str, data: Map| -> Result<(), Box<EvalAltResult>> {
let h = handle.clone();
let parsed_id = parse_doc_id(id)?;
let json = dynamic_to_json(&Dynamic::from(data));
let json = dynamic_to_json_capped(&Dynamic::from(data), MAX_JSON_MATERIALIZE_BYTES)?;
block_on("docs", async move {
h.service
.update(&h.cx, &h.collection, parsed_id, json)
@@ -265,7 +267,7 @@ fn register_group_create(engine: &mut RhaiEngine) {
"create",
|handle: &mut GroupDocsHandle, data: Map| -> Result<String, Box<EvalAltResult>> {
let h = handle.clone();
let json = dynamic_to_json(&Dynamic::from(data));
let json = dynamic_to_json_capped(&Dynamic::from(data), MAX_JSON_MATERIALIZE_BYTES)?;
let id = block_on("docs", async move {
h.service.create(&h.cx, &h.collection, json).await
})?;
@@ -293,7 +295,7 @@ fn register_group_find(engine: &mut RhaiEngine) {
"find",
|handle: &mut GroupDocsHandle, filter: Map| -> Result<Array, Box<EvalAltResult>> {
let h = handle.clone();
let json = dynamic_to_json(&Dynamic::from(filter));
let json = dynamic_to_json_capped(&Dynamic::from(filter), MAX_JSON_MATERIALIZE_BYTES)?;
let rows = block_on("docs", async move {
h.service.find(&h.cx, &h.collection, json).await
})?;
@@ -310,7 +312,7 @@ fn register_group_find_one(engine: &mut RhaiEngine) {
"find_one",
|handle: &mut GroupDocsHandle, filter: Map| -> Result<Dynamic, Box<EvalAltResult>> {
let h = handle.clone();
let json = dynamic_to_json(&Dynamic::from(filter));
let json = dynamic_to_json_capped(&Dynamic::from(filter), MAX_JSON_MATERIALIZE_BYTES)?;
let row = block_on("docs", async move {
h.service.find_one(&h.cx, &h.collection, json).await
})?;
@@ -325,7 +327,7 @@ fn register_group_update(engine: &mut RhaiEngine) {
|handle: &mut GroupDocsHandle, id: &str, data: Map| -> Result<(), Box<EvalAltResult>> {
let h = handle.clone();
let parsed_id = parse_doc_id(id)?;
let json = dynamic_to_json(&Dynamic::from(data));
let json = dynamic_to_json_capped(&Dynamic::from(data), MAX_JSON_MATERIALIZE_BYTES)?;
block_on("docs", async move {
h.service
.update(&h.cx, &h.collection, parsed_id, json)