fix(analysis): guardrails against model repetition loops
Small vision models (qwen3-vl-4b) sometimes loop the OCR array, running the response into the token ceiling (finish_reason: length) and failing to parse. Layered guardrails: - Prompts: explicit "transcribe each element once, never repeat/loop, at most N, STOP when done" in the OCR, grounding and combined prompts. - Schema: hard maxItems on ocr_results/tagging_results/content_type and maxLength on text/scene — LM Studio's grammar enforces these, so a loop is grammar-bounded rather than relying on max_tokens. - Sampling: a configurable frequency_penalty (ANALYSIS_FREQUENCY_PENALTY, default 0.3) sent with each request — the decode-time lever that actually breaks loops. - Code: sanitize() collapses runaway consecutive duplicates (same normalized text + kind) so a loop that slips through still can't flood the row. Tests: schema caps present, frequency_penalty included only when nonzero, sanitize collapses consecutive repeats; config default. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -141,6 +141,11 @@ pub struct AnalysisConfig {
|
||||
/// Output-constraint mode (`ANALYSIS_RESPONSE_FORMAT`):
|
||||
/// `json_schema` (default) | `json_object` | `none`.
|
||||
pub response_format: ResponseFormat,
|
||||
/// Sampling `frequency_penalty` sent with each request
|
||||
/// (`ANALYSIS_FREQUENCY_PENALTY`). A small positive value discourages
|
||||
/// the repetition loops that otherwise run small models into the token
|
||||
/// ceiling. `0` omits the field.
|
||||
pub frequency_penalty: f64,
|
||||
}
|
||||
|
||||
impl Default for AnalysisConfig {
|
||||
@@ -164,6 +169,7 @@ impl Default for AnalysisConfig {
|
||||
max_slices: 16,
|
||||
max_image_bytes: 8 * 1024 * 1024,
|
||||
response_format: ResponseFormat::JsonSchema,
|
||||
frequency_penalty: 0.3,
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -199,6 +205,7 @@ impl AnalysisConfig {
|
||||
response_format: std::env::var("ANALYSIS_RESPONSE_FORMAT")
|
||||
.map(|s| ResponseFormat::from_str(&s))
|
||||
.unwrap_or(d.response_format),
|
||||
frequency_penalty: env_f64("ANALYSIS_FREQUENCY_PENALTY", d.frequency_penalty),
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -619,6 +626,7 @@ mod tests {
|
||||
"ANALYSIS_TALL_ASPECT",
|
||||
"ANALYSIS_MAX_SLICES",
|
||||
"ANALYSIS_RESPONSE_FORMAT",
|
||||
"ANALYSIS_FREQUENCY_PENALTY",
|
||||
] {
|
||||
std::env::remove_var(k);
|
||||
}
|
||||
@@ -629,6 +637,7 @@ mod tests {
|
||||
assert_eq!(cfg.min_slice_height, 640);
|
||||
assert_eq!(cfg.max_slices, 16);
|
||||
assert_eq!(cfg.tall_aspect_threshold, 1.6);
|
||||
assert_eq!(cfg.frequency_penalty, 0.3);
|
||||
assert!(cfg.api_key.is_none());
|
||||
assert_eq!(cfg.response_format, ResponseFormat::JsonSchema);
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user