fix(server): default max_tokens 2048 -> 8192, which was starving the lesson

On a reasoning model the reasoning is spent from the same completion budget
and it goes first, so a small ceiling truncates the actual reply away. It
fails silently and looks exactly like a model that cannot follow the system
prompt, which is the expensive way to debug it.

Measured on gpt-oss-20b against the real ~3.5k-token system prompt, same
prompt and same model, only the ceiling changing:

  1400  an empty string, 0 bytes, after 27s
  2048  a truncated half-Korean fragment, none of the required blocks
  8000  99% English prose, no romanization, 0 batchim violations in the
        task lines, all three blocks, 11s

The old comment justified 2048 by worrying about 4k-context local models.
That reasoning was wrong: the system prompt alone is ~3.5k tokens, so such
a model cannot run this app at all and there was nothing to protect.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
MechaCat02
2026-09-10 07:17:16 +02:00
parent 9a925a5e3f
commit 988d33bd5f
3 changed files with 28 additions and 7 deletions

View File

@@ -28,9 +28,11 @@ HANKAN_OPENAI_BASE_URL=http://host.docker.internal:1234/v1
HANKAN_OPENAI_MODEL=local-model
# Local servers ignore this; hosted ones require it. Leave blank for local.
HANKAN_OPENAI_API_KEY=
# Completion ceiling. Kept modest because a small-context local model errors
# outright if asked for more than its context holds.
HANKAN_OPENAI_MAX_TOKENS=2048
# Completion ceiling. Do not lower this much: on a reasoning model the
# reasoning comes out of the same budget and goes first, so a small value
# truncates the actual lesson away. gpt-oss-20b returns an empty string at
# 1400 and a broken half-Korean fragment at 2048.
HANKAN_OPENAI_MAX_TOKENS=8192
# The docker network your existing Postgres and Caddy are on.
# docker network ls