On a reasoning model the reasoning is spent from the same completion budget
and it goes first, so a small ceiling truncates the actual reply away. It
fails silently and looks exactly like a model that cannot follow the system
prompt, which is the expensive way to debug it.
Measured on gpt-oss-20b against the real ~3.5k-token system prompt, same
prompt and same model, only the ceiling changing:
1400 an empty string, 0 bytes, after 27s
2048 a truncated half-Korean fragment, none of the required blocks
8000 99% English prose, no romanization, 0 batchim violations in the
task lines, all three blocks, 11s
The old comment justified 2048 by worrying about 4k-context local models.
That reasoning was wrong: the system prompt alone is ~3.5k tokens, so such
a model cannot run this app at all and there was nothing to protect.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
HANKAN_TUTOR_BACKEND=openai talks to anything serving
/chat/completions -- LM Studio, Ollama, llama.cpp, vLLM, LiteLLM,
OpenRouter, OpenAI. The TutorBackend seam already existed for this, so the
model becomes a config line rather than a code change.
Written against fetch rather than the openai package. The Anthropic SDK
alone is 14MB in the image, this backend uses one endpoint with no tools
and no retries, and local servers are the ones most likely to deviate from
an SDK's expectations. The real risk in hand-rolling it is SSE reassembly,
so that is where the tests are: a JSON payload split across two TCP reads,
an event whose blank-line terminator lands in the next read, heartbeat
comments, CRLF framing, and a stream that ends without [DONE]. The two
split cases both fail against a naive per-read parser, which is what makes
them worth having.
<think> blocks are stripped from the stream, tags split across chunks
included. Reasoning models served locally often emit chain-of-thought
inline in `content` rather than in a separate field, and left in it lands
in the lesson transcript where the block parser reads it as prose.
WHAT THIS COSTS: prompt caching. The Anthropic backend marks the ~12k
character gate as a cached prefix, so every turn after the first reads it
at a fraction of the input price. There is no portable equivalent, so
against a paid hosted endpoint the system prompt is re-billed every turn --
the biggest cost lever in the design, gone.
Against a local model it costs nothing, and the shape still pays: llama.cpp
and LM Studio reuse their KV cache for an unchanged prefix. Measured on a
6,948-character prompt against gpt-oss-20b, first token 1,563ms cold and
324ms warm, so the system prompt goes first and stays put here too.
Verified against LM Studio running openai/gpt-oss-20b, not only a fake: a
turn streams from the browser through this server to the model and back,
rendered in the chat, no page errors.
Also makes test/server/http.test.ts backend-agnostic. It asserted the echo
backend's wording and so failed the moment the server was pointed at a real
model -- precisely the case a transport test should survive.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The Android build always talks to the Pi cross-origin -- a Capacitor
webview serves the app from http://localhost, not from your domain -- so
the browser sends a preflight OPTIONS first. It is not permitted to attach
an Authorization header to that. Auth ran before anything else, so the
preflight came back 401 and the real request was never attempted.
There were no Access-Control-Allow-* headers either, so even a successful
preflight would not have helped. Verified from an actual page before the
fix: GET /api/sync and POST /api/tutor both "Failed to fetch" -- an opaque
network error that points at the network rather than at middleware order.
CORS now runs first and answers OPTIONS itself. Any origin is allowed by
default, which is not a hole: the gate is a bearer token rather than a
cookie, so a hostile page gains nothing from being allowed to send a
request it cannot authenticate. HANKAN_ALLOWED_ORIGINS narrows it.
backends/echo.ts is a keyless backend that reflects the request back in
chunks. Deploying involves a container, a reverse proxy, a token, CORS and
an SSE stream that has to survive compression -- five things that break
independently, none of which involve Anthropic. HANKAN_TUTOR_BACKEND=echo
proves all five from the phone before a key exists and before anything is
billed. CI now runs the server that way, so the tutor endpoint is
exercised over real HTTP rather than only against an injected mock.
test/server/http.test.ts covers the preflight, the allow-origin header on
real responses, Vary: Origin, that a bad token is still refused, and that
the SSE stream parses and terminates with a done event.
Verified end to end in a browser: the Pi configured through the settings
panel, sync pushing 2 rows and a second sync moving 0 (the pushedAt
watermark holding), the header switching from "local stand-in" to
"connected", and a turn streaming back over SSE with the 8,859-character
system prompt intact.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
POST /api/tutor takes the assembled system prompt, the transcript the
client owns, and the new message, and streams tokens back. It holds
nothing between requests, so a dropped connection costs one turn rather
than the conversation.
The seam moved first: Sample took the assembled prompt as messages[0] with
role user. It now has an explicit system field, which is what lets the
backend put it in the API's system parameter as a cached block. The gate
is ~12k characters and is byte-identical for as long as the learner stays
in one unit, so every turn after the first reads the prefix at a fraction
of the input price. That is the single biggest cost lever in the design,
and it was unreachable through the old shape.
prompt/tutor-system.md still ships unchanged; only where the string is
placed changed.
SSE has three rules that are silent when broken, and all three are
handled: every event ends with a blank line, payloads are JSON-encoded
because a raw newline in Korean text would break the framing, and a `:`
heartbeat every 15s keeps intermediaries from timing the stream out.
Cache-Control is set on the returned Response rather than inside
streamSSE, which writes its own and would overwrite it; `no-transform` is
there because compression, not buffering, is what usually makes SSE look
like it hangs behind a proxy.
The client uses fetch + getReader, not EventSource — EventSource cannot
POST, and the body is {system, history, message}. Aborting closes the
connection, the server aborts upstream, and a cancelled turn stops
billing. With no server configured the app falls back to the stub, so the
offline build is untouched.
backends/anthropic.ts is the default. backends/agent-sdk.ts is deliberately
unimplemented and documents why the plain API was chosen over PORT.md's
Agent SDK — chiefly that its prompt accepts only user-role messages, so
the transcript would have to be flattened into a single turn.
The endpoint's own tests use a mock backend and need no API key.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Hono and pg, run under --experimental-strip-types, so the deployed thing
is the source. GET /api/sync?cursor=N pages rows above the cursor;
POST /api/sync upserts last-write-wins. Bearer token on everything under
/api; /health is open, for the container healthcheck.
Rows are stored generically — primary key as text, body as JSONB —
because the server never reads inside a row. It stores and orders them and
the client interprets them, which keeps the two schemas from having to
move in lockstep.
change_seq is bumped by a BEFORE UPDATE trigger rather than by the write
path. A row edited after a client last pulled would otherwise keep its old
sequence, sit below that client's cursor, and never be delivered; putting
it in the database means no future write path can forget.
The last-write-wins comparison is in the ON CONFLICT clause itself, so a
losing row is not written at all and does not bump change_seq — a
conflict does not become traffic for every other device.
test/sync/roundtrip.test.ts runs two clients against a real Postgres and
asserts what actually goes wrong in sync: that a fresh client's seeded rows
cannot overwrite the server's history (the artifact's bug, as an executable
test), that a delete propagates, and that dict.loadedBands never crosses
the wire. It skips without HANKAN_TEST_SERVER, so npm test still runs
anywhere.
POST /api/test/reset exists only when HANKAN_TEST_MODE=1, so it cannot be
reached on the Pi even if the token leaks.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
