074f602494
HANKAN_TUTOR_BACKEND=openai talks to anything serving /chat/completions -- LM Studio, Ollama, llama.cpp, vLLM, LiteLLM, OpenRouter, OpenAI. The TutorBackend seam already existed for this, so the model becomes a config line rather than a code change. Written against fetch rather than the openai package. The Anthropic SDK alone is 14MB in the image, this backend uses one endpoint with no tools and no retries, and local servers are the ones most likely to deviate from an SDK's expectations. The real risk in hand-rolling it is SSE reassembly, so that is where the tests are: a JSON payload split across two TCP reads, an event whose blank-line terminator lands in the next read, heartbeat comments, CRLF framing, and a stream that ends without [DONE]. The two split cases both fail against a naive per-read parser, which is what makes them worth having. <think> blocks are stripped from the stream, tags split across chunks included. Reasoning models served locally often emit chain-of-thought inline in `content` rather than in a separate field, and left in it lands in the lesson transcript where the block parser reads it as prose. WHAT THIS COSTS: prompt caching. The Anthropic backend marks the ~12k character gate as a cached prefix, so every turn after the first reads it at a fraction of the input price. There is no portable equivalent, so against a paid hosted endpoint the system prompt is re-billed every turn -- the biggest cost lever in the design, gone. Against a local model it costs nothing, and the shape still pays: llama.cpp and LM Studio reuse their KV cache for an unchanged prefix. Measured on a 6,948-character prompt against gpt-oss-20b, first token 1,563ms cold and 324ms warm, so the system prompt goes first and stays put here too. Verified against LM Studio running openai/gpt-oss-20b, not only a fake: a turn streams from the browser through this server to the model and back, rendered in the chat, no page errors. Also makes test/server/http.test.ts backend-agnostic. It asserted the echo backend's wording and so failed the moment the server was pointed at a real model -- precisely the case a transport test should survive. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
38 lines
1.5 KiB
Plaintext
38 lines
1.5 KiB
Plaintext
# Copy to .env and fill in. Never commit .env.
|
|
|
|
# The `hankan` database in your existing Postgres. `postgres` here is the
|
|
# service name on the shared docker network, not a hostname on the Pi.
|
|
DATABASE_URL=postgres://hankan:CHANGE_ME@postgres:5432/hankan
|
|
|
|
# Shared secret between the app and this server. Generate one:
|
|
# openssl rand -base64 32
|
|
HANKAN_TOKEN=CHANGE_ME
|
|
|
|
# Which model serves the tutor: anthropic (default) | openai | echo.
|
|
# Omit the whole tutor config and sync still works — the app falls back to
|
|
# its local stand-in tutor.
|
|
HANKAN_TUTOR_BACKEND=anthropic
|
|
|
|
# For HANKAN_TUTOR_BACKEND=anthropic.
|
|
ANTHROPIC_API_KEY=
|
|
|
|
# For HANKAN_TUTOR_BACKEND=openai — anything speaking /chat/completions:
|
|
# LM Studio, Ollama, llama.cpp, vLLM, LiteLLM, OpenRouter, OpenAI.
|
|
# From a container, localhost is the container: use the host's LAN address
|
|
# or host.docker.internal, not 127.0.0.1.
|
|
# LM Studio http://<host>:1234/v1 model = the id shown in its UI
|
|
# Ollama http://<host>:11434/v1 model = e.g. qwen2.5:14b
|
|
# llama.cpp http://<host>:8080/v1
|
|
# OpenRouter https://openrouter.ai/api/v1
|
|
HANKAN_OPENAI_BASE_URL=http://host.docker.internal:1234/v1
|
|
HANKAN_OPENAI_MODEL=local-model
|
|
# Local servers ignore this; hosted ones require it. Leave blank for local.
|
|
HANKAN_OPENAI_API_KEY=
|
|
# Completion ceiling. Kept modest because a small-context local model errors
|
|
# outright if asked for more than its context holds.
|
|
HANKAN_OPENAI_MAX_TOKENS=2048
|
|
|
|
# The docker network your existing Postgres and Caddy are on.
|
|
# docker network ls
|
|
HANKAN_NETWORK=caddy_default
|