# server — sync and the tutor Two endpoints, both stateless. The client owns its database and its transcript; this process owns a Postgres table and an API key. **The app does not need this to work.** With no server configured it runs entirely offline against its own SQLite and a local stand-in tutor. Adding a server turns on two things: the real 선생님, and syncing between devices. ``` POST /api/tutor {system, systemTail, history, message} → SSE token stream GET /api/sync ?cursor=N → {epoch, rows, cursor, more}: a page of rows after N POST /api/sync {rows} → {epoch, applied, conflicts}: compare-and-swap writes GET /health → no auth, for the healthcheck ``` Everything under `/api` requires `Authorization: Bearer $HANKAN_TOKEN`, and `/api/sync` also requires `x-hankan-protocol: 2`. An older app gets **426 Upgrade Required** rather than a write the server would misread. See [How sync works](#how-sync-works). ### CORS — required for the phone The Android build talks to the Pi **cross-origin**: a Capacitor webview serves the app from its own origin (`http://localhost`), not from your domain. So the browser sends a preflight `OPTIONS` first, with no `Authorization` header — it is not permitted to attach one. Auth therefore has to run *after* CORS, or the preflight is answered 401 and the real request is never made. It surfaces as an opaque "Failed to fetch", which sends you looking at the network rather than at the middleware order. Any origin is allowed by default. That is not a hole: the gate is a bearer token rather than a cookie, so a hostile page gains nothing from being allowed to send a request it cannot authenticate. Set `HANKAN_ALLOWED_ORIGINS` to a comma-separated list to narrow it. ## Setting it up on the Pi ### 1. A database in the Postgres you already run ```sh docker exec -it psql -U postgres <<'SQL' CREATE ROLE hankan LOGIN PASSWORD 'pick-something-long'; CREATE DATABASE hankan OWNER hankan; SQL ``` The schema applies itself on boot. `server/sql/001-schema.sql` and `002-protocol-2.sql` are idempotent, so there is no migration step to run by hand. The first boot on protocol 2 drops any protocol-1 rows once; every device then hydrates and offers its own database back, which was always the source of truth. ### 2. Configure ```sh cd server cp .env.example .env $EDITOR .env # DATABASE_URL, HANKAN_TOKEN, ANTHROPIC_API_KEY docker network ls # find the network your Postgres and Caddy share ``` `compose.yaml` declares that network `external`, so it attaches to your existing stack rather than starting a second Postgres. Nothing is published to the host: Caddy reaches the container by name on the shared network. ### 3. Caddy ```caddyfile hankan.example.com { # Compression must not touch the tutor stream — see below. encode zstd gzip { match { not path /api/tutor* } } handle /api/tutor* { reverse_proxy hankan:8787 { flush_interval -1 transport http { read_timeout 0 write_timeout 0 } } } handle { reverse_proxy hankan:8787 } } ``` **Compression is what usually breaks SSE, not buffering.** `encode` delays the header flush until body bytes arrive and holds already-flushed events inside an unfinished compression frame, so the stream looks like it hangs. Excluding the tutor route is the important line; `flush_interval -1` is belt-and-braces (Caddy already auto-flushes `text/event-stream`) and the zero timeouts stop a long turn being cut off mid-lesson. Do not set `response_buffers` on this route. The server also sends `Cache-Control: no-cache, no-transform` and a `: ping` heartbeat every 15s, which defeat most intermediary caching and idle timeouts. ### 4. Connect the app In the app: 오늘 → ⚙ 설정 → Server and sync → the URL and the token. It syncs on connect, when the app regains focus, and every five minutes. ## How sync works Protocol 2 (`shared/sync-protocol.mjs`, `app/src/sync/`, `server/src/db.ts`). It is row-level, and every row carries three columns on the device: `base_seq`, the server version it last agreed with; `dirty`, set by every edit; and `rev`, which tells a push that lands after a further edit not to clear it. The artifact lost a week of work to its sync, and each of the three rules below closes one way that happened. **1. Hydrate before pushing.** A device pulls every page before it may push anything. A laptop last opened a week ago comes back, adopts the week of work the phone did, and only then offers its own edits. Changing the server, or the server's *epoch* (a random id replaced whenever its copy is thrown away), forces a full hydration again. **2. A counter, not a clock.** The server keeps a `change_seq` per row and writes a pushed row only if the push's `base` equals it: compare-and-swap. Anything else comes back as a conflict, with the server's current copy. `updated_at` is kept for display and decides nothing, so a device with its clock an hour out cannot win by it. Pushes are serialised and pulls take a shared advisory lock, so a pull can never skip a `change_seq` that a concurrent push is about to commit. **3. No silent shrinking.** A conflict goes to `app/src/sync/resolve.ts`, where the copy that holds more wins: a card with more reviews behind it, evidence with more observed, progress merged (done if either finished it, the higher answer count), the further unit on the roadmap, grammar notes and flags unioned, the better reading-drill round. A tie goes to the server, so every device lands on one copy. Shrinking is always declared, never inferred: - **Deliberate deletes** travel as tombstones, and a delete is obeyed against a concurrent edit: a card forgotten on one device stays forgotten. - **Resets** (clearing the lesson, resetting the roadmap or everything) raise a marker, `reset.`, a counter every device compares with the last one it applied. A device that had not heard of the reset empties the same things, its own offline work included, because a reset it did not know about still wins. - **Trimming the transcript is local.** A device keeps its last turns on screen, but the trim is never a deletion, and a turn not yet pushed is never trimmed. The artifact's trim tombstoned the other device's turns. Also fixed on the way: keys are JSON arrays, so a key containing a space (몇 명) no longer stops sync permanently; chat ids are UUIDv7, so two devices writing offline cannot collide; and the study log and peek tallies are kept per device and summed, so two devices' reviews of the same day both count. Rows are stored generically on the server, the primary key as text and the body as JSONB, because the server never reads inside a row. It stores, orders and versions them; the client interprets them. That keeps the two schemas from having to move in lockstep. ### What never syncs `meta` holds the learner's preferences *and* bookkeeping that describes one install, so an allowlist decides what may leave the device (`shared/sync-protocol.mjs`). `dict.loadedBands` is the dangerous one: replicating it would tell a phone that had loaded bands 0–2 that it holds every row the desktop has, and the word rail would then fail to find words it believes are present. `server.token`, `sync.*` and `schema_version` are excluded for related reasons. ### The bug this is shaped around The original artifact stamped a fresh device's empty defaults as newer than the server's real history, and clobbered it. Here a seeded row is never dirty, so it has nothing to push, and a device hydrates before it pushes at all. The week-old-laptop case, clock skew, offline chat on two devices, resets against devices that missed them, a forget racing a review, keys with spaces, and a server that lost its data are each a scenario in `test/sync/roundtrip.test.ts`, run against a real Postgres. ## The tutor `POST /api/tutor` takes the assembled system prompt, the transcript the client owns, and the new message; it holds nothing between requests, so a dropped connection costs one turn rather than the conversation. The system prompt comes in two parts. `system` is the shipped prompt with the gate filled in, ~12k characters that stay byte-identical for as long as the learner stays in one unit. It is sent as a cached block, so every turn after the first reads it at a fraction of the input price, the single biggest cost lever in the design. `systemTail` holds what changes every round (the practice set, the 다지기 checklist, the reason a draft was refused) and follows as a second, uncached block, so it never breaks the cached prefix. The OpenAI-compatible backend joins the two, prefix first. ### Choosing a model `HANKAN_TUTOR_BACKEND` picks one of three: | | | |---|---| | `anthropic` (default) | The Claude API. `ANTHROPIC_API_KEY`. | | `openai` | Anything speaking OpenAI's `/chat/completions`: LM Studio, Ollama, llama.cpp, vLLM, LiteLLM, OpenRouter, OpenAI. | | `echo` | No model at all. Reflects the request back, for proving a deployment. | For `openai`, set `HANKAN_OPENAI_BASE_URL` and `HANKAN_OPENAI_MODEL`; `HANKAN_OPENAI_API_KEY` is only needed by hosted endpoints. **From inside the container, `localhost` is the container** — a model server on the Pi itself is `http://host.docker.internal:1234/v1`, which `compose.yaml` maps for you. Verified against LM Studio serving `openai/gpt-oss-20b`: a turn streams end to end from the app, through this server, to the model and back. **What the OpenAI path costs you: prompt caching.** The Anthropic backend marks the ~12k-character gate as a cached prefix, so every turn after the first reads it at a fraction of the input price. There is no portable equivalent, so against a *paid hosted* endpoint the whole system prompt is re-billed every turn — the single biggest cost lever in the design, gone. Against a local model it costs nothing, and the shape still pays. llama.cpp and LM Studio reuse their KV cache for an unchanged prefix, and the system prompt is byte-identical for as long as the learner stays in one unit. Measured on a 6,948-character prompt against gpt-oss-20b: **first token 1,563ms cold, 324ms with the prefix already warm.** Which is why the system prompt goes first and stays put in this backend too. **Give a reasoning model room.** Its reasoning is spent from the same `max_tokens` budget and goes first, so too small a value truncates the lesson away entirely — and it fails silently, looking exactly like a model that cannot follow instructions. Measured on gpt-oss-20b with the real ~3.5k-token system prompt: an empty string at 1,400, a broken half-Korean fragment with none of the required blocks at 2,048, and a correct English lesson with all three at 8,000. `HANKAN_OPENAI_MAX_TOKENS` defaults to 8192 for that reason. A reasoning model served locally often emits chain-of-thought inline in `content` rather than in a separate field. `` blocks are stripped from the stream, including when the tags arrive split across chunks — otherwise they land in the lesson transcript and the block parser reads them as prose. `backends/` holds the seam. `anthropic.ts` is the Claude API and is the default. `openai.ts` is written against `fetch` rather than the `openai` package: the Anthropic SDK alone is 14MB in the image, this backend uses one endpoint with no tools and no retries, and local servers are the ones most likely to deviate from an SDK's expectations. `echo.ts` needs no API key and reflects the request back, chunk by chunk — set `HANKAN_TUTOR_BACKEND=echo` to prove a deployment (container, proxy, token, CORS, SSE through Caddy) from the phone before a key is involved and before anything is billed. Five things that can each break on their own, none of which involve Anthropic. `agent-sdk.ts` documents the subscription-billed path PORT.md originally specified and why it is not implemented — chiefly that its prompt accepts only user-role messages, so the transcript would have to be flattened into one turn. ## Running it locally ```sh docker run -d --name hankan-pg-test \ -e POSTGRES_PASSWORD=test -e POSTGRES_DB=hankan -p 55432:5432 postgres:16-alpine DATABASE_URL=postgres://postgres:test@localhost:55432/hankan \ HANKAN_TOKEN=test-token PORT=8788 HANKAN_TEST_MODE=1 \ node --experimental-strip-types server/src/main.ts # from the repo root, in another shell HANKAN_TEST_SERVER=http://localhost:8788 npm test ``` `HANKAN_TEST_MODE=1` adds `POST /api/test/reset`, which wipes the user's rows so each test starts clean. It exists only when that variable is set, so it cannot be reached on the Pi even if the token leaks. **Never set it in production.** The tutor's own tests run against a mock backend and need no API key.