Hankan/server/README.md

server — sync and the tutor

Two endpoints, both stateless. The client owns its database and its transcript; this process owns a Postgres table and an API key.

The app does not need this to work. With no server configured it runs entirely offline against its own SQLite and a local stand-in tutor. Adding a server turns on two things: the real 선생님, and syncing between devices.

POST /api/tutor   {system, systemTail, history, message} → SSE token stream
GET  /api/sync    ?cursor=N         → {epoch, rows, cursor, more}: a page of rows after N
POST /api/sync    {rows}            → {epoch, applied, conflicts}: compare-and-swap writes
GET  /health                        → no auth, for the healthcheck

Everything under /api requires Authorization: Bearer $HANKAN_TOKEN, and /api/sync also requires x-hankan-protocol: 2. An older app gets 426 Upgrade Required rather than a write the server would misread. See How sync works.

CORS — required for the phone

The Android build talks to the Pi cross-origin: a Capacitor webview serves the app from its own origin (http://localhost), not from your domain. So the browser sends a preflight OPTIONS first, with no Authorization header — it is not permitted to attach one. Auth therefore has to run after CORS, or the preflight is answered 401 and the real request is never made. It surfaces as an opaque "Failed to fetch", which sends you looking at the network rather than at the middleware order.

Any origin is allowed by default. That is not a hole: the gate is a bearer token rather than a cookie, so a hostile page gains nothing from being allowed to send a request it cannot authenticate. Set HANKAN_ALLOWED_ORIGINS to a comma-separated list to narrow it.

Setting it up on the Pi

1. A database in the Postgres you already run

docker exec -it <your-postgres-container> psql -U postgres <<'SQL'
CREATE ROLE hankan LOGIN PASSWORD 'pick-something-long';
CREATE DATABASE hankan OWNER hankan;
SQL

The schema applies itself on boot. server/sql/001-schema.sql and 002-protocol-2.sql are idempotent, so there is no migration step to run by hand. The first boot on protocol 2 drops any protocol-1 rows once; every device then hydrates and offers its own database back, which was always the source of truth.

2. Configure

cd server
cp .env.example .env
$EDITOR .env          # DATABASE_URL, HANKAN_TOKEN, ANTHROPIC_API_KEY
docker network ls     # find the network your Postgres and Caddy share

compose.yaml declares that network external, so it attaches to your existing stack rather than starting a second Postgres. Nothing is published to the host: Caddy reaches the container by name on the shared network.

3. Caddy

hankan.example.com {
    # Compression must not touch the tutor stream — see below.
    encode zstd gzip {
        match {
            not path /api/tutor*
        }
    }

    handle /api/tutor* {
        reverse_proxy hankan:8787 {
            flush_interval -1
            transport http {
                read_timeout  0
                write_timeout 0
            }
        }
    }

    handle {
        reverse_proxy hankan:8787
    }
}

Compression is what usually breaks SSE, not buffering. encode delays the header flush until body bytes arrive and holds already-flushed events inside an unfinished compression frame, so the stream looks like it hangs. Excluding the tutor route is the important line; flush_interval -1 is belt-and-braces (Caddy already auto-flushes text/event-stream) and the zero timeouts stop a long turn being cut off mid-lesson. Do not set response_buffers on this route.

The server also sends Cache-Control: no-cache, no-transform and a : ping heartbeat every 15s, which defeat most intermediary caching and idle timeouts.

4. Connect the app

In the app: 오늘 → ⚙ 설정 → Server and sync → the URL and the token. It syncs on connect, when the app regains focus, and every five minutes.

How sync works

Protocol 2 (shared/sync-protocol.mjs, app/src/sync/, server/src/db.ts). It is row-level, and every row carries three columns on the device: base_seq, the server version it last agreed with; dirty, set by every edit; and rev, which tells a push that lands after a further edit not to clear it.

The artifact lost a week of work to its sync, and each of the three rules below closes one way that happened.

1. Hydrate before pushing. A device pulls every page before it may push anything. A laptop last opened a week ago comes back, adopts the week of work the phone did, and only then offers its own edits. Changing the server, or the server's epoch (a random id replaced whenever its copy is thrown away), forces a full hydration again.

2. A counter, not a clock. The server keeps a change_seq per row and writes a pushed row only if the push's base equals it: compare-and-swap. Anything else comes back as a conflict, with the server's current copy. updated_at is kept for display and decides nothing, so a device with its clock an hour out cannot win by it. Pushes are serialised and pulls take a shared advisory lock, so a pull can never skip a change_seq that a concurrent push is about to commit.

3. No silent shrinking. A conflict goes to app/src/sync/resolve.ts, where the copy that holds more wins: a card with more reviews behind it, evidence with more observed, progress merged (done if either finished it, the higher answer count), the further unit on the roadmap, grammar notes and flags unioned, the better reading-drill round. A tie goes to the server, so every device lands on one copy.

Shrinking is always declared, never inferred:

• Deliberate deletes travel as tombstones, and a delete is obeyed against a concurrent edit: a card forgotten on one device stays forgotten.
• Resets (clearing the lesson, resetting the roadmap or everything) raise a marker, reset.<scope>, a counter every device compares with the last one it applied. A device that had not heard of the reset empties the same things, its own offline work included, because a reset it did not know about still wins.
• Trimming the transcript is local. A device keeps its last turns on screen, but the trim is never a deletion, and a turn not yet pushed is never trimmed. The artifact's trim tombstoned the other device's turns.

Also fixed on the way: keys are JSON arrays, so a key containing a space (몇 명) no longer stops sync permanently; chat ids are UUIDv7, so two devices writing offline cannot collide; and the study log and peek tallies are kept per device and summed, so two devices' reviews of the same day both count.

Rows are stored generically on the server, the primary key as text and the body as JSONB, because the server never reads inside a row. It stores, orders and versions them; the client interprets them. That keeps the two schemas from having to move in lockstep.

What never syncs

meta holds the learner's preferences and bookkeeping that describes one install, so an allowlist decides what may leave the device (shared/sync-protocol.mjs). dict.loadedBands is the dangerous one: replicating it would tell a phone that had loaded bands 0–2 that it holds every row the desktop has, and the word rail would then fail to find words it believes are present. server.token, sync.* and schema_version are excluded for related reasons.

The bug this is shaped around

The original artifact stamped a fresh device's empty defaults as newer than the server's real history, and clobbered it. Here a seeded row is never dirty, so it has nothing to push, and a device hydrates before it pushes at all. The week-old-laptop case, clock skew, offline chat on two devices, resets against devices that missed them, a forget racing a review, keys with spaces, and a server that lost its data are each a scenario in test/sync/roundtrip.test.ts, run against a real Postgres.

The tutor

POST /api/tutor takes the assembled system prompt, the transcript the client owns, and the new message; it holds nothing between requests, so a dropped connection costs one turn rather than the conversation.

The system prompt comes in two parts. system is the shipped prompt with the gate filled in, ~12k characters that stay byte-identical for as long as the learner stays in one unit. It is sent as a cached block, so every turn after the first reads it at a fraction of the input price, the single biggest cost lever in the design. systemTail holds what changes every round (the practice set, the 다지기 checklist, the reason a draft was refused) and follows as a second, uncached block, so it never breaks the cached prefix. The OpenAI-compatible backend joins the two, prefix first.

Choosing a model

HANKAN_TUTOR_BACKEND picks one of three:

anthropic (default)	The Claude API. ANTHROPIC_API_KEY.
openai	Anything speaking OpenAI's /chat/completions: LM Studio, Ollama, llama.cpp, vLLM, LiteLLM, OpenRouter, OpenAI.
echo	No model at all. Reflects the request back, for proving a deployment.

For openai, set HANKAN_OPENAI_BASE_URL and HANKAN_OPENAI_MODEL; HANKAN_OPENAI_API_KEY is only needed by hosted endpoints. From inside the container, localhost is the container — a model server on the Pi itself is http://host.docker.internal:1234/v1, which compose.yaml maps for you.

Verified against LM Studio serving openai/gpt-oss-20b: a turn streams end to end from the app, through this server, to the model and back.

What the OpenAI path costs you: prompt caching. The Anthropic backend marks the ~12k-character gate as a cached prefix, so every turn after the first reads it at a fraction of the input price. There is no portable equivalent, so against a paid hosted endpoint the whole system prompt is re-billed every turn — the single biggest cost lever in the design, gone.

Against a local model it costs nothing, and the shape still pays. llama.cpp and LM Studio reuse their KV cache for an unchanged prefix, and the system prompt is byte-identical for as long as the learner stays in one unit. Measured on a 6,948-character prompt against gpt-oss-20b: first token 1,563ms cold, 324ms with the prefix already warm. Which is why the system prompt goes first and stays put in this backend too.

Give a reasoning model room. Its reasoning is spent from the same max_tokens budget and goes first, so too small a value truncates the lesson away entirely — and it fails silently, looking exactly like a model that cannot follow instructions. Measured on gpt-oss-20b with the real ~3.5k-token system prompt: an empty string at 1,400, a broken half-Korean fragment with none of the required blocks at 2,048, and a correct English lesson with all three at 8,000. HANKAN_OPENAI_MAX_TOKENS defaults to 8192 for that reason.

A reasoning model served locally often emits chain-of-thought inline in content rather than in a separate field. <think> blocks are stripped from the stream, including when the tags arrive split across chunks — otherwise they land in the lesson transcript and the block parser reads them as prose.

backends/ holds the seam. anthropic.ts is the Claude API and is the default. openai.ts is written against fetch rather than the openai package: the Anthropic SDK alone is 14MB in the image, this backend uses one endpoint with no tools and no retries, and local servers are the ones most likely to deviate from an SDK's expectations. echo.ts needs no API key and reflects the request back, chunk by chunk — set HANKAN_TUTOR_BACKEND=echo to prove a deployment (container, proxy, token, CORS, SSE through Caddy) from the phone before a key is involved and before anything is billed. Five things that can each break on their own, none of which involve Anthropic. agent-sdk.ts documents the subscription-billed path PORT.md originally specified and why it is not implemented — chiefly that its prompt accepts only user-role messages, so the transcript would have to be flattened into one turn.

Running it locally

docker run -d --name hankan-pg-test \
  -e POSTGRES_PASSWORD=test -e POSTGRES_DB=hankan -p 55432:5432 postgres:16-alpine

DATABASE_URL=postgres://postgres:test@localhost:55432/hankan \
HANKAN_TOKEN=test-token PORT=8788 HANKAN_TEST_MODE=1 \
  node --experimental-strip-types server/src/main.ts

# from the repo root, in another shell
HANKAN_TEST_SERVER=http://localhost:8788 npm test

HANKAN_TEST_MODE=1 adds POST /api/test/reset, which wipes the user's rows so each test starts clean. It exists only when that variable is set, so it cannot be reached on the Pi even if the token leaks. Never set it in production.

The tutor's own tests run against a mock backend and need no API key.