Hankan/server/README.md

# server — sync and the tutor

Two endpoints, both stateless. The client owns its database and its
transcript; this process owns a Postgres table and an API key.

**The app does not need this to work.** With no server configured it runs
entirely offline against its own SQLite and a local stand-in tutor. Adding a
server turns on two things: the real 선생님, and syncing between devices.

```
POST /api/tutor   {system, history, message} → SSE token stream
GET  /api/sync    ?cursor=N                  → rows newer than the cursor
POST /api/sync    {rows}                     → upsert, last-write-wins
GET  /health                                 → no auth, for the healthcheck
```

Everything under `/api` requires `Authorization: Bearer $HANKAN_TOKEN`.

### CORS — required for the phone

The Android build talks to the Pi **cross-origin**: a Capacitor webview
serves the app from its own origin (`http://localhost`), not from your
domain. So the browser sends a preflight `OPTIONS` first, with no
`Authorization` header — it is not permitted to attach one. Auth therefore
has to run *after* CORS, or the preflight is answered 401 and the real
request is never made. It surfaces as an opaque "Failed to fetch", which
sends you looking at the network rather than at the middleware order.

Any origin is allowed by default. That is not a hole: the gate is a bearer
token rather than a cookie, so a hostile page gains nothing from being
allowed to send a request it cannot authenticate. Set
`HANKAN_ALLOWED_ORIGINS` to a comma-separated list to narrow it.

## Setting it up on the Pi

### 1. A database in the Postgres you already run

```sh
docker exec -it <your-postgres-container> psql -U postgres <<'SQL'
CREATE ROLE hankan LOGIN PASSWORD 'pick-something-long';
CREATE DATABASE hankan OWNER hankan;
SQL
```

The schema applies itself on boot — `server/sql/001-schema.sql` is idempotent,
so there is no migration step to run by hand.

### 2. Configure

```sh
cd server
cp .env.example .env
$EDITOR .env          # DATABASE_URL, HANKAN_TOKEN, ANTHROPIC_API_KEY
docker network ls     # find the network your Postgres and Caddy share
```

`compose.yaml` declares that network `external`, so it attaches to your
existing stack rather than starting a second Postgres. Nothing is published
to the host: Caddy reaches the container by name on the shared network.

### 3. Caddy

```caddyfile
hankan.example.com {
    # Compression must not touch the tutor stream — see below.
    encode zstd gzip {
        match {
            not path /api/tutor*
        }
    }

    handle /api/tutor* {
        reverse_proxy hankan:8787 {
            flush_interval -1
            transport http {
                read_timeout  0
                write_timeout 0
            }
        }
    }

    handle {
        reverse_proxy hankan:8787
    }
}
```

**Compression is what usually breaks SSE, not buffering.** `encode` delays the
header flush until body bytes arrive and holds already-flushed events inside
an unfinished compression frame, so the stream looks like it hangs. Excluding
the tutor route is the important line; `flush_interval -1` is belt-and-braces
(Caddy already auto-flushes `text/event-stream`) and the zero timeouts stop a
long turn being cut off mid-lesson. Do not set `response_buffers` on this
route.

The server also sends `Cache-Control: no-cache, no-transform` and a `: ping`
heartbeat every 15s, which defeat most intermediary caching and idle timeouts.

### 4. Connect the app

In the app: 오늘 → 서버 → the URL and the token. It syncs on connect, when the
tab regains focus, and every five minutes.

## How sync works

Row-level, last-write-wins on `updated_at`, cursor-based on a server-assigned
`change_seq`. One user, so the loser of a conflict is at worst one SRS grade.

Rows are stored generically — primary key as text, body as JSONB — because the
server never reads inside a row. It stores and orders them; the client
interprets them. That keeps the two schemas from having to move in lockstep.

Three things are load-bearing:

- **`change_seq` advances on every update, via a trigger.** A row edited after
  a client last pulled would otherwise sit below that client's cursor and
  never be delivered. Putting it in a trigger means no write path can forget.
- **The pull cursor advances only as rows are applied**, never from the push
  response. The server's newest `change_seq` includes rows this device has not
  seen; adopting it would skip them permanently, and nothing would ever ask
  for that range again.
- **Deletes travel as tombstones.** A deleted row leaves nothing to compare
  timestamps against, so without one the other device pushes its still-live
  copy back and the row silently returns.

### What never syncs

`meta` holds the learner's preferences *and* bookkeeping that describes one
install, so an allowlist decides what may leave the device
(`shared/sync-protocol.mjs`). `dict.loadedBands` is the dangerous one:
replicating it would tell a phone that had loaded bands 0–2 that it holds
every row the desktop has, and the word rail would then fail to find words it
believes are present. `server.token`, `sync.*` and `schema_version` are
excluded for related reasons.

### The bug this schema is shaped around

The original artifact stamped a fresh device's empty defaults as newer than
the server's real history, and clobbered it. Here seeded and defaulted rows
carry `updated_at = 0`, so they can never be dirty and can never win a
conflict. It is not avoided, it is unrepresentable —
`test/sync/roundtrip.test.ts` asserts it against a real Postgres.

## The tutor

`POST /api/tutor` takes the assembled system prompt, the transcript the client
owns, and the new message; it holds nothing between requests, so a dropped
connection costs one turn rather than the conversation.

The system prompt is passed as a cached block. It is ~12k characters of gate
and is byte-identical for as long as the learner stays in one unit — many
turns — so every turn after the first reads the prefix at a fraction of the
input price. This is the single biggest cost lever in the design.

### Choosing a model

`HANKAN_TUTOR_BACKEND` picks one of three:

| | |
|---|---|
| `anthropic` (default) | The Claude API. `ANTHROPIC_API_KEY`. |
| `openai` | Anything speaking OpenAI's `/chat/completions`: LM Studio, Ollama, llama.cpp, vLLM, LiteLLM, OpenRouter, OpenAI. |
| `echo` | No model at all. Reflects the request back, for proving a deployment. |

For `openai`, set `HANKAN_OPENAI_BASE_URL` and `HANKAN_OPENAI_MODEL`;
`HANKAN_OPENAI_API_KEY` is only needed by hosted endpoints. **From inside
the container, `localhost` is the container** — a model server on the Pi
itself is `http://host.docker.internal:1234/v1`, which `compose.yaml` maps
for you.

Verified against LM Studio serving `openai/gpt-oss-20b`: a turn streams end
to end from the app, through this server, to the model and back.

**What the OpenAI path costs you: prompt caching.** The Anthropic backend
marks the ~12k-character gate as a cached prefix, so every turn after the
first reads it at a fraction of the input price. There is no portable
equivalent, so against a *paid hosted* endpoint the whole system prompt is
re-billed every turn — the single biggest cost lever in the design, gone.

Against a local model it costs nothing, and the shape still pays. llama.cpp
and LM Studio reuse their KV cache for an unchanged prefix, and the system
prompt is byte-identical for as long as the learner stays in one unit.
Measured on a 6,948-character prompt against gpt-oss-20b: **first token
1,563ms cold, 324ms with the prefix already warm.** Which is why the system
prompt goes first and stays put in this backend too.

**Give a reasoning model room.** Its reasoning is spent from the same
`max_tokens` budget and goes first, so too small a value truncates the
lesson away entirely — and it fails silently, looking exactly like a model
that cannot follow instructions. Measured on gpt-oss-20b with the real
~3.5k-token system prompt: an empty string at 1,400, a broken half-Korean
fragment with none of the required blocks at 2,048, and a correct English
lesson with all three at 8,000. `HANKAN_OPENAI_MAX_TOKENS` defaults to
8192 for that reason.

A reasoning model served locally often emits chain-of-thought inline in
`content` rather than in a separate field. `<think>` blocks are stripped
from the stream, including when the tags arrive split across chunks —
otherwise they land in the lesson transcript and the block parser reads
them as prose.

`backends/` holds the seam. `anthropic.ts` is the Claude API and is the
default. `openai.ts` is written against `fetch` rather than the `openai`
package: the Anthropic SDK alone is 14MB in the image, this backend uses
one endpoint with no tools and no retries, and local servers are the ones
most likely to deviate from an SDK's expectations. `echo.ts` needs no API
key and reflects the request back, chunk by
chunk — set `HANKAN_TUTOR_BACKEND=echo` to prove a deployment (container,
proxy, token, CORS, SSE through Caddy) from the phone before a key is
involved and before anything is billed. Five things that can each break on
their own, none of which involve Anthropic. `agent-sdk.ts` documents the subscription-billed path PORT.md
originally specified and why it is not implemented — chiefly that its prompt
accepts only user-role messages, so the transcript would have to be flattened
into one turn.

## Running it locally

```sh
docker run -d --name hankan-pg-test \
  -e POSTGRES_PASSWORD=test -e POSTGRES_DB=hankan -p 55432:5432 postgres:16-alpine

DATABASE_URL=postgres://postgres:test@localhost:55432/hankan \
HANKAN_TOKEN=test-token PORT=8788 HANKAN_TEST_MODE=1 \
  node --experimental-strip-types server/src/main.ts

# from the repo root, in another shell
HANKAN_TEST_SERVER=http://localhost:8788 npm test
```

`HANKAN_TEST_MODE=1` adds `POST /api/test/reset`, which wipes the user's rows
so each test starts clean. It exists only when that variable is set, so it
cannot be reached on the Pi even if the token leaks. **Never set it in
production.**

The tutor's own tests run against a mock backend and need no API key.