diff --git a/.dockerignore b/.dockerignore new file mode 100644 index 0000000..1931a6e --- /dev/null +++ b/.dockerignore @@ -0,0 +1,8 @@ +node_modules +**/node_modules +app/dist +app/android +export +vendor +.git +*.tsbuildinfo diff --git a/server/.env.example b/server/.env.example new file mode 100644 index 0000000..1a81f84 --- /dev/null +++ b/server/.env.example @@ -0,0 +1,17 @@ +# Copy to .env and fill in. Never commit .env. + +# The `hankan` database in your existing Postgres. `postgres` here is the +# service name on the shared docker network, not a hostname on the Pi. +DATABASE_URL=postgres://hankan:CHANGE_ME@postgres:5432/hankan + +# Shared secret between the app and this server. Generate one: +# openssl rand -base64 32 +HANKAN_TOKEN=CHANGE_ME + +# For the tutor. Omit it and sync still works — the app falls back to its +# local stand-in tutor. +ANTHROPIC_API_KEY= + +# The docker network your existing Postgres and Caddy are on. +# docker network ls +HANKAN_NETWORK=caddy_default diff --git a/server/Dockerfile b/server/Dockerfile new file mode 100644 index 0000000..c2b41eb --- /dev/null +++ b/server/Dockerfile @@ -0,0 +1,26 @@ +# Node 22 runs TypeScript directly with --experimental-strip-types, so there +# is no build step and no compiled artefact to keep in sync with the source. +FROM node:22-alpine + +WORKDIR /app + +# Workspace manifests first, so a dependency change is the only thing that +# busts the install layer. +COPY package.json package-lock.json ./ +COPY server/package.json ./server/ +RUN npm ci --omit=dev --workspace @hankan/server --include-workspace-root + +# shared/ is imported by both the app and the server; the server needs the +# sync protocol at runtime. +COPY shared ./shared +COPY server ./server + +ENV NODE_ENV=production +ENV PORT=8787 +EXPOSE 8787 + +# The health route needs no token, precisely so this can reach it. +HEALTHCHECK --interval=30s --timeout=3s --start-period=10s \ + CMD node -e "fetch('http://127.0.0.1:'+(process.env.PORT||8787)+'/health').then(r=>process.exit(r.ok?0:1)).catch(()=>process.exit(1))" + +CMD ["node", "--experimental-strip-types", "server/src/main.ts"] diff --git a/server/README.md b/server/README.md index d90d07d..6e74f2f 100644 --- a/server/README.md +++ b/server/README.md @@ -1,25 +1,165 @@ -# server/ — not in this pass +# server — sync and the tutor -PORT.md steps 4–6 land here: the sync API, the tutor SSE endpoint, and the -Docker Compose deployment beside the Pi's existing Postgres and Caddy. +Two endpoints, both stateless. The client owns its database and its +transcript; this process owns a Postgres table and an API key. -Nothing here yet, on purpose. Steps 1–3 give a fully working offline app with -no server at all, and that is the version that gets used first. +**The app does not need this to work.** With no server configured it runs +entirely offline against its own SQLite and a local stand-in tutor. Adding a +server turns on two things: the real 선생님, and syncing between devices. -What is already shaped for it: +``` +POST /api/tutor {system, history, message} → SSE token stream +GET /api/sync ?cursor=N → rows newer than the cursor +POST /api/sync {rows} → upsert, last-write-wins +GET /health → no auth, for the healthcheck +``` -- every syncable table carries `updated_at`, and seeded rows are pinned to 0 - (see `app/src/db/writes.ts`) — the artifact's clobbering bug cannot be - expressed; -- `change_seq` is deliberately absent: it is server-assigned and arrives with - the sync layer; -- the dictionary is fetched by URL from `app/src/domain/dictionary.ts`, so - per-band deltas can come from the Pi instead of the bundle by changing a base - URL; -- the tutor goes through one `Sample` function - (`app/src/domain/stub-tutor.ts`). The real endpoint implements the same - contract — `onText` receives cumulative text, and an aborted turn stops - billing — so only that file changes. +Everything under `/api` requires `Authorization: Bearer $HANKAN_TOKEN`. -When the SSE endpoint lands, Caddy needs `flush_interval -1` on the proxy or -the stream buffers and the tutor appears to hang. +## Setting it up on the Pi + +### 1. A database in the Postgres you already run + +```sh +docker exec -it psql -U postgres <<'SQL' +CREATE ROLE hankan LOGIN PASSWORD 'pick-something-long'; +CREATE DATABASE hankan OWNER hankan; +SQL +``` + +The schema applies itself on boot — `server/sql/001-schema.sql` is idempotent, +so there is no migration step to run by hand. + +### 2. Configure + +```sh +cd server +cp .env.example .env +$EDITOR .env # DATABASE_URL, HANKAN_TOKEN, ANTHROPIC_API_KEY +docker network ls # find the network your Postgres and Caddy share +``` + +`compose.yaml` declares that network `external`, so it attaches to your +existing stack rather than starting a second Postgres. Nothing is published +to the host: Caddy reaches the container by name on the shared network. + +### 3. Caddy + +```caddyfile +hankan.example.com { + # Compression must not touch the tutor stream — see below. + encode zstd gzip { + match { + not path /api/tutor* + } + } + + handle /api/tutor* { + reverse_proxy hankan:8787 { + flush_interval -1 + transport http { + read_timeout 0 + write_timeout 0 + } + } + } + + handle { + reverse_proxy hankan:8787 + } +} +``` + +**Compression is what usually breaks SSE, not buffering.** `encode` delays the +header flush until body bytes arrive and holds already-flushed events inside +an unfinished compression frame, so the stream looks like it hangs. Excluding +the tutor route is the important line; `flush_interval -1` is belt-and-braces +(Caddy already auto-flushes `text/event-stream`) and the zero timeouts stop a +long turn being cut off mid-lesson. Do not set `response_buffers` on this +route. + +The server also sends `Cache-Control: no-cache, no-transform` and a `: ping` +heartbeat every 15s, which defeat most intermediary caching and idle timeouts. + +### 4. Connect the app + +In the app: 오늘 → 서버 → the URL and the token. It syncs on connect, when the +tab regains focus, and every five minutes. + +## How sync works + +Row-level, last-write-wins on `updated_at`, cursor-based on a server-assigned +`change_seq`. One user, so the loser of a conflict is at worst one SRS grade. + +Rows are stored generically — primary key as text, body as JSONB — because the +server never reads inside a row. It stores and orders them; the client +interprets them. That keeps the two schemas from having to move in lockstep. + +Three things are load-bearing: + +- **`change_seq` advances on every update, via a trigger.** A row edited after + a client last pulled would otherwise sit below that client's cursor and + never be delivered. Putting it in a trigger means no write path can forget. +- **The pull cursor advances only as rows are applied**, never from the push + response. The server's newest `change_seq` includes rows this device has not + seen; adopting it would skip them permanently, and nothing would ever ask + for that range again. +- **Deletes travel as tombstones.** A deleted row leaves nothing to compare + timestamps against, so without one the other device pushes its still-live + copy back and the row silently returns. + +### What never syncs + +`meta` holds the learner's preferences *and* bookkeeping that describes one +install, so an allowlist decides what may leave the device +(`shared/sync-protocol.mjs`). `dict.loadedBands` is the dangerous one: +replicating it would tell a phone that had loaded bands 0–2 that it holds +every row the desktop has, and the word rail would then fail to find words it +believes are present. `server.token`, `sync.*` and `schema_version` are +excluded for related reasons. + +### The bug this schema is shaped around + +The original artifact stamped a fresh device's empty defaults as newer than +the server's real history, and clobbered it. Here seeded and defaulted rows +carry `updated_at = 0`, so they can never be dirty and can never win a +conflict. It is not avoided, it is unrepresentable — +`test/sync/roundtrip.test.ts` asserts it against a real Postgres. + +## The tutor + +`POST /api/tutor` takes the assembled system prompt, the transcript the client +owns, and the new message; it holds nothing between requests, so a dropped +connection costs one turn rather than the conversation. + +The system prompt is passed as a cached block. It is ~12k characters of gate +and is byte-identical for as long as the learner stays in one unit — many +turns — so every turn after the first reads the prefix at a fraction of the +input price. This is the single biggest cost lever in the design. + +`backends/` holds the seam. `anthropic.ts` is the Claude API and is the +default. `agent-sdk.ts` documents the subscription-billed path PORT.md +originally specified and why it is not implemented — chiefly that its prompt +accepts only user-role messages, so the transcript would have to be flattened +into one turn. + +## Running it locally + +```sh +docker run -d --name hankan-pg-test \ + -e POSTGRES_PASSWORD=test -e POSTGRES_DB=hankan -p 55432:5432 postgres:16-alpine + +DATABASE_URL=postgres://postgres:test@localhost:55432/hankan \ +HANKAN_TOKEN=test-token PORT=8788 HANKAN_TEST_MODE=1 \ + node --experimental-strip-types server/src/main.ts + +# from the repo root, in another shell +HANKAN_TEST_SERVER=http://localhost:8788 npm test +``` + +`HANKAN_TEST_MODE=1` adds `POST /api/test/reset`, which wipes the user's rows +so each test starts clean. It exists only when that variable is set, so it +cannot be reached on the Pi even if the token leaks. **Never set it in +production.** + +The tutor's own tests run against a mock backend and need no API key. diff --git a/server/compose.yaml b/server/compose.yaml new file mode 100644 index 0000000..2fc857b --- /dev/null +++ b/server/compose.yaml @@ -0,0 +1,33 @@ +# Hankan's server, joining the Postgres and Caddy already running on the Pi. +# +# It brings up no database of its own: `hankan-net` is declared external, so +# this file attaches to the network those services are already on rather +# than standing up a parallel stack. Set the network name to whatever your +# existing compose project created — `docker network ls` will show it. +# +# cp .env.example .env # then fill it in +# docker compose up -d + +services: + hankan: + build: + context: .. + dockerfile: server/Dockerfile + container_name: hankan + restart: unless-stopped + environment: + # Reaches the existing Postgres by service name on the shared network. + DATABASE_URL: ${DATABASE_URL} + HANKAN_TOKEN: ${HANKAN_TOKEN} + ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY} + HANKAN_TUTOR_BACKEND: ${HANKAN_TUTOR_BACKEND:-anthropic} + PORT: 8787 + networks: + - hankan-net + # No ports published: Caddy is on the same network and proxies to + # hankan:8787 by name, so the service is never exposed directly. + +networks: + hankan-net: + external: true + name: ${HANKAN_NETWORK:-caddy_default}