Files
Hankan/README.md
MechaCat02 074f602494 feat(server): an OpenAI-compatible backend, so the model is yours to pick
HANKAN_TUTOR_BACKEND=openai talks to anything serving
/chat/completions -- LM Studio, Ollama, llama.cpp, vLLM, LiteLLM,
OpenRouter, OpenAI. The TutorBackend seam already existed for this, so the
model becomes a config line rather than a code change.

Written against fetch rather than the openai package. The Anthropic SDK
alone is 14MB in the image, this backend uses one endpoint with no tools
and no retries, and local servers are the ones most likely to deviate from
an SDK's expectations. The real risk in hand-rolling it is SSE reassembly,
so that is where the tests are: a JSON payload split across two TCP reads,
an event whose blank-line terminator lands in the next read, heartbeat
comments, CRLF framing, and a stream that ends without [DONE]. The two
split cases both fail against a naive per-read parser, which is what makes
them worth having.

<think> blocks are stripped from the stream, tags split across chunks
included. Reasoning models served locally often emit chain-of-thought
inline in `content` rather than in a separate field, and left in it lands
in the lesson transcript where the block parser reads it as prose.

WHAT THIS COSTS: prompt caching. The Anthropic backend marks the ~12k
character gate as a cached prefix, so every turn after the first reads it
at a fraction of the input price. There is no portable equivalent, so
against a paid hosted endpoint the system prompt is re-billed every turn --
the biggest cost lever in the design, gone.

Against a local model it costs nothing, and the shape still pays: llama.cpp
and LM Studio reuse their KV cache for an unchanged prefix. Measured on a
6,948-character prompt against gpt-oss-20b, first token 1,563ms cold and
324ms warm, so the system prompt goes first and stays put here too.

Verified against LM Studio running openai/gpt-oss-20b, not only a fake: a
turn streams from the browser through this server to the model and back,
rendered in the chat, no page errors.

Also makes test/server/http.test.ts backend-agnostic. It asserted the echo
backend's wording and so failed the moment the server was pointed at a real
model -- precisely the case a transport test should survive.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-09 19:39:31 +02:00

189 lines
8.9 KiB
Markdown

# Hankan · 한칸
A Korean reading tutor for manhwa. One codebase, two shells: a web app and an
Android app. **Works entirely offline — there is no server.**
Ported from a single-file Claude artifact. The curriculum, the tutor prompt and
the five logic modules came finished and tested; this repo is the application
built around them.
---
## What is here
```
data/ lib/ prompt/ validate.mjs verbatim from the export bundle — do not edit
shared/ band table + phonological ladder, shared by app and build
types/ TypeScript declarations for lib/ and shared/
tools/dict/ the dictionary build pipeline
app/ Vite + React + TypeScript, and the Capacitor shell
src/db/ one storage interface, two SQLite drivers
src/domain/ the gate, the lexicon, SRS, the tutor clients
src/sync/ push/pull against the Pi
src/ui/ six tabs, the review overlay, the 한글 keyboard
public/dict/ generated dictionary — committed, shipped
server/ sync + the tutor endpoint (optional)
tools/icons/ the 한칸 mark, generated
test/ lib goldens, driver conformance, domain, sync, SSE
```
`data/`, `lib/`, `prompt/` and `validate.mjs` are byte-identical to the export
and are excluded from lint and formatting. `lib/conjugation.js` encodes the seven
Korean irregular classes and is the reason no runtime morphological analyser is
needed; `lib/hangul.js` implements 두벌식 composition. Types are supplied
alongside in `types/`, so neither file had to be touched.
## Running it
```bash
npm install
npm run dict:build # only needed if app/public/dict/ is missing or stale
npm run dev # http://localhost:5173
```
```bash
npm run check # validate.mjs, typecheck, tests, roadmap assertion
npm run build # production build + service worker
```
### Android
The Capacitor project lives at `app/android/` and is committed. It needs the
Android SDK, which this repo does not install:
```bash
export ANDROID_HOME=/path/to/Android/Sdk
npm run build
npm run cap:sync
cd app/android && ./gradlew assembleDebug
```
`npx cap sync` copies `app/dist/` — including the dictionary — into the APK's
assets, so the phone build is as offline as the web one.
## How it works
### Storage — one interface, two drivers
`app/src/db/` exposes a single `Db` interface. `sqlite.web.ts` runs
`@sqlite.org/sqlite-wasm` in a dedicated Worker over the **OPFS SAHPool VFS**
(the plain OPFS VFS needs COOP/COEP headers, which neither a static host nor the
Capacitor webview reliably provides). `sqlite.native.ts` uses the Capacitor
SQLite plugin. Both apply the same migration array from `migrations.ts`.
**Seeded state never carries a write timestamp.** The artifact had a sync bug
where a fresh device stamped its own empty defaults as newer than the server's
real history and clobbered it. Three layers stop that from being expressible:
every syncable table declares `updated_at INTEGER NOT NULL DEFAULT 0`, so
forgetting the column is the safe failure; `db/writes.ts` splits every mutation
into `seedX()` (never stamps) and `editX()` (always stamps) and is the only file
allowed to read the clock; an ESLint rule enforces that, and the conformance
suite asserts a freshly seeded database has no non-zero `updated_at` anywhere.
### The dictionary
`npm run dict:build` merges a dictionary source, a frequency list, the curated
data in `data/`, and a hand-written grammar lexicon, then runs
`lib/conjugation.js → surfaceForms()` over every verb and adjective to fill the
`surface` table. Lookup of a conjugated form is therefore an index hit, not an
analysis — **no runtime morphological analyser ships**.
Sources are chosen automatically: KRDICT if a download has been vendored (it
has curated learner glosses and a graded 초급/중급/고급 level), otherwise the
kaikki.org Korean extract. KRDICT's download page is a JavaScript form behind
anti-bot protection, so it cannot be fetched by a script — see the comment at
the top of `tools/dict/fetch.mjs`.
Frequency needs care: a subtitle frequency list holds *surface* forms while a
dictionary holds lemmas, and a naive join on the headword gives every verb a
frequency of roughly zero. `tools/dict/freq-forms.mjs` inverts the join —
expanding each lemma into the forms it plausibly takes and summing — which
recovers 하다 from 118 to 89,041.
Output is one gzipped row dump per band, committed under `app/public/dict/`.
They are static assets in the bundle and in the APK's assets, so no server is
involved. Attribution is in [NOTICE.md](NOTICE.md); share-alike attaches to the
dictionary data, not to this code.
### The gate
`lib/gate.js` computes what the tutor may teach from the curriculum plus
progress, and `renderGate()` renders it into `{{GATE}}` in
`prompt/tutor-system.md`, which ships unchanged. `buildGate()` takes a
`vocabQuery` hook; `app/src/domain/gate.ts` fills it with a frequency-band query
so vocabulary grows as units are finished. Three refinements sit inside that
hook, all of them narrowing:
1. words a not-yet-finished unit is the first to introduce are excluded, so a
band ceiling cannot smuggle 3.4's material into 2.1;
2. during Phase 1 the results are filtered by the same phonological ladder
`validate.mjs` checks, so a band cannot hand the learner a 겹받침 at 1.4;
3. the list is capped at 800 by frequency, because `renderGate()` inlines it
into the prompt. The cap is strictly more restrictive than the band.
Confidence is clamped to `MAX_DELTA_PER_TURN` per turn. The artifact wrote the
model's number straight into the advancement gate, so one hallucinated
`::progress 95` could skip a unit.
### The tutor
`app/src/domain/stub-tutor.ts` stands in for the model and implements the exact
contract the real endpoint will (`onText` receives cumulative text; an aborted
turn keeps what it streamed). It rotates all four exercise types, builds its
`::words` from the current unit's real vocabulary, and climbs `::progress`
gradually, so every render path — including the 85% advancement banner — is
reachable with no server. Swapping in the Pi's SSE endpoint later touches only
that file.
## Checks
| Command | What it guards |
|---|---|
| `node validate.mjs` | the curriculum. **PASS — 0 blocking, 0 advisory** |
| `npm test` | lib golden tests, driver conformance, the seeded-timestamp rule, the gate |
| `npm run dict:assert` | **371/371 roadmap words resolve** (was 167 unglossable) |
| `npm run typecheck` · `npm run lint` | types, and the clock guard on `src/db/` |
CI runs them in that order, `validate.mjs` first.
## Sync and the tutor — optional, and genuinely optional
The app is complete without a server: its own SQLite, the shipped dictionary,
and a local stand-in tutor. Pointing it at a Pi adds two things — the real
선생님, and syncing between devices. Everything degrades to the offline
behaviour when the server is unreachable, and a sync failure is recorded in
settings rather than surfaced as an interruption.
Setup, the Caddy config, and how sync resolves conflicts: [server/README.md](server/README.md).
Three details worth knowing here:
- **Seeded rows carry `updated_at = 0`**, so a fresh device can neither push
its empty defaults nor win a conflict with them. The artifact's clobbering
bug is unrepresentable rather than merely avoided, and
`test/sync/roundtrip.test.ts` asserts it against a real Postgres.
- **An allowlist decides what leaves the device.** `meta` mixes preferences
with per-install bookkeeping; `dict.loadedBands` crossing the wire would
tell a phone it holds rows it never downloaded.
- **The system prompt is cached.** It is ~12k characters of gate, identical
for as long as the learner stays in one unit, so every turn after the first
reads it at a fraction of the input price.
- **The model is a config line, not a code change.** `HANKAN_TUTOR_BACKEND`
selects the Claude API, any OpenAI-compatible endpoint (LM Studio, Ollama,
llama.cpp, vLLM, OpenRouter), or a keyless echo backend for proving a
deployment. Keeping the system prompt first and unchanged is what makes
Anthropic's prefix cache work — and, it turns out, a local server's KV
cache too: 1,563ms to first token cold, 324ms warm.
## Not in this pass
Android beyond the existing Capacitor scaffold, and Play Store packaging.
## Known limitation
`lib/blocks.js` `parse()` does not close a `::gloss` block on its `=` line, so a
multi-sentence gloss collapses into one run-on line. Since `lib/` ships
unchanged, `app/src/domain/gloss.ts` splits the block and parses each sentence
separately. The behaviour is pinned in `test/lib/blocks.test.ts`; if the library
is ever revised, the one-line fix is `cur = null` and both workarounds go.