docs: the port as it now is — the enforced turn, the mobile shell, protocol 2

README: the bundle's checks and what they measure, stable lemma ids and
curriculum words as cards, the one resolver and the audit it matches, the
turn's rules (earned progress, recall evidence, the 다지기 checklist, the
letter-level check, the prompt's cached prefix), the five-destination shell
with history, answer mode and word lookup, and sync's three rules. The known
limitation is the one lib/blocks.js still has.

server/README: the endpoints as they are — systemTail, the paged pull, the
compare-and-swap push, the protocol header and its 426 — and how protocol 2
works: hydrate first, a counter not a clock, the copy that holds more wins,
shrinking only ever declared. The Caddy setup is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
MechaCat02
2026-09-16 22:14:25 +02:00
parent 57fd6e5087
commit e40aad637a
2 changed files with 227 additions and 126 deletions

241
README.md
View File

@@ -1,37 +1,43 @@
# Hankan · 한칸
A Korean reading tutor for manhwa. One codebase, two shells: a web app and an
Android app. **Works entirely offline — there is no server.**
Android app. **It works entirely offline; a server is optional.**
Ported from a single-file Claude artifact. The curriculum, the tutor prompt and
the five logic modules came finished and tested; this repo is the application
built around them.
Ported from a single-file Claude artifact, and migrated to its reworked 16 Sep
export. The curriculum, the tutor prompt, the logic modules and the checks
that measure them came finished and tested; this repo is the application built
around them.
---
## What is here
```
data/ lib/ prompt/ validate.mjs verbatim from the export bundle — do not edit
shared/ band table + phonological ladder, shared by app and build
data/ lib/ prompt/ fixtures/ verbatim from the export bundle — do not edit
validate.mjs audit-gate.mjs the bundle's own checks, and run-checks.sh to run both
shared/ band table, phonological ladder, lemma ids, sync protocol
types/ TypeScript declarations for lib/ and shared/
tools/dict/ the dictionary build pipeline
app/ Vite + React + TypeScript, and the Capacitor shell
src/db/ one storage interface, two SQLite drivers
src/domain/ the gate, the lexicon, SRS, the tutor clients
src/sync/ push/pull against the Pi
src/ui/ six tabs, the review overlay, the 한글 keyboard
src/db/ one storage interface, two SQLite drivers, the write rules
src/domain/ the gate, the resolver, the turn, the learner model, SRS
src/sync/ protocol 2: hydrate, compare-and-swap, resolve
src/ui/shell/ routes over history, the nav, layers, popovers
src/ui/tutor/ the lesson: conversation, exercises, answer mode, word lookup
src/ui/tabs/ 오늘, 설정, 단어, 문장, 문법, 활용 연습, 한글, 읽기 연습
src/ui/review/ the review screen
public/dict/ generated dictionary — committed, shipped
server/ sync + the tutor endpoint (optional)
tools/icons/ the 한칸 mark, generated
test/ lib goldens, driver conformance, domain, sync, SSE
test/ lib goldens, drivers, domain, the gate audit, sync, UI history
```
`data/`, `lib/`, `prompt/` and `validate.mjs` are byte-identical to the export
and are excluded from lint and formatting. `lib/conjugation.js` encodes the seven
Korean irregular classes and is the reason no runtime morphological analyser is
needed; `lib/hangul.js` implements 두벌식 composition. Types are supplied
alongside in `types/`, so neither file had to be touched.
`data/`, `lib/`, `prompt/`, `fixtures/`, `validate.mjs` and `audit-gate.mjs`
are byte-identical to the export and excluded from lint. Where the artifact's
inline code and `lib/` disagree, the port follows `lib/`, because `lib/` is
what `audit-gate.mjs` measures. Where `lib/` is looser than PORT.md, the
stricter rule sits at the call site and `lib/` stays untouched. Types for
`lib/` live alongside in `types/`.
## Running it
@@ -42,7 +48,7 @@ npm run dev # http://localhost:5173
```
```bash
npm run check # validate.mjs, typecheck, tests, roadmap assertion
npm run check # validate.mjs, the gate audit, typecheck, tests, roadmap assertion
npm run build # production build + service worker
```
@@ -58,122 +64,172 @@ npm run cap:sync
cd app/android && ./gradlew assembleDebug
```
`npx cap sync` copies `app/dist/` — including the dictionary into the APK's
assets, so the phone build is as offline as the web one.
`npx cap sync` copies `app/dist/`, dictionary included, into the APK's assets,
so the phone build is as offline as the web one.
## How it works
### Storage one interface, two drivers
### Storage: one interface, two drivers
`app/src/db/` exposes a single `Db` interface. `sqlite.web.ts` runs
`@sqlite.org/sqlite-wasm` in a dedicated Worker over the **OPFS SAHPool VFS**
(the plain OPFS VFS needs COOP/COEP headers, which neither a static host nor the
Capacitor webview reliably provides). `sqlite.native.ts` uses the Capacitor
SQLite plugin. Both apply the same migration array from `migrations.ts`.
SQLite plugin. Both apply the same migrations from `migrations.ts`.
**Seeded state never carries a write timestamp.** The artifact had a sync bug
where a fresh device stamped its own empty defaults as newer than the server's
real history and clobbered it. Three layers stop that from being expressible:
every syncable table declares `updated_at INTEGER NOT NULL DEFAULT 0`, so
forgetting the column is the safe failure; `db/writes.ts` splits every mutation
into `seedX()` (never stamps) and `editX()` (always stamps) and is the only file
allowed to read the clock; an ESLint rule enforces that, and the conformance
suite asserts a freshly seeded database has no non-zero `updated_at` anywhere.
real history and clobbered it. `db/writes.ts` splits every mutation into
`seedX()` (never stamps, never dirty) and `editX()` (stamps, marks the row
dirty, bumps its revision), and it is the only file allowed to read the clock.
An ESLint rule enforces that, and the conformance suite asserts a freshly seeded
database has nothing to push.
### The dictionary
`npm run dict:build` merges a dictionary source, a frequency list, the curated
data in `data/`, and a hand-written grammar lexicon, then runs
data in `data/` and a hand-written grammar lexicon, then runs
`lib/conjugation.js → surfaceForms()` over every verb and adjective to fill the
`surface` table. Lookup of a conjugated form is therefore an index hit, not an
analysis — **no runtime morphological analyser ships**.
`surface` table. Looking up a conjugated form is an index hit, not an analysis.
**No runtime morphological analyser ships.**
**A card names its word, not a build position.** A lemma's id is a hash of its
headword and part of speech (`shared/lemma-id.mjs`), so rebuilding the
dictionary can never quietly move a card onto a different word. The band files
carry no ids at all, and a changed build reloads its rows without touching
cards. Every word the roadmap introduces is exactly one card, tagged with its
unit, including the sixteen forms (봐, 갔어, 먹었어 …) no dictionary lists.
Sources are chosen automatically: KRDICT if a download has been vendored (it
has curated learner glosses and a graded 초급/중급/고급 level), otherwise the
kaikki.org Korean extract. KRDICT's download page is a JavaScript form behind
anti-bot protection, so it cannot be fetched by a script — see the comment at
the top of `tools/dict/fetch.mjs`.
anti-bot protection, so a script cannot fetch it. The comment at the top of
`tools/dict/fetch.mjs` explains.
Frequency needs care: a subtitle frequency list holds *surface* forms while a
dictionary holds lemmas, and a naive join on the headword gives every verb a
frequency of roughly zero. `tools/dict/freq-forms.mjs` inverts the join
expanding each lemma into the forms it plausibly takes and summing — which
frequency of roughly zero. `tools/dict/freq-forms.mjs` inverts the join,
expanding each lemma into the forms it plausibly takes and summing. That
recovers 하다 from 118 to 89,041.
Output is one gzipped row dump per band, committed under `app/public/dict/`.
They are static assets in the bundle and in the APK's assets, so no server is
involved. Attribution is in [NOTICE.md](NOTICE.md); share-alike attaches to the
Attribution is in [NOTICE.md](NOTICE.md); share-alike attaches to the
dictionary data, not to this code.
### The gate
### The gate: what the tutor may use
`lib/gate.js` computes what the tutor may teach from the curriculum plus
`lib/gate.js` computes what the tutor may teach from the curriculum and
progress, and `renderGate()` renders it into `{{GATE}}` in
`prompt/tutor-system.md`, which ships unchanged. `buildGate()` takes a
`vocabQuery` hook; `app/src/domain/gate.ts` fills it with a frequency-band query
so vocabulary grows as units are finished. Three refinements sit inside that
hook, all of them narrowing:
`prompt/tutor-system.md`, which ships unchanged.
1. words a not-yet-finished unit is the first to introduce are excluded, so a
band ceiling cannot smuggle 3.4's material into 2.1;
2. during Phase 1 the results are filtered by the same phonological ladder
`validate.mjs` checks, so a band cannot hand the learner a 겹받침 at 1.4;
3. the list is capped at 800 by frequency, because `renderGate()` inlines it
into the prompt. The cap is strictly more restrictive than the band.
**One resolver.** "Which dictionary word is this form?" is asked when the
learner taps a word and when the gate checks the tutor. `lib/lexicon.js`
answers both, with the SQLite dictionary behind it only where it has no route
(`domain/resolver.ts`). The order was measured on the bundle's 54 real tutor
messages: the dictionary alone flags 4 and 2 (too weak; 마셔 becomes 마시다
again), `lib/lexicon.js` first flags **7 and 2**, the shipped gate's numbers.
`test/domain/gate-audit.test.ts` reproduces the audit through the app's own
database, loader and resolver.
Confidence is clamped to `MAX_DELTA_PER_TURN` per turn. The artifact wrote the
model's number straight into the advancement gate, so one hallucinated
`::progress 95` could skip a unit.
**What is allowed:** the words of finished units, the current unit's words and
revisits, every word he has met (any card out of "new"), and, from Phase 2, the
frequency band. The words the tutor is *told* it may use and the words it is
*checked* against are one set.
### The tutor
**The client enforces; the prompt only explains.** Every reply is scanned before
he sees it (`scanTask`, `proseIsKorean`). A refused draft is never stored,
shown or applied: the tutor is asked again and told exactly why, twice at most.
After that the reply is shown with its words flagged, and the next turn names
them.
`app/src/domain/stub-tutor.ts` stands in for the model and implements the exact
contract the real endpoint will (`onText` receives cumulative text; an aborted
turn keeps what it streamed). It rotates all four exercise types, builds its
`::words` from the current unit's real vocabulary, and climbs `::progress`
gradually, so every render path — including the 85% advancement banner — is
reachable with no server. Swapping in the Pi's SSE endpoint later touches only
that file.
### The turn: what marking may change
The turn lives in `domain/turn.ts`, where it can be tested without React.
- **Progress is earned.** `::progress` is ignored until the unit has an answer.
A rise is capped at +25 per message and a fall is honoured in full. The next
unit is offered only at 85% with three answers behind it, and in a 다지기
review only once nothing on its checklist is open. `advanceUnit()` enforces
this too, not only the banner.
- **Recall evidence.** `::result` feeds a per-word record kept apart from the
schedule. A word is *learned* after three correct answers in separate rounds,
at least five rounds apart, none after a lookup. The schedule takes at most
one good grade a day from marking; in the artifact, five good rounds in one
afternoon made a word "secure" by interval alone.
- **Phase reviews.** The client holds the 다지기 checklist, each unit's rule
and every word the phase introduced (132 items for Phase 1), worked in
batches of ten. `::confirmed` ticks a rule on the tutor's word but a word
only on evidence.
- **The letter-level check.** The model cannot see jamo, so a recall answer
carries the app's own comparison (`letterCheck`) for the tutor to quote.
- **The prompt keeps its cache.** The per-round sections (the practice set,
the checklist, retry notes) follow the shipped prompt as a second, uncached
system block, so the unchanged prefix still hits Anthropic's prompt cache and
a local server's KV cache.
### The screen
Mobile-first, after the reworked artifact. Five destinations: 선생님, 복습
(review), 오늘, 단어, 학습. They sit in a bottom bar under 600px, a 76px rail
to 840px and a labelled rail beyond. The page never scrolls; each route owns
one scroller.
- **Routes are history** (`ui/shell/history.ts`). The back gesture leaves a
sub-page, and the word sheet, a popover and the review screen are layers
whose entries Back closes first. The rules are tested against a history
whose traversals land late, as a browser's do.
- **The lesson** is one column: roadmap strip, conversation, quick replies,
composer. The word list is docked from 840px and a sheet with three detents
below it, taking its height from the shell so nothing being answered is
covered.
- **Answer mode.** Below 840px, focusing an exercise's answer strips the
screen back to the question, the field and an answer bar. The 한글 keyboard
follows whichever field was touched last and comes up by itself for a
recall exercise.
- **Any word can be looked up**, in three tiers: an underline on every Korean
word showing what it is to him, a popover beside it (and English → 한국어 on
an exercise's English), and the full word list. A chip glosses on a long
press or with 힌트. Every lookup counts as a lookup in the answer.
### The stand-in tutor
`app/src/domain/stub-tutor.ts` implements the same contract as the real
endpoint and exercises every path offline: all five exercise types, `::words`
from the current unit, `::result`, `::confirmed`, and progress reported only
after answers. The app is complete with no server.
## Checks
| Command | What it guards |
|---|---|
| `node validate.mjs` | the curriculum. **PASS — 0 blocking, 0 advisory** |
| `npm test` | lib golden tests, driver conformance, the seeded-timestamp rule, the gate |
| `npm run dict:assert` | **371/371 roadmap words resolve** (was 167 unglossable) |
| `npm run typecheck` · `npm run lint` | types, and the clock guard on `src/db/` |
| `sh run-checks.sh` | `validate.mjs` (the curriculum, **PASS — 0 blocking, 0 advisory**) and `audit-gate.mjs` (the word gate against 54 real tutor messages, **7 of 41 and 2 of 13 flagged**) |
| `npm test` | lib goldens, driver conformance, the seed rule, the gate audit through the port, the turn, sync against a real Postgres, UI history |
| `npm run dict:assert` | **371/371 roadmap words resolve**, each as exactly one card |
| `npm run typecheck` · `npm run lint` | types, hooks, and the clock guard on `src/db/` |
CI runs them in that order, `validate.mjs` first.
CI runs `run-checks.sh` first, before anything is installed. The sync tests need
Postgres and a running server; `.github/workflows/ci.yml` starts both, and
[server/README.md](server/README.md#running-it-locally) shows how locally.
## Sync and the tutor — optional, and genuinely optional
## Sync and the tutor: optional
The app is complete without a server: its own SQLite, the shipped dictionary,
and a local stand-in tutor. Pointing it at a Pi adds two things the real
선생님, and syncing between devices. Everything degrades to the offline
behaviour when the server is unreachable, and a sync failure is recorded in
settings rather than surfaced as an interruption.
and the stand-in tutor. Pointing it at a Pi adds two things, the real 선생님
and syncing between devices. When the server is unreachable, everything falls
back to offline behaviour, and a failed send puts the message back in the box.
Setup, the Caddy config, and how sync resolves conflicts: [server/README.md](server/README.md).
Sync is built around three rules, each bought with lost work in the artifact:
Three details worth knowing here:
1. **Hydrate before pushing.** A device pulls everything first. A week-old
laptop cannot overwrite a week of work on the phone.
2. **A counter, not a clock.** A row is written only if the device last saw
the server's current version of it. Clock skew decides nothing.
3. **No silent shrinking.** When two copies disagree, the one that holds more
wins: more reviews, more answers, the further unit, the union of notes. A
deliberate deletion (a reset, a cleared lesson, a forgotten card) is
declared and obeyed.
- **Seeded rows carry `updated_at = 0`**, so a fresh device can neither push
its empty defaults nor win a conflict with them. The artifact's clobbering
bug is unrepresentable rather than merely avoided, and
`test/sync/roundtrip.test.ts` asserts it against a real Postgres.
- **An allowlist decides what leaves the device.** `meta` mixes preferences
with per-install bookkeeping; `dict.loadedBands` crossing the wire would
tell a phone it holds rows it never downloaded.
- **The system prompt is cached.** It is ~12k characters of gate, identical
for as long as the learner stays in one unit, so every turn after the first
reads it at a fraction of the input price.
- **The model is a config line, not a code change.** `HANKAN_TUTOR_BACKEND`
selects the Claude API, any OpenAI-compatible endpoint (LM Studio, Ollama,
llama.cpp, vLLM, OpenRouter), or a keyless echo backend for proving a
deployment. Keeping the system prompt first and unchanged is what makes
Anthropic's prefix cache work — and, it turns out, a local server's KV
cache too: 1,563ms to first token cold, 324ms warm.
Setup, the Caddy config and the protocol: [server/README.md](server/README.md).
## Not in this pass
@@ -181,8 +237,9 @@ Android beyond the existing Capacitor scaffold, and Play Store packaging.
## Known limitation
`lib/blocks.js` `parse()` does not close a `::gloss` block on its `=` line, so a
multi-sentence gloss collapses into one run-on line. Since `lib/` ships
unchanged, `app/src/domain/gloss.ts` splits the block and parses each sentence
separately. The behaviour is pinned in `test/lib/blocks.test.ts`; if the library
is ever revised, the one-line fix is `cur = null` and both workarounds go.
`lib/blocks.js` `parse()` builds a message's prose by deleting each block with a
pattern that also eats the `::` opening the block after it. It also runs a block
on to the next `::`, so when a gloss block is missing its closing `::` (local
models forget it routinely) the paragraph after it is read as gloss rows. Since
`lib/` ships unchanged, `app/src/domain/gloss.ts` reads the prose and the gloss
blocks from the raw text; the header of that file has the details.

View File

@@ -8,13 +8,16 @@ entirely offline against its own SQLite and a local stand-in tutor. Adding a
server turns on two things: the real 선생님, and syncing between devices.
```
POST /api/tutor {system, history, message} → SSE token stream
GET /api/sync ?cursor=N → rows newer than the cursor
POST /api/sync {rows} → upsert, last-write-wins
POST /api/tutor {system, systemTail, history, message} → SSE token stream
GET /api/sync ?cursor=N → {epoch, rows, cursor, more}: a page of rows after N
POST /api/sync {rows} → {epoch, applied, conflicts}: compare-and-swap writes
GET /health → no auth, for the healthcheck
```
Everything under `/api` requires `Authorization: Bearer $HANKAN_TOKEN`.
Everything under `/api` requires `Authorization: Bearer $HANKAN_TOKEN`, and
`/api/sync` also requires `x-hankan-protocol: 2`. An older app gets
**426 Upgrade Required** rather than a write the server would misread. See
[How sync works](#how-sync-works).
### CORS — required for the phone
@@ -42,8 +45,11 @@ CREATE DATABASE hankan OWNER hankan;
SQL
```
The schema applies itself on boot `server/sql/001-schema.sql` is idempotent,
so there is no migration step to run by hand.
The schema applies itself on boot. `server/sql/001-schema.sql` and
`002-protocol-2.sql` are idempotent, so there is no migration step to run by
hand. The first boot on protocol 2 drops any protocol-1 rows once; every
device then hydrates and offers its own database back, which was always the
source of truth.
### 2. Configure
@@ -98,30 +104,62 @@ heartbeat every 15s, which defeat most intermediary caching and idle timeouts.
### 4. Connect the app
In the app: 오늘 → 서버 → the URL and the token. It syncs on connect, when the
tab regains focus, and every five minutes.
In the app: 오늘 → ⚙ 설정 → Server and sync → the URL and the token. It syncs
on connect, when the app regains focus, and every five minutes.
## How sync works
Row-level, last-write-wins on `updated_at`, cursor-based on a server-assigned
`change_seq`. One user, so the loser of a conflict is at worst one SRS grade.
Protocol 2 (`shared/sync-protocol.mjs`, `app/src/sync/`, `server/src/db.ts`).
It is row-level, and every row carries three columns on the device: `base_seq`,
the server version it last agreed with; `dirty`, set by every edit; and `rev`,
which tells a push that lands after a further edit not to clear it.
Rows are stored generically — primary key as text, body as JSONB — because the
server never reads inside a row. It stores and orders them; the client
interprets them. That keeps the two schemas from having to move in lockstep.
The artifact lost a week of work to its sync, and each of the three rules
below closes one way that happened.
Three things are load-bearing:
**1. Hydrate before pushing.** A device pulls every page before it may push
anything. A laptop last opened a week ago comes back, adopts the week of work
the phone did, and only then offers its own edits. Changing the server, or the
server's *epoch* (a random id replaced whenever its copy is thrown away),
forces a full hydration again.
- **`change_seq` advances on every update, via a trigger.** A row edited after
a client last pulled would otherwise sit below that client's cursor and
never be delivered. Putting it in a trigger means no write path can forget.
- **The pull cursor advances only as rows are applied**, never from the push
response. The server's newest `change_seq` includes rows this device has not
seen; adopting it would skip them permanently, and nothing would ever ask
for that range again.
- **Deletes travel as tombstones.** A deleted row leaves nothing to compare
timestamps against, so without one the other device pushes its still-live
copy back and the row silently returns.
**2. A counter, not a clock.** The server keeps a `change_seq` per row and
writes a pushed row only if the push's `base` equals it: compare-and-swap.
Anything else comes back as a conflict, with the server's current copy.
`updated_at` is kept for display and decides nothing, so a device with its
clock an hour out cannot win by it. Pushes are serialised and pulls take a
shared advisory lock, so a pull can never skip a `change_seq` that a
concurrent push is about to commit.
**3. No silent shrinking.** A conflict goes to `app/src/sync/resolve.ts`,
where the copy that holds more wins: a card with more reviews behind it,
evidence with more observed, progress merged (done if either finished it,
the higher answer count), the further unit on the roadmap, grammar notes and
flags unioned, the better reading-drill round. A tie goes to the server, so
every device lands on one copy.
Shrinking is always declared, never inferred:
- **Deliberate deletes** travel as tombstones, and a delete is obeyed against a
concurrent edit: a card forgotten on one device stays forgotten.
- **Resets** (clearing the lesson, resetting the roadmap or everything) raise
a marker, `reset.<scope>`, a counter every device compares with the last one
it applied. A device that had not heard of the reset empties the same
things, its own offline work included, because a reset it did not know about
still wins.
- **Trimming the transcript is local.** A device keeps its last turns on
screen, but the trim is never a deletion, and a turn not yet pushed is never
trimmed. The artifact's trim tombstoned the other device's turns.
Also fixed on the way: keys are JSON arrays, so a key containing a space
(몇 명) no longer stops sync permanently; chat ids are UUIDv7, so two devices
writing offline cannot collide; and the study log and peek tallies are kept per
device and summed, so two devices' reviews of the same day both count.
Rows are stored generically on the server, the primary key as text and the
body as JSONB, because the server never reads inside a row. It stores, orders
and versions them; the client interprets them. That keeps the two schemas from
having to move in lockstep.
### What never syncs
@@ -133,13 +171,15 @@ every row the desktop has, and the word rail would then fail to find words it
believes are present. `server.token`, `sync.*` and `schema_version` are
excluded for related reasons.
### The bug this schema is shaped around
### The bug this is shaped around
The original artifact stamped a fresh device's empty defaults as newer than
the server's real history, and clobbered it. Here seeded and defaulted rows
carry `updated_at = 0`, so they can never be dirty and can never win a
conflict. It is not avoided, it is unrepresentable —
`test/sync/roundtrip.test.ts` asserts it against a real Postgres.
the server's real history, and clobbered it. Here a seeded row is never dirty,
so it has nothing to push, and a device hydrates before it pushes at all. The
week-old-laptop case, clock skew, offline chat on two devices, resets against
devices that missed them, a forget racing a review, keys with spaces, and a
server that lost its data are each a scenario in `test/sync/roundtrip.test.ts`,
run against a real Postgres.
## The tutor
@@ -147,10 +187,14 @@ conflict. It is not avoided, it is unrepresentable —
owns, and the new message; it holds nothing between requests, so a dropped
connection costs one turn rather than the conversation.
The system prompt is passed as a cached block. It is ~12k characters of gate
and is byte-identical for as long as the learner stays in one unit — many
turns — so every turn after the first reads the prefix at a fraction of the
input price. This is the single biggest cost lever in the design.
The system prompt comes in two parts. `system` is the shipped prompt with the
gate filled in, ~12k characters that stay byte-identical for as long as the
learner stays in one unit. It is sent as a cached block, so every turn after
the first reads it at a fraction of the input price, the single biggest cost
lever in the design. `systemTail` holds what changes every round (the
practice set, the 다지기 checklist, the reason a draft was refused) and follows
as a second, uncached block, so it never breaks the cached prefix. The
OpenAI-compatible backend joins the two, prefix first.
### Choosing a model