Three unit-1.1 lessons with gpt-oss-20b through LM Studio and the server —
the first real model on the reworked turn. The letter-level check went out
right, and the +25 clamp held: a reported 80 on the first answer was stored
as 25. What failed was how the model wrote its blocks, a different way each
session. All three transcripts are in test/fixtures/, verbatim, and each
failure below is a test against them.
Marks lost. The prompt asks for `여덟 | wrong | 여덜`. The first session wrote
`we | wrong | 우라 → 우리`, English prompt first; the second wrote no ::result
at all and marked only in prose, `✗ 나 | I (humble) → 저`. evidence.ts keys on
the first field of a ::result row, so nothing was ever recorded — no
evidence, no schedule, no confusions, and a 다지기 review that could never
close. The artifact would have lost them the same way. domain/marking.ts
attaches each mark to its word only where that is unambiguous: one Korean
word first, or through a prompt of the exercise he answered, read via that
exercise's ::words as the letter check reads it. With no ::result block the
✓/✗ lines are read on the same terms, so a mark can never name a word the
exercise did not ask for; a mark on a whole sentence is still dropped. What
he mistook a word for is taken from what he actually wrote whenever the mark
itself gives no other word — the third session put the right answer there.
Its third session, marked through all of this: 20 evidence rows, 20 cards.
Progress on requests. The prompt allows marks, ::confirmed and ::progress
only in reply to an answer. The model wrote ::progress on every message, and
three requests for a new exercise took the unit from 50% to 80% with nothing
answered. A reply to anything but an answer now changes none of them.
Feedback swallowed. The model closed no blocks, so lib read what followed
each one as rows: "your score is about 5%" became a result row the student
never saw, and a "---" became a recall item he was asked to write in 한글.
Another session fenced every block in ```. gloss.ts now decides every
block's extent from the raw text — at "::", the next block, a rule or fence
line, a blank line with no row after it, or for the piped blocks the first
line without a "|" — and hands lib the blocks properly closed. The gate
audit, now run through the parser the lesson uses, still flags 7 and 2.
Answers given away. Translate rows came with their meanings ("나 | I") and
recall hints were the answers ("two | 이"). A translate row keeps only its
Korean line, and a recall hint that is the expected word, or any word the
message declares, is dropped.
Also: the spelling a recall prompt expects now keeps its qualifiers. With 나
"I, me (casual)" and 저 "I, me (humble)" in one list, "I (humble)" matched
나: no letter check was sent, and the mark for 저 was filed under 나.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The bundle's audit-gate.mjs measures the word gate against 54 real tutor
messages. Scored the same way, the port's gate was not the gate that was
measured:
port's allowed set 13 · 2 over-fires on words he had met
port's dictionary resolver 4 · 2 too weak: 마셔 → 마시다 again
lib/lexicon.js, DB behind it 7 · 2 the shipped gate
domain/resolver.ts makes lib/lexicon.js the one resolver for word taps and
the gate, built from the shipped data in the audit's order — roadmap words
first, so a scheduled form cannot inherit its stem's permission — with the
dictionary consulted only where lexicon.js has no route. That fallback is
what the port adds over the artifact: a word the curated data does not know
is still recognised as a real word.
The allowed set gains every word he has met (any card out of "new"), as
the shipped gate has it; the frequency band stays on top from Phase 2, as
PORT.md specifies. The words the tutor is TOLD it may use and the words it
is CHECKED against are one set, so it is never refused for a word it was
offered.
test/domain/gate-audit.test.ts reproduces the audit through the app's own
code — database, loader, resolver — and gets the shipped 149-word allowed
set and the same 7 and 2 messages, word for word.
Also: {{VARIETY}} and {{FOCUS}} take the artifact's wording and know the
fifth exercise type, recall; and the prompt's maintainer header, which
explains the placeholders, is no longer sent to the model.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
