Three unit-1.1 lessons with gpt-oss-20b through LM Studio and the server —
the first real model on the reworked turn. The letter-level check went out
right, and the +25 clamp held: a reported 80 on the first answer was stored
as 25. What failed was how the model wrote its blocks, a different way each
session. All three transcripts are in test/fixtures/, verbatim, and each
failure below is a test against them.
Marks lost. The prompt asks for `여덟 | wrong | 여덜`. The first session wrote
`we | wrong | 우라 → 우리`, English prompt first; the second wrote no ::result
at all and marked only in prose, `✗ 나 | I (humble) → 저`. evidence.ts keys on
the first field of a ::result row, so nothing was ever recorded — no
evidence, no schedule, no confusions, and a 다지기 review that could never
close. The artifact would have lost them the same way. domain/marking.ts
attaches each mark to its word only where that is unambiguous: one Korean
word first, or through a prompt of the exercise he answered, read via that
exercise's ::words as the letter check reads it. With no ::result block the
✓/✗ lines are read on the same terms, so a mark can never name a word the
exercise did not ask for; a mark on a whole sentence is still dropped. What
he mistook a word for is taken from what he actually wrote whenever the mark
itself gives no other word — the third session put the right answer there.
Its third session, marked through all of this: 20 evidence rows, 20 cards.
Progress on requests. The prompt allows marks, ::confirmed and ::progress
only in reply to an answer. The model wrote ::progress on every message, and
three requests for a new exercise took the unit from 50% to 80% with nothing
answered. A reply to anything but an answer now changes none of them.
Feedback swallowed. The model closed no blocks, so lib read what followed
each one as rows: "your score is about 5%" became a result row the student
never saw, and a "---" became a recall item he was asked to write in 한글.
Another session fenced every block in ```. gloss.ts now decides every
block's extent from the raw text — at "::", the next block, a rule or fence
line, a blank line with no row after it, or for the piped blocks the first
line without a "|" — and hands lib the blocks properly closed. The gate
audit, now run through the parser the lesson uses, still flags 7 and 2.
Answers given away. Translate rows came with their meanings ("나 | I") and
recall hints were the answers ("two | 이"). A translate row keeps only its
Korean line, and a recall hint that is the expected word, or any word the
message declares, is dropped.
Also: the spelling a recall prompt expects now keeps its qualifiers. With 나
"I, me (casual)" and 저 "I, me (humble)" in one list, "I (humble)" matched
나: no letter check was sent, and the mark for 저 was filed under 나.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The 16 Sep lib fixed both defects gloss.ts used to work around: a gloss
block closes at each "=" line, and every block is collected. So the
sentence splitting goes.
Three defects remain in how a block ENDS, measured on the new lib:
· deleting one block still deletes the "::" of the next, so a whole
::words block reaches the prose as "words" plus its rows;
· an unclosed gloss still swallows the following paragraph as parts;
· two gloss blocks written back to back share one terminator, and lib
never sees the second.
The old rebuild also only re-parsed the FIRST gloss block, which against
a lib that accumulates would have dropped every block after it.
One pass over the raw text now yields both the prose and each gloss
block's content, a gloss block ending where its rows end; lib parses each
block on its own, so the row format is still entirely lib's.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The artifact was reworked after real incidents: a week of lost data, a
student taught out of order, and spelling diagnoses the model invented.
This takes the new export in verbatim; the port catches up in the
commits that follow.
Copied byte-identical from the bundle:
lib/ lexicon.js and sync.js are new; gate.js gains enforcement,
hangul.js letter-level marking, srs.js recall evidence,
conjugation.js deconjugate(); blocks.js now takes the last
block, closes gloss at "=", and parses recall, ::result and
::confirmed
data/ curriculum.json v5 — six 다지기 phase reviews; the 371
roadmap words are unchanged and no band moves
prompt/ English-only rule, recall, LETTER-LEVEL CHECK, marking
audit-gate.mjs, run-checks.sh, fixtures/ — the word gate measured
against 54 real tutor messages
CI runs run-checks.sh in place of validate.mjs alone, and `npm run check`
gains the audit. Baselines: validate PASS 0/0; audit 7 of 41 and 2 of 13.
types/lib/ declares the new API, and test/lib/ pins it: letterCheck on
the prompt's own 짧다/빫다 case, deconjugation, the roadmap-first order
that keeps 마셔 out of Phase 1, sync's three gates, and recall evidence —
including the two ways lib's evidence is looser than PORT.md, pinned as
they are so the call site that tightens them is visibly needed.
TaskHost gains a plain recall renderer so the tree typechecks against the
wider Task union.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Reported again: "::task translate" and its four lines showing as text above
the exercise they had been rendered into. The previous fix dropped
everything from the first line beginning "::", and could not see this one,
because by the time it ran the colons were gone:
**Example sentence**
*나 바다*
task translate <- the "::" removed, the items left as prose
나 바다
lib/blocks.js removes each block by string surgery, and a block's
terminator `(?:\n::|$)` is inside its own match, so removing one block
takes the two colons belonging to the NEXT one with it. Last time the
casualty was ::task before ::words; this reply put ::gloss first and the
casualty was ::task. Repairing that body cannot be made to work in general.
So the body is no longer repaired, it is derived: proseOf() reads the RAW
text, where the markers are always intact, and keeps every line outside a
block. The grammar is small -- a line of exactly "::" closes, any other
"::" line opens, a block runs until closed, until the next opens, or to the
end -- and nothing compares exact strings, because this model ends every
line with markdown's two trailing spaces.
An unclosed ::gloss is handled too, and every local model tested forgets
that closer. lib then reads the following paragraph as gloss parts, so two
sentences of English rendered as Korean example text inside the card. A
gloss row is `한글 | English | note` and a translation starts with "=";
anything else ends the block. The stranded prose goes back into the
message, and the gloss is now rebuilt unconditionally rather than only for
multi-sentence blocks, since the single-sentence case is exactly where this
bites.
Separately, MessageBody degrades gracefully on markdown instead of showing
it raw. The prompt forbids all of it and says so outright, but the backend
is pluggable now and a local model ignores the rule: a bulleted list of the
ten consonants arrived as lines starting with a hyphen, "*What we learn:*"
kept its asterisks, "---" showed as three dashes. Bullets, headings, rules
and *italic* now render quietly.
Verified by serving the exact reported reply through the real app: 10
bullets rendered as rows, no literal markup, no directive text, the gloss
card holding only 나 | 바다 = "I sea", and the exercise as real UI.
Also worth recording: three browser checks in this session were reading a
stub-generated turn synced down from Postgres, not the model. The app boots
on 수업, so the local stand-in writes the opening turn before a server can
be configured, and a fresh client then pulls the old transcript. Sending a
message after connecting is what actually exercises the remote path.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Reported from the app: "::task translate" and its five sentences appeared
as text in the message, directly above the exercise those same lines had
been rendered into.
It is lib/blocks.js, not the model. parse() removes ::words by truncating
the body at its index, then removes ::task by substring:
if (w) body = body.slice(0, body.indexOf("::words"));
if (t) body = body.replace(t[0], "");
RE.task's terminator (?:\n::|$) is part of the match, so t[0] ends with the
"\n::" belonging to the ::words that follows -- the two colons the line
above just truncated away. The substring no longer occurs, replace() is a
no-op, and the whole task block stays in the body.
Order is the whole trigger. The stub tutor emits ::words before ::task and
is therefore fine; the local model emitted ::task first. Nothing in the
prompt requires either order, so this was always reachable -- Claude would
hit it too. It survived every test until a real model chose the other way.
lib/ ships unchanged, so the fix is at the call site, next to the ::gloss
workaround that is there for the same reason: parseMessage() drops the body
from the first surviving directive line on. Safe precisely because parse()
has already removed the blocks it handled correctly, so a "::" still in the
body is by definition one that leaked. That also subsumes the streaming
filter added earlier, which is now one rule instead of two.
A turn can also be nothing but blocks -- this model writes no prose around
an exercise at all -- which left an empty bubble above it. The bubble is
skipped when there is nothing to put in it, and the typing dots stay up
while the reply so far is only markup, since there is genuinely nothing to
read yet.
test/domain/block-leak.test.ts pins the lib behaviour as it is, and the
workaround against a verbatim capture of the model output that produced the
report.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
