fix(tutor): read the message's prose from the raw text, not parse()'s body
Reported again: "::task translate" and its four lines showing as text above
the exercise they had been rendered into. The previous fix dropped
everything from the first line beginning "::", and could not see this one,
because by the time it ran the colons were gone:
**Example sentence**
*나 바다*
task translate <- the "::" removed, the items left as prose
나 바다
lib/blocks.js removes each block by string surgery, and a block's
terminator `(?:\n::|$)` is inside its own match, so removing one block
takes the two colons belonging to the NEXT one with it. Last time the
casualty was ::task before ::words; this reply put ::gloss first and the
casualty was ::task. Repairing that body cannot be made to work in general.
So the body is no longer repaired, it is derived: proseOf() reads the RAW
text, where the markers are always intact, and keeps every line outside a
block. The grammar is small -- a line of exactly "::" closes, any other
"::" line opens, a block runs until closed, until the next opens, or to the
end -- and nothing compares exact strings, because this model ends every
line with markdown's two trailing spaces.
An unclosed ::gloss is handled too, and every local model tested forgets
that closer. lib then reads the following paragraph as gloss parts, so two
sentences of English rendered as Korean example text inside the card. A
gloss row is `한글 | English | note` and a translation starts with "=";
anything else ends the block. The stranded prose goes back into the
message, and the gloss is now rebuilt unconditionally rather than only for
multi-sentence blocks, since the single-sentence case is exactly where this
bites.
Separately, MessageBody degrades gracefully on markdown instead of showing
it raw. The prompt forbids all of it and says so outright, but the backend
is pluggable now and a local model ignores the rule: a bulleted list of the
ten consonants arrived as lines starting with a hyphen, "*What we learn:*"
kept its asterisks, "---" showed as three dashes. Bullets, headings, rules
and *italic* now render quietly.
Verified by serving the exact reported reply through the real app: 10
bullets rendered as rows, no literal markup, no directive text, the gloss
card holding only 나 | 바다 = "I sea", and the exercise as real UI.
Also worth recording: three browser checks in this session were reading a
stub-generated turn synced down from Postgres, not the model. The app boots
on 수업, so the local stand-in writes the opening turn before a server can
be configured, and a fresh client then pulls the old transcript. Sending a
message after connecting is what actually exercises the remote path.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -18,8 +18,9 @@
|
||||
The fixture is a verbatim capture from gpt-oss-20b through the server. */
|
||||
|
||||
import { describe, it, expect } from "vitest";
|
||||
import { readFileSync } from "node:fs";
|
||||
import { parse } from "@lib/blocks.js";
|
||||
import { parseMessage, stripLeakedBlocks } from "@app/domain/gloss.js";
|
||||
import { parseMessage, proseOf, glossContent } from "@app/domain/gloss.js";
|
||||
|
||||
import type { ParsedMessage } from "@lib/blocks.js";
|
||||
|
||||
@@ -79,14 +80,130 @@ describe("parseMessage — the workaround", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("stripLeakedBlocks", () => {
|
||||
describe("proseOf", () => {
|
||||
it("does nothing without a directive", () => {
|
||||
expect(stripLeakedBlocks("a\n\nb")).toBe("a\n\nb");
|
||||
expect(proseOf("a\n\nb")).toBe("a\n\nb");
|
||||
});
|
||||
it("cuts from the first directive line", () => {
|
||||
expect(stripLeakedBlocks("keep\n::task translate\ndrop\n::words\ndrop")).toBe("keep");
|
||||
|
||||
it("drops every block, not merely from the first one on", () => {
|
||||
expect(proseOf("keep\n::task translate\ndrop\n::words\ndrop")).toBe("keep");
|
||||
});
|
||||
|
||||
it("resumes the prose after a block closes — gloss blocks sit inline", () => {
|
||||
expect(proseOf("before\n::gloss\n물 | N | water |\n::\nafter")).toBe("before\nafter");
|
||||
});
|
||||
|
||||
it("does not trip on a colon that is not at the start of a line", () => {
|
||||
expect(stripLeakedBlocks("see:: this")).toBe("see:: this");
|
||||
expect(proseOf("see:: this")).toBe("see:: this");
|
||||
});
|
||||
|
||||
it("ignores the trailing whitespace this model puts on every line", () => {
|
||||
// markdown hard line breaks: "::task translate "
|
||||
expect(proseOf("keep\n::task translate \n나 바다 \n::words \n물 | water")).toBe("keep");
|
||||
});
|
||||
|
||||
it("treats ::progress, which has no closing ::, as running to the end", () => {
|
||||
expect(proseOf("keep\n::progress 40 | going well")).toBe("keep");
|
||||
});
|
||||
});
|
||||
|
||||
/* A verbatim capture of gpt-oss-20b answering "Start unit 1.1." against the
|
||||
real assembled prompt — the reply behind the second report of raw markup
|
||||
showing above the exercise.
|
||||
|
||||
It differs from the first capture in the way that mattered: here ::gloss
|
||||
comes first, and removing it took the "::" off the ::task that followed,
|
||||
so what leaked was the bare word "task translate" with its five sentences
|
||||
under it. A rule that looked for lines beginning "::" could not see that,
|
||||
which is why the body is now read from the raw text instead. */
|
||||
describe("gpt-oss-20b, unit 1.1 — the reported reply", () => {
|
||||
const RAW = readFileSync(
|
||||
new URL("../fixtures/gpt-oss-20b-unit-1.1.txt", import.meta.url),
|
||||
"utf8",
|
||||
);
|
||||
|
||||
it("puts the blocks in the order that breaks lib/blocks.js", () => {
|
||||
const order = RAW.split("\n")
|
||||
.filter((l) => /^\s*::\w/.test(l))
|
||||
.map((l) => l.trim().split(/\s/)[0]);
|
||||
expect(order).toEqual(["::gloss", "::task", "::words", "::progress"]);
|
||||
});
|
||||
|
||||
it("renders the exercise", () => {
|
||||
expect(parseMessage(RAW).task?.type).toBe("translate");
|
||||
});
|
||||
|
||||
it("leaves no directive, and no de-colonised directive, in the prose", () => {
|
||||
const body = parseMessage(RAW).body;
|
||||
expect(body).not.toMatch(/(^|\n)\s*::/);
|
||||
// The exact shape of this leak: the marker stripped, the word left.
|
||||
expect(body).not.toMatch(/(^|\n)\s*(task|words|gloss|progress)\b/);
|
||||
});
|
||||
|
||||
it("does not repeat the exercise as prose above it", () => {
|
||||
const parsed = parseMessage(RAW);
|
||||
const task = parsed.task;
|
||||
// Task is a union; only the non-match kinds carry `items`.
|
||||
const raw = task && "items" in task ? (task.items as { q?: string }[]) : [];
|
||||
const items = raw.map((i) => i.q).filter((q): q is string => Boolean(q));
|
||||
expect(items.length).toBeGreaterThan(1);
|
||||
|
||||
/* Not "no item appears": this reply uses 나 바다 as a worked example in
|
||||
the prose AND as the first exercise line, which is legitimate. A leak
|
||||
drags in every line, so that is what to measure. */
|
||||
const echoed = items.filter((q) => parsed.body.includes(q));
|
||||
expect(echoed.length).toBeLessThan(items.length);
|
||||
});
|
||||
|
||||
it("keeps the teaching prose that came before the blocks", () => {
|
||||
expect(parseMessage(RAW).body).toContain("Korean syllables are written as blocks");
|
||||
});
|
||||
});
|
||||
|
||||
/* A gloss block the model forgot to close. Every local model tested does
|
||||
this, and lib/blocks.js then reads the following paragraph as gloss
|
||||
parts — two sentences of English rendered as Korean example text inside
|
||||
the card, at example-line size. */
|
||||
describe("an unclosed ::gloss block", () => {
|
||||
const RAW = [
|
||||
"Here is an example.",
|
||||
"::gloss",
|
||||
"나 | S | I",
|
||||
"바다 | O | sea",
|
||||
"= I sea",
|
||||
"",
|
||||
"(Here 바다 is two blocks.)",
|
||||
"Now practice these.",
|
||||
"::task translate",
|
||||
"나 바다",
|
||||
].join("\n");
|
||||
|
||||
it("ends the block where its rows end", () => {
|
||||
expect(glossContent("나 | S | I\n= I sea\n\nprose here\nmore prose")).toBe("나 | S | I\n= I sea\n");
|
||||
});
|
||||
|
||||
it("keeps only the real gloss rows in the card", () => {
|
||||
const gloss = parseMessage(RAW).gloss ?? [];
|
||||
expect(gloss).toHaveLength(1);
|
||||
expect(gloss[0]!.parts.map((p) => p.ko)).toEqual(["나", "바다"]);
|
||||
expect(gloss[0]!.en).toBe("I sea");
|
||||
});
|
||||
|
||||
it("returns the stranded prose to the message", () => {
|
||||
const body = parseMessage(RAW).body;
|
||||
expect(body).toContain("Here is an example.");
|
||||
expect(body).toContain("(Here 바다 is two blocks.)");
|
||||
expect(body).toContain("Now practice these.");
|
||||
// and still no exercise text or markup
|
||||
expect(body).not.toContain("나 바다");
|
||||
expect(body).not.toMatch(/::/);
|
||||
});
|
||||
|
||||
it("leaves a properly closed block exactly as lib parses it", () => {
|
||||
const closed = "::gloss\n물 | N | water\n= Water.\n::\nAfter.";
|
||||
const g = parseMessage(closed).gloss ?? [];
|
||||
expect(g).toHaveLength(1);
|
||||
expect(g[0]!.parts.map((p) => p.ko)).toEqual(["물"]);
|
||||
expect(parseMessage(closed).body).toBe("After.");
|
||||
});
|
||||
});
|
||||
|
||||
57
test/fixtures/gpt-oss-20b-unit-1.1.txt
vendored
Normal file
57
test/fixtures/gpt-oss-20b-unit-1.1.txt
vendored
Normal file
@@ -0,0 +1,57 @@
|
||||
Korean syllables are written as blocks that combine an **initial consonant** and a **vowel**.
|
||||
In this unit we only use the plain consonants **ㄱ ㄴ ㄷ ㄹㅁㅂㅅㅇㅈㅎ** and the ten basic vowels **ㅏ ㅑ ㅓ ㅕ ㅗ ㅛ ㅜㅠㅡㅣ**.
|
||||
|
||||
Each block has no final consonant, so a word is simply a sequence of such blocks.
|
||||
For example:
|
||||
|
||||
- ㄱㅏ (가)
|
||||
- ㄴㅏ (나)
|
||||
- ㄷㅏ (다)
|
||||
- ㄹㅏ (라)
|
||||
- ㅁㅏ (마)
|
||||
- ㅂㅏ (바)
|
||||
- ㅅㅏ (사)
|
||||
- ㅇㅏ (아)
|
||||
- ㅈㅏ (자)
|
||||
- ㅎㅏ (하)
|
||||
|
||||
A word is read by looking at each block in order.
|
||||
The learner’s mistake often is to treat the whole word as a single unit and miss the internal blocks.
|
||||
|
||||
**Example sentence**
|
||||
*나 바다*
|
||||
::gloss
|
||||
나 | S | I
|
||||
바다 | O | sea
|
||||
= I sea
|
||||
|
||||
(Here “바다” is two blocks: 바 (ㅂ+아) + 다 (ㄷ+아).)
|
||||
|
||||
Now practice reading and translating these short phrases.
|
||||
|
||||
::task translate
|
||||
나 바다
|
||||
그 나무
|
||||
어디 소리
|
||||
하나 바다
|
||||
|
||||
::words
|
||||
ㄱㅏ | ㄱ + ㅏ
|
||||
ㄴㅏ | ㄴ + ㅏ
|
||||
ㄷㅏ | ㄷ + ㅏ
|
||||
ㄹㅏ | ㄹ + ㅏ
|
||||
ㅁㅏ | ㅁ + ㅏ
|
||||
ㅂㅏ | ㅂ + ㅏ
|
||||
ㅅㅏ | ㅅ + ㅏ
|
||||
ㅇㅏ | ㅇ + ㅏ
|
||||
ㅈㅏ | ㅈ + ㅏ
|
||||
ㅎㅏ | ㅎ + ㅏ
|
||||
나 | I
|
||||
바다 | sea
|
||||
그 | he
|
||||
나무 | tree
|
||||
어디 | where
|
||||
소리 | sound
|
||||
하나 | one
|
||||
|
||||
::progress 0-100 | starting unit, basics to be mastered.
|
||||
Reference in New Issue
Block a user