fix(tutor): read the message's prose from the raw text, not parse()'s body
Reported again: "::task translate" and its four lines showing as text above
the exercise they had been rendered into. The previous fix dropped
everything from the first line beginning "::", and could not see this one,
because by the time it ran the colons were gone:
**Example sentence**
*나 바다*
task translate <- the "::" removed, the items left as prose
나 바다
lib/blocks.js removes each block by string surgery, and a block's
terminator `(?:\n::|$)` is inside its own match, so removing one block
takes the two colons belonging to the NEXT one with it. Last time the
casualty was ::task before ::words; this reply put ::gloss first and the
casualty was ::task. Repairing that body cannot be made to work in general.
So the body is no longer repaired, it is derived: proseOf() reads the RAW
text, where the markers are always intact, and keeps every line outside a
block. The grammar is small -- a line of exactly "::" closes, any other
"::" line opens, a block runs until closed, until the next opens, or to the
end -- and nothing compares exact strings, because this model ends every
line with markdown's two trailing spaces.
An unclosed ::gloss is handled too, and every local model tested forgets
that closer. lib then reads the following paragraph as gloss parts, so two
sentences of English rendered as Korean example text inside the card. A
gloss row is `한글 | English | note` and a translation starts with "=";
anything else ends the block. The stranded prose goes back into the
message, and the gloss is now rebuilt unconditionally rather than only for
multi-sentence blocks, since the single-sentence case is exactly where this
bites.
Separately, MessageBody degrades gracefully on markdown instead of showing
it raw. The prompt forbids all of it and says so outright, but the backend
is pluggable now and a local model ignores the rule: a bulleted list of the
ten consonants arrived as lines starting with a hyphen, "*What we learn:*"
kept its asterisks, "---" showed as three dashes. Bullets, headings, rules
and *italic* now render quietly.
Verified by serving the exact reported reply through the real app: 10
bullets rendered as rows, no literal markup, no directive text, the gloss
card holding only 나 | 바다 = "I sea", and the exercise as real UI.
Also worth recording: three browser checks in this session were reading a
stub-generated turn synced down from Postgres, not the model. The app boots
on 수업, so the local stand-in writes the opening turn before a server can
be configured, and a fresh client then pulls the old transcript. Sending a
message after connecting is what actually exercises the remote path.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
MechaCat02
@@ -69,27 +69,92 @@ function splitSentences(body: string): string[] {
|
||||
* Use this everywhere instead of calling parse() directly.
|
||||
*/
|
||||
/**
|
||||
* Cut the body at the first directive line that survived parse().
|
||||
* The prose of a message: every line that is not inside a block.
|
||||
*
|
||||
* Exported for the test that pins the lib bug this exists for.
|
||||
* Computed from the RAW text, not from parse()'s body. Repairing that body
|
||||
* cannot be made to work, because parse() removes each block by string
|
||||
* surgery and the removals damage each other — a block's terminator
|
||||
* `(?:\n::|$)` is inside its own match, so removing one block takes the
|
||||
* two colons belonging to the NEXT one with it. What is left behind is not
|
||||
* even recognisable as a directive any more:
|
||||
*
|
||||
* **Example sentence**
|
||||
* *나 바다*
|
||||
* task translate ← the "::" is gone; the items follow as prose
|
||||
* 나 바다
|
||||
*
|
||||
* An earlier version of this dropped everything from the first line
|
||||
* starting with "::", which cannot see that. The raw text always has its
|
||||
* markers, so scanning it is not a repair at all — it is just reading.
|
||||
*
|
||||
* The grammar is small: a line of exactly "::" closes a block, any other
|
||||
* line beginning "::" opens one (::gloss, ::task translate, ::words,
|
||||
* ::progress 40 | note), and a block runs until it is closed, until the
|
||||
* next block opens, or to the end. Trailing whitespace is common — this
|
||||
* model ends every line with markdown's two spaces — so nothing here
|
||||
* compares exact strings.
|
||||
*/
|
||||
export function stripLeakedBlocks(body: string): string {
|
||||
/**
|
||||
* Is this a line of ::gloss content?
|
||||
*
|
||||
* A gloss row is `한글 | English | note` and a translation line starts with
|
||||
* `=`. Nothing else belongs. The check matters because a model that forgets
|
||||
* the closing `::` leaves its next paragraph inside the block, and lib then
|
||||
* reads that prose as gloss parts — which is how two sentences of English
|
||||
* ended up rendered as Korean example text inside a gloss card.
|
||||
*/
|
||||
function isGlossContent(line: string): boolean {
|
||||
const t = line.trim();
|
||||
return t === "" || t.includes("|") || t.startsWith("=");
|
||||
}
|
||||
|
||||
/** Where a gloss block's content really ends, closed properly or not. */
|
||||
export function glossContent(body: string): string {
|
||||
const lines = body.split("\n");
|
||||
const at = lines.findIndex((l) => l.trimStart().startsWith("::"));
|
||||
return at === -1 ? body : lines.slice(0, at).join("\n").trimEnd();
|
||||
const end = lines.findIndex((l) => !isGlossContent(l));
|
||||
return (end === -1 ? lines : lines.slice(0, end)).join("\n");
|
||||
}
|
||||
|
||||
export function proseOf(raw: string): string {
|
||||
const out: string[] = [];
|
||||
let block: string | null = null;
|
||||
|
||||
for (const line of raw.split("\n")) {
|
||||
const t = line.trim();
|
||||
if (t === "::") {
|
||||
block = null;
|
||||
continue;
|
||||
}
|
||||
if (t.startsWith("::")) {
|
||||
block = t.slice(2).split(/\s/)[0] ?? null;
|
||||
continue;
|
||||
}
|
||||
// An unclosed gloss ends where its rows do; what follows is prose again,
|
||||
// and belongs in the message rather than swallowed into the card.
|
||||
if (block === "gloss" && !isGlossContent(line)) block = null;
|
||||
if (!block) out.push(line);
|
||||
}
|
||||
|
||||
return out.join("\n").trim();
|
||||
}
|
||||
|
||||
export function parseMessage(text: string): ParsedMessage {
|
||||
const parsed = parse(text);
|
||||
const body = stripLeakedBlocks(parsed.body);
|
||||
// lib/ owns extracting the blocks; the prose around them is ours.
|
||||
const body = proseOf(text);
|
||||
|
||||
if (!parsed.gloss) return { ...parsed, body };
|
||||
|
||||
const match = text.match(GLOSS_BLOCK);
|
||||
if (!match?.[1]) return { ...parsed, body };
|
||||
|
||||
const sentences = splitSentences(match[1]);
|
||||
if (sentences.length < 2) return { ...parsed, body }; // common case
|
||||
/* Rebuilt unconditionally, not only when there are several sentences.
|
||||
The single-sentence case is where an unclosed block hurts: lib has
|
||||
already absorbed the following paragraph into it, and returning its
|
||||
gloss untouched would keep that prose inside the card. Re-parsing from
|
||||
glossContent() drops it, and proseOf() has put it back in the message.
|
||||
For a well-formed single-sentence block this reproduces lib exactly. */
|
||||
const sentences = splitSentences(glossContent(match[1]));
|
||||
|
||||
const blocks: GlossBlock[] = [];
|
||||
for (const s of sentences) {
|
||||
|
||||
@@ -21,17 +21,32 @@ function isKoreanLine(line: string): boolean {
|
||||
return korean / bare.length > 0.55;
|
||||
}
|
||||
|
||||
/** **bold** is the only inline markup the prompt permits. */
|
||||
/* The prompt permits **bold** and nothing else, and says so outright: "No
|
||||
headings, tables, code fences or bullet characters." A model that obeys
|
||||
never exercises anything below.
|
||||
|
||||
Rendering them anyway is not an invitation. Now that the backend is
|
||||
pluggable, the tutor may be a local model that ignores the rule — and the
|
||||
failure was ugly and confusing rather than merely untidy: a bulleted list
|
||||
of the ten consonants arrived as lines beginning with a hyphen, "*What we
|
||||
learn:*" kept its asterisks, and a "---" divider showed as three dashes.
|
||||
Degrading gracefully costs little and keeps a lesson readable. */
|
||||
|
||||
/** **bold**, and *italic* as a concession to models that use it. */
|
||||
function inline(text: string) {
|
||||
return text.split(/(\*\*[^*]+\*\*)/g).map((part, i) =>
|
||||
part.startsWith("**") && part.endsWith("**") && part.length > 4 ? (
|
||||
<strong key={i}>{part.slice(2, -2)}</strong>
|
||||
) : (
|
||||
<Fragment key={i}>{part}</Fragment>
|
||||
),
|
||||
);
|
||||
return text.split(/(\*\*[^*]+\*\*|\*[^*\n]+\*)/g).map((part, i) => {
|
||||
if (part.startsWith("**") && part.endsWith("**") && part.length > 4)
|
||||
return <strong key={i}>{part.slice(2, -2)}</strong>;
|
||||
if (part.startsWith("*") && part.endsWith("*") && part.length > 2)
|
||||
return <em key={i}>{part.slice(1, -1)}</em>;
|
||||
return <Fragment key={i}>{part}</Fragment>;
|
||||
});
|
||||
}
|
||||
|
||||
const RULE = /^(?:-{3,}|\*{3,}|_{3,})$/;
|
||||
const HEADING = /^#{1,6}\s+(.*)$/;
|
||||
const BULLET = /^[-*•]\s+(.*)$/;
|
||||
|
||||
export function MessageBody({ text }: { text: string }) {
|
||||
const lines = text.split("\n");
|
||||
|
||||
@@ -41,6 +56,27 @@ export function MessageBody({ text }: { text: string }) {
|
||||
const trimmed = line.trim();
|
||||
if (!trimmed) return <div className="gap" key={i} />;
|
||||
|
||||
if (RULE.test(trimmed)) return <hr className="mrule" key={i} />;
|
||||
|
||||
const heading = HEADING.exec(trimmed);
|
||||
if (heading) {
|
||||
return (
|
||||
<p className="mhead" key={i}>
|
||||
<strong>{inline(heading[1]!)}</strong>
|
||||
</p>
|
||||
);
|
||||
}
|
||||
|
||||
const bullet = BULLET.exec(trimmed);
|
||||
if (bullet) {
|
||||
const item = bullet[1]!;
|
||||
return (
|
||||
<p className={`mbullet${isKoreanLine(item) ? " kline ko" : ""}`} key={i}>
|
||||
{inline(item)}
|
||||
</p>
|
||||
);
|
||||
}
|
||||
|
||||
const mark = trimmed.startsWith("✓") ? "ok" : trimmed.startsWith("✗") ? "no" : null;
|
||||
const rest = mark ? trimmed.slice(1).trimStart() : trimmed;
|
||||
|
||||
|
||||
@@ -162,3 +162,30 @@
|
||||
padding: 3px 7px;
|
||||
font-size: 12.5px;
|
||||
}
|
||||
|
||||
/* Markdown a non-compliant model emits. The prompt forbids all of it; these
|
||||
rules exist so it degrades into something readable instead of showing as
|
||||
literal hyphens, hashes and asterisks. Deliberately quiet — this is not a
|
||||
style to encourage. */
|
||||
.bubble .mhead {
|
||||
font-size: 14.5px;
|
||||
margin-top: 8px;
|
||||
}
|
||||
|
||||
.bubble .mbullet {
|
||||
padding-left: 15px;
|
||||
position: relative;
|
||||
}
|
||||
|
||||
.bubble .mbullet::before {
|
||||
content: "·";
|
||||
position: absolute;
|
||||
left: 4px;
|
||||
color: var(--ink3);
|
||||
}
|
||||
|
||||
.bubble .mrule {
|
||||
border: 0;
|
||||
border-top: 1px solid var(--line);
|
||||
margin: 10px 0;
|
||||
}
|
||||
|
||||
@@ -18,8 +18,9 @@
|
||||
The fixture is a verbatim capture from gpt-oss-20b through the server. */
|
||||
|
||||
import { describe, it, expect } from "vitest";
|
||||
import { readFileSync } from "node:fs";
|
||||
import { parse } from "@lib/blocks.js";
|
||||
import { parseMessage, stripLeakedBlocks } from "@app/domain/gloss.js";
|
||||
import { parseMessage, proseOf, glossContent } from "@app/domain/gloss.js";
|
||||
|
||||
import type { ParsedMessage } from "@lib/blocks.js";
|
||||
|
||||
@@ -79,14 +80,130 @@ describe("parseMessage — the workaround", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("stripLeakedBlocks", () => {
|
||||
describe("proseOf", () => {
|
||||
it("does nothing without a directive", () => {
|
||||
expect(stripLeakedBlocks("a\n\nb")).toBe("a\n\nb");
|
||||
expect(proseOf("a\n\nb")).toBe("a\n\nb");
|
||||
});
|
||||
it("cuts from the first directive line", () => {
|
||||
expect(stripLeakedBlocks("keep\n::task translate\ndrop\n::words\ndrop")).toBe("keep");
|
||||
|
||||
it("drops every block, not merely from the first one on", () => {
|
||||
expect(proseOf("keep\n::task translate\ndrop\n::words\ndrop")).toBe("keep");
|
||||
});
|
||||
|
||||
it("resumes the prose after a block closes — gloss blocks sit inline", () => {
|
||||
expect(proseOf("before\n::gloss\n물 | N | water |\n::\nafter")).toBe("before\nafter");
|
||||
});
|
||||
|
||||
it("does not trip on a colon that is not at the start of a line", () => {
|
||||
expect(stripLeakedBlocks("see:: this")).toBe("see:: this");
|
||||
expect(proseOf("see:: this")).toBe("see:: this");
|
||||
});
|
||||
|
||||
it("ignores the trailing whitespace this model puts on every line", () => {
|
||||
// markdown hard line breaks: "::task translate "
|
||||
expect(proseOf("keep\n::task translate \n나 바다 \n::words \n물 | water")).toBe("keep");
|
||||
});
|
||||
|
||||
it("treats ::progress, which has no closing ::, as running to the end", () => {
|
||||
expect(proseOf("keep\n::progress 40 | going well")).toBe("keep");
|
||||
});
|
||||
});
|
||||
|
||||
/* A verbatim capture of gpt-oss-20b answering "Start unit 1.1." against the
|
||||
real assembled prompt — the reply behind the second report of raw markup
|
||||
showing above the exercise.
|
||||
|
||||
It differs from the first capture in the way that mattered: here ::gloss
|
||||
comes first, and removing it took the "::" off the ::task that followed,
|
||||
so what leaked was the bare word "task translate" with its five sentences
|
||||
under it. A rule that looked for lines beginning "::" could not see that,
|
||||
which is why the body is now read from the raw text instead. */
|
||||
describe("gpt-oss-20b, unit 1.1 — the reported reply", () => {
|
||||
const RAW = readFileSync(
|
||||
new URL("../fixtures/gpt-oss-20b-unit-1.1.txt", import.meta.url),
|
||||
"utf8",
|
||||
);
|
||||
|
||||
it("puts the blocks in the order that breaks lib/blocks.js", () => {
|
||||
const order = RAW.split("\n")
|
||||
.filter((l) => /^\s*::\w/.test(l))
|
||||
.map((l) => l.trim().split(/\s/)[0]);
|
||||
expect(order).toEqual(["::gloss", "::task", "::words", "::progress"]);
|
||||
});
|
||||
|
||||
it("renders the exercise", () => {
|
||||
expect(parseMessage(RAW).task?.type).toBe("translate");
|
||||
});
|
||||
|
||||
it("leaves no directive, and no de-colonised directive, in the prose", () => {
|
||||
const body = parseMessage(RAW).body;
|
||||
expect(body).not.toMatch(/(^|\n)\s*::/);
|
||||
// The exact shape of this leak: the marker stripped, the word left.
|
||||
expect(body).not.toMatch(/(^|\n)\s*(task|words|gloss|progress)\b/);
|
||||
});
|
||||
|
||||
it("does not repeat the exercise as prose above it", () => {
|
||||
const parsed = parseMessage(RAW);
|
||||
const task = parsed.task;
|
||||
// Task is a union; only the non-match kinds carry `items`.
|
||||
const raw = task && "items" in task ? (task.items as { q?: string }[]) : [];
|
||||
const items = raw.map((i) => i.q).filter((q): q is string => Boolean(q));
|
||||
expect(items.length).toBeGreaterThan(1);
|
||||
|
||||
/* Not "no item appears": this reply uses 나 바다 as a worked example in
|
||||
the prose AND as the first exercise line, which is legitimate. A leak
|
||||
drags in every line, so that is what to measure. */
|
||||
const echoed = items.filter((q) => parsed.body.includes(q));
|
||||
expect(echoed.length).toBeLessThan(items.length);
|
||||
});
|
||||
|
||||
it("keeps the teaching prose that came before the blocks", () => {
|
||||
expect(parseMessage(RAW).body).toContain("Korean syllables are written as blocks");
|
||||
});
|
||||
});
|
||||
|
||||
/* A gloss block the model forgot to close. Every local model tested does
|
||||
this, and lib/blocks.js then reads the following paragraph as gloss
|
||||
parts — two sentences of English rendered as Korean example text inside
|
||||
the card, at example-line size. */
|
||||
describe("an unclosed ::gloss block", () => {
|
||||
const RAW = [
|
||||
"Here is an example.",
|
||||
"::gloss",
|
||||
"나 | S | I",
|
||||
"바다 | O | sea",
|
||||
"= I sea",
|
||||
"",
|
||||
"(Here 바다 is two blocks.)",
|
||||
"Now practice these.",
|
||||
"::task translate",
|
||||
"나 바다",
|
||||
].join("\n");
|
||||
|
||||
it("ends the block where its rows end", () => {
|
||||
expect(glossContent("나 | S | I\n= I sea\n\nprose here\nmore prose")).toBe("나 | S | I\n= I sea\n");
|
||||
});
|
||||
|
||||
it("keeps only the real gloss rows in the card", () => {
|
||||
const gloss = parseMessage(RAW).gloss ?? [];
|
||||
expect(gloss).toHaveLength(1);
|
||||
expect(gloss[0]!.parts.map((p) => p.ko)).toEqual(["나", "바다"]);
|
||||
expect(gloss[0]!.en).toBe("I sea");
|
||||
});
|
||||
|
||||
it("returns the stranded prose to the message", () => {
|
||||
const body = parseMessage(RAW).body;
|
||||
expect(body).toContain("Here is an example.");
|
||||
expect(body).toContain("(Here 바다 is two blocks.)");
|
||||
expect(body).toContain("Now practice these.");
|
||||
// and still no exercise text or markup
|
||||
expect(body).not.toContain("나 바다");
|
||||
expect(body).not.toMatch(/::/);
|
||||
});
|
||||
|
||||
it("leaves a properly closed block exactly as lib parses it", () => {
|
||||
const closed = "::gloss\n물 | N | water\n= Water.\n::\nAfter.";
|
||||
const g = parseMessage(closed).gloss ?? [];
|
||||
expect(g).toHaveLength(1);
|
||||
expect(g[0]!.parts.map((p) => p.ko)).toEqual(["물"]);
|
||||
expect(parseMessage(closed).body).toBe("After.");
|
||||
});
|
||||
});
|
||||
|
||||
57
test/fixtures/gpt-oss-20b-unit-1.1.txt
vendored
Normal file
57
test/fixtures/gpt-oss-20b-unit-1.1.txt
vendored
Normal file
@@ -0,0 +1,57 @@
|
||||
Korean syllables are written as blocks that combine an **initial consonant** and a **vowel**.
|
||||
In this unit we only use the plain consonants **ㄱ ㄴ ㄷ ㄹㅁㅂㅅㅇㅈㅎ** and the ten basic vowels **ㅏ ㅑ ㅓ ㅕ ㅗ ㅛ ㅜㅠㅡㅣ**.
|
||||
|
||||
Each block has no final consonant, so a word is simply a sequence of such blocks.
|
||||
For example:
|
||||
|
||||
- ㄱㅏ (가)
|
||||
- ㄴㅏ (나)
|
||||
- ㄷㅏ (다)
|
||||
- ㄹㅏ (라)
|
||||
- ㅁㅏ (마)
|
||||
- ㅂㅏ (바)
|
||||
- ㅅㅏ (사)
|
||||
- ㅇㅏ (아)
|
||||
- ㅈㅏ (자)
|
||||
- ㅎㅏ (하)
|
||||
|
||||
A word is read by looking at each block in order.
|
||||
The learner’s mistake often is to treat the whole word as a single unit and miss the internal blocks.
|
||||
|
||||
**Example sentence**
|
||||
*나 바다*
|
||||
::gloss
|
||||
나 | S | I
|
||||
바다 | O | sea
|
||||
= I sea
|
||||
|
||||
(Here “바다” is two blocks: 바 (ㅂ+아) + 다 (ㄷ+아).)
|
||||
|
||||
Now practice reading and translating these short phrases.
|
||||
|
||||
::task translate
|
||||
나 바다
|
||||
그 나무
|
||||
어디 소리
|
||||
하나 바다
|
||||
|
||||
::words
|
||||
ㄱㅏ | ㄱ + ㅏ
|
||||
ㄴㅏ | ㄴ + ㅏ
|
||||
ㄷㅏ | ㄷ + ㅏ
|
||||
ㄹㅏ | ㄹ + ㅏ
|
||||
ㅁㅏ | ㅁ + ㅏ
|
||||
ㅂㅏ | ㅂ + ㅏ
|
||||
ㅅㅏ | ㅅ + ㅏ
|
||||
ㅇㅏ | ㅇ + ㅏ
|
||||
ㅈㅏ | ㅈ + ㅏ
|
||||
ㅎㅏ | ㅎ + ㅏ
|
||||
나 | I
|
||||
바다 | sea
|
||||
그 | he
|
||||
나무 | tree
|
||||
어디 | where
|
||||
소리 | sound
|
||||
하나 | one
|
||||
|
||||
::progress 0-100 | starting unit, basics to be mastered.
|
||||
Reference in New Issue
Block a user