fix(gate): the gate was being spliced into the prompt's header sentence

assemblePrompt used template.replace("{{GATE}}", ...), and String.replace
with a string argument substitutes only the FIRST occurrence. All three
placeholders appear twice in tutor-system.md, because the document names
them in its own header paragraph before using them:

    Assembled per turn. `{{GATE}}` is `renderGate()` from `lib/gate.js`;
    `{{VARIETY}}` and `{{FOCUS}}` are one-liners built from recent state.

So the rendered gate replaced the backticked mention mid-sentence, and the
real slot further down was sent to the model as the literal text "{{GATE}}".
The mechanism that decides what the tutor is allowed to teach was delivered
in the wrong place, with a template token standing where it belonged, and
the same for VARIETY and FOCUS. Confirmed by capturing what the app
actually put on the wire: three unfilled placeholders at lines 60, 86 and
141.

Substitution is now anchored to a whole line, which is what distinguishes a
slot from a mention -- the header's are inline and backticked. The
replacement is a function because renderGate() output contains "$"
sequences that String.replace would otherwise interpret.

The existing tests could not have caught this. They ran against a synthetic
template naming each placeholder exactly once, which is precisely the
property the shipped file lacks. The new ones run against
prompt/tutor-system.md itself: no slot may survive unfilled, the header
must come through intact, and the gate must land between the profile and
the pre-flight check. Both fail against the old code.

Also appends a HOUSE STYLE section after the shipped prompt -- an addition
by the app, not an edit to the file, which still ships byte-identical.
It covers two things the prompt leaves to inference. The language of
explanation is never actually stated: "한글 and English only" is a rule
about retiring romanization, and everything else only implies English. A
strong model infers it; gpt-oss-20b delivered a full grammar lesson in
Korean to a student on unit 1.1 who cannot yet read it. And no-markdown is
stated outright at line 96 and was ignored anyway, so it is restated where
the consequence is visible: the app renders **bold** and nothing else, so a
table arrives as rows of literal pipes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
MechaCat02
2026-09-10 07:09:59 +02:00
parent 7b9c92eb98
commit 9a925a5e3f
2 changed files with 105 additions and 4 deletions

View File

@@ -4,6 +4,7 @@
the prompt assembly and the confidence clamp. */
import { describe, it, expect } from "vitest";
import { readFileSync } from "node:fs";
import {
UNITS,
VOCAB_CAP,
@@ -158,6 +159,13 @@ describe("vocabQuery is never wider than the band", () => {
describe("the prompt", () => {
const TEMPLATE = "BEFORE\n{{GATE}}\nMIDDLE\n{{VARIETY}}\nAFTER\n{{FOCUS}}";
/* The shipped prompt, byte for byte. The synthetic TEMPLATE above names
each placeholder exactly once, and that is precisely the property the
real file does not have — it mentions all three in its own header
paragraph before using them, which is how a first-occurrence replace
went unnoticed. Anything about substitution has to be asserted here. */
const SHIPPED = readFileSync(new URL("../../prompt/tutor-system.md", import.meta.url), "utf8");
it("fills all three placeholders and leaves the rest alone", () => {
const gate = gateFor({ progress: at("2.1"), bandQuery: bandOffering("학교") });
const out = assemblePrompt({
@@ -172,6 +180,46 @@ describe("the prompt", () => {
expect(out).toContain("AFTER");
});
it("leaves no placeholder in the SHIPPED prompt, which names each one twice", () => {
const gate = gateFor({ progress: at("2.1"), bandQuery: bandOffering("학교") });
const out = assemblePrompt({ template: SHIPPED, gate, recent: ["translate"], focus: "sentence" });
// The header paragraph mentions all three inline, in backticks.
expect(SHIPPED.match(/\{\{GATE\}\}/g)).toHaveLength(2);
/* No placeholder may survive as a line of its own — that is a slot that
did not get filled. The backticked mentions in the header are prose
about how assembly works and are left exactly as written. */
for (const name of ["GATE", "VARIETY", "FOCUS"]) {
const unfilled = new RegExp(`^\\{\\{${name}\\}\\}[ \\t]*$`, "m");
expect(unfilled.test(out), `{{${name}}} slot was never filled`).toBe(false);
}
});
it("puts the gate in its own slot, not into the header sentence", () => {
const gate = gateFor({ progress: at("2.1"), bandQuery: bandOffering("학교") });
const out = assemblePrompt({ template: SHIPPED, gate, recent: [], focus: "auto" });
// The header must survive intact, with its backticked mentions.
expect(out).toContain("`{{GATE}}` is `renderGate()`");
// And the gate must land after the profile and before the pre-flight
// check, which is where the shipped file puts the slot.
const knows = out.indexOf("WHAT HE KNOWS");
expect(knows).toBeGreaterThan(out.indexOf("Register: manhwa is written"));
expect(knows).toBeLessThan(out.indexOf("PRE-FLIGHT CHECK"));
});
it("survives a gate containing $ sequences, which replace() would eat", () => {
const out = assemblePrompt({
template: "A\n{{GATE}}\nB\n{{VARIETY}}\n{{FOCUS}}",
gate: { ...gateFor({ progress: at("1.1"), bandQuery: bandOffering() }) },
recent: [],
focus: "auto",
});
expect(out).not.toContain("$&");
});
it("renders the gate's own headings — the model keys off them", () => {
const gate = gateFor({ progress: at("2.1"), bandQuery: bandOffering("학교") });
const out = assemblePrompt({ template: TEMPLATE, gate, recent: [], focus: "auto" });