7bd8507909
Closes REVIEW.md §2. 167 of 371 roadmap words had no lexicon entry, so the word rail silently showed nothing. Now: Roadmap words: 371/371 resolve Spiral targets: 35/35 resolve Deck words: 386/386 resolve and npm run dict:assert makes it a blocking build failure, not a silent empty rail. No runtime morphological analyser ships. lib/conjugation.js surfaceForms() runs at BUILD time over every verb and adjective, so looking up a conjugated form is an index hit on the surface table. The frequency join had to be inverted. A subtitle frequency list holds surface forms; a dictionary holds lemmas whose -다 citation form barely occurs in running text, so joining on headword gives verbs a frequency of roughly zero. Expanding each lemma into the forms it plausibly takes and summing recovers 하다 from 118 to 89,041. Forms claimed by more than one lemma are dropped rather than split, so homographs don't inherit each other's mass. Those expansions score frequency only — the surface table itself stays strictly surfaceForms() output plus the headword. Bands are one per curriculum phase. Phase 1 admits no frequency band at all: during the writing-system phase every word must be phonologically legal for the unit reached, and a rank ceiling would hand the learner a 겹받침 during unit 1.4. shared/phonology.mjs lifts validate.mjs's own feature ladder to enforce that; it agrees with the validator on all 371 words. Sources are chosen automatically — KRDICT when vendored, otherwise the kaikki.org extract. KRDICT's download is a JS form behind anti-bot protection, so it cannot be fetched by CI; the derived band files are committed instead, which the app needs offline regardless. Attribution and the share-alike terms are in NOTICE.md. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
70 lines
3.0 KiB
JavaScript
70 lines
3.0 KiB
JavaScript
/* The vocabulary bands — the mechanism that makes the gate scale.
|
|
|
|
PORT.md's idea: ship only the bands the learner has reached, and let
|
|
buildGate()'s vocabQuery hook replace 371 hand-typed words with a query.
|
|
One band per phase, widening as he goes.
|
|
|
|
Two signals decide which band a word lands in:
|
|
|
|
- the dictionary's own curated learner level (초급 / 중급 / 고급), which
|
|
is human-graded for exactly this purpose;
|
|
- frequency rank, as the tiebreaker inside a level.
|
|
|
|
Level leads because a frequency list built from subtitles is a list of
|
|
SURFACE forms, and a lemma-keyed dictionary joins onto it badly — see
|
|
tools/dict/freq-forms.mjs for how the ranks are recovered at all. When a
|
|
source carries no level, bandOf() degrades to frequency alone.
|
|
|
|
PHASE 1 ADMITS NO FREQUENCY BAND AT ALL. During the writing-system phase
|
|
every word must be phonologically legal for the unit reached — a rank
|
|
ceiling would hand the learner a 겹받침 during unit 1.4. Phase 1 gets the
|
|
curated words and nothing else; see shared/phonology.mjs. */
|
|
|
|
/** Bands 0-5 are gated vocabulary, one per curriculum phase. */
|
|
export const BANDS = [
|
|
{ band: 0, phase: 1, maxFreq: 0, levels: [] },
|
|
{ band: 1, phase: 2, maxFreq: 1500, levels: ["초급"] },
|
|
{ band: 2, phase: 3, maxFreq: 3000, levels: ["초급"] },
|
|
{ band: 3, phase: 4, maxFreq: 5000, levels: ["초급", "중급"] },
|
|
{ band: 4, phase: 5, maxFreq: 8000, levels: ["초급", "중급"] },
|
|
{ band: 5, phase: 6, maxFreq: 15000, levels: ["초급", "중급", "고급"] },
|
|
];
|
|
|
|
/**
|
|
* Everything the dictionary knows that no band admits. Shipped so the word
|
|
* rail can gloss an unfamiliar word the learner meets in the wild, but never
|
|
* returned by vocabQuery — the tutor cannot reach it.
|
|
*/
|
|
export const REFERENCE_BAND = 6;
|
|
|
|
/** Sources that are always available, whatever the frequency data says. */
|
|
export const ALWAYS_AVAILABLE = new Set(["curated", "grammar", "sentence", "sfx"]);
|
|
|
|
/** Curriculum phase (1-6) → band. */
|
|
export const bandForPhase = (phase) =>
|
|
Math.max(0, Math.min(BANDS.length - 1, Math.round(phase) - 1));
|
|
|
|
/** Unit id ("3.4") → band. */
|
|
export const bandForUnit = (unitId) => bandForPhase(Number.parseInt(String(unitId), 10) || 1);
|
|
|
|
/** The frequency ceiling a learner at this band may draw on. */
|
|
export const ceilingForBand = (band) => BANDS[Math.max(0, Math.min(BANDS.length - 1, band))].maxFreq;
|
|
|
|
/**
|
|
* The lowest band that admits this word, or REFERENCE_BAND if none does.
|
|
* `level` may be null when the source does not grade its entries.
|
|
*/
|
|
export function bandOf({ source, freqRank, level }) {
|
|
if (ALWAYS_AVAILABLE.has(source)) return 0;
|
|
|
|
for (const b of BANDS) {
|
|
if (b.band === 0) continue; // phase 1 takes curated words only
|
|
if (freqRank == null || freqRank > b.maxFreq) continue;
|
|
// A graded source must also clear the level gate; an ungraded one is
|
|
// admitted on frequency alone.
|
|
if (level && b.levels.length && !b.levels.includes(level)) continue;
|
|
return b.band;
|
|
}
|
|
return REFERENCE_BAND;
|
|
}
|