Learn from Corrections
Spec for the Jot for iPhone and Jot for Windows agents. Shipped in Jot for Mac 1.23 (2026-09-29). Source of truth: jot/docs/vocabulary-learn-from-edits/design.md (Revision 2) and jot-shared 42e2428: VocabularyLearning.swift, DecoderVocabulary.swift, EditLearner.swift, CorrectionStore.swift, VocabularyGate.swift.
Verified end to end through the real Mac pipeline on the owner's 7 "Claude" recordings: before, "cloud"; after one correction, 7/7 "Claude".
1. The rule: one correction, one path
- A correction is one thing: "when I say X, write Y".
- Stored once, visibly: as a sounds-like (alias) on the term in the vocabulary list. That list is the only source the decoder reads.
- Every surface calls one function,
VocabularyLearning.apply(Correction). No per-surface learning logic. A grep for store/alias writes outside JotVocabCore and the list adapter must return nothing. - No learned auto-replace rules. No "always replace", no "confirmed once, replace silently". A word changes only when audio supports it (decoder pair, or the CTC gate hearing the term). "Keep original" only ever blocks.
- Up and down: a correction turns a pair on; "keep original" or editing it back pauses it (last explicit signal wins); deleting the sounds-like removes it.
2. Who implements what
| Piece | iPhone (Swift) | Windows (C#) |
|---|---|---|
Correction, VocabularyLearning.apply, receipts/undo | Pull jot-shared 42e2428 | Port; pass the shared JSON fixtures |
EditLearner (edit diff → lessons) | Pull | Port (§4) |
VocabularyListWriting adapter (your vocabulary store, by row id) | Implement | Implement |
Wire every correction surface to apply | Implement (keyboard: queue Codable Corrections for the app; it can't write the app's list) | Implement |
Decoder pairs + weights (DecoderVocabulary.derive) | n/a until decode bias (Parakeet) | Port if your decoder has bias (§6) |
| Gate: no learned auto-apply; paused pair blocks | Pull (fixtures updated) | Port; fixtures changed (§7) |
3. Correction surfaces (Mac inventory — mirror yours)
| Surface | Call |
|---|---|
| Edit → Done | EditLearner lessons → .correct (userCasing) / .keepOriginal / .recase |
| Add to Vocabulary (select text) | .correct(heard, term, userCasing: true) |
| Review list: pick term / keep / undo | .correct / .keepOriginal / .undo(receipt) — provenance bookkeeping stays per surface; receipt stored per verdict |
| Popup: confirm / wider-phrase / keep | .correct / .correct / .keepOriginal; timeout teaches nothing |
| Settings: add / remove sounds-like chip | .correct(userCasing: true, termID) / .forget (routed by row id) |
| Settings: delete / rename term | plain list write; pairs vanish with the term |
Casing: only hand-typed surfaces (userCasing: true) may change the term's spelling. Popup/review proposals can be sentence-start capitalized ("EBay"); they resolve to the list's existing spelling.
Undo: reverses exactly what one correction added — its sounds-like first; the term row only if nothing else is on it and its text is unchanged; flags compare-and-restore.
4. Hand-edit rules (EditLearner.learn)
trim identical start/end; > 2000 changed words → nothing word-align before vs after (punctuation-insensitive) per changed hunk: pure insert / delete → skip outside 1–3 → 1–4 words → skip punctuation- or digit-only → skip case-only → .recase (not first-letter at sentence start) before-side is exactly a vocabulary term → .keepOriginal (user undid a term) before-side has a term among other words → skip both sides everyday AND replacement has no capitalized word → skip (a→the, your→you're) not sound-alike (VocabularyGate.plausible) → skip (tomorrow → "on Monday") else → .correct(original, term, heardByModel: original ∈ raw, userCasing: true) one lesson per pair per edit session; languages without an everyday-word list: learn nothing
5. Store (corrections.json, per mapping)
editsToward, editsAway : Int // up/down history
editHeard : Bool // model actually wrote the original
pairPaused : Bool // last explicit signal was keep/edit-back
// absent key ⇒ editsAway > editsToward
confirmations/reverts/net // ask-ranking prior ONLY, never an apply
alwaysReplace, editStreak // REMOVED: still decoded for old files, ignored, dropped on next write
6. Decoder (derived from the list; nothing stored twice)
pairs(term) = each sounds-like of term that is
single word, ≥ 3 letters, Latin script,
SAME FIRST LETTER as the term, not paused
weight(term) = 5.0 if term has an active pair, else 3.5
pair push = +10 on the term's continuation, −10 on the original's diverging token,
only where the letters so far are shared; never inside/right after the term
near-completion: a pushed word one trailing letter short of the term (≥ 4 letters),
not the original itself, is emitted as the term in its own casing
("claud" → "Claude")
Measurements (LibriSpeech test-clean, N=100 lists)
| Setting | Result |
|---|---|
| One learned term at 5 / 6 / 7 / 8 | false insertions 0 / 5–8 / 19–56 / 81–251 of 2620 → ceiling 5 |
| Pair push 10 + near-completion, same-first-letter natural pairs | corrected-word recall 0.5% → 49.5%; 0/640 word-count changes; U-WER 1.982 → 1.989 |
| Different first letter, everyday original ("chart" → Jot shape) | conversion 0.4%; 21/2620 utterances damaged → excluded (stays with the gate) |
Windows: your decoder uses start 2.0 / continue 3.0 — keep the ratio and re-measure; do not assume these numbers transfer.
7. Gate + fixture changes (cross-platform contract)
- Step 0 no longer auto-applies (no grant, no net ≥ 1 override, no net ≤ −1 demotion). Only a paused pair → BLOCK. Everything else goes through the evidence steps.
- Fixtures now use
pausedinstead ofalwaysReplace. Rewritten: "grant auto-applies" → "learned common pair never auto-applies"; "demotion beats grant" → "paused pair blocks"; ask "grant excluded" now expects the ask; store "grant revoke" → "old grant key decodes and is ignored". Added: "confirmed rare pair (net 3) needs evidence", "paused pair blocks".
8. Tests to port
- Each
Correctioncase; receipt undo is an exact inverse (pre-existing term/alias untouched, later sounds-likes survive). - cloud → Claude via every surface produces the same list + store state.
- Deleting a sounds-like leaves no pair and no replace rule.
- Pair eligibility: cloud→Claude, Vini→Vineet yes; chart→Jot, beneath→Vineet no; multi-word, < 3 letters, non-Latin no.
- Old
corrections.jsonwithalwaysReplace/editStreakloads; old provenance without receipts loads. - Users with no sounds-likes and no corrections: identical decoder input to before.
9. Status
MacReleased 1.23 (public + Sony). jot-shared 42e2428. FluidAudio fork vineetu/FluidAudio branch jot/learned-pairs @ 0de0032e.
iPhonePull jot-shared 42e2428; adapter + wire surfaces; keyboard queues Corrections.
WindowsPort §1–§7 against the shared fixtures; re-measure decoder numbers.