You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Experiment: choose between prompt lookup and MTP drafts by measured acceptance and cost #37
Idea from Strata by @Niko1221 (MIT): when the reply repeats the context (code edits, quoted text), prompt lookup drafts up to 5 tokens from the earlier copy, but only where its measured acceptance and cost say it pays. Strata reports code edits 6-11% faster and other text unchanged (HOW_IT_WORKS.md, "Guess, then check").
Where we are
The fork has n-gram drafters (ngram-simple, ngram-map-k, ngram-map-k4v, ngram-mod, ngram-cache), and --spec-type takes a comma-separated list.
With several drafters the first one that produces a draft wins (common_speculative_draft, common/speculative.cpp). Nothing tracks acceptance or verify cost.
Coding-agent traffic copies a lot: edits of quoted files, tool outputs, diffs. MTP alone already gets acceptance 0.80-0.91 on the edit-code task of mtp-bench.py, against 0.53-0.64 on prose.
Experiment
Baseline: draft-mtp alone vs ngram-mod,draft-mtp vs draft-mtp,ngram-mod (first wins) on the 4 tasks of mtp-bench.py. Add an agent-like task that rewrites a long quoted file with small changes. Both temperature 1.0 and greedy.
A per-sequence chooser: keep a running estimate of each drafter's accepted tokens per draft and the cost of its draft plus verify window. Take the drafter with the most expected tokens per ms for the next step; an n-gram draft can be longer than --spec-draft-n-max.
Keep the MTP context in sync when an n-gram draft was used (catch-up of the accepted tokens).
Done when
Code-edit tasks gain clearly (target: Strata's 6-11%) and prose/list tasks do not lose (within noise, interleaved pairs).
Activity