Feature Request: Expose Speculative Decoding (MTP / Draft) in public C API (llama.h) for downstream bindings #27469

Feature Description

Expose Multi-Token Prediction (MTP) and speculative decoding capabilities in the public C API (llama.h) and export them in dynamic shared libraries (llama.dll, libllama.so, libllama.dylib).

Specifically, this feature request entails:

1. Public C API Access: Making speculative decoding workflows—currently encapsulated in common/speculative.cpp—accessible via stable C functions in llama.h.
2. High-Level Speculative Orchestration: Providing a high-level speculative context API to handle draft generation, verification, KV cache synchronization, and multi-pass decoding natively, rather than requiring wrapper languages to implement complex tensor plumbing downstream.
3. Downstream Interop: Enabling language bindings (C#, Python, Rust, Swift, Go) to run multi-token prediction and draft verification natively through the exported shared library without depending on internal common/ headers or custom forks.

Motivation

Currently, downstream bindings (like LLamaSharp, llama-cpp-python, and Rust wrappers) are unable to utilize the incredible speedups provided by Speculative Decoding and Multi-Token Prediction (MTP).

The recent unifications for MTP (e.g., --spec-type draft-mtp) and other speculative methods (EAGLE, n-gram, DFlash) are fantastic, but the orchestration logic lives entirely inside common/speculative.cpp. Because common/ is not exported to the shared library (llama.dll / libllama.so), pure C API consumers are locked out of these features.

Initially, we attempted to implement the MTP drafting loop manually in C# for LLamaSharp. However, after reviewing speculative.cpp, we realized the native MTP implementation is too complex to replicate safely downstream:

• It requires a dual-context architecture (ctx_tgt and a dedicated ctx_dft with LLAMA_CONTEXT_TYPE_MTP).

• It relies on extracting and passing dense hidden state embeddings directly into the evaluation batch (batch.embd / llama_get_embeddings_nextn_ith) instead of just token IDs.

• It dynamically handles layer offsets (llama_set_nextn_layer_offset) for chain_heads architectures.

Attempting to rebuild this intricate, multi-pass autoregressive logic across P/Invoke boundaries in wrapper languages is highly brittle and creates severe technical debt.

Possible Implementation

We are hoping the maintainers can consider formalizing speculative decoding into llama.h to help binding authors:

The Ideal Solution (A Stable Speculative API)
Introduce a formal llama_speculative_* or llama_draft_* API suite into llama.h. Similar to how the modern llama_sampler API abstracts the sampling chain, an opaque speculative context could handle the draft-and-verify loop, RoPE shifts, and KV cache rollbacks internally. Bindings would simply feed the prompt and receive verified tokens.

Conceptual Example:

// Initialize a speculative context managing both target and draft models
LLAMA_API struct llama_speculative_context * llama_speculative_init(
    struct llama_context * ctx_tgt, 
    struct llama_context * ctx_dft,
    const struct llama_speculative_params * params
);

// A unified decode/sample loop that handles the draft/verify internally
LLAMA_API void llama_speculative_decode(
    struct llama_speculative_context * spec_ctx,
    struct llama_batch * batch
);

// Extract the accepted tokens after verification
LLAMA_API int32_t llama_speculative_get_accepted_tokens(
    struct llama_speculative_context * spec_ctx,
    llama_token * out_tokens,
    int32_t max_tokens
);

With models like DeepSeek V3/R1, Qwen 3.6, and Gemma 4 relying heavily on MTP for performance, this is becoming a critical feature for the broader ecosystem. If you are open to exposing an experimental API, we would love to collaborate by immediately testing it downstream in C# and reporting back!

Prerequisites

• I am running the latest code. Mention the version if possible as well.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new and useful enhancement to share.