[Piggidragon]

The MTP draft head of --spec-draft-vocab (#36) was a copy of the list's rows of the output head: 168 / 100 MiB on the 4070 / 3060 for the 66k-token list of Qwen3.8 27B. With 262k context in two slots that did not fit: a 95k-token prompt next to a generating slot ran the 3060 out of memory (cudaGraphInstantiate), so the +7% could not be deployed. This makes the draft head a view of the output head, without memory of its own.

How

• At load the rows of the output head are written again in an order where the list's tokens come first in each device's slice (the tokens of each kind in ascending order, the list spread over the devices like the head's rows, so both GPUs keep their share of the draft head). The draft head is then the first rows of each slice.
• ggml-backend-meta: ggml_backend_meta_tensor_n_slices / ggml_backend_meta_tensor_slice give the device tensors of a split weight, and ggml_backend_meta_tensor_init_row_views makes a weight whose slice j is a view of the first rows[j] rows of another weight's slice j (the split state callback reports its rows per device). Without split mode tensor the draft head is a plain ggml_view_2d.
• The target's logits now come in the head's row order. llama_context puts them in token order when they are first read (llama_get_logits_ith / llama_get_logits, all rows at once so a row pointer is followed by the next rows as before), into a buffer of its own (~60 us per row of 248k tokens); a decode whose logits nobody reads costs nothing, and the samplers are unchanged.
• The view needs the head as a tensor of its own in the plain buffer type of a device whose buffers keep a weight's rows as they are: CUDA, ROCm, MUSA, Vulkan, Metal, or a meta device over them under -sm tensor. Tied heads, heads in host memory, row-split buffers and OpenCL/CANN/SYCL (they store some weights transformed as a whole) keep the copy.
• Under -sm tensor tokens move between the GPUs (the list is spread like the head's rows, so both keep their share of the draft head). On GPUs of different types the logits of moved tokens differ from those without the list in the last bits: up to ~2e-6 on the 4070 + 3060, argmax unchanged; on one GPU they are bitwise equal.
• Guards: backend samplers are not used with such a model (their device logits are in row order), LoRA adapters of the output head are refused, and the model saver asserts (the rows in the buffers are reordered).
• A context with llama_context_params::draft_vocab no longer has to be an MTP context (its output head projects onto the draft vocabulary); the test uses that.
• ggml-meta fix found by the new test: a tensor with one element along its split dimension kept the full stride on a device without elements, and reading it went past its end (draft logits of a one-token list under -sm tensor).

Tried and dropped: reordering every row in place when the context synchronizes (~90 us per row, -1.4% against the copy), and reporting the rows in head order with their tokens through the sampled-candidates API (no reorder at all, but -11%: the penalties sampler's fast path needs the candidates in vocabulary order).

Results (Qwen3.8 27B UD-Q4_K_S, 4070 + 3060, -sm tensor, MTP depth 2, production settings)

mtp-bench (single slot, 384 tokens, 2 reps; mean over write-code, edit-code, explain, list), two runs each for view and copy:

	production sampling	greedy
no list	59.2 t/s	63.2
copy (#36)	63.7	66.6
view	63.6	66.7

German letter (production sampling): no list 47.2 -> view 50.0 t/s. Acceptance of view and copy is identical in every task.

• Greedy outputs identical with the copy, the view and no list on these prompts (English, German, Japanese - the last drafts over the whole vocabulary through the fallback); the MTP draft candidates of copy and view identical (1332 of 1332 lines).
• VRAM after the requests: view 11177 / 11365 MiB = no list; copy 11345 / 11465.
• 98.6k-token prompt next to a generating slot (production preset with the list): min free 291 / 120 MiB, the same as without the list; the copy ran out of memory here.
• test-llama-archs -a '^(llama|qwen35|qwen35moe)$' (CPU, each GPU, the Meta configs): all 64 passed, with a new check: a model loaded with a draft vocabulary gives the logits of the same model without it (bitwise on one device, nmse under -sm tensor), and a draft_vocab context gives the full head's logits at the list's tokens, for a list of a third of the vocabulary and one of a single token.
• After the review fixes: view 63.3 / 66.7 t/s (unchanged), outputs and candidates as above; the reviewer's probes: rows walked from one llama_get_logits_ith pointer now match (before: rows 2.. were zeros).

🤖 Generated with Claude Code

[Piggidragon]

Review fixes in 26544da: (1) the -sm tensor rounding on mixed GPUs is documented and the test tolerates it, (2) the logits API orders all rows at once, so a row pointer walks the next rows again, (3) the view only on CUDA/ROCm/MUSA/Vulkan/Metal (and meta over them), (4) the test now multiplies with the draft head itself; it found a ggml-meta stride bug for slices without elements, fixed in the same commit. test-llama-archs llama|qwen35|qwen35moe: 64/64.

🤖 Generated with Claude Code