Qwen3.5-122B-A10B drafter: 2–12% acceptance under llama.cpp with a quantized target, while Qwen3.6-35B-A3B holds 50–59% in an identical setup #152

First: thank you for DFlash — on the same machine described below, your Qwen3.6 drafters are delivering excellent results (details at the end). This report is about the one model where we could not make it work, with what we hope is a useful elimination matrix.

Setup

• Engine: llama.cpp mainline draft-dflash (PR #22105 implementation), commit 956973c, ROCm backend (AMD Strix Halo / gfx1151, Windows)
• Target: unsloth/Qwen3.5-122B-A10B-MTP-GGUF UD-Q4_K_XL (73 GB), ctx 65536, --parallel 1 -sm layer
• Drafters tested (both converted with current-master convert_hf_to_gguf.py --target-model-dir <stock Qwen3.5-122B-A10B metadata>):
  • z-lab/Qwen3.5-122B-A10B-DFlash @ 2274357 (the 2026-06-19 Modal retrain; 6L, 8 extraction layers, SWA 4096)
  • z-lab/Qwen3.5-122B-A10B-DFlash @ 6c7242c (the original April weights; 4L, 6 extraction layers, SWA 2048)
• Flags: --spec-type draft-dflash --spec-draft-n-max 6 (block_size=16 read from metadata; init logs confirm n_extract=8 / n_extract=6 respectively)

Results

For comparison, on the same machine, engine build, quant level (UD-Q4_K_XL), and flags:

So the conversion pipeline, engine build, and target quant level are each validated by a working control; only the 122B pair fails.

What we ruled out

• Conversion pipeline — re-converting the 35B drafter with the exact pipeline used for the 122B reproduces healthy acceptance (48–59%).
• Drafter precision — F16 vs Q8_0 makes no difference for either 122B drafter generation (table above).
• Vocab mismatch — drafter and target tokenizers are identical (N=248320, probe-verified token-for-token, both pairs).
• Extraction-layer indexing — GGUF dflash.target_layers carries the same +1 convention in the working 35B GGUF ([2,7,12,17,23,28,33,38] from HF [1,6,...]) and ours ([2,8,15,21,27,33,40,46] from HF [1,7,...]).
• MTP/nextn layer placement in the unsloth repacks — nextn is the last block in both targets (blk.48 of 49 for the 122B, blk.40 of 41 for the 35B); real-layer indices are unshifted.
• Target weight revisions — Qwen/Qwen3.5-122B-A10B weights are unchanged since February; the unsloth repack (May) and both drafter generations derive from the same checkpoint.
• Prefill feature injection — the implementation's pos_max < N-1 ("Drafts may degrade") diagnostic never fires; the injection path runs on every prefill ubatch.
• Drafter GGUF structure is byte-convention identical to the working 35B conversion (69 tensors, same keys).

Hypothesis and questions

The pattern — the tightly-fitted retrain degrades more (2%) than the looser original (10%) against the same Q4 target — looks like sensitivity of the extraction-layer hidden states to target quantization on this fine-grained MoE, rather than a plumbing bug. Your published accept lengths (3.2–8.7) are measured on SGLang with high-precision targets; we cannot test a BF16 target locally (245 GB > our 110 GB).

1. Was the 122B drafter ever validated against a quantized target (llama.cpp or otherwise)? Is quantized-target feature drift a known failure mode at this scale?
2. Is there anything llama.cpp's implementation should be doing differently for this drafter family (e.g. feature normalization, the SWA pattern) that would not affect the 35B?
3. Your README mentions the training recipe is planned for open-sourcing — is there an ETA? We would run a quant-aware adaptation (teacher = dequantized Q4 features) and report back.

We are currently running a target-precision ladder (UD-Q3_K_XL / UD-Q4_K_XL / UD-Q5_K_M) to test the hypothesis directly and will post the acceptance-vs-precision curve here when it completes.

The good news: on this same consumer hardware, your Qwen3.6-27B drafter gives +48% end-to-end decode over the model's own MTP heads, the 35B +26%, and gemma-4-31B-it 3.0× over no speculation — all in production use. Thank you.