Qwen3.8-27B DFlash2: accepted length ~3 on llama.cpp vs ~5.5 on vLLM with the same drafter — expected from target pairing, or a selector setting? #170

Question: expected accepted length for the Qwen3.8-27B DFlash2 drafter on llama.cpp vs vLLM?

Measured on 2× AMD Radeon AI PRO R9700 (gfx1201), ROCm 7.2.3, same drafter weights on two engines, same day:

llama.cpp (ggml-org/llama.cpp master + PR #27858 for tensor split, 3286b033c; details and numbers in my comment there: ggml-org/llama.cpp#27858 (comment))

• target: unsloth/Qwen3.8-27B-GGUF UD-Q6_K_XL, q8_0 KV, -sm tensor -fa on
• drafter: z-lab/Qwen3.8-27B-DFlash2-GGUF Q8_0 (the reconverted file with dflash.rope.dimension_sections)
• common_speculative_impl_draft_dflash: block_size=8, mask_token_id=248070, n_extract=5, sample_from_anchor=true (defaults, untuned)
• slot timings: draft acceptance 0.41–0.65 at n_max 3, 0.29–0.43 at n_max 5, 0.21–0.33 at n_max 7; mean accepted len ≈2.2–3.3 at every depth
• greedy output is byte-identical at n3/n5/n7 and to the built-in MTP head, so the verify path looks correct; it just doesn't accept more than MTP does (MTP: 2.4–2.9). Result: DFlash n3 53 tok/s vs MTP n3 56, no-spec 30.5.

vLLM (0.27.1 with the Radiance RDNA4 kernels, tcclaviger/Qwen3.8-27B-DFlash2-FP8, an FP8 requant of your checkpoint, target amd/Qwen3.8-27B-Quark-AWQ-MXFP4, num_speculative_tokens 7, greedy draft):

• ≈5.5 accepted tokens/step; 134 tok/s single-stream vs 55 without the drafter.

Is a ~3 vs ~5.5 accepted-length gap expected from the target pairing (Q6_K GGUF vs the MXFP4 AWQ checkpoint, both distinct from the bf16 you trained against), or is there a selector / n_extract / sample_from_anchor setting the llama.cpp path should be using? Happy to run whatever helps, including the bf16 GGUF drafter or other targets, and to share the JSON/logs.

Thanks for DFlash2 and for keeping the GGUFs current.