[Regression]: draft-mtp decode ~10-13% slower on master vs a02c7f58c (RADV, RDNA3) — verify-step slowdown + cache-reuse-dependent acceptance instability #30638

Summary

Paired A/B between a02c7f58c (Sep 26) and current master 2851b3bcb on a
7900 XTX (RADV) shows the speculative draft-mtp path regressed ~10–13% while
the raw decode path did not:

• llama-bench, same commit pair: pp512 +0.9%, tg128 −0.2% (4/4
  counterbalanced pairs) — raw kernels unaffected.
• llama-server + --spec-type draft-mtp --spec-draft-n-max 2: −12.7%
  paired mean (4/4 pairs), decomposing into two independent effects:
  • (a) ~+10% verify-step time at identical work — present with and
    without --cache-reuse.
  • (b) within-session acceptance instability — only with
    --cache-reuse > 0.

Not yet bisected. Two disjoint commit ranges are implicated (details below).

Environment

• AMD RX 7900 XTX (NAVI31, RDNA3), headless (display on iGPU), 7800X3D, 32 GB RAM
• Arch Linux, Mesa RADV 26.2.4 (vulkan-radeon), loader 1.4.363
• Both builds: default CMake Release, Vulkan backend
  • A: a02c7f58c, llama-server md5 1c810682934d0545ea8395b148baacc3
  • B: 2851b3bcb (clean tree), llama-server md5 00e0d0dc645225203c8742c334169350
• GGML_VK_ALLOW_GRAPHICS_QUEUE=1 in both arms (constant; cancels in the
  comparison)
• Model: 27B dense, IQ3_S (3.4375 bpw) with MTP draft head, 11.28 GiB
• Flags: -ngl 99 -c 131072 -np 1 --cache-reuse 256 -t 16 -fa on -ctk q4_0 -ctv q4_0 --spec-type draft-mtp --spec-draft-n-max 2 --jinja

Method

4 counterbalanced pairs (AB|BA|AB|BA), fresh server per cycle, GPU edge-temp
gate ≤50 °C, discarded 300-token warmup per cycle, 3 × 300-token greedy
completions per cycle (identical 41-token prompt). Decode rate from
wall-clock; acceptance and exact target-decode counts ("steps") recovered
from server logs. Per-run md5/env/cmdline provenance.

Results (production flags, --cache-reuse 256)

Effect (a): pair 1 ran the same number of target decodes with equal
acceptance (397.86 vs 397.87 steps; 0.626 vs 0.629) yet took 13.1 s vs 11.9 s
→ 32.9 vs 29.9 ms per decode step (+10%). Consistent in every master cycle
(32.5–32.9 ms vs 29.3–29.9 ms on a02c).

Effect (b): master acceptance within one server session is bimodal at rep
level (e.g. cycle 2 reps: 0.46 / 0.62 / 0.44) while a02c sessions stay uniform
(0.55–0.70). Degraded reps offer more drafts (draft_n 302–315 vs ~250–285)
with fewer accepted → steps inflate.

Probe: --cache-reuse 0

• (b) disappears without reuse → tracks the KV-reuse / slot-state path.
• (a) persists: 32.6 vs 29.6 ms/step (+10.1%) → independent of reuse.
• cr0 is not a workaround (master still −9.4% at cr0).

Candidate ranges (in a02c..2851b3b, not bisected)

• For (b): server: save the draft state in the slot file #30574 (save draft state in slot file), server: prefer empty slots and keep states overwritten by child tasks #30567 (slot preference /
  state overwrites).
• For (a): vulkan : split matmul dispatch when it would exceed maxComputeWorkGroupCount #29533 (matmul dispatch split), vulkan : reuse descriptor sets when bindings are constant #29280 (descriptor-set reuse),
  vulkan: pack FMA for f16 accumulation, no dot product op #29877 (f16 FMA packing) — speculative-loop kernel/scheduling changes.
• Also in range, unexamined: vulkan: fix amd iGPU slow checkpoint read #30049, vulkan : copy strided f16 KV once on AMD so reads spread across memory channels #27703, server: fix the off-by-two in the SWA reuse threshold #30580.

Notes

• Between-session acceptance variance exists on both builds (a02c 0.584–0.629)
  — baseline run-to-run (GPU non-determinism under greedy); the distinctive
  master failure is the within-session swing with reuse on.
• Happy to run a bisect or collect more reps on request.

Disclosure: measurement harness, analysis and this report were prepared with
AI assistance; all benchmark runs were performed and verified locally by me.