[akaanupkumar]

Overview

Fuse the FFN gate + SwiGLU/GEGLU into the mul_mat_q write-back epilogue. llama.cpp already fuses this pattern for decode (mul_mat_vec_q). This closes that gap for MMQ. The gate matmul keeps its result in registers and applies silu/gelu(gate) * up in the epilogue, so the gate tensor avoids round-trips through DRAM.

The scope of this PR is currently limited to Q4_K, NVIDIA MMA (Turing+), FFN, prefill, SWIGLU/GEGLU.

Additional information

Performance data with RTX 5090 (Blackwell) ( CUDA 13.3, -ngl 999 -fa 1 -r 10, clocks locked 2227/14001 MHz )

llama.cpp defaults (n_batch 2048, n_ubatch 512)

model	test	t/s master	t/s PR	speedup
llama 8B Q4_K_M	pp512	11807.18	11952.58	+1.2 %
llama 8B Q4_K_M	pp2048	11214.46	11331.88	+1.0 %
llama 8B Q4_K_M	pp8192	10307.66	10406.00	+1.0 %
gemma4 12B Q4_K_M	pp512	7422.58	7578.93	+2.1 %
gemma4 12B Q4_K_M	pp2048	6940.84	7076.76	+2.0 %
gemma4 12B Q4_K_M	pp8192	6661.56	6785.78	+1.9 %
qwen3.6 27B Q4_K_M	pp512	3109.04	3158.78	+1.6 %
qwen3.6 27B Q4_K_M	pp2048	3039.65	3082.55	+1.4 %
qwen3.6 27B Q4_K_M	pp8192	2980.33	3022.20	+1.4 %

Microbatch 4096(-b 4096 -ub 4096)

model	test	t/s master	t/s PR	speedup
llama 8B Q4_K_M	pp2048	12324.15	12764.57	+3.6 %
llama 8B Q4_K_M	pp8192	11118.91	11543.19	+3.8 %
gemma4 12B Q4_K_M	pp2048	7368.25	7640.74	+3.7 %
gemma4 12B Q4_K_M	pp8192	6725.42	6974.55	+3.7 %
qwen3.6 27B Q4_K_M	pp2048	3141.85	3216.65	+2.4 %
qwen3.6 27B Q4_K_M	pp8192	3073.55	3153.45	+2.6 %

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: Yes. Paired with Claude.

[lexasub]

may be additionaly constexpr has_fusion (for filter in compile time)

[anujj]

#28702 PR will take care of this fusion