[gaetan-puleo]

Overview

This PR improves Qwen3.8-Flash-Next PP2048 prompt-processing performance on AMD Strix Halo for both ROCm and Vulkan.

The ROCm changes optimize the production shapes used by:

• Quantized dense matrix multiplication
• Top-10 routing across 512 experts
• MoE expert dispatch and weighted output
• Gated DeltaNet
• Recurrent-state operations
• Reductions and elementwise kernels

The Vulkan changes optimize:

• Wide AMD prompt matmul selection
• Large top-10 MoE matmul selection
• Transposed recurrent-state concat
• Contiguous elementwise multiplication

The changes are restricted to measured AMD or Strix Halo paths and production Qwen3.8 shapes. Other devices retain their existing selection logic.

This PR does not include the commits, corpus fixtures, or AGENTS.md changes from correctness/vulkan-mmv-and-agent-flow.

Measurements

Device: Ryzen AI Max+ 395 / Radeon 8060S
Memory: 128 GB unified LPDDR5X
Power: not recorded
BIOS: UMA split not recorded
Kernel: 7.1.8-200.fc44.x86_64
Backends: ROCm 7.14, Vulkan RADV
ROCm build: Release, GGML_HIP=ON, AMDGPU_TARGETS=gfx1151
Vulkan: Release, GGML_VULKAN=ON
Baseline: 6c84c7d
Change: a7ad7b7
Branch: qwen-3.8-flash-pp-improvement
Model: Qwen3.8-Flash-Next-UD-IQ4_XS, 176.9B parameters, 4.25 bpw
Settings: PP2048, batch=2048, ubatch=2048, F16 K/V, Flash Attention enabled
Repeats: 4

ROCm

Depth	Stock tok/s	After tok/s	Gain
0	441.37 +/- 1.85	639.03 +/- 4.84	+44.78%
12000	323.14 +/- 1.14	405.70 +/- 1.71	+25.55%
32000	235.37 +/- 1.01	271.34 +/- 0.96	+15.28%
64000	167.44 +/- 1.49	189.98 +/- 0.88	+13.46%

Vulkan

Depth	Stock tok/s	After tok/s	Gain
0	435.45 +/- 1.52	469.69 +/- 1.69	+7.86%
12000	303.82 +/- 0.77	321.59 +/- 1.02	+5.85%
32000	206.80 +/- 1.05	216.54 +/- 0.77	+4.71%
64000	128.13 +/- 2.04	130.82 +/- 1.69	+2.10%

Benchmark command:

llama-bench -m MODEL -p 2048 -d 0,12000,32000,64000 -b 2048 -ub 2048 -n 0 -r 4 -ngl 99 -fa on -ctk f16 -ctv f16 --load-mode none -o jsonl

Implementation

ROCm changes include:

• RDNA 3.5 MMQ tile and dispatch tuning
• Qwen top-10 MoE routing specialization
• Parallel deterministic top-10 mm_ids processing
• J48 dispatch for 512-expert/top-10 MoE
• Weighted top-10 expert-output fusion
• Q4_K, Q5_K, and Q6_K production-shape kernels
• Gated DeltaNet tuning
• Recurrent concat, copy, quantization, reduction, and elementwise specializations

Vulkan changes include:

• Cooperative-matrix dequant selection for wide AMD prompt matmuls
• Cooperative-matrix dequant selection for large top-10 MoE batches
• Tiled transpose reuse for recurrent-state concat
• Direct indexing for same-shape contiguous multiplication

Correctness

The optimized builds were compared with separately built reference binaries using identical model, prompt, tokenization, batch, KV, Flash Attention, and offload settings.

Check	Result
ROCm prompt tokens	Byte-identical
ROCm complete final logits	Byte-identical
ROCm logits SHA-256	4631827e942084c116fcea4fb5e30df4f37585797e4e6a7f8880ad24dd71b681
ROCm deterministic decode	Identical
ROCm backend operations	5798/5798 passed
Vulkan prompt tokens	Byte-identical
Vulkan complete final logits	Byte-identical
Vulkan logits SHA-256	aaa573fbcea43753939c6dfd7b4d271bbd1585140c55e6e61fd5abad6311dcde
Vulkan candidate versus fork-stock decode	Identical

The complete Vulkan suite reported 17335/17338. The same three exact-batch matmul failures reproduce on the stock implementation with the same tests, so no candidate-only failure was found.

Faster Flash Attention tile experiments were rejected because they changed final logits.

The pre-existing Vulkan MMV split behavior from ca94157f7 is addressed by a separate correctness PR and is intentionally not included here.

Additional information

What was not verified:

• Non-Strix Halo AMD GPUs
• NVIDIA or Intel GPUs
• AMD proprietary Vulkan
• Windows
• End-to-end speculative decoding with llama-benchy
• Power-profile sensitivity because sustained TDP was not recorded
• A benchmark rerun against the latest post-merge master; the recorded baseline is 6c84c7d5d

Requirements

• I have read and agree with the contributing guidelines
• This change is Strix Halo specific and supported by measurements on Strix Halo
• AI usage disclosure: AGENT-AUTHORED - OpenCode assisted with implementation, profiling, validation, benchmark analysis, and this PR description
• What was NOT verified: non-Strix hardware, proprietary Vulkan, Windows, speculative decoding throughput, and a benchmark rerun against the latest merge-base