[aslater3]

Title

Issue: #1545, #1606 (Arc A-series decode)

Summary

The wide MMVQ kernels (Q6_K a2 and the generic native_mmvq_wide) spill registers at 2-8 columns with the default 128 GRFs on an Arc A770, and their bandwidth collapses as the column count grows (Q6_K 2560x10240: 248 GB/s at 1 column, 74 GB/s at 4, 8 GB/s at 8). On Alchemist they now launch in large-GRF mode (256) at 2-8 columns: 1.3-7.4x faster at 2-8 columns, bit-identical, decode +10-17% on an A770 (the verify window runs these at 2-6 columns every round).

Large GRF halves the threads resident per Xe core, so it is the default on Alchemist only (strata::intel_gpu_gen). Battlemage and every other device, and 1 column everywhere, launch the same kernels as today. STRATA_MMVQ_LARGE_GRF=1 forces it on, =0 off.

What changed

• sycl/src/kernels/cuda/native_mmvq.dp.cpp: mmvq_large_grf(queue) (once per process, Alchemist by default, env override); in launch_q6k_wide (a2 path) and launch_wide, if constexpr (NCOLS > 1) + the runtime switch launch a grf_size<256> copy of the same kernel, else the existing launch, unchanged.
• sycl/src/kernels/mmvq_bench.cpp: prints a checksum of the output (FNV-1a over the float bits), so the modes can be compared bit for bit.

How it was checked

Arc A770 16 GB, xe, Linux 7.0, oneAPI 2026.1 (JIT), Ryzen 7 3800X, 30 GB RAM, same day.

mmvq_bench <n_in> <n_out> <cols>, Q6_K, main vs this PR (us; checksums identical with STRATA_MMVQ_LARGE_GRF=0 and =1 in all 15 cases):

shape	1 col	2 cols	4 cols	6 cols	8 cols
2560 x 2560	33.5 -> 33.4	55.2 -> 30.0	97.0 -> 36.0	150.7 -> 44.5	631.1 -> 153.1
2560 x 10240	86.6 -> 86.1	157.4 -> 79.4	289.3 -> 92.9	552.8 -> 110.2	2726.5 -> 370.5
10240 x 2560	83.7 -> 83.9	119.4 -> 88.9	237.4 -> 103.1	393.9 -> 129.8	1704.4 -> 465.2

End to end, a0116b27 + #2190 + #2193 + #2203 with and without this PR, Coder IQ1_M, 8K-token prompt, 256 tokens, greedy, --resident-experts --max-context 131072 --kv int8 --kv-resident 20480 --spec 4 --mtp, STRATA_VERIFY_STEPPED=1:

• decode 23.47 / 23.50 / 23.73 / 23.77 -> 25.99 / 27.10 / 27.49 tok/s
• prompt unchanged (472-502 tok/s both, within run-to-run spread)
• output ids identical (72b762d5b2a3) in every run with the same expert placement; one run had a different cache placement (hits 0.7605 vs 0.7457-0.7468) and different ids, the CPU-vs-GPU expert rounding seen on main too.

Not built or run on Battlemage or i915. SYCL-only files.

Extra Notes

Several other runs did not finish because of the decode hang in #2194 (both builds); they are left out.

🤖 Generated with Claude Code

[aslater3]

Re-measured on top of v0.1.44.1 (which now has #2190 #2193 #2203 #2199 and the #2194 fix; this branch merges onto it without conflicts), same A770 (xe), Coder IQ1_M, 8K prompt, 256 tokens, interleaved runs, 2026-10-14:

build	decode tok/s	verify window: wait for rings
v0.1.44.1	21.82, 22.16, 22.22, 22.38, 22.61	53.7-54.3 ms/round
v0.1.44.1 + this PR	24.37, 25.21, 25.59	43.3-44.1 ms/round

+10-15%, prompt unchanged (485-510 vs 493-505 tok/s). Output ids identical except one run with a different cache placement (the CPU-vs-GPU rounding). The first run of each build is left out as cold (20.73 and 16.42). No run hung.

🤖 Generated with Claude Code

[aslater3]

Added a89d4e7a: 1-column MMVQ in large GRF as well, for rows up to 4096 wide (STRATA_MMVQ_GRF1, on by default with mmvq_large_grf; =0 turns it off). The 128-register 1-column Q6_K kernel spilled 1.4 KB per SIMD32 thread on the A770 (unitrace).

dense_bench, 1 column, A770 (hashes identical):

matrix	before	after
Q6_K qkv	288 GB/s	316 GB/s
Q6_K z	255	292
Q6_K QSA q	295	325
Q6_K k/v	60	72
Q5_K z	269	294
Q5_K k/v	67	88
Q4_K z	249	269

• Excluded: 6144-wide rows (the out-proj) are unchanged or slower in large GRF, so they keep 128 registers, and so does IQ4_XS (-1 to -8%).
• Engine: the effect is within noise; 1-column windows and the drafter are a small share of a round. Drafting time went 8.1-8.2 -> 7.9-8.2 ms per round.

🤖 Generated with Claude Code

[Niko1221]

Thank you, nice find and a clear write-up. On our Arc A750 the Q6_K wide kernel at 4 columns goes from 175 us to 102 us (2560x10240) and from 2.29 ms to 1.78 ms on the 2560x248320 head with the large-GRF launch, the checksums are identical with the switch on and off, and the engine output is bit-identical to main (first-token logits md5 equal). End to end our A750 run is too noisy to isolate it (the model we have has no Q6_K head), so we are going on your A770 numbers plus the kernel timings. It is in the 0.1.45 candidate, default on for Alchemist only as you wrote it, authorship kept. We also applied the same rule to the new planar Q6_K head kernel from #2340, which spilled without it.

[Niko1221]

Thanks for the pull request.

Your change is already in main for 0.2.0-beta, with your credit, so there is nothing left for you to do here. We're closing it because we're resetting the tracker for 0.2.0-beta: there are more than 1,300 open issues and pull requests, and we're closing everything that isn't being worked on right now.

If part of your work did not make it in and you want it considered, open a new pull request on top of main that follows CONTRIBUTING.md, or comment /reopen after fixing the title area prefix and the template.