vulkan: tune Q4_0/Q5_K/TQ2_0 and DeltaNet kernels; benchmark qwen3.5/3.8 (0.8B, 4B, 27B)

[geisten]

Follow-up to #458: the qwen35 Vulkan kernels are correct (bit-identical to cpu_scalar on an FP32 KV cache) but not tuned.

Not measured / not tuned

• Q4_0 / Q4_1 / Q8_0 / Q5_K / TQ2_0 matvec + GEMM: first-cut kernels (size-agnostic matvec, 8 rows × 32 batch rows GEMM). No comparison with llama.cpp Vulkan on the same GGUFs, no occupancy/vectorization work; Q4_0 SoA loads not benchmarked against the native block layout.
• Gated-DeltaNet mixer (deltanet_conv_f32, deltanet_delta_f32): one workgroup per v-head, thread j owns column j of S in global memory, shared-memory tree reductions with ~19 barriers per token per head; prefill runs the delta rule serially over the sequence. Candidates: keep S in registers/shared memory, subgroup reductions with a size specialization constant, chunked delta rule for prefill.
• caps.dn_subchunk is not declared for Vulkan; the effect of chunking at GEIST_M_MAX 64/128 on prefill is unmeasured.
• Only the 0.8B was benchmarked end to end (pp64 ≈ 1100 t/s, tg ≈ 256 t/s on the 2080 Ti). No numbers for 4B (2080 Ti) or 27B (RADV) beyond "works".

Proposal

Add the qwen3.5-0.8B/4B/27B rows to benchmark/results/VULKAN.md (same protocol as #364), profile with GEIST_VK_PROFILE=1, and attack the top pipes.

Acceptance

• Documented pp/tg per model and device; a ratio against llama.cpp Vulkan for each; PRs per optimisation with before/after.

[geisten]

Data point from the Ternary-Bonsai-2-27B Vulkan port (#472/#473/#477): on the RTX 2080 Ti matmul_pq2_0 is ~8 of ~13 TFLOP/s FP32 peak (2.86 ms per 5120×17408 × 128 rows) → tensor cores (coopmat, ternary codes expanded to f16 in shared memory, as mm_q4k_cm does for Q4_K) are the next prefill step; decode matvec_pq2_0 is ~170 GB/s of ~616 GB/s (16 lanes per 128-element block, x re-read per row group → a multi-row variant like metal's matvec_pq2_n8). On the RADV iGPU the same model runs at 1.5 t/s for prefill and 1.1 t/s for decode; prefill is #471, but decode is ~an order of magnitude below the iGPU's memory bandwidth and is not explained by it — needs a look at the memory type the weight buffers land in and the matvec access pattern on wave64. Details: benchmark/results/TERNARY.md (PR #477).

[geisten]

Head-to-head on the RTX 2080 Ti, same GGUF (PrismML-Eng/llama.cpp adfffbe, llama-bench -ngl 99 -p 128,512 -n 64 -r 2): fork CUDA pp128 577 / pp512 781 / tg64 44.4 t/s; fork Vulkan (KHR_coopmat, int dot: 0) 524 / 556 / 29.7; geist Vulkan 39 / 38 / 23.1. Decode is at 78 % of the fork's Vulkan (52 % of CUDA) — kernel efficiency (multi-row matvec, subgroup reductions), not A8 vs A32, since the fork does not use int-dot here either. Prefill is 14× behind: the fork's GEMM runs on the tensor cores, ours is register-tiled FP32 at ~8 of 13 TFLOP/s. A matmul_pq2_0_cm (coopmat, ternary codes → f16 in shared memory, like mm_q4k_cm) is the change that matters most. Full table: benchmark/results/TERNARY.md (PR #477).

[geisten]

Update: PR #487 (PQ2_0 coopmat GEMM + register-resident DeltaNet state) takes Bonsai pp512 on the RTX 2080 Ti from 38 to 388 t/s (fork Vulkan 556, CUDA 781). Steps: coopmat GEMM 198 → packed k-contiguous shared tiles + arithmetic dequant 247 → DeltaNet state in registers 316 → 128×64 tile 350 → next-step global loads in flight over the MMAs 388. Tried and dropped: BK=64 (191, shared 36 KB → 1 WG/SM), two-deep register prefetch (369). GEMM is ~25 TFLOP/s (was 8). Decode is unchanged (22.5 t/s vs fork Vulkan 29.7) and is the next target.

[geisten]

Status on main (bce22ae, 2026-10-01). Done: DeltaNet state in registers (#487/#489), subgroup reductions in deltanet_delta_f32 (#505), Vulkan declares dn_subchunk (#493), PQ2_0 coopmat GEMM (#487/#493).

Remaining: Q4_0/Q4_1/Q8_0/Q5_K/TQ2_0 matvec/GEMM tuning (shaders unchanged since 37df686/c935b18); a chunked delta rule for prefill (still serial); qwen3.5 0.8B/4B/27B rows plus the llama.cpp ratio in benchmark/results/VULKAN.md.

---

Generated by Claude Code

[geisten]

Status from the ticket sweep: the tuning and the benchmark rows need the 2080 Ti / RADV and the qwen3.5 GGUFs, which this container doesn't have. One part could be done here: the chunked delta rule for DeltaNet prefill, checked against cpu_scalar on the synthetic qwen35 fixture (tests/deltanet_ref.h from #609). Whether it is faster could only be seen on the card.

Question: should I build the chunked delta rule blind (correctness only, behind a switch until it is measured), or leave the whole ticket for a GPU session?

---

Generated by Claude Code