[eelgaev]

The prompt path dequantizes each dense Q8_0 projection (GDN qkv/gate/out, QSA q/k/v/out, shared expert, hc) into the GEMM scratch before cuBLAS. dequant_kernel runs one thread per 32-value block, so a warp's 2-byte stores land 64 bytes apart: about 70 GB/s of writes on a V100.

dequant_q8_0_kernel gives each block to four threads, 8 values each, written with 16-byte stores: a warp reads 8 consecutive blocks and writes 512 consecutive bytes. The arithmetic per value is unchanged (d * q), so the output is the same bits. It is used for Q8_0 only when the output rows are contiguous (ld == cols) and the pointers are aligned (blocks to 2 bytes, output to 16); everything else (including the padded ld scratch) takes dequant_kernel as before. One file, 32 lines added, nothing else changed.

Bitwise

Every 2-D Q8_0 tensor of Qwen3.8-Flash-Next UD-Q4_K_XL (498 tensors) through dequant_f32, dequant_bf16 and dequant_f16, built from main (fb58e0d) and from this branch: an FNV-1a hash over all outputs is equal for all three output types. dequant_bf16_test covers Q8_0 against the CPU reference as before.

Speed (V100-SXM2-16GB, each call timed on its own)

	before	after
attn_qkv 10240 x 2560, FP16	0.740 ms (70 GB/s written)	0.106 ms (495 GB/s)
attn_q 12288 x 2560, FP16	0.887 ms	0.126 ms
ssm_out 2560 x 6144, FP16	0.451 ms	0.065 ms
all 498 tensors, FP16	138.5 ms	20.4 ms
all 498 tensors, BF16	140.8 ms	20.5 ms
all 498 tensors, FP32	190.9 ms	35.4 ms

Whole prompts on 4x V100, 4,096-token chunks (where this was first used): 18K 4,537 -> 4,708, 65K 6,540 -> 6,766, 123K 6,783 -> 6,985 tok/s (+3-4%). The kernel is plain CUDA (no architecture-specific instructions), so newer cards take the same path.

[G0-0000]

Independent verification from a V100-SXM2-32GB box (Strata v0.1.41 main + your patch applied cleanly — only dequant_bf16.cu, +32/-0):

• elementwise_parity: 0 failures — 108 row cases, 0 bit mismatches, 0 guard failures; f32→bf16 bulk 0 of 4096 differ. The fast path is bit-identical on our setup.
• Our model pack is all-Q8_0 dense (503 tensors), so the guard (type==8, ld==cols, 2-byte/16-byte alignment) takes the fast path exactly where it should.
• We could not resolve an end-to-end prompt gain on this box (cross-shot variance ~18%), so no speed claim from us — but the correctness side is solid: clean apply, no conflicts, bitwise parity green.

Looks good from our side.

[Niko1221]

Ported with your authorship. dequant_bf16_test passes (10 types, f32/bf16 0 off) on an RTX A4000, and the prompt residual dumps at 4096 and 20000 tokens are identical to the build without it on an RTX 3060 and on the A4000 box. On the RTX 3060 the prompt speed moved by +0.3 to +0.7%, which is within noise there, so we make no speed claim; we could not test the V100 case you describe.