[CUDA] Add INT4 paged KV cache with per-channel scales - #32515

Merged

Tianlei Wu (tianleiwu) merged 12 commits into

mainfrom

tlwu/20260909/int4_kv_per_channel

Sep 10, 2026

Commits on Sep 9, 2026

Commits on Sep 10, 2026

[CUDA] Normalize the PER_CHANNEL K scale folded into Q for paged XQA (#32520)
Tianlei Wu (tianleiwu)
authored
4879861
lintrunner

Tianlei Wu (tianleiwu)
and
github-actions[bot]
authored
40e4bb0
fix(cuda): preserve paged KV scale range
Tianlei Wu (tianleiwu)
committed
f049ffc
lintrunner

Tianlei Wu (tianleiwu)
and
github-actions[bot]
authored
5261110
test(cuda): assert per-channel K decode takes the portable kernel
Tianlei Wu (tianleiwu)
committed
20e1d56
Restore XQA for per-channel KV with a normalized fold, behind a switch
Tianlei Wu (tianleiwu)
committed
5b88ad9
test(cuda): cover the bounded normalizer exponent on XQA
Tianlei Wu (tianleiwu)
committed
395661b