Repository navigation

CUDA: improve top-k algorithm selection - #28713

Merged

ORippler merged 15 commits into

ggml-org:masterfrom

praneshgo:pgonegandla/topk-radix-only

Oct 8, 2026

Merged

CUDA: improve top-k algorithm selection#28713
ORippler merged 15 commits into
ggml-org:masterfrom
praneshgo:pgonegandla/topk-radix-only

Commits on Sep 10, 2026
- CUDA: radix top-k for large row counts
  praneshgo
  committed
  a011ccd
Commits on Oct 6, 2026
- Merge branch 'master' into pgonegandla/topk-radix-only
  Pranesh Gonegandla
  committed
  04e682a
- CUDA: select the TOP_K implementation by shape
  Pranesh Gonegandla
  committed
  e4683d8
- CUDA: make top-k comments less verbose
  Pranesh Gonegandla
  committed
  0d4ee05
Commits on Oct 7, 2026
- CUDA: remove the TOP_K width limit from supports_op
  Pranesh Gonegandla
  committed
  cb68893
- CUDA: use DeviceTopK for single-row TOP_K if available
  Pranesh Gonegandla
  committed
  975bf81
Commits on Oct 8, 2026
- CUDA: avoid ncols overflow in the TOP_K bitonic check
  Pranesh Gonegandla
  committed
  d2f31e4
- CUDA: share the row chunking helper between argsort and top-k
  Pranesh Gonegandla
  committed
  036b9f4
- CUDA: do the TOP_K radix blocks_per_row math in int64_t
  Pranesh Gonegandla
  committed
  61c3682
- CUDA: rename GGML_CUDA_TOP_K_NROWS_THRESHOLD_DEVICETOPK to GGML_CUDA_TOP_K_NROWS_THRESHOLD
  Pranesh Gonegandla
  committed
  0454fc4
- CUDA: share one sort helper between the bitonic and CUB TOP_K paths
  Pranesh Gonegandla
  committed
  8fb941a
- CUDA: update the TOP_K TODO, threshold and chunking comments
  Pranesh Gonegandla
  committed
  4bdba64
- tests: add TOP_K cases that span several row chunks
  Pranesh Gonegandla
  committed
  ebcdb64
- CUDA: use int64_t col in the TOP_K radix loops, fix threshold comment
  Pranesh Gonegandla
  committed
  a406e45
- CUDA: limit TOP_K and ARGSORT support to ne[0] <= INT_MAX
  Pranesh Gonegandla
  committed
  45211ef

Commits

Commits on Sep 10, 2026

Commits on Oct 6, 2026

Commits on Oct 7, 2026

Commits on Oct 8, 2026