Skip to content

feat(cuda): make GPU dispatch mode configurable#7621

Merged
0ax1 merged 1 commit into
developfrom
ad/cuda-dispatch-mode
Apr 24, 2026
Merged

feat(cuda): make GPU dispatch mode configurable#7621
0ax1 merged 1 commit into
developfrom
ad/cuda-dispatch-mode

Conversation

@0ax1

@0ax1 0ax1 commented Apr 24, 2026

Copy link
Copy Markdown
Contributor

Allow for explicitly choosing dyn dispatch or standalone kernels. We primarily do this to be able to test and bench the standalone CUDA kernels.

In a follow up, I'll change the logic to prefer standalone kernels for auto mode, if available for an array tree, over dyn dispatch.

Allow for explicitly choosing dyn dispatch or standalone kernels.

Signed-off-by: Alexander Droste <alexander.droste@protonmail.com>
@0ax1 0ax1 added the changelog/feature A new feature label Apr 24, 2026
@0ax1

0ax1 commented Apr 24, 2026

Copy link
Copy Markdown
Contributor Author

Proof by bench:

Warning: It is not recommended to reduce nresamples below 1000.
Benchmarking bitunpack_cuda_u8/bitunpack/3bw: Warming up for 1.0000 ns
Warning: Unable to complete 10 samples in 1.0ns. You may wish to increase target time to 4.2s.
bitunpack_cuda_u8/bitunpack/3bw
                        time:   [72.466 µs 78.387 µs 78.355 µs]
                        thrpt:  [1188.6 GiB/s 1188.1 GiB/s 1285.2 GiB/s]
                 change:
                        time:   [-51.700% -44.557% -44.078%] (p = 0.00 < 0.05)
                        thrpt:  [+78.822% +80.364% +107.04%]
                        Performance has improved.
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high severe

Benchmarking bitunpack_cuda_u16/bitunpack/5bw: Warming up for 1.0000 ns
Warning: Unable to complete 10 samples in 1.0ns. You may wish to increase target time to 4.2s.
bitunpack_cuda_u16/bitunpack/5bw
                        time:   [82.442 µs 82.816 µs 83.530 µs]
                        thrpt:  [2229.9 GiB/s 2249.1 GiB/s 2259.3 GiB/s]
                 change:
                        time:   [-39.171% -38.913% -38.575%] (p = 0.00 < 0.05)
                        thrpt:  [+62.801% +63.702% +64.394%]
                        Performance has improved.

@0ax1 0ax1 requested review from a10y and robert3005 April 24, 2026 11:57
@0ax1 0ax1 enabled auto-merge (squash) April 24, 2026 11:57
@0ax1 0ax1 merged commit 7c2df2c into develop Apr 24, 2026
63 of 64 checks passed
@0ax1 0ax1 deleted the ad/cuda-dispatch-mode branch April 24, 2026 20:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/feature A new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants