Repository navigation
ci(cuda): detect runner GPU architectures before building - #8347
Merged
mohanchen merged 1 commit intoNov 21, 2026
Merged
Conversation
Query compute capabilities in the CI allocation and pass unique numeric architectures to CMake. Keep single-architecture builds on homogeneous runners and preserve the existing compiler cache and test steps.
mohanchen
approved these changes
Nov 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The CUDA workflow can run on Tesla P100 (sm_60), but it currently builds only for sm_70. Detecting the runner's compute capabilities lets P100 build for 60 and V100 build for 70, avoiding CUDA error 209 without compiling unused architectures.
Linked Issue
No separate issue is needed: the failing develop CUDA job records a Tesla P100-PCIE-16GB, CUDA 12.2.140,
CMAKE_CUDA_ARCHITECTURES=70, andno kernel image is available for execution on the deviceinsnap_psibeta_kernel.cu.What's changed?
Query
nvidia-smi --query-gpu=compute_cap --format=csv,noheaderinside the CI job, validate the output, and pass sorted, unique numeric architectures to CMake. Repeated devices produce one target; mixed allocations compile for each architecture present. Empty, invalid, or failed detection stops configuration.Detection is limited to CI, where compilation and testing use the same GPU allocation. Keep the current build parallelism, shared compiler cache, and GPU test steps. Explicit numeric nvcc arguments distinguish CUDA objects while preserving C++ cache reuse across GPU types.
Unit Tests and/or Case Tests for my changes
Exercised the actual detection block with 12 GPU responses, including P100, V100, repeated/mixed devices, reordered devices, A100, two-digit major capabilities, empty output, malformed values, and query failure. All passed; the architecture list reaches CMake as one argument.
CUDA 12.2.140 / GCC 12.4.0: the detected V100 target produces only sm_70 and a CUDA probe runs successfully on Tesla V100-SXM2-16GB. Simulated P100 input
6.0produces target 60 and only sm_60 machine code. P100 execution is not yet verified.Built the existing ABACUS CUDA targets with detected architecture 70 using CUDA 12.9.86 / GCC 13.3.0.
cuobjdump --list-elfconfirms that the actualsnap_psibeta_kernel.cu.ocontains only sm_70. Build commands:With
OMP_NUM_THREADS=1on Tesla V100-SXM2-16GB, the following command passed all three targets: 8 snap tests, 5 radial-table tests, and 1 radial-interpolation CUDA test (14 tests total):ctest --test-dir build -V --timeout 1700 \ -R '^(MODULE_LCAO_tddft_snap_psibeta_half_test|MODULE_LCAO_radial_tables_gpu|MODULE_LCAO_tddft_rad_interp_cuda_test)$'With shared ccache 4.14.3 / CUDA 12.2.140, compiling the same source for
70 -> 60 -> 70produced two misses and one direct hit; object inspection confirmed the requested architecture each time.Bash, actionlint, focused ShellCheck,
git diff --check, and repository governance checks passed. Whole-workflow ShellCheck reports the same four existing SC2162 informational findings as develop.Still required: a complete CUDA workflow run on the upstream P100 runner. Full-workflow build timing and all GPU Autotest suites were not reproduced locally.
Governance Notes
No INPUT or core-module behavior changes; no documentation update or governance exception required. Existing CUDA tests exercise the resulting builds.