Plug-and-play wheels: vendored FKL headers + arch/CUDA auto-detection (pip install and go) - #5
Merged
morousg merged 2 commits intoJun 11, 2026
Conversation
New publish-pypi job: on v* tags (after build + codegen compile checks pass) the sdist+wheel are uploaded to PyPI with pypa/gh-action-pypi-publish using OIDC Trusted Publishing — no API tokens or secrets stored in GitHub. One-time setup required on pypi.org (documented in the workflow): add a pending publisher for project 'fkl-python' pointing at this repository, workflow wheels.yml, environment 'pypi'. The 'fkl-python' name is currently unclaimed on PyPI (verified via the simple index). skip-existing=true makes tag re-runs idempotent; the GitHub Release job is unchanged and runs in parallel.
…tion 'pip install fkl-python' now works with ZERO user steps: no FKL checkout, no FKL_INCLUDE/FKL_ROOT/FKL_ARCH env vars. - scripts/vendor_fkl.py copies the FKL headers (header-only, Apache-2.0, 704KB/67 files) into fkl/_vendor/ with LICENSE + upstream commit recorded; pyproject packages them as package-data; the CI build job vendors from LTS-C++17 before building, and the wheel smoke test asserts the headers are present inside the installed wheel. - backend.py resolution cascade: FKL_INCLUDE env > vendored > dev checkout. Resolved lazily so import never fails (CI builds without CUDA still pass the import test). - FKL_ARCH now auto-detects: env > compute capability of GPU 0 via the CUDA driver API (pure ctypes, no nvidia-smi/nvcc needed) > sm_75 floor. - CUDA_HOME auto-detects: env > /usr/local/cuda > nvcc on PATH. Verified end-to-end: wheel installed in a clean venv with env -i (fake HOME, zero FKL env vars) runs a real fused GPU pipeline: arch sm_120 auto-detected, headers resolved from inside the wheel, correct results. Existing test matrix re-verified (vertical/circular/thread-fusion spot checks green).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Makes
pip install fkl-pythontruly plug and play: zero user steps after install — no FKL checkout, noFKL_INCLUDE/FKL_ROOT/FKL_ARCHenvironment variables.(Includes the PyPI Trusted Publishing commit from #4 — merging this supersedes that PR.)
How
Vendored headers: FKL is header-only and Apache-2.0, so
scripts/vendor_fkl.pycopiesinclude/(67 files, ~700KB) intofkl/_vendor/FusedKernelLibrary/with the upstream LICENSE and the exact commit recorded inVENDOR_INFO.txt.pyproject.tomlpackages them as package-data; the CI build job vendors fromLTS-C++17before building and the wheel smoke test asserts the headers are present inside the installed wheel.Resolution cascade (lazily, so
import fklnever fails on CUDA-less machines):FKL_INCLUDEenv var → vendored headers in the wheel → sibling dev checkout. Developers keep full control; users need nothing.GPU arch auto-detection:
FKL_ARCHenv → compute capability of GPU 0 queried via the CUDA driver API with pure ctypes (no nvidia-smi/nvcc needed) →sm_75floor. On the dev box this resolvessm_120correctly.CUDA toolkit auto-detection:
CUDA_HOME/CUDA_PATHenv →/usr/local/cuda→nvccon PATH.Verification (real end-to-end)
Installed the built wheel in a clean venv and ran under
env -iwith a fake HOME (so the dev checkout is unreachable) and zero FKL env vars:Remaining machine requirements (unavoidable for a JIT design): an NVIDIA driver and a CUDA toolkit (nvcc) or clang++. The vendored-headers + JIT approach is what keeps ONE pure-python wheel serving every GPU arch and CUDA version — the alternative (precompiled binaries) would defeat FKL's whole compile-time fusion model.