Repository navigation
Pascal build runs (STRATA_EXPERIMENTAL_SM60) - #295
Closed
giostrives wants to merge 1 commit into
Closed
giostrives wants to merge 1 commit into
giostrives wants to merge 1 commit into
Conversation
…to 6.0; setup accepts it -DSTRATA_EXPERIMENTAL_SM60=ON built for Pascal but the engine still refused the card at start (src/core/device.cu). The flag now reaches the code and lowers that check to 6.0. setup.py accepts 6.x cards when STRATA_EXPERIMENTAL_SM60=1 and passes the flag to CMake when it compiles for one. Tested on 4 x GTX 1080 Ti (sm_61, CUDA 12.9, driver 580), IQ2_XS: one card 27 tok/s decode, 262 tok/s on a 28K-token prompt; needle found at 2K-28K on 1-4 cards. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Owner
|
Thanks! The engine/CMake part shipped in 0.1.31 (the flag and the device check). We're taking the setup part, extended to sm_70 (setup admits 6.x/7.0 only with STRATA_EXPERIMENTAL_SM60=1 and builds with CUDA 12.x). |
Contributor
Author
|
Thank you for your work! It must be exhausting with all these PRs. Okay, so I don't understand: do you need me to fix the conflicts or will you simply copy part of the config from here? I also updated the other pr which was closed (I tagged you as I can't reopen it): #294 |
Niko1221
added a commit
that referenced
this pull request
Oct 1, 2026
…the community build with CUDA 12.x (#295) The engine/CMake part shipped in 0.1.31. Setup now admits compute capability 6.x and 7.0 only when STRATA_EXPERIMENTAL_SM60=1 is set (gpu_problem; without it the message says so), compiles the engine with -DSTRATA_EXPERIMENTAL_SM60=ON for cards below sm_75 (engine_defs), and takes the newest CUDA 12.x toolkit (find_nvcc(below=(13, 0)); CUDA 13 cannot build sm_60/sm_70) - stopping with a hint when there is none, or when an RTX 50 card (CUDA 13) shares the model with such a card. Without the variable nothing changes. Test: tools/test_setup_choices.py (mocked toolkits and cards). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0154bwiad8gY7wRehCXy27pN
Owner
|
0.1.32: setup admits 6.x/7.0 with |
Closed
Contributor
|
A data point and a build note for Pascal on Ubuntu 26.04 (glibc 2.43), v0.1.39 (
Disclosure: prepared with an AI assistant (Claude Sonnet 5.5, medium effort) under my direction. |
Niko1221
added a commit
that referenced
this pull request
Oct 6, 2026
…the community build with CUDA 12.x (#295) The engine/CMake part shipped in 0.1.31. Setup now admits compute capability 6.x and 7.0 only when STRATA_EXPERIMENTAL_SM60=1 is set (gpu_problem; without it the message says so), compiles the engine with -DSTRATA_EXPERIMENTAL_SM60=ON for cards below sm_75 (engine_defs), and takes the newest CUDA 12.x toolkit (find_nvcc(below=(13, 0)); CUDA 13 cannot build sm_60/sm_70) - stopping with a hint when there is none, or when an RTX 50 card (CUDA 13) shares the model with such a card. Without the variable nothing changes. Test: tools/test_setup_choices.py (mocked toolkits and cards).
anon761
pushed a commit
to anon761/Strata-upstream
that referenced
this pull request
Oct 6, 2026
…the community build with CUDA 12.x (Niko1221#295) The engine/CMake part shipped in 0.1.31. Setup now admits compute capability 6.x and 7.0 only when STRATA_EXPERIMENTAL_SM60=1 is set (gpu_problem; without it the message says so), compiles the engine with -DSTRATA_EXPERIMENTAL_SM60=ON for cards below sm_75 (engine_defs), and takes the newest CUDA 12.x toolkit (find_nvcc(below=(13, 0)); CUDA 13 cannot build sm_60/sm_70) - stopping with a hint when there is none, or when an RTX 50 card (CUDA 13) shares the model with such a card. Without the variable nothing changes. Test: tools/test_setup_choices.py (mocked toolkits and cards).
anon761
pushed a commit
to anon761/Strata-upstream
that referenced
this pull request
Oct 6, 2026
…the community build with CUDA 12.x (Niko1221#295) The engine/CMake part shipped in 0.1.31. Setup now admits compute capability 6.x and 7.0 only when STRATA_EXPERIMENTAL_SM60=1 is set (gpu_problem; without it the message says so), compiles the engine with -DSTRATA_EXPERIMENTAL_SM60=ON for cards below sm_75 (engine_defs), and takes the newest CUDA 12.x toolkit (find_nvcc(below=(13, 0)); CUDA 13 cannot build sm_60/sm_70) - stopping with a hint when there is none, or when an RTX 50 card (CUDA 13) shares the model with such a card. Without the variable nothing changes. Test: tools/test_setup_choices.py (mocked toolkits and cards). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0154bwiad8gY7wRehCXy27pN
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I am running this on my machine with 4 × GTX 1080 Ti: I had some issues with the previous commit:
-DSTRATA_EXPERIMENTAL_SM60=ON (01bca8a) builds for Pascal, but the engine still refused the card at startup (src/core/device.cu). The flag now reaches the code and lowers that check to 6.0. setup.py accepts 6.x cards when STRATA_EXPERIMENTAL_SM60=1 and passes the flag to CMake. Nothing changes without the flag.
The speed for the "small" model goes from 27 tok/s to about 35 tok/s on 3 or 4 cards.
The prompt processing speed slows down on multiple GPUs (it is an old machine) and the memory management needs some fixes.
Once this and my other PR for multi-gpu are merged I'll run a second optimisation run for both my machines.