Repository navigation
helper cache: configurable VRAM allowance with allocation-floor checks - #1458
Closed
saikiran-rs wants to merge 1 commit into
Closed
saikiran-rs wants to merge 1 commit into
saikiran-rs wants to merge 1 commit into
Conversation
Owner
|
Thanks. No speed claim, so it is outside this speed round; it can go in as a plain setting if you rebase it. |
Owner
|
Thanks for the pull request. We're resetting the tracker for 0.2.0-beta. There are more than 1,300 open issues and pull requests, too many for us to review properly, so we're closing everything that isn't being worked on right now. Closed here does not mean rejected. To reopen it takes about two minutes:
The rules are in CONTRIBUTING.md. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Helper expert caches use a fixed 512 MiB allowance during admission. Add
STRATA_REMOTE_RESERVE_MIBso a measured setup can explicitly choose a different allowance, while retaining 512 MiB when unset.What changed
This changes helper cache admission only. The historical 128 MiB setting was used on one Linux RX 7900 XTX + RX 6800 XT machine; the default remains 512 MiB. Its combined tuning results are documented in the companion benchmark report, with no isolated allowance speedup claimed.
Validation
Based on upstream
d5ea7133741e67743c0e886bb426c0ce8d69cf6c(0.1.40.3):ctest -R '^remote_reserve_test$': passed.remote_experts.cppcompiled against the HIP SDK in a fresh isolated build configuration: passed.git diff --check: passed.No new GPU/model inference run was performed on this isolated PR. Historical full-context/quality results refer to the earlier combined research build. CUDA and Windows runtime testing remain unmeasured.
Related #1184 handles primary-device selection and resident-RAM helper support; #848 handles a layer split's reserve. Their changes are not duplicated here.