Repository navigation
Conversation
At 1 or more the usage counts never decay: they grow every adaptation, reach inf (2.0: after 128 adaptations) and the swap gains become inf - inf = NaN, which the swap sort then compares. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VZy1yKaDDiA8a7svdwaHio
txiki739
added a commit
to txiki739/Strata3090-5060TI
that referenced
this pull request
Oct 5, 2026
…iko1221#591: at 1 or more the counts grow without end) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
txiki739
added a commit
to txiki739/Strata3090-5060TI
that referenced
this pull request
Oct 5, 2026
…1221#838 q8_1 finite, Niko1221#863, Niko1221#851, Niko1221#591) and what they gave
txiki739
added a commit
to txiki739/Strata3090-5060TI
that referenced
this pull request
Oct 5, 2026
…of RAM The 64 GB section now has the same measurements as the 128 GB ones (the engine limited to 60 GiB by a cgroup): the decode per prompt and the prefill of both models with one and two GPUs, long prompts of 32K to 200K, and the agent sessions. UD-IQ4_XS fits at the cap and loses 2-4% (81.5 / 104.7 tok/s); UD-Q4_K_XL reads its experts from the NVMe (21.4 / 25.2 tok/s, varying a lot between sessions) and needs 44 min for a 200K prompt. The CPU-worker and --adapt-decay charts are re-measured on the current engine (two runs per point; 1.0 is refused since PR Niko1221#591), and the agent sessions with 128 GB updated (78.1 / 59.9 tok/s).
Contributor
Author
|
Reopened as #1047, the same branch and commit. GitHub closed this one when my fork was made private by mistake, which took it out of the fork network. |
txiki739
added a commit
to txiki739/Strata3090-5060TI
that referenced
this pull request
Oct 12, 2026
…iko1221#591: at 1 or more the counts grow without end) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
eddoursul
pushed a commit
to eddoursul/Lamina
that referenced
this pull request
Oct 12, 2026
…iko1221#591: at 1 or more the counts grow without end) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The range check from #407, alone on 0.1.38, as offered there. 0.1.38's
--adapt-decayaccepts any value.What goes wrong at 1 or more.
usageis multiplied by the decay after every adaptation, so with 1 or more the counts never shrink. At 2, an expert routed early in the conversation passesFLT_MAXafter 128 adaptations, which is about 128 verify rounds with--adapt-every 1or ~500 with the default 4. Then, inadapt:cand[i].first < vict[i].first + 1.5fis false for inf against inf, so the pair is pushed withgain = inf - inf = NaN.std::sort(swaps, gain > gain)then compares NaNs. That breaks the strict weak orderingstd::sortrequires, which is undefined behaviour.Measured on v0.1.38 (RTX 5090, IQ2_XS, fixed
--expert-cache 14900, a 1,000-token chat,--adapt-every 1, two runs each):--adapt-decayAt 2 the tier swapped 3.2x as many experts. Nothing visible broke in a run this short (same output both times, no crash), so the case for refusing is the undefined sort, not a measured failure.
The change: the flag parse refuses values outside (0, 1) with "--adapt-decay must be between 0 and 1 (exclusive)" and exit code 2 (checked with 1.5). Values inside the range behave as before.
🤖 Generated with Claude Code
https://claude.ai/code/session_01VZy1yKaDDiA8a7svdwaHio