You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
tools/make_profile.py --base cannot reorder a complete base (shipped profile reports 0 from the traces) #587
tools/make_profile.py is documented to rank "the base profile's ranking, then the pairs your routing traces used, most frequent first" so a --dump-routing trace of your workload seeds the cache with the experts it actually routes. But the base is only ever appended to, and the shipped data/expert-profile.bin already ranks all 24,576 (layer, expert) pairs — so with it the trace cannot change anything:
$ python tools/make_profile.py coding-trace.bin --base data/expert-profile.bin --out /tmp/prof.bin
wrote /tmp/prof.bin: 24576 ranked pairs (24576 from the base, 0 from the traces, 0 filled in)
/tmp/prof.bin is byte-identical to data/expert-profile.bin. The only way to let a trace order the top is --no-base, which throws the shipped ranking away entirely.
In rank_profile/main, take(read_profile(base)) fills all 24,576 slots before the trace pairs are considered, and take() skips anything already seen. This matters because --expert-cache auto fills the VRAM tier from the profile in order; a workload-specific profile therefore never gets to put its most-routed experts in the slots that fit. (Measured on an RTX A3000 12 GB / Swift 1.5 IQ3_XXS: the shipped base is 2,107 resident slots; a workload-ranked profile raised the fresh-conversation hit rate 0.468 → 0.528, but only via --no-base.)
Proposed fix
Add --reorder: rank the trace's pairs first (most frequent first), then let the base fill the rest, then the fill — so a trace orders the top while the base still covers everything the trace did not see. Default behaviour is unchanged. PR: #589.
Fixed in 0.1.39 by #589: tools/make_profile.py --reorder ranks your --dump-routing trace's pairs first, then the base's, then the rest. Without the flag the output is unchanged.
Activity
docs: the layer split's per-card dense weights (PR #639), STRATA_AREN…
Niko1221 commentedon Oct 4, 2026
Fixed in 0.1.39 by #589:
tools/make_profile.py --reorderranks your--dump-routingtrace's pairs first, then the base's, then the rest. Without the flag the output is unchanged.docs: the layer split's per-card dense weights (PR #639), STRATA_AREN…
docs: the layer split's per-card dense weights (PR Niko1221#639), STR…
docs: the layer split's per-card dense weights (PR Niko1221#639), STR…
DeepSeek --parallel: the prompt read a piece at a time with the other…
DeepSeek --parallel on an event scheduler: prompts read a piece at a …