Skip to content

tools/make_profile.py --base cannot reorder a complete base (shipped profile reports 0 from the traces) #587

Closed

Description

@yannickloth

tools/make_profile.py is documented to rank "the base profile's ranking, then the pairs your routing traces used, most frequent first" so a --dump-routing trace of your workload seeds the cache with the experts it actually routes. But the base is only ever appended to, and the shipped data/expert-profile.bin already ranks all 24,576 (layer, expert) pairs — so with it the trace cannot change anything:

$ python tools/make_profile.py coding-trace.bin --base data/expert-profile.bin --out /tmp/prof.bin
wrote /tmp/prof.bin: 24576 ranked pairs (24576 from the base, 0 from the traces, 0 filled in)

/tmp/prof.bin is byte-identical to data/expert-profile.bin. The only way to let a trace order the top is --no-base, which throws the shipped ranking away entirely.

In rank_profile/main, take(read_profile(base)) fills all 24,576 slots before the trace pairs are considered, and take() skips anything already seen. This matters because --expert-cache auto fills the VRAM tier from the profile in order; a workload-specific profile therefore never gets to put its most-routed experts in the slots that fit. (Measured on an RTX A3000 12 GB / Swift 1.5 IQ3_XXS: the shipped base is 2,107 resident slots; a workload-ranked profile raised the fresh-conversation hit rate 0.468 → 0.528, but only via --no-base.)

Proposed fix

Add --reorder: rank the trace's pairs first (most frequent first), then let the base fill the rest, then the fill — so a trace orders the top while the base still covers everything the trace did not see. Default behaviour is unchanged. PR: #589.

Activity

  1. added a commit that references this issue on Oct 4, 2026
  2. Niko1221 commented on Oct 4, 2026

    @Niko1221
    Owner

    Fixed in 0.1.39 by #589: tools/make_profile.py --reorder ranks your --dump-routing trace's pairs first, then the base's, then the rest. Without the flag the output is unchanged.

  3. added a commit that references this issue on Oct 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions