Repository navigation

serve: --moe-slots auto, --moe-prefetch (off by default) - #169

Merged

bong-water-water-bong merged 3 commits into

mainfrom

Sep 27, 2026

Merged

serve: --moe-slots auto, --moe-prefetch (off by default)#169
bong-water-water-bong merged 3 commits into
mainfrom
moe/serve-auto

Commits on Sep 26, 2026
- serve: --moe-slots auto sizes the expert cache from free memory; --moe-prefetch, off by default
  
  bong-water-water-bong
  and
  claude
  committed
  d029aa5
- docs: --moe-slots auto, --moe-prefetch, and the slow prompt path when streaming
  
  bong-water-water-bong
  and
  claude
  committed
  4139fae
- Merge branch 'main' into moe/serve-auto
  bong-water-water-bong
  authored
  0e72711