You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
DeepSeek and MiMo Flash as alternative to GLM Flash #4
Thanks, those are a good size for this. Each new architecture needs its own model code in the engine, and it reads llama.cpp's GGUF, so the first thing I'll check is how well llama.cpp supports them. They'd also need a lot of RAM, around 150GB+ at 3-4 bit. Keeping this open.
KV cache compression helps with long contexts, but at ~300B it’s the weights that don’t fit, the cache is the small part. Lower quants would do more. That’s how GLM 5.3 Flash runs here: the smallest one is a REAP50 Q3_K_M, 79GB with half the routed experts pruned, and it needs ~60GB of RAM next to a 32GB card. I’d rather start from published GGUFs than make my own, since checking that a 2-bit quant still answers well is a project of its own. If a good IQ2 or REAP version of DeepSeek V4 Flash or MiMo Flash shows up, that’s what I’d go for. Still need llama.cpp support first.
Nobody done aggressive REAP of DeepSeek v4.1 or DeepSeek v4 Pro or Kimi K3 or Kimi K2.7 or MiMo v2.6 Pro
Some more qa work on my Mimo flash and glm flash REAPS is worth pursuing. Reducing active experts could help decode speed, but I'm not sure that its worth the potential intelligence cost. I also fine tuned the draft head for the 0xsero 0731 REAP. I need to see how it transfers to gguf. I'm interested in making an ambitious REAP to fit Mimo pro on 128gb; if it can work, it would be valuable, if slow.
DeepSeek V4 Flash has been in since 0.1.44.1-poly.2 (docs/DEEPSEEK.md). I'm closing this and moving MiMo Flash to the vote in #16, with the other model requests.
Activity
VecSzn commentedon Oct 10, 2026
Thanks, those are a good size for this. Each new architecture needs its own model code in the engine, and it reads llama.cpp's GGUF, so the first thing I'll check is how well llama.cpp supports them. They'd also need a lot of RAM, around 150GB+ at 3-4 bit. Keeping this open.
TomLucidor commentedon Oct 11, 2026
@VecSzn if you can quant this + use KV cache compression that might do the trick?
VecSzn commentedon Oct 11, 2026
KV cache compression helps with long contexts, but at ~300B it’s the weights that don’t fit, the cache is the small part. Lower quants would do more. That’s how GLM 5.3 Flash runs here: the smallest one is a REAP50 Q3_K_M, 79GB with half the routed experts pruned, and it needs ~60GB of RAM next to a 32GB card. I’d rather start from published GGUFs than make my own, since checking that a 2-bit quant still answers well is a project of its own. If a good IQ2 or REAP version of DeepSeek V4 Flash or MiMo Flash shows up, that’s what I’d go for. Still need llama.cpp support first.
TomLucidor commentedon Oct 12, 2026
Okay for similar weights (assuming REAP50 or smaller)
If I want to dream ambitious, I would pick crazy REAP for:
patrickbdevaney commentedon Oct 12, 2026
Some more qa work on my Mimo flash and glm flash REAPS is worth pursuing. Reducing active experts could help decode speed, but I'm not sure that its worth the potential intelligence cost. I also fine tuned the draft head for the 0xsero 0731 REAP. I need to see how it transfers to gguf. I'm interested in making an ambitious REAP to fit Mimo pro on 128gb; if it can work, it would be valuable, if slow.
VecSzn commentedon Oct 17, 2026
DeepSeek V4 Flash has been in since 0.1.44.1-poly.2 (docs/DEEPSEEK.md). I'm closing this and moving MiMo Flash to the vote in #16, with the other model requests.
cpu pool: sleep between requests instead of spinning (fixes simplexai…
Merge pull request simplexai-labs#9: CPU pool: sleep between requests…
engine: fix a race in the CPU expert pool, and never hang silently ag…