[bong-water-water-bong]

1bit serve --moe-slots passed only -ot exps=CPU, so llama.cpp kept a full RAM copy of every expert, which is exactly what streaming is meant to avoid. On Qwen3-Coder-30B:

flags	expert buffer
-ot exps=CPU	Vulkan_Host (pinned) 16.9 GiB
+ --no-host	CPU 3.9 GiB + CPU_REPACK 12.7 GiB
+ --no-host -nr	CPU 16.9 GiB (the default load mode reads the file)
+ --no-host -nr --load-mode mmap	CPU_Mapped: the process holds 259 MiB of anonymous memory

Perplexity is identical in every case (21.1029). Decode never used the copy, so the streamed speeds already measured still hold, but the memory saving did not.

• The serve path now passes all three flags.
• Verified with 1bit serve --moe-slots 1536: the chat answers, and serve plus llama-server hold 128 MiB of anonymous memory. The rest of their RSS is file-backed page cache.
• docs/moe-streaming.md corrects its "memory-mapped" claims and explains the three flags.

This is also what blocked streaming Flash-Next (71.7 GiB of experts): the memory guard fired on the copy.

🤖 Generated with Claude Code

[context7]

Docs7 for 1bit-monster/engine

Result	Status	Action
Deployment	➖ Not used	—
Content review	➖ Did not run. This site has no agent runs available this month. Wait for the monthly reset or check your Docs7 plan.	—

Commit 74368e8

[github-actions]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪

🧪 No relevant tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Missing Error Handling for MOE Flags The code adds new command-line arguments for MOE handling but does not validate that the required flags are correctly set. If --no-host, --no-repack, and --load-mode mmap are not properly handled by the underlying llama.cpp implementation, it could lead to unexpected behavior or runtime errors. The change assumes these flags will work as intended without explicit checks. // the experts stay file-backed: without these, llama.cpp copies every one into RAM // (a pinned Vulkan host buffer, or a CPU repack), which is what streaming avoids argv.insert(argv.end(), {"-ot", "exps=CPU", "--no-host", "--no-repack", "--load-mode", "mmap"}); Inconsistent Flag Description The documentation states that all three flags (-ot exps=CPU, --no-host, --no-repack, --load-mode mmap) are needed, but the description of the previous behavior only mentions -ot exps=CPU alone. This could be misleading to users who are not familiar with the full context of how these flags interact with each other. All three flags are needed. With `-ot exps=CPU` alone, llama.cpp puts the experts in a pinned Vulkan host buffer (Coder-30B: 16.9 GiB of RAM); `--no-host` alone gets a CPU repack (12.7 GiB); and this build's default load mode reads the file into memory. With all three, the experts are a `CPU_Mapped` buffer and the process holds 259 MiB of anonymous memory. `1bit serve --moe-slots` passes all three. Before 2026-09-26 it passed only `-ot exps=CPU`, so the streamed runs below also held a full RAM copy of the experts. Decode never used that copy, so the speeds hold, but the memory saving did not.