You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Peer-expert tier (--peer-device 1) crashes on first prompt: "iq_dequant_gu_f16: an illegal memory access" #2666
Bug: two-GPU peer-expert tier (--peer-device 1) crashes on the first prompt — "iq_dequant_gu_f16: an illegal memory access"
Engine: 0.1.41 (locally built for this CPU) OS: Windows 11 (WDDM) Hardware: 2x NVIDIA RTX 3060 12 GB, Intel Xeon E5-2697 v2 (no AVX2), 128 GB RAM Model: swift-1.5-iq2_xs (Qwen3.8-Flash-Next MoE), context 128000, KV q4_0, --kv-resident 32768, STRATA_ARENA_PIN_GIB=26
Reproduction
Run on two GPUs (config gpu: [0, 1]) with --peer-device 1, so CUDA1 becomes an adaptive expert-cache tier instead of a layer split. The engine starts cleanly and fills the peer tier, but the first ~5.3k-token prompt crashes it deterministically:
strata generate: --trim-stage-weights needs a layer split across GPUs (ignored)
strata generate: expert cache 3353 slots, 4.49 GiB of VRAM; policy is
strata generate: peer GPU 1: 7723 experts, 10.36 GiB (filled in 4.5 s); with the primary's 3353 that is 11076 of 24576 on the GPUs
strata generate: session is up (engine 0.1.41)
strata serve: prompt chunk auto: 7936 tokens, a 96-slot ring
strata prefill: peer GPU 1 computes its experts' rows of each prompt chunk (up to 79360 rows per layer, no P2P: through mapped host memory in FP16, one weighted sum per token back, compact group buffers); 0 MiB left free on it
strata serve: 443 MiB of VRAM free with everything loaded
iq_dequant_gu_f16: an illegal memory access was encountered
Crash site: src/prefill/prefill.cpp:3679 (iq_dequant_gu_f16 on the P6 native-pack dequant path).
Not a VRAM-shortage (already ruled out)
All three crash identically on the same first prompt:
--vram-reserve-mib 1800 (primary reserve raised): 1497 MiB free on CUDA0 — same crash
--prefill auto:8192 (smaller prompt chunks, per the engine's own startup hint) + --peer-reserve-mib 1200: same crash, peer card 679 MiB free
With no peer tier the same model+prompts run fine on the same two cards as a layer split.
Why this matters
The peer tier is a big win for small cards: primary 3353 + peer 7723 = 11076 of 24576 experts on the GPUs, versus ~5,700 on the two-card layer split for the same model. If the peer prefill path could run without the illegal access, small-VRAM dual-GPU boxes would get a substantial decode lift.
We're resetting the tracker for 0.2.0-beta. There are more than 1,300 open issues and pull requests, too many for us to read properly, so we're closing everything that isn't being worked on right now. Closed here does not mean rejected.
To reopen it takes about two minutes:
Re-test on 0.2.0-beta. It may already be fixed. Write down what you see.
Start the title with its area: [Engine], [GUI], [TUI], [Benchmark], [Docs] or [Other].
Fill in the parts of the issue form: what happened, the Strata version, your graphics card, memory and system.
Comment /reopen. A check runs and either reopens the issue or lists what is still missing.
Activity
peer experts: peer access is used only if a test copy each way arrive…
docs: SECOND_GPU.md - the peer tier tests peer access at start, keep …
Merge f-145-fix3: #2622 PLE gather, #2267 config args, #2644 split sh…
fix(HIP compat): cudaMemcpyPeer and cudaDeviceDisablePeerAccess for the
Merge upstream main (477ac13, 0.1.45): the server, setup, docs and up…
Niko1221 commentedon Oct 19, 2026
Thanks for taking the time to report this.
We're resetting the tracker for 0.2.0-beta. There are more than 1,300 open issues and pull requests, too many for us to read properly, so we're closing everything that isn't being worked on right now. Closed here does not mean rejected.
To reopen it takes about two minutes:
/reopen. A check runs and either reopens the issue or lists what is still missing.The rules are in CONTRIBUTING.md.
peer experts: peer access is used only if a test copy each way arrive…