Repository navigation
not stable start with 3+ cards #841
Description
hello! about 10-15 starts and only 2 sucsees. Getting error out of memory for 1 2 or all cards.
try: fixex expert cashe, placing cache at 3rd card, spliting layers like 16,32.
able to start only with 2 cards option w/o vision and not stable.
3060-12gb X3
json file auto created with 2 cars choose have strata-swift-iq2_xs.json "gpu": [ 0, 1, 2 ]
re running setup causing sometimes auto filling options like 1 1 2 2 1 e.t.c without keys press
[strata] experts loaded: 33.02 GiB at 0.07 GiB/s (579 s so far)
[strata] filling the GPU's expert cache (4884 experts, 6.33 GiB of VRAM) ...
[strata] still starting (599 s) - please wait ...
Traceback (most recent call last):
File "F:!strada\serve\server.py", line 4145, in
sys.exit(main())
~~~~^^
File "F:!strada\serve\server.py", line 4001, in main
engine = StrataEngine(exe, engine_args(cfg) + (effort_end or []), cwd=cfg.get("cwd"), log=cfg.get("log"),
env=env, lazy=lazy)
File "F:!strada\serve\server.py", line 478, in init
raise RuntimeError("the engine exited before it was ready" + (f" (see {log})" if log else "") +
start_failure_hint(log, log_start) + start_log_tail(log, log_start))
RuntimeError: the engine exited before it was ready (see F:!strada\strata-swift-iq2_xs.log)
the engine log's last lines:
strata generate: PLE on, table 320001536 rows of F:!strada\Strata-data\models\swift-IQ2_XS\Swift-Qwen3.8-Flash-Next-GSQ-RCO-IQ2_XS-00001-of-00002.gguf
strata generate: layer split: CUDA1 holds its weights, session [16, 33); 7.79 GiB free
strata generate: layer split: CUDA2 holds its weights, session [33, 48) and the head; 7.47 GiB free
strata mtp: draft layer loaded, 839 MiB of VRAM (experts 675, dense 111), files read in 9.59 s (82 MiB/s)
strata generate: GPU 0: NVIDIA GeForce RTX 3060, compute capability 8.6
strata generate: multi-GPU under WDDM: at most 8 GiB of the expert arena is pinned (STRATA_ARENA_PIN_GIB changes it)
strata generate: expert arena read unbuffered (0 of 32 probe reads from the file cache; 16.8 GiB available, 63.5 GiB of files)
strata generate: expert arena: locked 25626 MiB via working-set minimum + VirtualLock; cudaHostRegister limited to 8 GiB by the engine (multi-GPU under WDDM, or remote experts); 12 slices pinned (7 GiB); large pages refused for 35456548864 B (GetLargePageMinimum=2097152, VirtualAlloc error 1314); using 4 KB pages
strata generate: loaded 33.02 GiB at 0.07 GiB/s
strata generate: hint: ~24x below what this hardware streams from a normal launch. If Strata is started by Task Scheduler or a service, register the task with Priority 4 (Normal) and 'Run with highest privileges' - the scheduler's defaults (Below normal + a least-privilege token) throttle the load. See docs/DETAILS.md ('Running it at startup').
strata generate: expert cache 4884 slots, 6.33 GiB of VRAM; policy is
strata generate: the GPU computes the experts in the cache; it rounds differently from the CPU,
so a reply can differ slightly from a run without the cache (same quality:
bench/results/2026-09-27-cache-parity).
PROFILE, ranked by routing frequency, no eviction.
strata generate: pre-filled 4884 of 4884 slots from the profile; slot 0 verified
strata generate: layer split, CUDA1: 7.79 GiB free of 12.00, room for experts 7.01 GiB
strata generate: layer split: CUDA1 runs layers 16-32, expert cache 5198 slots (7.01 GiB), 5198 of its 8704 profiled pairs; slot 0 verified
strata generate: layer split, CUDA2: 6.65 GiB free of 12.00, room for experts 5.87 GiB
strata generate: layer split, CUDA2 expert cache: ExpertCache: cudaMalloc(5.87 GiB) failed: out of memory
Activity
BGonnermann commentedon Oct 4, 2026
The failure occurs when CUDA2 allocates its 5.87 GiB expert cache, despite reporting 6.65 GiB free. Windows commit exhaustion is one possibility; Strata's troubleshooting guide recommends a System managed page file for this symptom. Enable it, reboot, and retry before changing cache settings.
Your JSON also enables all three GPUs. For a clean two-card test, regenerate explicitly with:
START-HERE.bat --setup --gpus 0,1 --vision noConfirm the resulting config contains
"gpu": [0,1].The 0.07 GiB/s loading speed needs separate investigation. Test a direct terminal launch. If using Task Scheduler, apply the Normal priority and highest-privileges settings named in the log.
If allocation still fails, include total RAM and Task Manager's Committed used/limit figures alongside the engine log.
yotadrivers-blip commentedon Oct 5, 2026
swap was system. also try to 20gb swap (60g ram)
speed is not issue tested with ide 8 tokens at start and 35 at and of promt after context was used got error
after this i got this at 3 card config (below) and verify: instantiate: out of memory at 2 cards config.
RuntimeError: the engine exited before it was ready (see F:!strada\strata-swift-iq2_xs.log)
the engine log's last lines:
strata generate: expert cache 5477 slots, 7.10 GiB of VRAM; policy is
strata generate: the GPU computes the experts in the cache; it rounds differently from the CPU,
so a reply can differ slightly from a run without the cache (same quality:
bench/results/2026-09-27-cache-parity).
PROFILE, ranked by routing frequency, no eviction.
strata generate: pre-filled 5477 of 5477 slots from the profile; slot 0 verified
strata generate: layer split, CUDA1: 7.79 GiB free of 12.00, room for experts 7.01 GiB
strata generate: layer split: CUDA1 runs layers 16-32, expert cache 5198 slots (7.01 GiB), 5198 of its 8704 profiled pairs; slot 0 verified
strata generate: layer split, CUDA2: 6.65 GiB free of 12.00, room for experts 5.87 GiB
strata generate: layer split: CUDA2 runs layers 33-47, expert cache 4239 slots (5.87 GiB), 4239 of its 7680 profiled pairs; slot 0 verified
strata generate: layer split: CUDA0 runs layers 0-15
strata generate: R4 hit path ON - resident experts are computed on the GPU
strata generate: 19 expert-pool workers + the host thread
strata generate: session is up (engine 0.1.39)
strata generate: token graph hit path: 14914 resident experts, decided on the device
strata serve: prompt chunk auto: 8192 tokens, a 96-slot ring
strata serve: the prompt path borrows 2907 CUDA0 cache slots (3.77 GiB)
strata serve: CUDA1 prompt path borrows 2800 of its 5198 slots (3.77 GiB)
strata serve: CUDA2 prompt path borrows 2720 of its 4239 slots (3.77 GiB)
yotadrivers-blip commentedon Oct 5, 2026
p.s looks like ouf of memory N2 investigated. chat opened with old erros and good then create a new dialog. but 3 cards still not loading well
setup: quick fixes and OS items (#657 #737 #670 #798 #775 #779 #841 #881
generate: warn when the profile caps an explicit --expert-cache; retr…
Niko1221 commentedon Oct 6, 2026
0.1.40: a split stage whose expert cache fails to open is retried with 90% of the slots, up to 3 times, each try in the log, and setup no longer takes keys typed while it was busy as answers. If it still fails, please paste the log and Task Manager's Committed used/limit.
setup: quick fixes and OS items (#657 #737 #670 #798 #775 #779 #841 #881
generate: warn when the profile caps an explicit --expert-cache; retr…
yotadrivers-blip commentedon Oct 6, 2026
updated to 1.40 3 cards setup geting:
F:!strada\Strata-data\models\swift-IQ2_XS\Swift-Qwen3.8-Flash-Next-GSQ-RCO-IQ2_XS-00001-of-00002.gguf
strata generate: layer split: CUDA1 holds its weights, session [16, 33); 7.71 GiB free
strata generate: layer split: CUDA2 holds its weights, session [33, 48) and the head; 7.40 GiB free
strata mtp: draft layer loaded, 835 MiB of VRAM (experts 675, dense 111), files read in 9.63 s (82 MiB/s)
strata generate: GPU 0: NVIDIA GeForce RTX 3060, compute capability 8.6
strata generate: multi-GPU under WDDM: at most 8 GiB of the expert arena is pinned (STRATA_ARENA_PIN_GIB changes it)
strata generate: expert arena read unbuffered (0 of 32 probe reads from the file cache; 13.8 GiB available, 63.5 GiB of files)
strata generate: expert arena: locked 25626 MiB via working-set minimum + VirtualLock; cudaHostRegister limited to 8 GiB by the engine (multi-GPU under WDDM, or remote experts); 12 slices pinned (7 GiB); large pages refused for 35456548864 B (GetLargePageMinimum=2097152, VirtualAlloc error 1314); using 4 KB pages
strata generate: loaded 33.02 GiB at 0.07 GiB/s
strata generate: hint: ~24x below what this hardware streams from a normal launch. If Strata is started by Task Scheduler or a service, register the task with Priority 4 (Normal) and 'Run with highest privileges' - the scheduler's defaults (Below normal + a least-privilege token) throttle the load. See docs/DETAILS.md ('Running it at startup').
strata generate: expert cache auto: 7.71 GiB free, 700 MiB reserved (+0 MiB for the draft head) -> 4998 slots
strata generate: only 0 MiB free once the slots are written (reserve 700 MiB); shrinking the expert cache
strata generate: only 630 MiB free once the slots are written (reserve 700 MiB); shrinking the expert cache
strata generate: expert cache 4550 slots, 5.90 GiB of VRAM; policy is
strata generate: the GPU computes the experts in the cache; it rounds differently from the CPU,
so a reply can differ slightly from a run without the cache (same quality:
bench/results/2026-09-27-cache-parity).
PROFILE, ranked by routing frequency, no eviction.
strata generate: pre-filled 4550 of 4550 slots from the profile; slot 0 verified
strata generate: layer split, CUDA1: 7.71 GiB free of 12.00, room for experts 6.93 GiB
strata generate: layer split: CUDA1 runs layers 16-32, expert cache 5145 slots (6.93 GiB), 5145 of its 8704 profiled pairs; slot 0 verified
strata generate: layer split, CUDA2: 6.58 GiB free of 12.00, room for experts 5.80 GiB
strata generate: layer split, CUDA2 expert cache: ExpertCache: cudaMalloc(5.80 GiB) failed: out of memory
[strata] experts loaded: 33.02 GiB at 0.07 GiB/s (567 s so far)
[strata] filling the GPU's expert cache (4550 experts, 5.90 GiB of VRAM) ...
[strata] still starting (589 s) - please wait ...
Traceback (most recent call last):
File "F:!strada\serve\server.py", line 4145, in
sys.exit(main())
~~~~^^
File "F:!strada\serve\server.py", line 4001, in main
engine = StrataEngine(exe, engine_args(cfg) + (effort_end or []), cwd=cfg.get("cwd"), log=cfg.get("log"),
env=env, lazy=lazy)
File "F:!strada\serve\server.py", line 478, in init
raise RuntimeError("the engine exited before it was ready" + (f" (see {log})" if log else "") +
start_failure_hint(log, log_start) + start_log_tail(log, log_start))
RuntimeError: the engine exited before it was ready (see F:!strada\strata-swift-iq2_xs.log)
the engine log's last lines:
strata mtp: draft layer loaded, 835 MiB of VRAM (experts 675, dense 111), files read in 9.63 s (82 MiB/s)
strata generate: GPU 0: NVIDIA GeForce RTX 3060, compute capability 8.6
strata generate: multi-GPU under WDDM: at most 8 GiB of the expert arena is pinned (STRATA_ARENA_PIN_GIB changes it)
strata generate: expert arena read unbuffered (0 of 32 probe reads from the file cache; 13.8 GiB available, 63.5 GiB of files)
strata generate: expert arena: locked 25626 MiB via working-set minimum + VirtualLock; cudaHostRegister limited to 8 GiB by the engine (multi-GPU under WDDM, or remote experts); 12 slices pinned (7 GiB); large pages refused for 35456548864 B (GetLargePageMinimum=2097152, VirtualAlloc error 1314); using 4 KB pages
strata generate: loaded 33.02 GiB at 0.07 GiB/s
strata generate: hint: ~24x below what this hardware streams from a normal launch. If Strata is started by Task Scheduler or a service, register the task with Priority 4 (Normal) and 'Run with highest privileges' - the scheduler's defaults (Below normal + a least-privilege token) throttle the load. See docs/DETAILS.md ('Running it at startup').
strata generate: expert cache auto: 7.71 GiB free, 700 MiB reserved (+0 MiB for the draft head) -> 4998 slots
strata generate: only 0 MiB free once the slots are written (reserve 700 MiB); shrinking the expert cache
strata generate: only 630 MiB free once the slots are written (reserve 700 MiB); shrinking the expert cache
strata generate: expert cache 4550 slots, 5.90 GiB of VRAM; policy is
strata generate: the GPU computes the experts in the cache; it rounds differently from the CPU,
so a reply can differ slightly from a run without the cache (same quality:
bench/results/2026-09-27-cache-parity).
PROFILE, ranked by routing frequency, no eviction.
strata generate: pre-filled 4550 of 4550 slots from the profile; slot 0 verified
strata generate: layer split, CUDA1: 7.71 GiB free of 12.00, room for experts 6.93 GiB
strata generate: layer split: CUDA1 runs layers 16-32, expert cache 5145 slots (6.93 GiB), 5145 of its 8704 profiled pairs; slot 0 verified
strata generate: layer split, CUDA2: 6.58 GiB free of 12.00, room for experts 5.80 GiB
strata generate: layer split, CUDA2 expert cache: ExpertCache: cudaMalloc(5.80 GiB) failed: out of memory
yotadrivers-blip commentedon Oct 6, 2026
p.s also found update.bat showing 1.39 verstion. re downloaded executables but still showing try to download full package next
yotadrivers-blip commentedon Oct 6, 2026
re downloaded, re-compiled, re-run and same error sys.exit(main())
~~~~^^
File "F:!strada\serve\server.py", line 4854, in main
engine = StrataEngine(exe, engine_args(cfg) + (effort_end or []), cwd=cfg.get("cwd"), log=cfg.get("log"),
env=env, lazy=lazy)
File "F:!strada\serve\server.py", line 631, in init
raise RuntimeError("the engine exited before it was ready" + (f" (see {log})" if log else "") +
start_failure_hint(log, log_start) + start_log_tail(log, log_start))
RuntimeError: the engine exited before it was ready (see F:!strada\strata-swift-iq2_xs.log)
the engine log's last lines:
strata generate: pre-filled 4536 of 4536 slots from the profile in 8.4 s (756 MB/s); slot 0 verified
strata generate: layer split, CUDA1: 7.71 GiB free of 12.00, room for experts 6.93 GiB
strata generate: layer split: CUDA1 runs layers 16-32, expert cache 5141 slots (6.93 GiB), 5141 of its 8704 profiled pairs; slot 0 verified
strata generate: layer split, CUDA2: 6.57 GiB free of 12.00, room for experts 5.80 GiB
strata generate: layer split, CUDA2 expert cache: ExpertCache: cudaMalloc(5.80 GiB) failed: out of memory; trying 3766 of 4185 slots (try 1 of 3)
strata generate: layer split, CUDA2 expert cache: ExpertCache: cudaMalloc(5.21 GiB) failed: out of memory; trying 3389 of 3766 slots (try 2 of 3)
strata generate: layer split, CUDA2 expert cache: ExpertCache: cudaMalloc(4.69 GiB) failed: out of memory; trying 3050 of 3389 slots (try 3 of 3)
strata generate: layer split: CUDA2 runs layers 33-47, expert cache 3050 slots (4.22 GiB), 3050 of its 7680 profiled pairs; slot 0 verified
strata generate: layer split: CUDA0 runs layers 0-15
strata generate: R4 hit path ON - resident experts are computed on the GPU
strata generate: 19 expert-pool workers + the host thread
strata generate: session is up (engine 0.1.40)
strata generate: token graph hit path: 12727 resident experts, decided on the device
strata serve: WARNING: prompt chunk 7424 tokens, not 8192: CUDA2 (NVIDIA GeForce RTX 3060) has 3050 expert-cache slots, and a 8192-token chunk borrows 2821 of them (it must keep 128, and lend at most 85%) - prompts read slower than on CUDA0 alone (#448)
strata serve: a card this small can serve as a helper expert cache instead of a split stage: without --layer-split, with --expert-cache-device1..3 N, in order (docs/SECOND_GPU.md)
strata serve: prompt chunk auto: 7424 tokens, a 96-slot ring
strata serve: the prompt path borrows 2753 CUDA0 cache slots (3.57 GiB)
strata serve: CUDA1 prompt path borrows 2656 of its 5141 slots (3.57 GiB)
strata serve: CUDA2 prompt path borrows 2586 of its 3050 slots (3.57 GiB)
strata serve: layer split, CUDA1 prompt path: prefill: the layer split's hand-off buffers
Для продолжения нажмите любую клавишу . . .
[strata] 2026-10-06 15:08:33 engine started: F:!strada\engine\strata.exe --serve --pack F:!strada\Strata-data\packs\swift-iq2_xs --native F:!strada\Strata-data\models\swift-IQ2_XS\Swift-Qwen3.8-Flash-Next-GSQ-RCO-IQ2_XS-00001-of-00002.gguf --ple-gguf F:!strada\Strata-data\models\swift-IQ2_XS\Swift-Qwen3.8-Flash-Next-GSQ-RCO-IQ2_XS-00001-of-00002.gguf --expert-profile F:!strada\data\expert-profile.bin --expert-cache auto --prefill auto --spec 4 --mtp F:!strada\Strata-data\mtp\rt --max-context 131072 --kv int8 --kv-resident 32768 --vision --vram-reserve-mib 700 --remote-expert-opt --layer-split auto --spec-min-p 0.5
strata generate: layer split across 3 GPUs: CUDA0, then CUDA1,2 (split auto)
strata generate: EXPERIMENTAL older-CPU build (ggml-cpu for avx): this CPU has no AVX2, so the CPU's experts run on ggml-cpu's kernels - expect it to be slow
strata generate: PCIe probe: 0.8 GB/s host->device (best of 0.8 0.8 0.8 0.8) -> pcie_frac 0.02 (default 0.55)
strata generate: this CPU has no AVX-512: the expert kernels run on ggml-cpu vec_dot (no AVX2: the older-CPU build) (multi-token for the i-quant gate/up rows)
strata generate: native pack: F:!strada\Strata-data\packs\swift-iq2_xs experts (largest blob 1.51 MB), token embedding IQ4_XS in mapped host memory (322 MiB, 2.1 s)
strata generate: 1466 MiB of weights loaded from F:!strada\Strata-data\packs\swift-iq2_xs in 2.5 s (302 canonical tensors skipped: served natively)
strata generate: 300 native projection matrices, 1474.85 MiB of weights, in 13.5 s
strata generate: the SSD is kept awake while rows are read: one page of the table after 100 ms without a read, until 60 s after the last request (STRATA_SSD_KEEPALIVE=0 turns it off)
strata generate: layer split: CUDA1 PCIe probe 0.8 GB/s (best of 0.8 0.8 0.8 0.8) -> pcie_frac 0.02
strata generate: layer split: CUDA1 holds its weights; 7.98 GiB free (its session follows the split search)
strata generate: layer split: CUDA2 PCIe probe 0.8 GB/s (best of 0.8 0.8 0.8 0.8) -> pcie_frac 0.02
strata generate: layer split: CUDA2 holds its weights; 7.98 GiB free (its session follows the split search)
strata generate: profile F:!strada\data\expert-profile.bin: 24576 ranked pairs, built for 24576 slots
strata generate: layer split auto: CUDA0 28 SMs at 1.81 GHz -> 1.43 ms per layer, 7.29 GiB free before its session carve
strata generate: layer split auto: CUDA1 28 SMs at 1.81 GHz -> 1.43 ms per layer, 7.20 GiB free before its session carve
strata generate: layer split auto: CUDA2 28 SMs at 1.81 GHz -> 1.43 ms per layer, 6.22 GiB free before its session carve
strata generate: layer split auto: K=16,33 - predicted 71.5 ms per decode window; the caches hold 14861 of 24576 profiled pairs (~98.6% of the routed mass), best of 1035 placements
strata generate: KV streaming: 32768 of 131072 cells per QSA layer in VRAM, the K/V in 0.52 GiB of pinned RAM
strata generate: PLE on, table 320001536 rows of F:!strada\Strata-data\models\swift-IQ2_XS\Swift-Qwen3.8-Flash-Next-GSQ-RCO-IQ2_XS-00001-of-00002.gguf
strata generate: layer split: CUDA1 holds its weights, session [16, 33); 7.71 GiB free
strata generate: layer split: CUDA2 holds its weights, session [33, 48) and the head; 7.39 GiB free
strata mtp: draft layer loaded, 835 MiB of VRAM (experts 675, dense 111), files read in 1.58 s (496 MiB/s)
strata generate: GPU 0: NVIDIA GeForce RTX 3060, compute capability 8.6
strata generate: multi-GPU under WDDM: at most 8 GiB of the expert arena is pinned (STRATA_ARENA_PIN_GIB changes it)
strata generate: expert arena read unbuffered (0 of 32 probe reads from the file cache; 14.6 GiB available, 63.5 GiB of files)
strata generate: expert arena: locked 25626 MiB via working-set minimum + VirtualLock; cudaHostRegister limited to 8 GiB by the engine (multi-GPU under WDDM, or remote experts); 12 slices pinned (7 GiB); large pages refused for 35456548864 B (GetLargePageMinimum=2097152, VirtualAlloc error 1314); using 4 KB pages
strata generate: loaded 33.02 GiB at 0.06 GiB/s
strata generate: hint: ~24x below what this hardware streams from a normal launch. If Strata is started by Task Scheduler or a service, register the task with Priority 4 (Normal) and 'Run with highest privileges' - the scheduler's defaults (Below normal + a least-privilege token) throttle the load. See docs/DETAILS.md ('Running it at startup').
strata generate: 19 pool workers on logical processors 2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38, the host thread on 0 (draining too) (--host-core first)
strata generate: expert cache auto: 7.71 GiB free, 700 MiB reserved (+0 MiB for the draft head) -> 4994 slots
strata generate: expert cache 5419 slots, 7.02 GiB of VRAM; policy is
strata generate: the GPU computes the experts in the cache; it rounds differently from the CPU,
so a reply can differ slightly from a run without the cache (same quality:
bench/results/2026-09-27-cache-parity).
PROFILE, ranked by routing frequency, no eviction.
strata generate: pre-filled 5419 of 5419 slots from the profile in 9.9 s (761 MB/s); slot 0 verified
strata generate: layer split, CUDA1: 7.71 GiB free of 12.00, room for experts 6.93 GiB
strata generate: layer split: CUDA1 runs layers 16-32, expert cache 5141 slots (6.93 GiB), 5141 of its 8704 profiled pairs; slot 0 verified
strata generate: layer split, CUDA2: 6.57 GiB free of 12.00, room for experts 5.80 GiB
strata generate: layer split: CUDA2 runs layers 33-47, expert cache 4185 slots (5.80 GiB), 4185 of its 7680 profiled pairs; slot 0 verified
strata generate: layer split: CUDA0 runs layers 0-15
strata generate: R4 hit path ON - resident experts are computed on the GPU
strata generate: 19 expert-pool workers + the host thread
strata generate: session is up (engine 0.1.40)
strata generate: token graph hit path: 14745 resident experts, decided on the device
strata serve: prompt chunk auto: 8192 tokens, a 96-slot ring
strata serve: the prompt path borrows 3009 CUDA0 cache slots (3.90 GiB)
strata serve: CUDA1 prompt path borrows 2899 of its 5141 slots (3.90 GiB)
strata serve: CUDA2 prompt path borrows 2814 of its 4185 slots (3.90 GiB)
strata serve: layer split, CUDA1 prompt path: prefill: the layer split's hand-off buffers
load_hparams: Qwen-VL models require at minimum 1024 image tokens to function correctly on grounding tasks
load_hparams: if you encounter problems with accuracy, try adding --image-min-tokens 1024
load_hparams: more info: https://github.com/ggml-org/llama.cpp/issues/16842
strata-vision: model files loaded in 20.0 s
strata-vision: warmed up at 1024 image tokens in 1.4 s
[strata] 2026-10-06 15:19:43 engine started: F:!strada\engine\strata.exe --serve --pack F:!strada\Strata-data\packs\swift-iq2_xs --native F:!strada\Strata-data\models\swift-IQ2_XS\Swift-Qwen3.8-Flash-Next-GSQ-RCO-IQ2_XS-00001-of-00002.gguf --ple-gguf F:!strada\Strata-data\models\swift-IQ2_XS\Swift-Qwen3.8-Flash-Next-GSQ-RCO-IQ2_XS-00001-of-00002.gguf --expert-profile F:!strada\data\expert-profile.bin --expert-cache auto --prefill auto --spec 4 --spec-min-p 0.5 --mtp F:!strada\Strata-data\mtp\rt --max-context 131072 --kv int8 --kv-resident 32768 --vision --vram-reserve-mib 700 --remote-expert-opt --layer-split auto
strata generate: layer split across 3 GPUs: CUDA0, then CUDA1,2 (split auto)
strata generate: EXPERIMENTAL older-CPU build (ggml-cpu for avx): this CPU has no AVX2, so the CPU's experts run on ggml-cpu's kernels - expect it to be slow
strata generate: PCIe probe: 0.8 GB/s host->device (best of 0.8 0.8 0.8 0.8) -> pcie_frac 0.02 (default 0.55)
strata generate: this CPU has no AVX-512: the expert kernels run on ggml-cpu vec_dot (no AVX2: the older-CPU build) (multi-token for the i-quant gate/up rows)
strata generate: native pack: F:!strada\Strata-data\packs\swift-iq2_xs experts (largest blob 1.51 MB), token embedding IQ4_XS in mapped host memory (322 MiB, 2.2 s)
strata generate: 1466 MiB of weights loaded from F:!strada\Strata-data\packs\swift-iq2_xs in 9.8 s (302 canonical tensors skipped: served natively)
strata generate: 300 native projection matrices, 1474.85 MiB of weights, in 14.1 s
strata generate: the SSD is kept awake while rows are read: one page of the table after 100 ms without a read, until 60 s after the last request (STRATA_SSD_KEEPALIVE=0 turns it off)
strata generate: layer split: CUDA1 PCIe probe 0.8 GB/s (best of 0.8 0.8 0.8 0.8) -> pcie_frac 0.02
strata generate: layer split: CUDA1 holds its weights; 7.98 GiB free (its session follows the split search)
strata generate: layer split: CUDA2 PCIe probe 0.8 GB/s (best of 0.8 0.8 0.8 0.8) -> pcie_frac 0.02
strata generate: layer split: CUDA2 holds its weights; 7.98 GiB free (its session follows the split search)
strata generate: profile F:!strada\data\expert-profile.bin: 24576 ranked pairs, built for 24576 slots
strata generate: layer split auto: CUDA0 28 SMs at 1.81 GHz -> 1.43 ms per layer, 7.29 GiB free before its session carve
strata generate: layer split auto: CUDA1 28 SMs at 1.81 GHz -> 1.43 ms per layer, 7.20 GiB free before its session carve
strata generate: layer split auto: CUDA2 28 SMs at 1.81 GHz -> 1.43 ms per layer, 6.22 GiB free before its session carve
strata generate: layer split auto: K=16,33 - predicted 71.5 ms per decode window; the caches hold 14861 of 24576 profiled pairs (~98.6% of the routed mass), best of 1035 placements
strata generate: KV streaming: 32768 of 131072 cells per QSA layer in VRAM, the K/V in 0.52 GiB of pinned RAM
strata generate: PLE on, table 320001536 rows of F:!strada\Strata-data\models\swift-IQ2_XS\Swift-Qwen3.8-Flash-Next-GSQ-RCO-IQ2_XS-00001-of-00002.gguf
strata generate: layer split: CUDA1 holds its weights, session [16, 33); 7.71 GiB free
strata generate: layer split: CUDA2 holds its weights, session [33, 48) and the head; 7.39 GiB free
strata mtp: draft layer loaded, 835 MiB of VRAM (experts 675, dense 111), files read in 7.01 s (112 MiB/s)
strata generate: GPU 0: NVIDIA GeForce RTX 3060, compute capability 8.6
strata generate: multi-GPU under WDDM: at most 8 GiB of the expert arena is pinned (STRATA_ARENA_PIN_GIB changes it)
strata generate: expert arena read unbuffered (0 of 32 probe reads from the file cache; 14.1 GiB available, 63.5 GiB of files)
strata generate: expert arena: locked 25626 MiB via working-set minimum + VirtualLock; cudaHostRegister limited to 8 GiB by the engine (multi-GPU under WDDM, or remote experts); 12 slices pinned (7 GiB); large pages refused for 35456548864 B (GetLargePageMinimum=2097152, VirtualAlloc error 1314); using 4 KB pages
strata generate: loaded 33.02 GiB at 0.06 GiB/s
strata generate: hint: ~24x below what this hardware streams from a normal launch. If Strata is started by Task Scheduler or a service, register the task with Priority 4 (Normal) and 'Run with highest privileges' - the scheduler's defaults (Below normal + a least-privilege token) throttle the load. See docs/DETAILS.md ('Running it at startup').
strata generate: 19 pool workers on logical processors 2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38, the host thread on 0 (draining too) (--host-core first)
strata generate: expert cache auto: 7.71 GiB free, 700 MiB reserved (+0 MiB for the draft head) -> 4994 slots
strata generate: only 0 MiB free once the slots are written (reserve 700 MiB); shrinking the expert cache
strata generate: only 619 MiB free once the slots are written (reserve 700 MiB); shrinking the expert cache
strata generate: expert cache 4536 slots, 5.88 GiB of VRAM; policy is
strata generate: the GPU computes the experts in the cache; it rounds differently from the CPU,
so a reply can differ slightly from a run without the cache (same quality:
bench/results/2026-09-27-cache-parity).
PROFILE, ranked by routing frequency, no eviction.
strata generate: pre-filled 4536 of 4536 slots from the profile in 8.4 s (756 MB/s); slot 0 verified
strata generate: layer split, CUDA1: 7.71 GiB free of 12.00, room for experts 6.93 GiB
strata generate: layer split: CUDA1 runs layers 16-32, expert cache 5141 slots (6.93 GiB), 5141 of its 8704 profiled pairs; slot 0 verified
strata generate: layer split, CUDA2: 6.57 GiB free of 12.00, room for experts 5.80 GiB
strata generate: layer split, CUDA2 expert cache: ExpertCache: cudaMalloc(5.80 GiB) failed: out of memory; trying 3766 of 4185 slots (try 1 of 3)
strata generate: layer split, CUDA2 expert cache: ExpertCache: cudaMalloc(5.21 GiB) failed: out of memory; trying 3389 of 3766 slots (try 2 of 3)
strata generate: layer split, CUDA2 expert cache: ExpertCache: cudaMalloc(4.69 GiB) failed: out of memory; trying 3050 of 3389 slots (try 3 of 3)
strata generate: layer split: CUDA2 runs layers 33-47, expert cache 3050 slots (4.22 GiB), 3050 of its 7680 profiled pairs; slot 0 verified
strata generate: layer split: CUDA0 runs layers 0-15
strata generate: R4 hit path ON - resident experts are computed on the GPU
strata generate: 19 expert-pool workers + the host thread
strata generate: session is up (engine 0.1.40)
strata generate: token graph hit path: 12727 resident experts, decided on the device
strata serve: WARNING: prompt chunk 7424 tokens, not 8192: CUDA2 (NVIDIA GeForce RTX 3060) has 3050 expert-cache slots, and a 8192-token chunk borrows 2821 of them (it must keep 128, and lend at most 85%) - prompts read slower than on CUDA0 alone (#448)
strata serve: a card this small can serve as a helper expert cache instead of a split stage: without --layer-split, with --expert-cache-device1..3 N, in order (docs/SECOND_GPU.md)
strata serve: prompt chunk auto: 7424 tokens, a 96-slot ring
strata serve: the prompt path borrows 2753 CUDA0 cache slots (3.57 GiB)
strata serve: CUDA1 prompt path borrows 2656 of its 5141 slots (3.57 GiB)
strata serve: CUDA2 prompt path borrows 2586 of its 3050 slots (3.57 GiB)
strata serve: layer split, CUDA1 prompt path: prefill: the layer split's hand-off buffers
setup: quick fixes and OS items (Niko1221#657 Niko1221#737 Niko1221#670
generate: warn when the profile caps an explicit --expert-cache; retr…
Niko1221 commentedon Oct 7, 2026
Please retest on 0.1.40, which retries with 90% of the cache and flushes more at setup. If three cards still fail, send the full start log, the Windows commit charge and pagefile size at that moment. Until then, two cards is the working setup.
setup: quick fixes and OS items (Niko1221#657 Niko1221#737 Niko1221#670
generate: warn when the profile caps an explicit --expert-cache; retr…
setup: 先打ちのキーを質問の前に捨て、PLE の表のシャードが長すぎたら止め、手で足した学習済みのプロファイルを残す（upstream …
明示した --expert-cache がプロファイルの長さで頭打ちになるときに警告し、層の分割の段のキャッシュが開けなければ 90% で…
Niko1221 commentedon Oct 13, 2026
Thanks for the report. To keep the tracker current we are closing issues older than three days: 0.1.44 is out with a long list of fixes, and many older reports are covered by it or by 0.1.43.1.
If this still happens on 0.1.44, please leave a comment here with your engine log (and the version line it prints), and we will reopen it. A new issue that links this one works too.
melodictsk commentedon Oct 14, 2026
System: Ryzen 9950x3d, 64 GB RAM, 3070m 16GB (first in system), 4080s 32GB (second in system).
When launching on the 3070m, everything starts without errors. If the 3070m is disabled, launching on the 4080s also proceeds without errors. However, when launching on both graphics cards simultaneously, or on the 4080s alone without disabling the 3070m, an error occurs:
loading the vision encoder ...
loading the model (the first start takes a minute or two) ...
[strata] starting the engine: reading the model's weights ...
[strata] loading the experts into RAM (about 55 GB) and locking part of them for the GPU.
YOUR PC CAN BE SLOW OR STOP RESPONDING FOR 1-3 MINUTES NOW - this is normal.
Please wait and don't close this window; the browser opens when it is ready.
[strata] experts loaded: 46.84 GiB at 4.85 GiB/s (15 s so far)
[strata] filling the GPU's expert cache (5045 experts, 9.60 GiB of VRAM) ...
[strata] almost ready ...
Traceback (most recent call last):
File "C:\Strata-main\serve\server.py", line 6976, in
sys.exit(main())
^^^^^^
File "C:\Strata-main\serve\server.py", line 6753, in main
engine = StrataEngine(exe, engine_args(cfg) + (effort_end or []), cwd=cfg.get("cwd"), log=cfg.get("log"),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Strata-main\serve\server.py", line 889, in init
raise EngineDied(why + (f" (see {log})" if log else "") + note +
EngineDied: the engine exited before it was ready (see C:\Strata-main\strata-iq3_s.log)
the engine process ended: exit code 1
the engine log's last lines:
strata generate: ExpertCache: cudaMalloc(22.76 GiB) failed: out of memory (Windows has 11.1 GiB of commit left: RAM + page file); trying a smaller expert cache: 8999 slots
strata generate: ExpertCache: cudaMalloc(17.06 GiB) failed: out of memory (Windows has 11.1 GiB of commit left: RAM + page file); trying a smaller expert cache: 6743 slots
strata generate: ExpertCache: cudaMalloc(12.80 GiB) failed: out of memory (Windows has 11.1 GiB of commit left: RAM + page file); trying a smaller expert cache: 5045 slots
strata generate: expert cache: 5045 slots (9.60 GiB) after 3 smaller tries - a bigger page file lets it use more of the free VRAM
strata generate: expert cache 5045 slots, 9.60 GiB of VRAM; policy is
strata generate: the GPU computes the experts in the cache; it rounds differently from the CPU,
so a reply can differ slightly from a run without the cache (same quality:
bench/results/2026-09-27-cache-parity).
PROFILE, ranked by routing frequency, no eviction.
strata generate: pre-filled 5045 of 5045 slots from the profile in 0.4 s (23671 MB/s); slot 0 verified
strata generate: R4 hit path ON - resident experts are computed on the GPU
strata generate: 15 expert-pool workers + the host thread
strata generate: session is up (engine 0.1.44.1)
strata generate: token graph hit path: 5045 resident experts, decided on the device
strata serve: Windows commit: 1.6 GiB left of a 82.4 GiB limit (RAM + page file)
strata serve: WARNING the Windows commit limit is 82.4 GiB (RAM + page file) and 1.6 GiB of it is left, but this run plans 2.0 GiB of host allocations and keeps 4.0 GiB free: it can end at the limit with no message (exit code 0xC0000409, #1607) while RAM is still free. Raise the page file (System managed, or RAM-sized) or lower --conversation-cache-mib / --max-context / --prefill
strata: the CPU prefill share is not armed by default: only 1.6 GiB of the Windows commit limit is left (the share needs 0.30 GiB and 4.0 GiB left over). STRATA_PREFILL_CPU_SHARE=auto turns it on anyway
strata serve: prompt chunk auto: 8192 tokens, a 384-slot ring
strata serve: the prompt path borrows 2186 CUDA0 cache slots (4.12 GiB)
strata serve: prefill gemm: cublasCreate: cuBLAS status 1
I suspect this is because the memory allocation is calculated based on the size of the larger VRAM, and it fails to allocate that amount on the graphics card with less VRAM. Somewhere in the code, the graphics card numbering is mixed up.
yotadrivers-blip commentedon Oct 14, 2026
sorry for late reply as for me i had 30gb fixed swap and 50gb ram. Fixed after making about 50gb swap, after that will work with 3 cards/auto with no errors
melodictsk commentedon Oct 14, 2026
increase swap from 30gb to 50 gb and its now working.
LOCAL: size the expert cache before pipeline-windows' second verifier