Skip to content

RuntimeError("the engine exited before it was ready" + (f" (see {log})" if log else "")) #496

Closed
Closed
RuntimeError("the engine exited before it was ready" + (f" (see {log})" if log else ""))#496

Description

@AlexBekhtin
AlexBekhtin
opened on Oct 2, 2026

With Coder model result is same.
Windows 11

Strata - Qwen3.8-Flash-Next on a normal PC (a GPU + system RAM + CPU)
  Found your earlier install in D:\STRATA\Strata-main_old (q2_0): setting up this copy the same way - the model files are reused, nothing big is downloaded.

=== Step 1: checking your PC ===
  [ok] GPU: NVIDIA GeForce RTX 3060 Laptop GPU, 6.0 GB VRAM, compute capability 8.6, driver 617.14
  [!]  less than 12 GB of VRAM: Strata will run, but most experts stay on the CPU and it will be slow
  [ok] RAM: 64 GB
  [ok] CPU: Intel(R) Core(TM) i5-10500H CPU @ 2.50GHz (AVX2)

=== Step 2: your choices ===
  [ok] model: Qwen3.8-Flash-Next

  1) Q2_0     2-bit, the fastest; download 66 GB, uses ~34 GB of RAM
  2) IQ2_XS   2-bit i-quant, a little better quality, close in speed; download 68 GB, uses ~36 GB of RAM
  3) IQ3_XXS  3-bit i-quant, better quality, slower (more CPU work per token); download 76 GB, uses ~43 GB of RAM
  4) IQ3_S    3.5-bit i-quant, the best quality (matches the full model), the slowest; needs a 64 GB PC with little else running; download 84 GB, uses ~50 GB of RAM
  [ok] size: Q2_0
  [ok] context: 32768 tokens
  [ok] KV cache: 8-bit
  [ok] images: off
  [ok] experimental speed projection: off

=== Step 3: Python packages ===
  Installing numpy, jinja2, regex, pyyaml, tqdm, requests, cmake, ninja, pillow, psutil ...
  > D:\STRATA\Strata\.venv\Scripts\python.exe -m pip install --quiet --disable-pip-version-check numpy==2.5.3; python_version >= "3.12" numpy==2.4.6; python_version == "3.11" numpy==2.2.6; python_version < "3.11" jinja2==3.1.6 regex==2026.9.10 pyyaml==6.0.3 tqdm==4.70.1 requests==2.34.2 cmake==4.4.3 ninja==1.13.2 pillow==12.3.0 psutil==7.2.2 markupsafe==3.0.3 certifi==2026.7.22 charset-normalizer==3.5.1 idna==3.20 urllib3==2.8.0 colorama==0.4.6; sys_platform == "win32"
  [ok] numpy, jinja2, regex, pyyaml, tqdm, requests, cmake, ninja, pillow, psutil installed

=== Step 4: the Strata engine ===
  llama.cpp source:    39.6 MB
  [ok] llama.cpp source downloaded
  [ok] llama.cpp 3cf0325 (gguf-py, ggml, mtmd)
  Downloading the ready-made Strata engine ...
  Strata engine:   0.12 / 0.12 GB (94%)
  [ok] Strata engine downloaded
  [ok] ready-made engine 0.1.35 for sm_75, sm_86, sm_89, sm_120 (CUDA 13.0)
  Installing NVIDIA CUDA libraries (cuBLAS, CUDA runtime; ~0.4 GB) ...
  > D:\STRATA\Strata\.venv\Scripts\python.exe -m pip install --quiet --disable-pip-version-check nvidia-cublas==13.0.2.14 nvidia-cuda-runtime==13.0.96
  [ok] NVIDIA CUDA libraries (cuBLAS, CUDA runtime; ~0.4 GB) installed
  [ok] engine: D:\STRATA\Strata\engine\strata.exe

=== Step 5: downloading Qwen3.8-Flash-Next Q2_0 ===
  [ok] Qwen3.8-Flash-Next-GSQ-RCO-Q2_0-00001-of-00002.gguf already downloaded
  [ok] Qwen3.8-Flash-Next-GSQ-RCO-Q2_0-00002-of-00002.gguf already downloaded
  [ok] model files present

=== Step 6: preparing the model for Strata ===
  [ok] model prepared: D:\STRATA\Strata-data\packs\q2_0
  [ok] MTP draft layer: D:\STRATA\Strata-data\mtp\rt

=== Step 7: writing the start script ===
  [ok] start script: run-q2_0.bat

All set.
  API (OpenAI):     http://127.0.0.1:8080/v1   (any API key; model name: anything)
  API (Anthropic):  http://127.0.0.1:8080/v1/messages
  Next time:        just run START-HERE.bat (or run-q2_0.bat) - it starts right away
  [ok] GPU: GPU 0 (NVIDIA GeForce RTX 3060 Laptop GPU, 6 GB)

  ----------------------------------------------------------------------------------------------------
  Starting qwen3.8-flash-next-q2_0: it loads about 38 GB into RAM and locks part of it for the GPU.
  While it does, YOUR PC CAN BE SLOW OR STOP RESPONDING FOR 1-3 MINUTES (longer the first time after a
  restart). That is normal: please wait and don't close this window - the browser opens when it is ready.
  Later, closing this window stops the model.
  ----------------------------------------------------------------------------------------------------
loading the model (the first start takes a minute or two) ...
[strata] starting the engine: reading the model's weights ...
[strata] loading the experts into RAM (about 38 GB) and locking part of them for the GPU.
         YOUR PC CAN BE SLOW OR STOP RESPONDING FOR 1-3 MINUTES NOW - this is normal.
         Please wait and don't close this window; the browser opens when it is ready.
[strata] experts loaded: 31.64 GiB at 3.17 GiB/s (15 s so far)
[strata] almost ready ...
Traceback (most recent call last):
  File "D:\STRATA\Strata\serve\server.py", line 2619, in <module>
    sys.exit(main())
             ~~~~^^
  File "D:\STRATA\Strata\serve\server.py", line 2496, in main
    engine = StrataEngine(exe, engine_args(cfg), cwd=cfg.get("cwd"), log=cfg.get("log"), env=env, lazy=lazy)
  File "D:\STRATA\Strata\serve\server.py", line 262, in __init__
    raise RuntimeError("the engine exited before it was ready" + (f" (see {log})" if log else ""))
RuntimeError: the engine exited before it was ready (see D:\STRATA\Strata\strata-q2_0.log)

Activity

  1. Niko1221 commented on Oct 2, 2026

    @Niko1221
    Niko1221
    on Oct 2, 2026
    Owner

    Thanks. The console stops right after session is up, which is where a 6 GB card sets up the draft head and the verify buffers. On 0.1.35 the engine's own reason isn't shown there. Could you post the last ~40 lines of strata-q2_0.log from the Strata folder? Meanwhile, please try 0.1.36 (it names the draft-head case), and START-HERE.bat --setup --draft-vocab en or a smaller context. The next release also shows the end of the engine log in that error for every kind of failure.

  2. AlexBekhtin commented on Oct 2, 2026

    @AlexBekhtin
    AlexBekhtin
    on Oct 2, 2026
    Author

    strata-q2_0.log

    strata generate: PCIe probe: 12.1 GB/s host->device -> pcie_frac 0.26 (default 0.55)
strata generate: this CPU has no AVX-512: the expert kernels run on AVX-2 (multi-token for the i-quant gate/up rows)
strata generate: native pack: D:\STRATA\Strata-data\packs\q2_0 experts (largest blob 1.38 MB), token embedding Q3_K in mapped host memory (260 MiB)
strata generate: 1416 MiB of weights loaded from D:\STRATA\Strata-data\packs\q2_0 (303 canonical tensors skipped: served natively)
strata generate: 301 native projection matrices, 1376.20 MiB of weights
strata generate: the SSD is kept awake while rows are read: one page of the table after 100 ms without a read, until 60 s after the last request (STRATA_SSD_KEEPALIVE=0 turns it off)
strata generate: profile D:\STRATA\Strata\data\expert-profile.bin: 24576 ranked pairs, built for 24576 slots
strata generate: PLE on, table 320001536 rows of D:\STRATA\Strata-data\models\Q2_0\Qwen3.8-Flash-Next-GSQ-RCO-Q2_0-00002-of-00002.gguf
strata mtp: draft layer loaded, 839 MiB of VRAM (experts 675, dense 111), files read in 0.56 s (1413 MiB/s)
strata generate: GPU 0: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6
strata generate: expert arena: cudaHostRegister PORTABLE ok; large pages (2097152 B)
strata generate: loaded 31.64 GiB at 3.17 GiB/s
strata generate: experimental native Q5_K head, 437043200 bytes
strata generate: expert cache auto: 0.29 GiB free, 700 MiB reserved (+184 MiB for the draft head) -> 0 slots
strata generate: no VRAM is left for the expert cache: lower --max-context, use --kv k8v4, run images on the CPU, or close other programs that use the GPU
strata generate: pre-filled 0 of 0 slots from the profile; slot 0 verified
strata generate: 5 expert-pool workers + the host thread
strata generate: session is up (engine 0.1.35)
strata serve: the prompt path allocates its own buffers (too few cache slots to borrow)
strata hc: CUDA0: the hyper-connection read runs as staged (the norm per token and stream, the down projection's activations staged ahead by cp.async); checked bit for bit against the plain read on this card (STRATA_HC_SPLIT=1 or 0 for the earlier ones)
strata serve: verify: needs the profile-filled VRAM expert tier (--expert-profile and --expert-cache); with --expert-cache auto, no VRAM was left for it: lower --max-context, use --kv k8v4 or images on the CPU

  3. AlexBekhtin commented on Oct 2, 2026

    @AlexBekhtin
    AlexBekhtin
    on Oct 2, 2026
    Author

    Reducing the context didn’t help.
    I updated to 0.1.36 using git pull.

    D:\STRATA\Strata>START-HERE.bat --setup --draft-vocab en
Strata - Qwen3.8-Flash-Next on a normal PC (a GPU + system RAM + CPU)

=== Step 1: checking your PC ===
  [ok] GPU: NVIDIA GeForce RTX 3060 Laptop GPU, 6.0 GB VRAM, compute capability 8.6, driver 617.14
  [!]  less than 12 GB of VRAM: Strata will run, but most experts stay on the CPU and it will be slow
  [ok] RAM: 64 GB
  [ok] CPU: Intel(R) Core(TM) i5-10500H CPU @ 2.50GHz (AVX2)

=== Step 2: your choices ===
  1) Qwen3.8-Flash-Next   Qwen; GSQ-RCO quants by ISTA-DASLab - the original model
  2) Swift 1.5            UkisAI's fine-tune of Qwen3.8-Flash-Next - thinks much shorter (-63% thinking tokens, 1.8x sooner answers by its authors' numbers)
  3) Qwen3.8-Flash-Next Coder ISTA-DASLab's coding version - half the experts (code, tools, images kept): needs ~32 GB of RAM, faster; weaker outside coding
  4) Qwen3.8-Flash-Next (Unsloth) Unsloth's 4-bit quantization (EXPERIMENTAL) - 4-bit, 111 GB download, most experts read from the SSD: slow (7-8.5 tokens/s on a 64 GB PC)   [experimental]
Which model? [1]: 1
  [ok] model: Qwen3.8-Flash-Next

  1) Q2_0     2-bit, the fastest; download 66 GB, uses ~34 GB of RAM
  2) IQ2_XS   2-bit i-quant, a little better quality, close in speed; download 68 GB, uses ~36 GB of RAM
  3) IQ3_XXS  3-bit i-quant, better quality, slower (more CPU work per token); download 76 GB, uses ~43 GB of RAM
  4) IQ3_S    3.5-bit i-quant, the best quality (matches the full model), the slowest; needs a 64 GB PC with little else running; download 84 GB, uses ~50 GB of RAM
Which size? [3]: 1
  [ok] size: Q2_0

  Context length = how much text the model can see at once (your chat, files, tool output).
  Longer needs more VRAM for it, so fewer experts fit on the GPU:
  1) 8K tokens
  2) 32K tokens   (recommended for your GPU)
  3) 64K tokens
  4) 128K tokens
  5) 256K tokens
  6) 384K tokens   (experimental: setup adds rope scaling)
  7) 512K tokens   (experimental: setup adds rope scaling)
Context? [2]: 1
  [ok] context: 8192 tokens

  Images: the model can also read pictures (screenshots, photos, scanned pages). This adds a 0.9 GB
  download and keeps ~1.4 GB of VRAM free for the image encoder, so text is a few % slower.
Do you want images? [n]:
  [ok] images: off

  EXPERIMENTAL - speed projection: a small control vector applied while the model runs (layers 4-44).
  It changes how the model answers: its package describes it as a refusal-direction projection (the
  model declines far fewer requests). Off unless you choose it; when on, the web app can switch it off
  per chat. Details: data/experimental-speed-projection/README.md
Turn on the experimental speed projection? [n]:
  [ok] experimental speed projection: off

=== Step 3: Python packages ===
  [ok] numpy, jinja2, regex, pyyaml, tqdm, requests, cmake, ninja, pillow, psutil already installed

=== Step 4: the Strata engine ===
  [ok] llama.cpp 3cf0325 (gguf-py, ggml, mtmd)
  [ok] ready-made engine already installed
  [ok] NVIDIA CUDA libraries (cuBLAS, CUDA runtime; ~0.4 GB) already installed
  [ok] engine: D:\STRATA\Strata\engine\strata.exe

=== Step 5: downloading Qwen3.8-Flash-Next Q2_0 ===
  [ok] Qwen3.8-Flash-Next-GSQ-RCO-Q2_0-00001-of-00002.gguf already downloaded
  [ok] Qwen3.8-Flash-Next-GSQ-RCO-Q2_0-00002-of-00002.gguf already downloaded
  [ok] model files present

=== Step 6: preparing the model for Strata ===
  [ok] model prepared: D:\STRATA\Strata-data\packs\q2_0
  [ok] MTP draft layer: D:\STRATA\Strata-data\mtp\rt

=== Step 7: writing the start script ===
Tune Strata for this PC now? It measures a few engine settings (about 5-10 minutes; the PC is busy meanwhile; later: START-HERE --calibrate) [y]:

  Tuning Strata for this PC: the output speed is measured with a few engine settings (the PCIe share, the
  draft depth, the CPU threads). It takes about 5-10 minutes; the PC is busy meanwhile.
  Loading the model for the measurements ...
[strata] starting the engine: reading the model's weights ...
[strata] loading the experts into RAM (about 38 GB) and locking part of them for the GPU.
         YOUR PC CAN BE SLOW OR STOP RESPONDING FOR 1-3 MINUTES NOW - this is normal.
         Please wait and don't close this window; the browser opens when it is ready.
[strata] experts loaded: 31.64 GiB at 3.12 GiB/s (16 s so far)
[strata] almost ready ...
  [!]  the tuning did not finish (the engine exited before it was ready (see D:\STRATA\Strata\strata-q2_0.log)): the default settings stay
       the engine said: strata serve: verify: needs the profile-filled VRAM expert tier (--expert-profile and --expert-cache); with --expert-cache auto, no VRAM was left for it: lower --max-context, use --kv k8v4 or images on the CPU
  [ok] start script: run-q2_0.bat

All set.
  API (OpenAI):     http://127.0.0.1:8080/v1   (any API key; model name: anything)
  API (Anthropic):  http://127.0.0.1:8080/v1/messages
  Next time:        just run START-HERE.bat (or run-q2_0.bat) - it starts right away
  Tuning:           FAILED (the reason is above): the default settings stay - START-HERE.bat --calibrate tries again
  [ok] GPU: GPU 0 (NVIDIA GeForce RTX 3060 Laptop GPU, 6 GB)

  ----------------------------------------------------------------------------------------------------
  Starting qwen3.8-flash-next-q2_0: it loads about 38 GB into RAM and locks part of it for the GPU.
  While it does, YOUR PC CAN BE SLOW OR STOP RESPONDING FOR 1-3 MINUTES (longer the first time after a
  restart). That is normal: please wait and don't close this window - the browser opens when it is ready.
  Later, closing this window stops the model.
  ----------------------------------------------------------------------------------------------------
loading the model (the first start takes a minute or two) ...
[strata] starting the engine: reading the model's weights ...
[strata] loading the experts into RAM (about 38 GB) and locking part of them for the GPU.
         YOUR PC CAN BE SLOW OR STOP RESPONDING FOR 1-3 MINUTES NOW - this is normal.
         Please wait and don't close this window; the browser opens when it is ready.
[strata] experts loaded: 31.64 GiB at 3.13 GiB/s (15 s so far)
[strata] almost ready ...
Traceback (most recent call last):
  File "D:\STRATA\Strata\serve\server.py", line 2703, in <module>
    sys.exit(main())
             ~~~~^^
  File "D:\STRATA\Strata\serve\server.py", line 2580, in main
    engine = StrataEngine(exe, engine_args(cfg), cwd=cfg.get("cwd"), log=cfg.get("log"), env=env, lazy=lazy)
  File "D:\STRATA\Strata\serve\server.py", line 291, in __init__
    raise RuntimeError("the engine exited before it was ready" + (f" (see {log})" if log else "") +
                       start_failure_hint(log, log_start))
RuntimeError: the engine exited before it was ready (see D:\STRATA\Strata\strata-q2_0.log)
  4. Niko1221
    added a commit that references this issue on Oct 2, 2026

    serve: the engine-exited-before-READY error shows this start's last 2…

    f7df4ef
  5. Niko1221
    closed this as completedin db4f91aon Oct 2, 2026

  6. Niko1221 commented on Oct 2, 2026

    @Niko1221
    Niko1221
    on Oct 2, 2026
    Owner

    0.1.37 shows the end of the engine's log in that error for every kind of failure, so updating and starting again will show the reason directly. Please post it here.

  7. Niko1221 commented on Oct 2, 2026

    @Niko1221
    Niko1221
    on Oct 2, 2026
    Owner

    Thanks for the log, it shows the cause. It isn't the context: on a 6 GB laptop RTX 3060, after the weights (1.4 GB), the draft layer (839 MiB) and the reserve (700 MiB + the draft head), only 0.29 GiB of VRAM is left, so the expert cache gets 0 slots. The verify step needs at least some of them, so the engine stops.

    To get it running now, edit strata-q2_0.json in the Strata folder, in "args":

    1. Lower the reserve: change the value after "--vram-reserve-mib" to 300 (or add "--vram-reserve-mib", "300"). That frees ~400 MB.
    2. If the log still says 0 slots: remove "--mtp", "<path>" and the "--spec", "4" / "--spec-min-p", ... pairs. Turning off the draft layer frees ~1 GB of VRAM (answers come out slower per step, but it runs).
    3. Keep images off (--vision no in setup) on a 6 GB card.

    It will still be slow on 6 GB, since most experts run on the CPU (setup warns about it). On our side this should not need hand edits: setup should pick these settings for a card under 8 GB, and the engine should start with a tiny cache instead of stopping. Reopening this to track that for the next release.

  8. Niko1221
    reopened this on Oct 2, 2026
  9. BlueKingMuch
    added a commit that references this issue on Oct 2, 2026

    Engine 0.1.37: a silent engine is restarted (Niko1221#481), Windows A…

    db4f91a

  10. AlexBekhtin commented on Oct 2, 2026

    @AlexBekhtin
    AlexBekhtin
    on Oct 2, 2026
    Author

    The advice regarding the “--vram-reserve-mib”, “300” limit turned out to be useful. Now everything works. I’ll also keep the other tips in mind. Thank you very much!

  11. AlexBekhtin commented on Oct 2, 2026

    @AlexBekhtin
    AlexBekhtin
    on Oct 2, 2026
    Author

    If I use "--vram-reserve-mib", "300" and a context of 8K, the model starts, but doesn’t complete the task.
    I tried removing the parameters "--mtp", "" separately and together with the "--spec" pair; it doesn’t even start for a 8K context, with the following error:

    D:\STRATA\Strata>run-q2_0.bat
loading the model (the first start takes a minute or two) ...
[strata] starting the engine: reading the model's weights ...
Traceback (most recent call last):
  File "D:\STRATA\Strata\serve\server.py", line 2703, in <module>
    sys.exit(main())
             ~~~~^^
  File "D:\STRATA\Strata\serve\server.py", line 2580, in main
    engine = StrataEngine(exe, engine_args(cfg), cwd=cfg.get("cwd"), log=cfg.get("log"), env=env, lazy=lazy)
  File "D:\STRATA\Strata\serve\server.py", line 291, in __init__
    raise RuntimeError("the engine exited before it was ready" + (f" (see {log})" if log else "") +
                       start_failure_hint(log, log_start))
RuntimeError: the engine exited before it was ready (see D:\STRATA\Strata\strata-q2_0.log)

  12. XeonG commented on Oct 2, 2026

    @XeonG
    XeonG
    on Oct 2, 2026

    I didn't see this duplicate but my own thread #517

    same problem, I didn't have this issue days ago not sure what build version last worked for me now or what changed since to effect this, but rolling back should be easier given the amount of changes coming in almost daily to this project.

  13. Niko1221
    added 3 commits that reference this issue on Oct 3, 2026

    A 6 GB card starts with a small expert cache, or says what is short (#…

    e1ee248

    #496: the engine chooses a small card's reserve; setup only gives the…

    4731a9b

    Engine 0.1.38: faster prompts (#372 #374 #413 #452), Q5_0 experts on …

    5123c97

  14. 13 remaining items

  15. anon761
    added 6 commits that reference this issue on Oct 6, 2026

    Engine 0.1.38: faster prompts (Niko1221#372 Niko1221#374 Niko1221#413 N…

    209de46

    serve: the engine-exited-before-READY error shows this start's last 2…

    f2628fd

    Engine 0.1.37: a silent engine is restarted (Niko1221#481), Windows A…

    f72a351

    A 6 GB card starts with a small expert cache, or says what is short (N…

    2bf05e0

    Niko1221#496: the engine chooses a small card's reserve; setup only g…

    3dbdedb

    Engine 0.1.38: faster prompts (Niko1221#372 Niko1221#374 Niko1221#413 N…

    38bdc08
  16. k0valik
    added 2 commits that reference this issue on Oct 15, 2026

    replay(githubproxy.fjygbaifeng.eu.org-eelgaev-Strata-AC922-ac922): src/core from ac922

    1c72985

    replay(githubproxy.fjygbaifeng.eu.org-eelgaev-Strata-AC922-ac922): src/core from ac922

    9075a5d
  17. j-luwierski
    added a commit that references this issue on Oct 17, 2026

    prefill: the advisory VRAM plan - the startup arithmetic says what it…

    703592a
  18. corbinjurgens
    added a commit that references this issue on Oct 19, 2026

    feat: opt-in shared prefill stage under Windows CUDA VRAM pressure

    1357a2d
  19. DarthCY-K
    added a commit that references this issue on Oct 20, 2026

    low-RAM mode: the auto expert cache keeps 1500 MiB of VRAM free under…

    78279d3
  20. gopinath87607
    added 2 commits that reference this issue on Oct 20, 2026

    prefill: STRATA_PREFILL_KEEP (128) is the slots a loan leaves the cache

    badfccd

    prefill: STRATA_PREFILL_KEEP (128) is the slots a loan leaves the cache

    9231c50
  21. TheSmokeDev
    added a commit that references this issue on Oct 28, 2026

    DeepSeek engine: Project Maya's one-GPU decode path (the PCIe lane, i…

    5dc9982
  22. Niko1221
    added a commit that references this issue on Oct 29, 2026

    Qwen: the automatic expert cache leaves the reserve it was asked to k…

    cfa197e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions