Repository navigation
Misc. bug: E llama_init_from_model: failed to initialize the context: Gemma4Assistant #24343
Description
Name and Version
version: 9110 (ef22b3e)
built with GNU 12.3.0 for Linux x86_64
Operating systems
Linux
Which llama.cpp modules do you know to be affected?
llama-server
Command line
~/projects/llama.cpp/build/bin/llama-server -m ./models/gemma-4-12B-it-qat-UD-Q4_K_XL.gguf --model-draft ./models/gemma-4-12B-it-qat-assistant-MTP-Q8_0.gguf --spec-type draft-mtp --spec-draft-n-max 4 --parallel 1 --ctx-size 131072 --temp 1.0 --top-p 0.95 --top-k 64 --tools all --webui-mcp-proxy --jinja --chat-template-file ~/AI/models/jinja_templates/custom_pub_chat_template_gemma4.jinjaProblem description & steps to reproduce
I get this error when launching llama-server:
E llama_init_from_model: failed to initialize the context: Gemma4Assistant requires ctx_other to be set (this is normal during memory fitting)
Note the error goes away if I add -fit off to the command line above.
First Bad Commit
No response
Relevant log output
Logs
```0.00.123.654 I log_info: verbosity = 3 (adjust with the -lv N CLI arg)
0.00.123.657 I device_info:
0.00.215.039 I - CUDA0 : NVIDIA GeForce RTX 3080 Ti (11912 MiB, 11640 MiB free)
0.00.215.049 I - CPU : 12th Gen Intel(R) Core(TM) i7-12700K (62986 MiB, 62986 MiB free)
0.00.215.124 I system_info: n_threads = 8 (n_threads_batch = 8) / 20 | CUDA : ARCHS = 860 | USE_GRAPHS = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX_VNNI = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.00.215.165 I srv init: running without SSL
0.00.215.188 I srv init: using 19 threads for HTTP server
0.00.215.273 W srv llama_server: -----------------
0.00.215.274 W srv llama_server: CORS proxy is enabled, do not expose server to untrusted environments
0.00.215.274 W srv llama_server: This feature is EXPERIMENTAL and may be removed or changed in future versions
0.00.215.274 W srv llama_server: -----------------
0.00.215.280 W srv llama_server: -----------------
0.00.215.280 W srv llama_server: Built-in tools are enabled, do not expose server to untrusted environments
0.00.215.281 W srv llama_server: This feature is EXPERIMENTAL and may be changed in the future
0.00.215.281 W srv llama_server: -----------------
0.00.215.284 I srv start: binding port with default address family
0.00.216.449 I srv llama_server: loading model
0.00.216.464 I srv load_model: loading model './models/gemma-4-12B-it-qat-UD-Q4_K_XL.gguf'
0.00.613.937 E llama_init_from_model: failed to initialize the context: Gemma4Assistant requires ctx_other to be set (this is normal during memory fitting)
0.00.657.037 W srv load_model: [spec] failed to measure draft model memory: failed to create llama_context from model
0.00.657.049 I common_init_result: fitting params to device memory ...
0.00.657.049 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.01.369.923 W load: control-looking token: 212 '' was not control-type; this is probably a bug in the model. its type will be overridden
0.01.370.244 W load: control-looking token: 50 '<|tool_response>' was not control-type; this is probably a bug in the model. its type will be overridden
0.01.389.089 W load: special_eog_ids contains '<|tool_response>', removing '' token from EOG list
0.02.349.477 W llama_context: n_ctx_seq (131072) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
</details>
<!-- Long logs that you upload as files go here, outside the "console" area -->
Activity
edt-xx commentedon Jun 10, 2026
I also see this with b9587 using unsloth/gemma-4-31B-it-Q4_0-MTP.gguf, --fit off is not helping in this case.
[46437] 0.00.634.848 E llama_init_from_model: failed to initialize the context: Gemma4Assistant requires ctx_other to be set (this is normal during memory fitting)
goupdate commentedon Jun 10, 2026
C:\ai-llama>llama-server --reasoning off --model C:\ai-koboldcpp\gemma-4-12B-it-qat-UD-Q4_K_XL.gguf --model-draft C:\ai-koboldcpp\mtp-gemma-4-12B-it.gguf -c 262144 -n 131072 --temp 1.0 --top-p 0.95 --top-k 64
←[34m0.00.354.624←[0m ←[32mI ←[0mlog_info: verbosity = 3 (adjust with the
-lv NCLI arg)←[34m0.00.354.642←[0m ←[32mI ←[0mdevice_info:
←[34m0.00.356.149←[0m ←[32mI ←[0m - Vulkan0 : NVIDIA GeForce RTX 4080 SUPER (16261 MiB, 15491 MiB free)
←[34m0.00.356.162←[0m ←[32mI ←[0m - CPU : Intel(R) Xeon(R) CPU E5-2698 v3 @ 2.30GHz (65440 MiB, 46792 MiB free)
←[34m0.00.356.319←[0m ←[32mI ←[0msystem_info: n_threads = 32 (n_threads_batch = 32) / 64 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | ←[34m0.00.356.352←[0m ←[32mI ←[0msrv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true ←[34m0.00.356.536←[0m ←[32mI ←[0msrv init: running without SSL ←[34m0.00.356.671←[0m ←[32mI ←[0msrv init: using 63 threads for HTTP server ←[34m0.00.357.336←[0m ←[32mI ←[0msrv start: binding port with default address family ←[34m0.00.373.286←[0m ←[32mI ←[0msrv llama_server: loading model ←[34m0.00.373.313←[0m ←[32mI ←[0msrv load_model: loading model 'C:\ai-koboldcpp\gemma-4-12B-it-qat-UD-Q4_K_XL.gguf' ←[34m0.02.710.101←[0m ←[31mE llama_init_from_model: failed to initialize the context: Gemma4Assistant requires ctx_other to be set (this is normal during memory fitting) ←[0m←[34m0.02.849.821←[0m ←[35mW srv load_model: [spec] failed to measure draft model memory: failed to create llama_context from model ←[0m←[34m0.02.849.862←[0m ←[32mI ←[0mcommon_init_result: fitting params to device memory ...
Fix Gemma 4 MTP: bump llama-swap to v224-cuda-b9592, pin main GGUF
llama : suppress misleading Gemma4Assistant error during memory fitting
nhtua commentedon Jun 18, 2026
I saw this issue too
rchildre3 commentedon Jun 22, 2026
hitting this now with on llama.cpp version:
52b3df002 (tag: b9754) common/peg : implement ac parser for stricter grammar generation (#24869)built with:
cmake \ -G Ninja \ -B build \ -DCMAKE_BUILD_TYPE=RelWithDebInfo \ -DCMAKE_CXX_FLAGS="-march=native" \ -DCMAKE_EXPORT_COMPILE_COMMANDS=1 \ -DCMAKE_INTERPROCEDURAL_OPTIMIZATION=1 \ -DGGML_VULKAN=1 \ -DGGML_NATIVE=1Hitting a segfault with:
gdb -q --args ./build/bin/llama-server -hf unsloth/gemma-4-E2B-it-qat-GGUF:Q4_0 --threads 3 --spec-type draft-mtp --spec-draft-n-max 2rchildre3 commentedon Jun 23, 2026
Still hitting this on:
be4a6a63e (HEAD -> master, tag: b9775) server : check draft context creation error (#24922)(even with #24922 merged in)
jmorth commentedon Jun 25, 2026
Also seeing this with the latest
llama-serverand Gemma4 31B:khimaros commentedon Jul 2, 2026
seeing this with all gemma4 MTP models from unsloth with vulkan
emk commentedon Jul 4, 2026
Confirmed with CUDA on
b9871-ef2d77011.But the 26B A4B works fine, with just the first non-fatal warning:
Snegovik commentedon Jul 10, 2026
Тоже имею эту проблему с Gemma-4-26B-A4B-qat. requires ctx_other to be set
AlvaroBalbin commentedon Jul 11, 2026
The error is coming from the memory-fitting probe, not from real context init. In common/fit.cpp the fit routine loads the draft model and calls llama_init_from_model with cparams.ctx_other == nullptr just to measure its footprint, and for this arch the LLM_ARCH_GEMMA4_ASSISTANT branch in src/llama-context.cpp (around line 119) deliberately throws "requires ctx_other to be set", because the target context only gets wired up later in common_speculative_init_result. That is why -fit off makes it go away: it skips the probe. So the E line is expected and non-fatal, and there's already a TODO right above the throw to turn it into a non-printing exception so it stops showing up as an error. Can you confirm whether the server actually fails to start, or does it log this line and then continue serving normally? If it continues, this is just misleading log noise rather than a real init failure.
pamaral-git commentedon Jul 20, 2026
@AlvaroBalbin at least for me the server still runs and I'm able to continue almost always
controlaltnerd commentedon Aug 13, 2026
I can confirm this appears to be a real init failure with Meta Muse Glimmer's draft assistant model, based on the logs I get in LM Studio:
I brought it up with the LM Studio devs directly before realizing it was probably happening with llama.ccp itself.
github-actions commentedon Sep 27, 2026
This issue was closed because it has been inactive for 14 days since being marked as stale.