Repository navigation
llama : move no_alloc out of llama_hparams - #31141
Merged
Merged
Conversation
Assisted-by: dsh:llama.cpp/ggml-org/GLM-5.3-Flash-GGUF (2x DGX Spark (RDMA, TP=2))
Assisted-by: dsh:llama.cpp/ggml-org/GLM-5.3-Flash-GGUF (2x DGX Spark (RDMA, TP=2))
Assisted-by: dsh:llama.cpp/ggml-org/GLM-5.3-Flash-GGUF (2x DGX Spark (RDMA, TP=2))
CISC
approved these changes
Oct 26, 2026
S95F
added a commit
to S95F/llama.cpp
that referenced
this pull request
Oct 26, 2026
Upstream moved no_alloc off llama_hparams in ggml-org#31141 and threaded it through the llama_moe_cache constructor instead. The merge updated the signature and the callers, but the host expert pinning added on this branch still read model.hparams.no_alloc, so the file stopped compiling as soon as it was touched. The argument is already in scope right there and the buffer allocation a few lines above uses it, so use it. Assisted-by: Claude Opus 5
This was referenced Oct 27, 2026
auRose94
pushed a commit
to auRose94/llama.cpp
that referenced
this pull request
Oct 28, 2026
* llama : move no_alloc out of llama_hparams Assisted-by: dsh:llama.cpp/ggml-org/GLM-5.3-Flash-GGUF (2x DGX Spark (RDMA, TP=2)) * cont : clean-up * cont : clean-up Assisted-by: dsh:llama.cpp/ggml-org/GLM-5.3-Flash-GGUF (2x DGX Spark (RDMA, TP=2)) * tests : add a no_alloc case to test-llama-context Assisted-by: dsh:llama.cpp/ggml-org/GLM-5.3-Flash-GGUF (2x DGX Spark (RDMA, TP=2))
HEchternacht
added a commit
to HEchternacht/llama.cpp
that referenced
this pull request
Oct 28, 2026
…, realtime Conflicts resolved by hand: - ggml-cpu.c: keep fork's i02 < 0 skip (MoE hcache uses negative ids) - vecdotq.cuh: keep fork's PrismML 128-block Q1_0/Q2_0 vec_dot - server.cpp: take upstream entrypoint refactor, re-add fork /grammar routes - tests/CMakeLists.txt: take upstream test layout, keep test-kv-mean-center - llama-context.cpp: hcache gate uses model.no_alloc() (ggml-org#31141)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
no_allocis a load/context option, not a model hyperparameter. Removellama_hparams::no_allocand pass it explicitly instead:bool no_allocparameter right after the model reference; wrappers forward it to their sub-cachesllama_memory_paramsgains ano_allocfield socreate_memorythreads it without reading it from the modelllama_memory_bufs(llama-memory.h/.cpp): owns the {ggml context, backend buffer} pairs of a memory instance and centralizes what every leaf previously duplicated - buffer allocation, theno_allocdummy-buffer path (size-0 buffer with the tensors attached so the scheduler won't allocate them), buffer clear, andmemory_breakdown()(real buffer sizes, or simulated sizes viaggml_backend_alloc_ctx_tensors_from_buft_sizewhenno_alloc). Used byllama_kv_cache,llama_memory_recurrentandllama_dsv4_comp_statellama_moe_cachetakesno_allocas a parameter and reusesllama_memory_bufs::alloc_buffor its own buffer layoutllama_memory_get_buffers()returns an empty vector for a null memory (memory-less archs)Resolves the TODO added in #31087.
Additional information
test-llama-archs: 538 passedtest-llama-contextover all 135 dummy models generated bytest-llama-archs -o: 135/135 passedno_alloc=truesweep loading every dummy model withLLAMA_LOAD_MODE_NONE: all report a non-zero simulated memory breakdown and only null-base buffersRequirements
dsh:llama.cpp