Repository navigation
Eval bug: VRAM usage increased from b10680 (SYCL backend) #29063
Description
Name and Version
Docker based llama.cpp (ghcr.io/ggml-org/llama.cpp:server-intel) :
Build 11011 (aa39d7a) with IntelLLVM 2025.3.3 for Linux x86_64
Operating systems
Linux
GGML backends
SYCL
Hardware
AMD Ryzen 9 5900XT + 2 x Intel Arc B580
Models
Qwen3.8-27B-Q4_K_M
Problem description & steps to reproduce
Increased VRAM usage starting with builds > b10680.
With llama.cpp build 106080 (ghcr.io/ggml-org/llama.cpp:server-intel-b10680), VRAM usage:
llama-cpp | [57651] 0.00.604.904 I common_params_fit_impl: - SYCL0 (Intel(R) Arc(TM) B580 Graphics): 12216 total, 9362 used, 1162 free vs. target of 1024
llama-cpp | [57651] 0.00.604.905 I common_params_fit_impl: - SYCL1 (Intel(R) Arc(TM) B580 Graphics): 12216 total, 10397 used, 1391 free vs. target of 1024
llama-cpp | [57651] 0.00.604.905 I common_params_fit_impl: projected to use 19759 MiB of device memory vs. 22313 MiB of free device memory
With any later build:
llama-cpp | [33301] 0.00.582.005 I common_params_fit_impl: - SYCL0 (Intel(R) Arc(TM) B580 Graphics): 12216 total, 9685 used, 840 free vs. target of 1024
llama-cpp | [33301] 0.00.582.006 I common_params_fit_impl: - SYCL1 (Intel(R) Arc(TM) B580 Graphics): 12216 total, 10720 used, 1068 free vs. target of 1024
llama-cpp | [33301] 0.00.582.006 I common_params_fit_impl: projected to use 20405 MiB of device memory vs. 22314 MiB of free device memory
The model and all used parameters are the same.
...
--gpu-layers all
--threads 16
--split-mode layer
--models-preset /config/config.ini
--flash-attn on
--cache-type-k q8_0
--cache-type-v q8_0
--main-gpu 1
--log-verbosity 4
...
[Qwen3.8-27B-Q4_K_M]
model = /models/lmstudio-community/Qwen3.8-27B-GGUF/Qwen3.8-27B-Q4_K_M.gguf
temp = 1.0
top-p = 0.95
top-k = 20
ctx-size = 90212
First Bad Commit
Any docker build later than b10680.
Activity
arthw commentedon Sep 18, 2026
@alex-rk86
Looks like the llama.cpp enbale some new features impact the buffer size, instead of SYCL backend.
#28981
b10680/b10681 (both are same data)
two factors:
If want to get the detailed cause, you need to confirm which commit lead to this issue.
Thank you!
alex-rk86 commentedon Sep 18, 2026
Hi @arthw,
Thank you, I haven't noticed those differences in logs.
I have just re-tested, disabling reasoning preserve (--no-reasoning-preserve) makes no impact to the compute buffer size.
My understanding that a compute buffer size depends on model size, context length, quantization and batch size. That should be the same in my cases, or allocation logic has been changed. I will try to go over commits to see anything related to it, not sure if it is an actual bug or intended behavior.
As workaround for newer versions I've reduced my batch sizes since dropping the context length is not desirable.
Regards,
Alex
arthw commentedon Sep 19, 2026
@alex-rk86
OK, it's great!
Thank you!