Eval bug: Vision not working with Qwen 27B 3.6/3.8 on AMD AI Max

Name and Version

C:\LLM\vLLM>llamacpp\llama-server --version
version: 0.1.0-dev (build 10437, commit 16d222f)
built with Clang 20.1.8 for Windows x86_64

Operating systems

Windows

GGML backends

Vulkan

Hardware

AMD Ryzen AI MAX+ 395 @ 128GB shared memory

Models

https://huggingface.co/ggml-org/Qwen3.8-27B-GGUF:Q8_0 with vision enabled

Problem description & steps to reproduce

When providing an image via the webui and simply asking what it sees in the image, the server silently crashes even with verbose logging.

First Bad Commit

No response

Relevant log output

C:\LLM\vLLM>llamacpp\llama-server -hf ggml-org/Qwen3.8-27B-GGUF:Q8_0 --hf-token <token_deleted> --jinja -ngl 99 -c 65536
0.00.476.362 I cmn common_param: common_params_print_info: verbosity = 3 (adjust with the -lv N CLI arg)
0.00.477.937 W srv llama_server: -----------------
0.00.477.943 W srv llama_server: CORS is set to allow all origins ('*') and no API key is set
0.00.477.944 W srv llama_server: this can be a security risk (cross-origin attacks)
0.00.477.944 W srv llama_server: more info: #25655
0.00.477.945 W srv llama_server: -----------------
Downloading mmproj-Qwen3.8-27B-Q8_0.gguf ─────────────────────────── 100%
Downloading Qwen3.8-27B-Q8_0.gguf ────────────────────────────────── 100%
68.09.817.504 W finalize_file: failed to create symlink: A required privilege is not held by the client.
68.09.817.510 W finalize_file: switching to degraded mode
68.09.833.809 I srv load_model: loading model 'ggml-org/Qwen3.8-27B-GGUF:Q8_0'
68.18.031.614 I cmn init: llama threadpool init, n_threads = 16
68.18.284.342 W load_hparams: Qwen-VL models require at minimum 1024 image tokens to function correctly on grounding tasks
68.18.284.348 W load_hparams: if you encounter problems with accuracy, try adding --image-min-tokens 1024
68.18.284.348 W load_hparams: more info: #16842

68.18.730.138 I srv load_model: loaded multimodal model, 'C:\Users<username_deleted>.cache\huggingface\hub\models--ggml-org--Qwen3.8-27B-GGUF\snapshots\0669b98607d47046c7c2b3f801011d54a08cfccf\mmproj-Qwen3.8-27B-Q8_0.gguf'
68.19.172.410 I srv load_model: initializing, n_slots = 4, n_ctx_slot = 65536, kv_unified = 'true'
68.19.181.947 I srv init: chat template supports preserving reasoning, consider enabling it via --reasoning-preserve
68.19.181.972 I srv llama_server: model loaded
68.19.181.977 I srv llama_server: listening on http://127.0.0.1:8080
68.19.181.977 W srv llama_server: NOTICE: server default port will be changed to :9931 in a future release
68.19.181.978 W srv llama_server: ref: #26508
68.54.927.911 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1
68.54.929.055 I slot launch_slot_: id 3 | task 0 | processing task, is_child = 0
68.59.734.505 I slot print_timing: id 3 | task 0 | prompt processing, n_tokens = 46, progress = 0.02, t = 4.81 s / 9.57 tokens per second
C:\LLM\vLLM>

To clarify, this is not a new bug. It also happened with Qwen3.6 from unsloth and Qwen3.8 from unsloth as well.

Text alone with MTP works

Text alone with MTP works

Log say 68.18.284.342 W load_hparams: Qwen-VL models require at minimum 1024 image tokens to function correctly on grounding tasks
68.18.284.348 W load_hparams: if you encounter problems with accuracy, try adding --image-min-tokens 1024
Did u test to add it ? Because on many model without this args vision not working ;)

Yes,

llamacpp\llama-server ^
  --model models/Qwen3.8-27B/Qwen3.8-27B-Q6_K.gguf ^
  --mmproj models/Qwen3.8-27B/mmproj-BF16.gguf ^
  --alias "unsloth/Qwen3.8-27B-Q6_K-MTP" ^
  --threads 16 ^
  --split-mode none ^
  --n-gpu-layers all ^
  --flash-attn on ^
  --temp 0.6 ^
  --top-p 0.95 ^
  --top-k 20 ^
  --min-p 0.00 ^
  --presence-penalty 0.00 ^
  --repeat-penalty 1.00 ^
  --spec-type draft-mtp ^
  --spec-draft-n-max 2 ^
  --batch-size 512 ^
  --ctx-size 262144 ^
  --image-min-tokens 1024 ^
  --ctx-checkpoints 64 ^
  --swa-checkpoints 32 ^
  --cache-ram -1 ^
  --host 0.0.0.0 ^
  --port 8101 ^

Result:

[Sat 08/15/2026 16:04:04.46] Starting Qwen3.8-27B Server 
0.00.062.523 I cmn  common_param: common_params_print_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.064.014 W srv  llama_server: -----------------
0.00.064.020 W srv  llama_server: CORS is set to allow all origins ('*') and no API key is set
0.00.064.020 W srv  llama_server: this can be a security risk (cross-origin attacks)
0.00.064.021 W srv  llama_server: more info: https://github.com/ggml-org/llama.cpp/pull/25655
0.00.064.021 W srv  llama_server: -----------------
0.00.080.495 I srv    load_model: loading model 'models/Qwen3.8-27B/Qwen3.8-27B-Q6_K.gguf'
0.09.109.794 I cmn          init: llama threadpool init, n_threads = 16
0.09.981.435 I common_speculative_init_result: creating MTP draft context against the target model 'models/Qwen3.8-27B/Qwen3.8-27B-Q6_K.gguf'
0.13.548.893 I srv    load_model: loaded multimodal model, 'models/Qwen3.8-27B/mmproj-BF16.gguf'
0.15.445.056 I srv    load_model: initializing, n_slots = 4, n_ctx_slot = 262144, kv_unified = 'true'
0.15.654.255 I srv          init: chat template supports preserving reasoning, consider enabling it via --reasoning-preserve
0.15.654.282 I srv  llama_server: model loaded
0.15.654.286 I srv  llama_server: listening on http://0.0.0.0:8101
1.03.023.779 I slot get_availabl: id  3 | task -1 | selected slot by LRU, t_last = -1
1.03.023.850 I slot launch_slot_: id  3 | task 0 | processing task, is_child = 0
1.07.893.509 I slot print_timing: id  3 | task 0 | prompt processing, n_tokens =     46, progress = 0.04, t =   4.76 s / 9.67 tokens per second
[Sat 08/15/2026 16:05:14.52] Exiting Qwen3.8-27B Server with code -1073740791

And also without MTP

[Sat 08/15/2026 16:07:02.41] Starting Qwen3.8-27B Server 
0.00.060.058 I cmn  common_param: common_params_print_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.061.450 W srv  llama_server: -----------------
0.00.061.455 W srv  llama_server: CORS is set to allow all origins ('*') and no API key is set
0.00.061.455 W srv  llama_server: this can be a security risk (cross-origin attacks)
0.00.061.456 W srv  llama_server: more info: https://github.com/ggml-org/llama.cpp/pull/25655
0.00.061.456 W srv  llama_server: -----------------
0.00.074.095 I srv    load_model: loading model 'models/Qwen3.8-27B/Qwen3.8-27B-Q6_K.gguf'
0.00.994.931 W model has unused tensor blk.64.attn_norm.weight (size = 20480 bytes) -- ignoring
0.00.994.944 W model has unused tensor blk.64.post_attention_norm.weight (size = 20480 bytes) -- ignoring
0.00.994.959 W model has unused tensor blk.64.attn_q.weight (size = 51609600 bytes) -- ignoring
0.00.994.964 W model has unused tensor blk.64.attn_k.weight (size = 4300800 bytes) -- ignoring
0.00.994.969 W model has unused tensor blk.64.attn_v.weight (size = 4300800 bytes) -- ignoring
0.00.994.987 W model has unused tensor blk.64.attn_output.weight (size = 25804800 bytes) -- ignoring
0.00.994.992 W model has unused tensor blk.64.attn_q_norm.weight (size = 1024 bytes) -- ignoring
0.00.994.997 W model has unused tensor blk.64.attn_k_norm.weight (size = 1024 bytes) -- ignoring
0.00.995.003 W model has unused tensor blk.64.ffn_gate.weight (size = 73113600 bytes) -- ignoring
0.00.995.008 W model has unused tensor blk.64.ffn_down.weight (size = 73113600 bytes) -- ignoring
0.00.995.013 W model has unused tensor blk.64.ffn_up.weight (size = 73113600 bytes) -- ignoring
0.00.995.021 W model has unused tensor blk.64.nextn.eh_proj.weight (size = 55705600 bytes) -- ignoring
0.00.995.027 W model has unused tensor blk.64.nextn.enorm.weight (size = 20480 bytes) -- ignoring
0.00.995.032 W model has unused tensor blk.64.nextn.hnorm.weight (size = 20480 bytes) -- ignoring
0.00.995.049 W model has unused tensor blk.64.nextn.shared_head_norm.weight (size = 20480 bytes) -- ignoring
0.08.312.902 I cmn          init: llama threadpool init, n_threads = 16
0.09.853.878 I srv    load_model: loaded multimodal model, 'models/Qwen3.8-27B/mmproj-BF16.gguf'
0.11.545.512 I srv    load_model: initializing, n_slots = 4, n_ctx_slot = 262144, kv_unified = 'true'
0.11.550.572 I srv          init: chat template supports preserving reasoning, consider enabling it via --reasoning-preserve
0.11.550.601 I srv  llama_server: model loaded
0.11.550.605 I srv  llama_server: listening on http://0.0.0.0:8101
0.15.528.214 I slot get_availabl: id  3 | task -1 | selected slot by LRU, t_last = -1
0.15.528.297 I slot launch_slot_: id  3 | task 0 | processing task, is_child = 0
0.20.236.655 I slot print_timing: id  3 | task 0 | prompt processing, n_tokens =     46, progress = 0.04, t =   4.71 s / 9.77 tokens per second
[Sat 08/15/2026 16:07:24.67] Exiting Qwen3.8-27B Server with code -1073740791

Here's the snip of the chat

same silent crash :(

Also worth mentioning: same setup works on an NVIDIA 4080

On the same AMD AI Max machine using the ROCm version the test runs as well, but at very slow speeds because its not using the GPU

So it seems to be related to the Vulcan backend

On a related note: disabling flash attention, even at minimal context length of 8K and no vision, also leads to a crash

--no-mmproj-offload fixes it, then the server uses the CPU instead

On my AMD Ryzen AI MAX+ 395, I am able to input images and the model correctly describes their content.

Command-line arguments:

> .\llama-server ^
    -m ..\..\my_models\unsloth\Qwen3.8-27B-GGUF\Qwen3.8-27B-IQ4_NL.gguf ^
    --spec-type draft-mtp ^
    --spec-draft-n-max 3 ^
    -bs ^
    --mmproj ..\..\my_models\unsloth\Qwen3.8-27B-GGUF\mmproj-BF16.gguf ^
    --batch-size 4096 ^
    --ubatch-size 4096 ^
    --image-min-tokens 1024 ^
    -ngl 99 ^
    -fa on ^
    -np 1 ^
    --temp 1 ^
    --repeat-penalty 1.1 ^
    --host 0.0.0.0 ^
    --port 1234

Logs during execution:

0.08.474.546 I cmn  common_param: common_params_print_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.08.652.249 W srv  llama_server: -----------------
0.08.652.267 W srv  llama_server: CORS is set to allow all origins ('*') and no API key is set
0.08.652.268 W srv  llama_server: this can be a security risk (cross-origin attacks)
0.08.652.269 W srv  llama_server: more info: https://github.com/ggml-org/llama.cpp/pull/25655
0.08.652.270 W srv  llama_server: -----------------
0.08.665.975 I srv    load_model: loading model '..\..\my_models\unsloth\Qwen3.8-27B-GGUF\Qwen3.8-27B-IQ4_NL.gguf'
0.16.500.818 I common_speculative_init_result: creating MTP draft context against the target model '..\..\my_models\unsloth\Qwen3.8-27B-GGUF\Qwen3.8-27B-IQ4_NL.gguf'
0.19.557.596 I srv    load_model: loaded multimodal model, '..\..\my_models\unsloth\Qwen3.8-27B-GGUF\mmproj-BF16.gguf'
0.19.835.145 I srv    load_model: initializing, n_slots = 1, n_ctx_slot = 262144, kv_unified = 'false'
0.19.836.625 W llama_sampler_backend_support: device 'ROCm0' does not have support for op TOP_K needed for sampler 'top-k'
0.19.899.452 I srv          init: chat template supports preserving reasoning, consider enabling it via --reasoning-preserve
0.19.900.179 I srv  llama_server: model loaded
0.19.900.186 I srv  llama_server: listening on http://0.0.0.0:1234
0.45.990.399 I slot get_availabl: id  0 | task -1 | selected slot by LRU, t_last = -1

However, when processing an image, several warnings appear such as:
W find_slot: non-consecutive token position 46 after 45 for sequence 0 with N new tokens
These do not seem to cause any actual issues in the output.

0.45.991.036 I slot launch_slot_: id  0 | task 0 | processing task, is_child = 0
0.48.262.077 W find_slot: non-consecutive token position 46 after 45 for sequence 0 with 1247 new tokens
0.48.262.679 W find_slot: non-consecutive token position 46 after 45 for sequence 0 with 1247 new tokens
0.48.279.703 W find_slot: non-consecutive token position 98 after 46 for sequence 0 with 10 new tokens
0.48.279.791 W find_slot: non-consecutive token position 98 after 46 for sequence 0 with 10 new tokens
0.58.935.086 I slot print_timing: id  0 | task 0 | n_gen =    103, tg =  16.26 t/s, tg_3s =  16.41 t/s
1.01.939.540 I slot print_timing: id  0 | task 0 | n_gen =    146, tg =  15.63 t/s, tg_3s =  14.31 t/s
1.05.072.907 I slot print_timing: id  0 | task 0 | n_gen =    194, tg =  15.55 t/s, tg_3s =  15.32 t/s
1.08.112.907 I slot print_timing: id  0 | task 0 | n_gen =    241, tg =  15.53 t/s, tg_3s =  15.46 t/s
1.11.241.553 I slot print_timing: id  0 | task 0 | n_gen =    284, tg =  15.23 t/s, tg_3s =  13.74 t/s
1.14.367.789 I slot print_timing: id  0 | task 0 | n_gen =    324, tg =  14.88 t/s, tg_3s =  12.79 t/s
1.17.491.025 I slot print_timing: id  0 | task 0 | n_gen =    365, tg =  14.66 t/s, tg_3s =  13.13 t/s
1.20.617.100 I slot print_timing: id  0 | task 0 | n_gen =    405, tg =  14.45 t/s, tg_3s =  12.80 t/s
1.23.626.599 I slot print_timing: id  0 | task 0 | n_gen =    451, tg =  14.53 t/s, tg_3s =  15.28 t/s
1.26.692.919 I slot print_timing: id  0 | task 0 | n_gen =    504, tg =  14.78 t/s, tg_3s =  17.28 t/s
1.29.720.367 I slot print_timing: id  0 | task 0 | n_gen =    554, tg =  14.92 t/s, tg_3s =  16.52 t/s
1.32.849.133 I slot print_timing: id  0 | task 0 | n_gen =    604, tg =  15.00 t/s, tg_3s =  15.98 t/s
1.35.862.680 I slot print_timing: id  0 | task 0 | n_gen =    658, tg =  15.21 t/s, tg_3s =  17.92 t/s
1.38.902.693 I slot print_timing: id  0 | task 0 | n_gen =    720, tg =  15.55 t/s, tg_3s =  20.39 t/s
1.41.980.047 I slot print_timing: id  0 | task 0 | n_gen =    781, tg =  15.82 t/s, tg_3s =  19.82 t/s
1.45.002.211 I slot print_timing: id  0 | task 0 | n_gen =    829, tg =  15.82 t/s, tg_3s =  15.88 t/s
1.46.731.398 I slot print_timing: id  0 | task 0 | prompt eval time =    6668.43 ms /  1307 tokens (    5.10 ms per token,   196.00 tokens per second)
1.46.731.416 I slot print_timing: id  0 | task 0 |        eval time =   54062.99 ms /   853 tokens (   63.45 ms per token,    15.76 tokens per second)
1.46.731.419 I slot print_timing: id  0 | task 0 |       total time =   60731.43 ms /  2160 tokens
1.46.731.421 I slot print_timing: id  0 | task 0 |    graphs reused =        408
1.46.731.436 I slot print_timing: id  0 | task 0 | draft acceptance = 0.35929 (  443 accepted /  1233 generated), mean len =  2.08
1.46.735.429 I slot      release: id  0 | task 0 | stop processing: n_tokens = 2161, truncated = 0

If I may ask, Linux or MS Host? Terminal looks like Linux

It's actually Windows 11 Pro 25H2.

Is this the ROCm version?

 llama_sampler_backend_support: device 'ROCm0' does not have support for op TOP_K needed for sampler 'top-k'

Did you install any ROCm drivers? because when I use the ROCm version --list_devices does not report any

I tried your command and it still crashes on me :/

Yes, I am using the ROCm version (Release build b10410). Please refer to this issue for instructions on how to use it:
#26996

You, Sir, are my hero! It works! 💃

same symptom on a different model: qwen3.8-flash-next. but with one caveat:
llama-mtmd-cli handles the image. llama-server does not. same model, same mmproj, same build, same gpu.

that rules out the weights, the projector and the vulkan backend, and leaves the server path.

environment:

build 10737 (36b101543), vulkan, msvc x64
Radeon 8060S (gfx1151), driver 32.0.31007.5012, windows 11
Qwen3.8-Flash-Next (qwen4exp), UD-Q4_K_XL, mmproj F16
no MTP, no speculative decoding, one slot

the model differs from the one in the report, so the shared factor is not the architecture.

the failing invocation:

llama-server -m Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf \
  --mmproj mmproj-Qwen3.8-Flash-Next-F16.gguf \
  --port 1235 -c 262144 -ngl 99 -np 1 -fa on \
  -ctk q8_0 -ctv q8_0 -lm mmap --lazy-mode on \
  --jinja --no-context-shift -lv 4

the server's last lines before the exit:

I image decoded (batch 1/1) in 29 ms
W find_slot: non-consecutive token position 21 after 13 for sequence 0 with 6 new tokens
W find_slot: non-consecutive token position 21 after 13 for sequence 0 with 6 new tokens

the process ends there. the http request returns a connection reset.

the same model and projector under llama-mtmd-cli complete, and the answer describes the image correctly:

I encoding mtmd batch, n_chunks = 1 (done = 1, total = 3)
I mtmd batch encoding done in 682 ms
W find_slot: non-consecutive token position 4 after 3 for sequence 0 with 9 new tokens
W find_slot: non-consecutive token position 24 after 4 for sequence 0 with 18 new tokens

what the failure is not:

not the find_slot warnings. they appear in the successful llama-mtmd-cli run as well.
not image size. a 1472x1472 image (4015 image tokens, two batches) and a 64x64 image (9 image tokens, one batch) fail at the same point.
not the image encode. image decoded is logged for every batch before the exit.

negative checks, with what each could have seen:

no GGML_ASSERT in 420 lines of -lv 4 output. that log does carry asserts. one appears in the same file for an unrelated failure on the same build.
no minidump. %LOCALAPPDATA%\CrashDumps holds dumps for this binary from earlier unrelated faults, so collection works.
no Application event log entry with id 1000.

on this build the exit is not an unhandled exception.

one extra observation. after a failed image request, the next plain text request to the same server also ends the process. text requests issued before any image are stable across many turns.

if you need i'm willing to test a patch build on this hardware, or let me know if you need further logs or a higher verbosity capture.

update: --no-mmproj-offload fixes this on vulkan, confirmed on a second architecture.

the workaround posted earlier in this thread also works for qwen3.8-flash-next (qwen4exp), build 10737, vulkan, Radeon 8060S. the projector runs on cpu, the server describes the image correctly, and a plain text request sent immediately afterwards also succeeds. the process stays alive.

llama-server -m Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf \
  --mmproj mmproj-Qwen3.8-Flash-Next-F16.gguf \
  --no-mmproj-offload --image-min-tokens 1024 \
  -c 262144 -ngl 99 -np 1 -fa on -ctk q8_0 -ctv q8_0

so the failure is in the gpu path for the projector, not in the weights, the projector file, or the model architecture. llama-mtmd-cli was already working with the same model and mmproj before this change, which points the same way.

cost of the workaround, measured on this hardware:

the cpu projector is not the bottleneck. it took 1931 ms, against 29 ms with gpu offload.
image derived tokens prefill at 22.9 tok/s. text prefills at roughly 150 tok/s on the same server.
--image-min-tokens 1024 raises even a 64x64 image to 1024 tokens, so any image costs about a minute.
a 1472x1472 photo is roughly 4000 image tokens and did not finish within 9 minutes.

resizing images before sending is the practical mitigation. dropping --image-min-tokens is the other lever where grounding accuracy is not needed.

one note for anyone arriving here from a search: this issue is closed, but the vulkan path is not fixed. the close followed a move to a rocm build. on vulkan the crash still reproduces on current master unless --no-mmproj-offload is passed.

Navigation Menu

Repository navigation

Eval bug: Vision not working with Qwen 27B 3.6/3.8 on AMD AI Max #27124

Name and Version

Operating systems

GGML backends

Hardware

Models

Problem description & steps to reproduce

First Bad Commit

Relevant log output

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Eval bug: Vision not working with Qwen 27B 3.6/3.8 on AMD AI Max #27124

Description

Name and Version

Operating systems

GGML backends

Hardware

Models

Problem description & steps to reproduce

First Bad Commit

Relevant log output

Activity

AriatneAI commented on Aug 15, 2026

AriatneAI commented on Aug 15, 2026

cobra91 commented on Aug 15, 2026

AriatneAI commented on Aug 15, 2026

AriatneAI commented on Aug 15, 2026

AriatneAI commented on Aug 15, 2026

AriatneAI commented on Aug 15, 2026

AriatneAI commented on Aug 15, 2026

satoyon commented on Aug 16, 2026

AriatneAI commented on Aug 16, 2026

satoyon commented on Aug 16, 2026

AriatneAI commented on Aug 16, 2026

AriatneAI commented on Aug 16, 2026

satoyon commented on Aug 16, 2026

AriatneAI commented on Aug 16, 2026

bitlamas commented on Sep 2, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions