Eval bug: Metal backend silently produces corrupted output on Intel Mac with AMD GPU (no error, no CPU fallback) #30702

Name and Version

build : b11669-620d2225c

Operating systems

Mac

GGML backends

Metal

Hardware

• Hardware: Intel Core i9-9880H / AMD Radeon Pro 5500M 8 GB / Intel UHD Graphics 630

Models

• Model: Qwen2.5-7B-Instruct GGUF Q4_K_M

Problem description & steps to reproduce

Description

On this Intel Mac, the default CMake build enables the Metal backend and
offloads to the GPU by default. The Metal path produces consistently garbled
output, with no error and no visible warning, while the same model file,
same binary runs correctly with -ngl 0 (CPU only). A GGML_METAL=OFF
build also runs correctly.

The issue is not that Metal doesn't support this GPU — it's that it fails
silently: a user on this configuration gets plausible-looking but wrong
results, and has to bisect across backends themselves to discover the GPU
path is unsupported.

Suggested behavior: at startup, detect unsupported GPU families and either
(a) refuse to initialize with a clear error, or
(b) fall back to CPU with a prominent warning.

Reproduction

cmake -B build && cmake --build build -j

./build/bin/llama-cli -m <model>.gguf -p "你好" -n 128 -st
# → garbled output, e.g. "That negóивкиonestlyardy ..."

./build/bin/llama-cli -m <model>.gguf -p "你好" -n 128 -st -ngl 0
# → correct output

Expected behavior

Explicit error at startup (e.g. "Metal backend does not support ;
use -ngl 0 or rebuild with GGML_METAL=OFF"), or automatic CPU fallback.

Actual behavior

Model loads, generation runs, output is deterministic garbage. No error.

#19563 documents that Metal produces silent garbage on AMD discrete GPUs (Intel Macs). The root-cause fixes went stale, but regardless of when the kernels are fixed, the backend should detect non-Apple-Silicon / unsupported GPU families at init and (a) error out with guidance (-ngl 0 or GGML_METAL=OFF) or (b) fall back to CPU with a warning. Several downstream projects (#19563 comments) now ship their own workarounds, suggesting users keep hitting this.

First Bad Commit

No response

Relevant log output

yshen@192 week4 % ./llama.cpp/build/bin/llama-cli -m ./models/qwen2.5-7b-instruct-q4_k_m-00001-of-00002.gguf -p "你好" -n 128 -st -ngl 0


Loading model...  

▄▄ ▄▄
██ ██
██ ██  ▀▀█▄ ███▄███▄  ▀▀█▄    ▄████ ████▄ ████▄
██ ██ ▄█▀██ ██ ██ ██ ▄█▀██    ██    ██ ██ ██ ██
██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀
                                    ██    ██
                                    ▀▀    ▀▀

build      : b11669-620d2225c
model      : ./models/qwen2.5-7b-instruct-q4_k_m-00001-of-00002.gguf
ftype      : Q4_K - Medium
modalities : text

> 你好
你好！有什么可以帮助你的吗？

[ Prompt: 37.2 t/s | Generation: 5.7 t/s ]


Exiting...
yshen@192 week4 % ./llama.cpp/build/bin/llama-cli -m ./models/qwen2.5-7b-instruct-q4_k_m-00001-of-00002.gguf -p "你好" -n 128 -st       
Loading model...  

▄▄ ▄▄
██ ██
██ ██  ▀▀█▄ ███▄███▄  ▀▀█▄    ▄████ ████▄ ████▄
██ ██ ▄█▀██ ██ ██ ██ ▄█▀██    ██    ██ ██ ██ ██
██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀
                                    ██    ██
                                    ▀▀    ▀▀

build      : b11669-620d2225c
model      : ./models/qwen2.5-7b-instruct-q4_k_m-00001-of-00002.gguf
ftype      : Q4_K - Medium
modalities : text

> 你好
That
 

 negóировкиonestlyardy 
 
 Mein masked.strstr

stown

Ray
yshen@192 week4 %