[brainiac13]

Overview

This fixes multi-GPU selection in the Metal backend on Macs with more than one physical GPU, for example an Intel Mac with an eGPU.

Previously, all Metal backend devices ended up on the system default GPU, which meant that adding multiple MTL devices didn’t actually allow using multiple physical GPUs.

With this change, backend device indices are mapped to the available physical GPUs, sorted by VRAM (largest first). So MTL0 uses the GPU with the most VRAM, MTL1 the next one, etc.

Existing behavior is kept for GGML_METAL_DEVICES and for device indices that don’t map to a physical GPU. Those continue to use virtual devices on the default GPU.

This also makes it possible to split layers across physical GPUs, for example:

-dev MTL0,MTL1

Tensor copies between different physical GPUs use the host staging path. Metal private buffers can’t be accessed directly from a GPU other than the one that created them.

I also disabled concurrent dispatch on non-Apple GPU families as a workaround for #25866.

Fixes #28565

Additional information

Tested on macOS 26.5.1 on a 2019 Intel MacBook Pro with:

• Radeon Pro 5600M (8 GB)
• RX 6900 XT (16 GB) connected as an eGPU over TB3

The branch has been rebased onto current master.

Benchmarks with Gemma 4 12B UD-Q4_K_XL (pp / tg, t/s):

• RX 6900 XT (MTL0): 62 / 34
• Radeon Pro 5600M (MTL1): 19 / 14
• RX 6900 XT + Radeon Pro 5600M: 45 / 21

There are two limitations I ran into while testing that appear to be unrelated to this change:

• Large prompts on a single eGPU can hit the existing buffer upload issue tracked in Eval bug: Metal: GGML_ASSERT(buf_src) failed in ggml_metal_buffer_set_tensor on Intel Mac with AMD discrete GPU (newBufferWithBytesNoCopy page-alignment requirement) #26949.
• Sustained VRAM overcommit on the internal AMD GPU can cause multi-second stalls in the macOS AMD driver. This appears to be an OS/driver issue rather than something specific to llama.cpp.

Requirements

• I have read and agree with die contributing guidelines
• AI usage disclosure: YES — I used AI assistance while working on parts of the implementation and PR description. I reviewed the resulting changes, tested them on the hardware described above, and verified that I understand the code being submitted.

[ggml-gh-bot]

Hi @brainiac13, thanks for your contribution!

Per our contribution guidelines, the automated PR checker found the following issue(s) that need your attention:

• PR Template not respected: Please respect the template when creating a new pull request. Make sure to fill out all required sections.

• AI-generated content: While code is allowed to be generated by AI, please write the PR description and commit messages on your own without the help of AI.

---

Please note that maintainers reserve the right to make final decisions on PRs. If you believe there is a mistake, please comment below.