Repository navigation
Compile bug: Metal tensor API silently inert on M5+/A19+ — MTLCompileOptions.languageVersion never set #27473
Description
Name and Version
version: 10150 (dee2a84)
built with AppleClang 21.0.0.21000099 for Darwin arm64
Operating systems
Mac
Which llama.cpp modules do you know to be affected?
llama-cli, llama-server, llama-bench, Test code, Other (Please specify in the next section), whisper.cpp, any other consumer of ggml
Command line
cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_METAL=ON
cmake --build build -j
./build/bin/llama-server --versionProblem description & steps to reproduce
Slower than expected prefills on M5 Max led me down this path.
Apple Silicon's TPU equivalent have not been usable via llama.cpp since 5b180c3 (2025-11-06). This was by design. However, there are now products capable of using this technology.
Apple's M5 and A19 chips have dedicated cores capable of speeding up matmul operations, in a meaningful way. Using the new hardware is currently not possible due to the issue being reported.
- Requirements
- Apple Silicon M5, or A19 (ggml disables the tensor API by name on anything older, so the bug is inert on those machines)
- macOS 26+
- Build main with Metal
git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp
cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_METAL=ON
cmake --build build -j
-
Run any binary that initialises the Metal device
./build/bin/llama-server --version -
Observe
E ggml_metal_library_init_from_source: error compiling source
W ggml_metal_device_init: - the tensor API is not supported in this environment - disabling
has_tensor is now false despite the hardware supporting it. test-backend-ops prints the resulting state directly: ggml_metal_device_init: has tensor = false.
- Reveal the compiler error
ggml_metal_device_init()callsggml_metal_library_init_from_source(dev, src_tensor_f16, false), and that last argument is verbose. Pass true to surface it:
program_source:6:17: error: use of undeclared identifier 'mpp'
using namespace mpp::tensor_ops;
program_source:7:46: error: use of undeclared identifier 'dextents'
The issue was isolated/confirmed by setting MTLCompileOptions.languageVersion explicitly and recompiling:
languageVersion setting: outcome
unset (default): FAIL — use of undeclared identifier 'mpp'
(3<<16)+2 (3.2): FAIL, same as above
(4<<16) (4.0): PASS
First Bad Commit
technically this commit introduced the bug: 5b180c3
PR #16634
However, it is only a bug because newer hardware is being limited due to the default language version.
Related
This does break protocol but there is a PR to address this issue, just in case it helps to clarify the issue and expedite the fix. here
Relevant log output
Logs
E ggml_metal_library_init_from_source: error compiling source
W ggml_metal_device_init: - the tensor API is not supported in this environment - disabling
Activity
[-]Misc. bug: [metal] "Input types must match cooperative tensor types" (M5+/A19+ only)[/-][+]Misc. bug: [metal] "Input types must match cooperative tensor types" (Apple Silicon)[/+][-]Misc. bug: [metal] "Input types must match cooperative tensor types" (Apple Silicon)[/-][+]Misc. bug: [metal] "the tensor API is not supported in this environment" on M5+/A19+ hardware[/+][-]Misc. bug: [metal] "the tensor API is not supported in this environment" on M5+/A19+ hardware[/-][+]Compiler bug: [metal] "the tensor API is not supported in this environment" on M5+/A19+ hardware[/+][-]Compiler bug: [metal] "the tensor API is not supported in this environment" on M5+/A19+ hardware[/-][+]Compile bug: [metal] "the tensor API is not supported in this environment" on M5+/A19+ hardware[/+][-]Compile bug: [metal] "the tensor API is not supported in this environment" on M5+/A19+ hardware[/-][+]Compile bug: Metal tensor API silently inert on M5+/A19+ — MTLCompileOptions.languageVersion never set[/+]leecrossley commentedon Aug 26, 2026
I have also seen and can confirm this correctness failure on Apple M5 Max with current master
macOS 26.5.1, Xcode 26.6, Metal 32023.883, GGML_METAL_EMBED_LIBRARY=OFF
Runtime probe reports
has tensor = true, but CMake builds the externaldefault.metallibwithoutGGML_METAL_HAS_TENSORI believe that mismatch produces incorrect Metal results
and disabling tensor dispatch fixes the identical test
so the precompiled-library guard in #27461 is a correctness fix (not only a performance fix) I'd suggest adding an external-metallib Q6_K MUL_MAT regression test
JamesFranc commentedon Aug 28, 2026
Thanks for reproducing @leecrossley
Agreed on the need for a test. Regression test added for Q6_K MUL_MAT in #27461 in the most recent commit
github-actions commentedon Oct 13, 2026
This issue was closed because it has been inactive for 14 days since being marked as stale.