Repository navigation
Compile bug: Vulkan shaders not compiling any more on Debian Stable (12/bookworm) #11052
Description
Git commit
$ git rev-parse HEAD
5437d4a
Operating systems
Linux
GGML backends
Vulkan
Problem description & steps to reproduce
On an up-to-date Debian Bookworm, Vulkan shaders do not compile any more. After some digging, this seems to be related to changes introduced in b4280 (3df784b) where the GL_KHR_cooperative_matrix extension is being used.
Without being familiar with Vulkan, my understanding is that these extensions started to be introduced with Vulkan 1.3.255 (see https://www.phoronix.com/news/Vulkan-1.3.255), but Vulkan on the current Debian Stable has version 1.3.239. Here are the versions of various packages which may be related:
$ sudo apt list libvulkan1 mesa-vulkan-drivers glslc
glslc/stable,now 2023.2-1 amd64 [installed]
libvulkan1/stable,now 1.3.239.0-1 amd64 [installed,automatic]
mesa-vulkan-drivers/stable,now 22.3.6-1+deb12u1 amd64 [installed]
In order to reproduce:
git checkout b4280
rm -rf build/
cmake -B build -DGGML_VULKAN=ON -DGGML_CCACHE=OFF
cmake --build build
The build properly compiles vulkan-shaders-gen, but then it hangs with multiple errors following this pattern (see full log below):
...
[ 5%] Linking CXX executable ../../../../bin/vulkan-shaders-gen
[ 5%] Built target vulkan-shaders-gen
[ 6%] Generate vulkan shaders
ggml_vulkan: Generating and compiling shaders to SPIR-V
cannot compile matmul_f32_f16_coopmat
/bin/glslc -fshader-stage=compute --target-env=vulkan1.2 -O llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f32_f16_coopmat.spv -DACC_TYPE=float -DB_TYPE=float16_t -DCOOPMAT=1 -DDATA_A_F32=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:11: warning: '#extension' : extension not supported: GL_KHR_cooperative_matrix
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: 'coopmat' : undeclared identifier
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: '' : syntax error, unexpected COMMA, expecting LEFT_PAREN
1 warning and 2 errors generated.
The previous tag (b4279) builds fine with the same setup.
Is it possible to disable this extension?
If not, and if Vulkan is not supported any more on such platforms, it could be useful to have a version check in CMake, so that people can understand right away that this is a problem with the version of a dependency.
First Bad Commit
Relevant log output
$ cmake -B build -DGGML_VULKAN=ON -DGGML_CCACHE=OFF
-- The C compiler identification is GNU 12.2.0
-- The CXX compiler identification is GNU 12.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Git: /usr/bin/git (found version "2.39.5")
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- Including CPU backend
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")
-- Adding CPU backend variant ggml-cpu: -march=native
-- Found Vulkan: /usr/lib/x86_64-linux-gnu/libvulkan.so (found version "1.3.239") found components: glslc glslangValidator
-- Vulkan found
-- Including Vulkan backend
-- Configuring done
-- Generating done
-- Build files have been written to: xxx/llama.cpp/build
$ $ cmake --build build
...
[ 4%] Linking CXX shared library libggml-base.so
[ 4%] Built target ggml-base
[ 4%] Building CXX object ggml/src/ggml-vulkan/vulkan-shaders/CMakeFiles/vulkan-shaders-gen.dir/vulkan-shaders-gen.cpp.o
[ 5%] Linking CXX executable ../../../../bin/vulkan-shaders-gen
[ 5%] Built target vulkan-shaders-gen
[ 6%] Generate vulkan shaders
ggml_vulkan: Generating and compiling shaders to SPIR-V
cannot compile matmul_f32_f16_coopmat
/bin/glslc -fshader-stage=compute --target-env=vulkan1.2 -O llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f32_f16_coopmat.spv -DACC_TYPE=float -DB_TYPE=float16_t -DCOOPMAT=1 -DDATA_A_F32=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:11: warning: '#extension' : extension not supported: GL_KHR_cooperative_matrix
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: 'coopmat' : undeclared identifier
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: '' : syntax error, unexpected COMMA, expecting LEFT_PAREN
1 warning and 2 errors generated.
cannot compile matmul_f32_f16_aligned_coopmat
/bin/glslc -fshader-stage=compute --target-env=vulkan1.2 -O llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f32_f16_aligned_coopmat.spv -DACC_TYPE=float -DALIGNED=1 -DB_TYPE=f16mat2x4 -DCOOPMAT=1 -DDATA_A_F32=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=8 -DLOAD_VEC_B=8
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:11: warning: '#extension' : extension not supported: GL_KHR_cooperative_matrix
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: 'coopmat' : undeclared identifier
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: '' : syntax error, unexpected COMMA, expecting LEFT_PAREN
1 warning and 2 errors generated.
cannot compile matmul_f16_aligned_coopmat
/bin/glslc -fshader-stage=compute --target-env=vulkan1.2 -O llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f16_aligned_coopmat.spv -DACC_TYPE=float -DALIGNED=1 -DB_TYPE=f16mat2x4 -DCOOPMAT=1 -DDATA_A_F16=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=8 -DLOAD_VEC_B=8
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:11: warning: '#extension' : extension not supported: GL_KHR_cooperative_matrix
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: 'coopmat' : undeclared identifier
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: '' : syntax error, unexpected COMMA, expecting LEFT_PAREN
1 warning and 2 errors generated.
cannot compile matmul_f16_coopmat
/bin/glslc -fshader-stage=compute --target-env=vulkan1.2 -O llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f16_coopmat.spv -DACC_TYPE=float -DB_TYPE=float16_t -DCOOPMAT=1 -DDATA_A_F16=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:11: warning: '#extension' : extension not supported: GL_KHR_cooperative_matrix
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: 'coopmat' : undeclared identifier
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: '' : syntax error, unexpected COMMA, expecting LEFT_PAREN
1 warning and 2 errors generated.
cannot compile matmul_f32_f32_coopmat
/bin/glslc -fshader-stage=compute --target-env=vulkan1.2 -O llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f32_f32_coopmat.spv -DACC_TYPE=float -DB_IS_FLOAT=1 -DB_TYPE=float -DCOOPMAT=1 -DDATA_A_F32=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:11: warning: '#extension' : extension not supported: GL_KHR_cooperative_matrix
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: 'coopmat' : undeclared identifier
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: '' : syntax error, unexpected COMMA, expecting LEFT_PAREN
1 warning and 2 errors generated.
cannot compile matmul_f32_f32_aligned_coopmat
/bin/glslc -fshader-stage=compute --target-env=vulkan1.2 -O llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f32_f32_aligned_coopmat.spv -DACC_TYPE=float -DALIGNED=1 -DB_IS_FLOAT=1 -DB_TYPE=mat2x4 -DCOOPMAT=1 -DDATA_A_F32=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=8 -DLOAD_VEC_B=8
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:11: warning: '#extension' : extension not supported: GL_KHR_cooperative_matrix
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: 'coopmat' : undeclared identifier
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: '' : syntax error, unexpected COMMA, expecting LEFT_PAREN
1 warning and 2 errors generated.
cannot compile matmul_f16_f32_coopmat
/bin/glslc -fshader-stage=compute --target-env=vulkan1.2 -O llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f16_f32_coopmat.spv -DACC_TYPE=float -DB_IS_FLOAT=1 -DB_TYPE=float -DCOOPMAT=1 -DDATA_A_F16=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:11: warning: '#extension' : extension not supported: GL_KHR_cooperative_matrix
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: 'coopmat' : undeclared identifier
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: '' : syntax error, unexpected COMMA, expecting LEFT_PAREN
1 warning and 2 errors generated.
cannot compile matmul_q4_0_f32_coopmat
/bin/glslc -fshader-stage=compute --target-env=vulkan1.2 -O llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q4_0_f32_coopmat.spv -DACC_TYPE=float -DB_IS_FLOAT=1 -DB_TYPE=float -DCOOPMAT=1 -DDATA_A_Q4_0=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=2
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:11: warning: '#extension' : extension not supported: GL_KHR_cooperative_matrix
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: 'coopmat' : undeclared identifier
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: '' : syntax error, unexpected COMMA, expecting LEFT_PAREN
1 warning and 2 errors generated.
cannot compile matmul_f16_f32_aligned_coopmat
/bin/glslc -fshader-stage=compute --target-env=vulkan1.2 -O llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f16_f32_aligned_coopmat.spv -DACC_TYPE=float -DALIGNED=1 -DB_IS_FLOAT=1 -DB_TYPE=mat2x4 -DCOOPMAT=1 -DDATA_A_F16=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=8 -DLOAD_VEC_B=8
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:11: warning: '#extension' : extension not supported: GL_KHR_cooperative_matrix
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: 'coopmat' : undeclared identifier
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: '' : syntax error, unexpected COMMA, expecting LEFT_PAREN
1 warning and 2 errors generated.
cannot compile matmul_q4_0_f32_aligned_coopmat
/bin/glslc -fshader-stage=compute --target-env=vulkan1.2 -O llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q4_0_f32_aligned_coopmat.spv -DACC_TYPE=float -DALIGNED=1 -DB_IS_FLOAT=1 -DB_TYPE=mat2x4 -DCOOPMAT=1 -DDATA_A_Q4_0=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=2 -DLOAD_VEC_B=8
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:11: warning: '#extension' : extension not supported: GL_KHR_cooperative_matrix
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: 'coopmat' : undeclared identifier
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: '' : syntax error, unexpected COMMA, expecting LEFT_PAREN
1 warning and 2 errors generated.
cannot compile matmul_q4_0_f16_aligned_coopmat
/bin/glslc -fshader-stage=compute --target-env=vulkan1.2 -O llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q4_0_f16_aligned_coopmat.spv -DACC_TYPE=float -DALIGNED=1 -DB_IS_FLOAT=1 -DB_TYPE=f16mat2x4 -DCOOPMAT=1 -DDATA_A_Q4_0=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=2 -DLOAD_VEC_B=8
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:11: warning: '#extension' : extension not supported: GL_KHR_cooperative_matrix
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: 'coopmat' : undeclared identifier
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: '' : syntax error, unexpected COMMA, expecting LEFT_PAREN
1 warning and 2 errors generated.
cannot compile matmul_q4_0_f16_coopmat
/bin/glslc -fshader-stage=compute --target-env=vulkan1.2 -O llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q4_0_f16_coopmat.spv -DACC_TYPE=float -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DCOOPMAT=1 -DDATA_A_Q4_0=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=2
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:11: warning: '#extension' : extension not supported: GL_KHR_cooperative_matrix
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: 'coopmat' : undeclared identifier
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: '' : syntax error, unexpected COMMA, expecting LEFT_PAREN
1 warning and 2 errors generated.
cannot compile matmul_q4_1_f32_coopmat
/bin/glslc -fshader-stage=compute --target-env=vulkan1.2 -O llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q4_1_f32_coopmat.spv -DACC_TYPE=float -DB_IS_FLOAT=1 -DB_TYPE=float -DCOOPMAT=1 -DDATA_A_Q4_1=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=2
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:11: warning: '#extension' : extension not supported: GL_KHR_cooperative_matrix
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: 'coopmat' : undeclared identifier
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: '' : syntax error, unexpected COMMA, expecting LEFT_PAREN
1 warning and 2 errors generated.
cannot compile matmul_q4_1_f16_aligned_coopmat
/bin/glslc -fshader-stage=compute --target-env=vulkan1.2 -O llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q4_1_f16_aligned_coopmat.spv -DACC_TYPE=float -DALIGNED=1 -DB_IS_FLOAT=1 -DB_TYPE=f16mat2x4 -DCOOPMAT=1 -DDATA_A_Q4_1=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=2 -DLOAD_VEC_B=8
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:11: warning: '#extension' : extension not supported: GL_KHR_cooperative_matrix
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: 'coopmat' : undeclared identifier
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: '' : syntax error, unexpected COMMA, expecting LEFT_PAREN
1 warning and 2 errors generated.
cannot compile matmul_q4_1_f32_aligned_coopmat
/bin/glslc -fshader-stage=compute --target-env=vulkan1.2 -O llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q4_1_f32_aligned_coopmat.spv -DACC_TYPE=float -DALIGNED=1 -DB_IS_FLOAT=1 -DB_TYPE=mat2x4 -DCOOPMAT=1 -DDATA_A_Q4_1=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=2 -DLOAD_VEC_B=8
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:11: warning: '#extension' : extension not supported: GL_KHR_cooperative_matrix
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: 'coopmat' : undeclared identifier
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: '' : syntax error, unexpected COMMA, expecting LEFT_PAREN
1 warning and 2 errors generated.
cannot compile matmul_q4_1_f16_coopmat
/bin/glslc -fshader-stage=compute --target-env=vulkan1.2 -O llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q4_1_f16_coopmat.spv -DACC_TYPE=float -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DCOOPMAT=1 -DDATA_A_Q4_1=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=2
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:11: warning: '#extension' : extension not supported: GL_KHR_cooperative_matrix
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: 'coopmat' : undeclared identifier
llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:195: error: '' : syntax error, unexpected COMMA, expecting LEFT_PAREN
1 warning and 2 errors generated.
Activity
0cc4m commentedon Jan 3, 2025
There is a check in CMake for coopmat support that should be preventing this: https://github.com/ggerganov/llama.cpp/blob/f66f5829276650cd83a087ab2cfed1a760183ea1/ggml/src/ggml-vulkan/CMakeLists.txt#L11-L23
But I don't see the output of that check in your log. Are you sure it was using the master branch?
mbaudier commentedon Jan 3, 2025
Yes, I have seen the test you are refering to, but it was added slightly later, in b4287 (ecc93d0), while the logs I have provided were from the first failing build (b4280).
As I understand it, this tests for GL_NV_cooperative_matrix2, not GL_KHR_cooperative_matrix.
To be on the safe side, I have tried with the current state of master (e7da954) :
and the same issue appears.
0cc4m commentedon Jan 3, 2025
You are correct, I guess I wasn't awake enough yet...
Basically you need newer Vulkan headers and a newer glslc version. You could use a ppa or build them yourself. It's unfortunate that the versions Debian provides are that old.
Maybe that should also be checked in a similar way to coopmat2, but for now that has not been implemented.
mbaudier commentedon Jan 3, 2025
Thank you for your quick answer! We can live without Vulkan support until the next Debian Stable.
pepijndevos commentedon Jan 6, 2025
So are all debian bookworm users just expected to stay on b4279?
It seems an environment variable GGML_VK_DISABLE_COOPMAT is introduced, why not a build define?
This does seem to happen for GGML_VULKAN_COOPMAT2_GLSLC_SUPPORT.
0cc4m commentedon Jan 6, 2025
As mentioned earlier, you can install glslc and Vulkan headers separately from your distro's versions. You can install the whole Vulkan SDK.
The environment variable was introduced because it's a single line of code, and required for performance comparison tests. A build define or a check similar to the one for coopmat2 would be fine, you're welcome to contribute it if you need it quickly.
pepijndevos commentedon Jan 6, 2025
Maybe a better solution would be to add a newer vulkan to bookworm-backports, I might look into that.
mbaudier commentedon Jan 6, 2025
Following @0cc4m's recommendation, I reproduced the GGML_VULKAN_COOPMAT2_GLSLC_SUPPORT logic and disabled the parts related to coopmat in:
until I could get it to build locally on my Debian 12/bookworm workstation:
I could then do a quick test:
The changes are available in this branch of my forked repo:
https://github.com/mbaudier/llama.cpp/tree/debian-bookworm-vulkan
This should definitely be reviewed carefully and tested on an environment where these Vulkan extensions are actually available. As already mentioned, I have no experience at all with Vulkan development, but I am used to hack and patch third-party software for packaging and compatibility purposes.
Do you think that it goes in the right direction?
How can I check whether my GPU would support these extensions with a newer Vulkan SDK?
This would definitely be great to be able to have standards-based GPU support on plain Debian Stable.
0cc4m commentedon Jan 6, 2025
@mbaudier Yeah, that looks good. Just open a PR with the changes and I'll test it, I don't think more than minor changes would be needed to merge.
https://vulkan.gpuinfo.org/ is a good resource on Vulkan extension support. But in this case, it's very simple: Only Nvidia RTX and AMD RX 7000 GPUs support the coopmat extension. Intel ARC kinda does, but it's not mature yet.