You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It may be helpful to support SYCL in addition to the native CUDA and ROCm implementations. This may be as simple as some testing and documentation, although code changes for an additional backend are also entirely possible.
One thing to watch: the VAE uses two custom ops (ggml_snake and ggml_col2im_1d) that are implemented in the CUDA, Vulkan and CPU backends but not yet in SYCL. The scheduler will fall back to CPU for those, so the DiT and LM should run on SYCL but the VAE decode may be slower. If you can build and confirm everything runs, I can write the SYCL kernels for those two ops.
One thing to watch: the VAE uses two custom ops (ggml_snake and ggml_col2im_1d) that are implemented in the CUDA, Vulkan and CPU backends but not yet in SYCL. The scheduler will fall back to CPU for those, so the DiT and LM should run on SYCL but the VAE decode may be slower. If you can build and confirm everything runs, I can write the SYCL kernels for those two ops.
The SYCL backend build, but on my system require a fixed VAE buffer reserved with ggml_backend_sched_reserve (otherwise libze explode).
I can confirm the slow CPU VAE decoding (on my system is too slow to be usefull).
Can you try --vae-chunk 512 ? Default is 1024 and require a least 8GB RAM/VRAM free
My system have 32gb of shared ram, I'm using an iGPU (while in this specific aspect it use CPU only).
I went down to 256 chunk size but was still too slow.
For the libze explosion, the problem was the realloc.
There's another problem. (Reducing chunk size is just to save memory at the expense of performance; 1024 is nominal speed, anything beyond that doesn't provide any performance benefit.)
Thanks for the logs, very useful. Traced it through ggml: the root cause is upstream, not the tiling.
Since ggml_snake and ggml_col2im_1d have no SYCL kernel, the VAE graph splits across SYCL0 + SYCL_Host -> multi-buffer. On a larger tile ggml grows the buffers via the sched reserve path, which is fine on CUDA/Vulkan but kills the device on SYCL at the readback (ggml-sycl.cpp:597) -> DEVICE_LOST.
So the real bug is that the SYCL backend does not survive a mid-session buffer free+realloc. Your reserve patch is a valid workaround (it removes the realloc), but the durable fix belongs in ggml-sycl. I will fix it there so every backend behaves the same.
Note: 512 chunk only survives because the realloc is smaller, not a real fix.
I will also be able to write native SYCL kernels for ggml_snake and ggml_col2im_1d so the VAE runs fully on GPU with no CPU fallback. That keeps the graph single-buffer and makes the decode actually usable on your Arc. I would just need to test a few iterations on real hardware to validate the kernels.
Hi @ServeurpersoCom I have an Arc Pro B60 (Battlemage, 24 GB) on Linux and can do the hardware testing you mentioned.
I’ve been running YuE2 through audio.cpp’s vendored ggml on SYCL (oneAPI 2026.1) and hit the same wall: col2im_1d had no SYCL kernel. On top of that, ggml-sycl’s own CONV_TRANSPOSE_1D was so slow there that a single Oobleck upsample layer took 21 s. Here’s what I have working now:
COL2IM_1D SYCL kernel, F32/F16, a port of the Vulkan shader. Conv-transpose goes through matmul + col2im, the same path CUDA and Vulkan use. The whole VAE stays on the GPU, and a full song’s VAE decode takes about 4.5 s.
Fused SDPA for FLASH_ATTN_EXT via oneDNN Graph, with GQA handled natively. The existing SYCL flash kernel doesn’t use XMX. With oneDNN SDPA the YuE2 NAR attention goes from about 74 ms to 3.3 ms per layer.
Smaller ggml-sycl gaps: bf16↔f32 copies, 16-bit CONCAT, and a contiguous fast path for unary ops (SILU 4× faster).
End to end on the B60, a 4:20 song renders in 250 s, against 500 s on Vulkan, and the output sounds right.
I haven’t built acestep.cpp on SYCL yet, and I haven’t written a snake kernel because audio.cpp’s YuE2 VAE builds snake from primitive ops. I’m happy to:
build acestep.cpp with -DGGML_SYCL=ON on the B60 and report back, including whether the realloc/DEVICE_LOST issue reproduces on a discrete card;
test your snake/col2im_1d kernels, or send mine as a PR to wherever suits you best: here, or the ggml repo you track.
14. Hi @ServeurpersoCom I have an Arc Pro B60 (Battlemage, 24 GB) on Linux and can do the hardware testing you mentioned.
I’ve been running YuE2 through audio.cpp’s vendored ggml on SYCL (oneAPI 2026.1) and hit the same wall: col2im_1d had no SYCL kernel. On top of that, ggml-sycl’s own CONV_TRANSPOSE_1D was so slow there that a single Oobleck upsample layer took 21 s. Here’s what I have working now:
COL2IM_1D SYCL kernel, F32/F16, a port of the Vulkan shader. Conv-transpose goes through matmul + col2im, the same path CUDA and Vulkan use. The whole VAE stays on the GPU, and a full song’s VAE decode takes about 4.5 s.
Fused SDPA for FLASH_ATTN_EXT via oneDNN Graph, with GQA handled natively. The existing SYCL flash kernel doesn’t use XMX. With oneDNN SDPA the YuE2 NAR attention goes from about 74 ms to 3.3 ms per layer.
Smaller ggml-sycl gaps: bf16↔f32 copies, 16-bit CONCAT, and a contiguous fast path for unary ops (SILU 4× faster).
End to end on the B60, a 4:20 song renders in 250 s, against 500 s on Vulkan, and the output sounds right.
I haven’t built acestep.cpp on SYCL yet, and I haven’t written a snake kernel because audio.cpp’s YuE2 VAE builds snake from primitive ops. I’m happy to:
build acestep.cpp with -DGGML_SYCL=ON on the B60 and report back, including whether the realloc/DEVICE_LOST issue reproduces on a discrete card;
test your snake/col2im_1d kernels, or send mine as a PR to wherever suits you best: here, or the ggml repo you track.
Hi @skipzx-lab, thanks, a B60 is exactly what this issue was missing.
Good news: acestep.cpp needs no extra SYCL kernel. I wrote and upstreamed the COL2IM_1D and Snake ops with their kernels for every backend, Snake fusion included (ggml-org/llama.cpp#25264 for the SYCL col2im kernel). Snake stays a chain of primitive ops (mul, sin, sqr, mul, add) that each backend fuses, so the whole VAE should fit on SYCL0 with no CPU split at all.
That split is what triggered the realloc and then the DEVICE_LOST StefanoBalocco hit on his iGPU. The most useful test: ./buildsycl.sh, a full run with --device SYCL0 at the default --vae-chunk 1024, and the log. If DEVICE_LOST comes back with a single backend graph, it is a real ggml-sycl bug and I will handle it there.
Since you already run YuE2, you can also test my yue2.cpp: same design as acestep.cpp, same ./buildsycl.sh, and it ships an audio encoder (MERT and its tokenizer) that audio.cpp does not have, to continue a song of your own or turn it into a cover.
The gaps you hit do not come from ggml but from the ggml vendored in audio.cpp: it is a late June snapshot that was never resynced. Upstream already has the SYCL col2im, oneDNN flash attention with XMX (ggml-org/llama.cpp#25222), the bf16 copies and the 16 bit CONCAT. It would be worth reporting to audio.cpp: rebasing their ggml module would bring all of that at once, and they would also gain from moving every deconvolution to mul_mat + COL2IM_1D instead of CONV_TRANSPOSE_1D, and from writing Snake exactly in the fusable form.
Thanks, that clears up a lot. I'll raise the ggml resync with audio.cpp.
The test you asked for:./buildsycl.sh (icx/icpx, oneAPI 2026.1, Ubuntu 26.04, Arc Pro B60 24 GB, Level Zero 1.14.37020), your examples/full.json (240 s, turbo Q8_0, 8 steps), --device SYCL0, default --vae-chunk 1024. No DEVICE_LOST. Every stage ran on SYCL0, and the VAE did 7 tiles in a single backend graph. Logs: b60-sycl0-lm.log and b60-sycl0-synth.log.
A real bug did turn up, in ggml-sycl itself, and it's still on llama.cpp master. The LM stopped early, and erratically. With seed 42 across 5 runs it produced 16, 299, 388, 521 and 1,300 codes for a 240 s target (about 1,200). Vulkan on the same request produced 1,197. GGML_SYCL_ENABLE_OPT=0 fixed it (1,196 / 1,200 / 1,198) but made decode 4× slower.
Cause: the tied embedding.
embed_tokens (Q8_0) is the src0 of the LM-head mul_mat. With a single-token batch, opt_for_reorder rearranges it in place into the qs-then-d layout.
The same tensor also feeds ggml_get_rows for the token embeddings. ggml_sycl_op_get_rows never checks extra->optimized_feature.reorder, so after the first decode step every embedding lookup reads the reordered bytes as block_q8_0.
I checked master's ggml-sycl/getrows.cpp: there's no reorder handling there either, so any model with a tied Q4_0/Q8_0 embedding on SYCL should hit this.
Fix: a reorder-aware path in ggml_sycl_op_get_rows for Q8_0/Q4_0 that decodes the qs-then-d layout (+45 lines, sycl-getrows-reorder-fix.patch). With it, and the optimisation left on, 4/4 runs gave 1,197–1,199 codes.
Timings, same B60, same request, Vulkan vs SYCL with the fix:
Stage
Vulkan
SYCL
LM decode, ~1,200 codes
66.5 s
21.6 s
LM prefill
2.8 s
5.2 s
DiT, 8 steps
12.8 s
11.3 s
VAE, 7 tiles × 1024
5.6 s
16.0 s
So SYCL wins overall, but the VAE is about 3× slower than on Vulkan. I haven't profiled it yet and can if that's useful.
I'll open the get_rows fix as a PR on llama.cpp, unless you'd rather route it differently. I'll also try yue2.cpp, since MERT continuation of my own recordings is exactly what I'm after.
Great report, thanks! Glad to see the DEVICE_LOST gone with a single backend graph, it confirms the CPU split was the only cause.
Good catch on get_rows: the reorder mutates in place a tensor that get_rows also reads, and acestep shares the same embed_tokens between the lm_head and the lookup. Yes, open the PR on llama.cpp, that is the right route, and I will sync it into my fork afterwards. Two notes for the review: in llama.cpp itself the input embedding lives on the CPU and a tied output is loaded as a separate copy, so the bug mostly hits ggml programs that share a single tensor on the device; and ggml-sycl also reorders Q4_K, Q2_K, Q3_K, Q5_K and Q6_K, so a reviewer will probably ask to cover every reorderable type.
For the VAE being 3x slower than on Vulkan, I think I know the cause: I wrote the Snake fusion for CUDA, Vulkan and Metal, but not for SYCL, since I had no hardware to test it. On your B60, each Snake therefore runs as 5 separate kernels instead of one. If you are up for testing it on your card, I can write it; a profile of the VAE would help confirm this first.
And I am curious to hear how yue2.cpp and the MERT continuation of your recordings work out for you.
So there are two things to do, writing the Snake fusion for SYCL and integrating your fix, and I can probably land both on my ggml fork before this weekend.
PR is up: ggml-org/llama.cpp#30430. As you predicted a reviewer would ask, it covers every type ggml-sycl reorders for 2D weights (Q4_0, Q8_0, Q2_K, Q3_K, Q4_K, Q5_K, Q6_K), with a test-backend-ops case that runs MUL_MAT then GET_ROWS on the same weight. It fails on master for all 7 on SYCL and passes with the fix (also passes on Vulkan).
VAE profile (acestep.cpp, one 1024-frame tile on the B60, ~2.4 s per tile, 7 tiles):
Op
per tile
share
IM2COL f16
856 ms
36%
MUL_MAT f16
733-776 ms
31%
ADD + MUL + SIN + SQR (mostly the Snake chain)
~650 ms
~27%
COL2IM_1D
108 ms
5%
So Snake fusion should help, but IM2COL looks like the bigger target on SYCL. Happy to test either on the card.
yue2.cpp on SYCL: builds clean with ./buildsycl.sh.
yue-transcribe (SheetSage2 Q8_0) failed with FATAL: unexpected token -1 - the same get_rows reorder bug. GGML_SYCL_ENABLE_OPT=0 fixed it, and with the PR's fix applied to your ggml it transcribes with the optimisation on, byte-identical to the OPT=0 score.
Tokenize + MERT continuation of a 12 s voice memo works. For my use it is the wrong tool though: my ideas are rough sketches and I want them as inspiration for a clean performance, not the original audio carried into the song. The transcribe -> score route fits that much better.
Speed on the B60: AR around 10-13 ms/token. The NAR is slower than I expected: 32 steps took 61 s at 2439 frames and 176 s at 6480 frames (about 1.9 and 5.5 s/step). For comparison, audio.cpp's YuE2 on the same card with oneDNN SDPA for the attention does about 2.0 s/step at ~6500 frames, so roughly 2.7x. I can profile the yue2.cpp NAR per op if that helps.
Thanks a lot @skipzx-lab, this is exactly the kind of help I needed since I have no Intel hardware to test on. I cherry-picked your get_rows fix (ggml-org/llama.cpp#30430) into the ggml module used by my music and vocoder projects, so acestep.cpp and yue2.cpp get the full speed SYCL path right now without waiting for the upstream merge. audio.cpp should benefit too once it resyncs its ggml, since several of its decoders share one tensor between the token embedding and the lm_head.
Next on my side is the Snake fusion for SYCL, and your VAE profile shows IM2COL is worth a look too. I'll ping you when there is something to test on the B60.
Activity
ServeurpersoCom commentedon Apr 13, 2026
Hi @iwr-redmond, GGML already has a SYCL backend and our architecture is backend-agnostic, so it should work out of the box. To build:
Probably need a "source /opt/intel/oneapi/setvars.sh" ?
mkdir build && cd build
cmake .. -DGGML_SYCL=ON
cmake --build . --config Release -j $(nproc)
One thing to watch: the VAE uses two custom ops (ggml_snake and ggml_col2im_1d) that are implemented in the CUDA, Vulkan and CPU backends but not yet in SYCL. The scheduler will fall back to CPU for those, so the DiT and LM should run on SYCL but the VAE decode may be slower. If you can build and confirm everything runs, I can write the SYCL kernels for those two ops.
iwr-redmond commentedon Apr 13, 2026
I use an RTX 4060 at the moment. Perhaps this FR can be left open so that a user with an Intel GPU can do the basic testing you need.
iwr-redmond commentedon May 25, 2026
Introduced in 4922ed1 but not yet added to the Readme.
StefanoBalocco commentedon Jun 21, 2026
The SYCL backend build, but on my system require a fixed VAE buffer reserved with ggml_backend_sched_reserve (otherwise libze explode).
I can confirm the slow CPU VAE decoding (on my system is too slow to be usefull).
ServeurpersoCom commentedon Jun 21, 2026
Can you try --vae-chunk 512 ? Default is 1024 and require a least 8GB RAM/VRAM free
StefanoBalocco commentedon Jun 21, 2026
My system have 32gb of shared ram, I'm using an iGPU (while in this specific aspect it use CPU only).
I went down to 256 chunk size but was still too slow.
For the libze explosion, the problem was the realloc.
ServeurpersoCom commentedon Jun 21, 2026
There's another problem. (Reducing chunk size is just to save memory at the expense of performance; 1024 is nominal speed, anything beyond that doesn't provide any performance benefit.)
ServeurpersoCom commentedon Jun 21, 2026
Can you send me the full log?
StefanoBalocco commentedon Jun 21, 2026
I need to compile it again because I moved to vulkan implementation (less hassle to compile/run it).
I'll give you some feedback tomorrow.
StefanoBalocco commentedon Jun 22, 2026
Ok, I replicate the crash problem only on 1024 chunk.
Using 512 chunk is working.
The log-1024-fixedvae.txt is the one of patched source code to use reserve.
The time to decode the vae is around a minute and half, using 256 chunk the time is doubled.
log-512.txt
log-1024.txt
log-1024-fixedvae.txt
Build flags:
-DGGML_CPU_ALL_VARIANTS=ON
-DGGML_BACKEND_DL=ON
-DGGML_SYCL=ON
-DCMAKE_SKIP_INSTALL_ALL_DEPENDENCY=ON
-DCMAKE_C_COMPILER=$(ONEAPI_ROOT)/compiler/$(ONEAPI_VER)/bin/icx
-DCMAKE_CXX_COMPILER=$(ONEAPI_ROOT)/compiler/$(ONEAPI_VER)/bin/icpx
-DGGML_BACKEND_DIR=/usr/lib/$(DEB_HOST_MULTIARCH)/acestep.cpp
The reserved vae buffers patch
StefanoBalocco@0a40fdb
ServeurpersoCom commentedon Jun 22, 2026
Thanks for the logs, very useful. Traced it through ggml: the root cause is upstream, not the tiling.
Since ggml_snake and ggml_col2im_1d have no SYCL kernel, the VAE graph splits across SYCL0 + SYCL_Host -> multi-buffer. On a larger tile ggml grows the buffers via the sched reserve path, which is fine on CUDA/Vulkan but kills the device on SYCL at the readback (ggml-sycl.cpp:597) -> DEVICE_LOST.
So the real bug is that the SYCL backend does not survive a mid-session buffer free+realloc. Your reserve patch is a valid workaround (it removes the realloc), but the durable fix belongs in ggml-sycl. I will fix it there so every backend behaves the same.
Note: 512 chunk only survives because the realloc is smaller, not a real fix.
ServeurpersoCom commentedon Jun 22, 2026
I will also be able to write native SYCL kernels for ggml_snake and ggml_col2im_1d so the VAE runs fully on GPU with no CPU fallback. That keeps the graph single-buffer and makes the decode actually usable on your Arc. I would just need to test a few iterations on real hardware to validate the kernels.
skipzx-lab commentedon Oct 12, 2026
Hi @ServeurpersoCom I have an Arc Pro B60 (Battlemage, 24 GB) on Linux and can do the hardware testing you mentioned.
I’ve been running YuE2 through audio.cpp’s vendored ggml on SYCL (oneAPI 2026.1) and hit the same wall: col2im_1d had no SYCL kernel. On top of that, ggml-sycl’s own CONV_TRANSPOSE_1D was so slow there that a single Oobleck upsample layer took 21 s. Here’s what I have working now:
COL2IM_1D SYCL kernel, F32/F16, a port of the Vulkan shader. Conv-transpose goes through matmul + col2im, the same path CUDA and Vulkan use. The whole VAE stays on the GPU, and a full song’s VAE decode takes about 4.5 s.
Fused SDPA for FLASH_ATTN_EXT via oneDNN Graph, with GQA handled natively. The existing SYCL flash kernel doesn’t use XMX. With oneDNN SDPA the YuE2 NAR attention goes from about 74 ms to 3.3 ms per layer.
Smaller ggml-sycl gaps: bf16↔f32 copies, 16-bit CONCAT, and a contiguous fast path for unary ops (SILU 4× faster).
End to end on the B60, a 4:20 song renders in 250 s, against 500 s on Vulkan, and the output sounds right.
I haven’t built acestep.cpp on SYCL yet, and I haven’t written a snake kernel because audio.cpp’s YuE2 VAE builds snake from primitive ops. I’m happy to:
build acestep.cpp with -DGGML_SYCL=ON on the B60 and report back, including whether the realloc/DEVICE_LOST issue reproduces on a discrete card;
test your snake/col2im_1d kernels, or send mine as a PR to wherever suits you best: here, or the ggml repo you track.
ServeurpersoCom commentedon Oct 12, 2026
Hi @skipzx-lab, thanks, a B60 is exactly what this issue was missing.
Good news: acestep.cpp needs no extra SYCL kernel. I wrote and upstreamed the COL2IM_1D and Snake ops with their kernels for every backend, Snake fusion included (ggml-org/llama.cpp#25264 for the SYCL col2im kernel). Snake stays a chain of primitive ops (mul, sin, sqr, mul, add) that each backend fuses, so the whole VAE should fit on SYCL0 with no CPU split at all.
That split is what triggered the realloc and then the DEVICE_LOST StefanoBalocco hit on his iGPU. The most useful test: ./buildsycl.sh, a full run with --device SYCL0 at the default --vae-chunk 1024, and the log. If DEVICE_LOST comes back with a single backend graph, it is a real ggml-sycl bug and I will handle it there.
Since you already run YuE2, you can also test my yue2.cpp: same design as acestep.cpp, same ./buildsycl.sh, and it ships an audio encoder (MERT and its tokenizer) that audio.cpp does not have, to continue a song of your own or turn it into a cover.
The gaps you hit do not come from ggml but from the ggml vendored in audio.cpp: it is a late June snapshot that was never resynced. Upstream already has the SYCL col2im, oneDNN flash attention with XMX (ggml-org/llama.cpp#25222), the bf16 copies and the 16 bit CONCAT. It would be worth reporting to audio.cpp: rebasing their ggml module would bring all of that at once, and they would also gain from moving every deconvolution to mul_mat + COL2IM_1D instead of CONV_TRANSPOSE_1D, and from writing Snake exactly in the fusable form.
skipzx-lab commentedon Oct 12, 2026
Thanks, that clears up a lot. I'll raise the ggml resync with audio.cpp.
The test you asked for:
./buildsycl.sh(icx/icpx, oneAPI 2026.1, Ubuntu 26.04, Arc Pro B60 24 GB, Level Zero 1.14.37020), yourexamples/full.json(240 s, turbo Q8_0, 8 steps),--device SYCL0, default--vae-chunk 1024. No DEVICE_LOST. Every stage ran on SYCL0, and the VAE did 7 tiles in a single backend graph. Logs: b60-sycl0-lm.log and b60-sycl0-synth.log.A real bug did turn up, in ggml-sycl itself, and it's still on llama.cpp master. The LM stopped early, and erratically. With seed 42 across 5 runs it produced 16, 299, 388, 521 and 1,300 codes for a 240 s target (about 1,200). Vulkan on the same request produced 1,197.
GGML_SYCL_ENABLE_OPT=0fixed it (1,196 / 1,200 / 1,198) but made decode 4× slower.Cause: the tied embedding.
embed_tokens(Q8_0) is thesrc0of the LM-headmul_mat. With a single-token batch,opt_for_reorderrearranges it in place into the qs-then-d layout.ggml_get_rowsfor the token embeddings.ggml_sycl_op_get_rowsnever checksextra->optimized_feature.reorder, so after the first decode step every embedding lookup reads the reordered bytes asblock_q8_0.ggml-sycl/getrows.cpp: there's no reorder handling there either, so any model with a tied Q4_0/Q8_0 embedding on SYCL should hit this.Fix: a reorder-aware path in
ggml_sycl_op_get_rowsfor Q8_0/Q4_0 that decodes the qs-then-d layout (+45 lines, sycl-getrows-reorder-fix.patch). With it, and the optimisation left on, 4/4 runs gave 1,197–1,199 codes.Timings, same B60, same request, Vulkan vs SYCL with the fix:
So SYCL wins overall, but the VAE is about 3× slower than on Vulkan. I haven't profiled it yet and can if that's useful.
I'll open the get_rows fix as a PR on llama.cpp, unless you'd rather route it differently. I'll also try yue2.cpp, since MERT continuation of my own recordings is exactly what I'm after.
ServeurpersoCom commentedon Oct 12, 2026
Great report, thanks! Glad to see the DEVICE_LOST gone with a single backend graph, it confirms the CPU split was the only cause.
Good catch on get_rows: the reorder mutates in place a tensor that get_rows also reads, and acestep shares the same embed_tokens between the lm_head and the lookup. Yes, open the PR on llama.cpp, that is the right route, and I will sync it into my fork afterwards. Two notes for the review: in llama.cpp itself the input embedding lives on the CPU and a tied output is loaded as a separate copy, so the bug mostly hits ggml programs that share a single tensor on the device; and ggml-sycl also reorders Q4_K, Q2_K, Q3_K, Q5_K and Q6_K, so a reviewer will probably ask to cover every reorderable type.
For the VAE being 3x slower than on Vulkan, I think I know the cause: I wrote the Snake fusion for CUDA, Vulkan and Metal, but not for SYCL, since I had no hardware to test it. On your B60, each Snake therefore runs as 5 separate kernels instead of one. If you are up for testing it on your card, I can write it; a profile of the VAE would help confirm this first.
And I am curious to hear how yue2.cpp and the MERT continuation of your recordings work out for you.
ServeurpersoCom commentedon Oct 12, 2026
So there are two things to do, writing the Snake fusion for SYCL and integrating your fix, and I can probably land both on my ggml fork before this weekend.
skipzx-lab commentedon Oct 13, 2026
PR is up: ggml-org/llama.cpp#30430. As you predicted a reviewer would ask, it covers every type ggml-sycl reorders for 2D weights (Q4_0, Q8_0, Q2_K, Q3_K, Q4_K, Q5_K, Q6_K), with a test-backend-ops case that runs MUL_MAT then GET_ROWS on the same weight. It fails on master for all 7 on SYCL and passes with the fix (also passes on Vulkan).
VAE profile (acestep.cpp, one 1024-frame tile on the B60, ~2.4 s per tile, 7 tiles):
So Snake fusion should help, but IM2COL looks like the bigger target on SYCL. Happy to test either on the card.
yue2.cpp on SYCL: builds clean with
./buildsycl.sh.yue-transcribe(SheetSage2 Q8_0) failed withFATAL: unexpected token -1- the same get_rows reorder bug.GGML_SYCL_ENABLE_OPT=0fixed it, and with the PR's fix applied to your ggml it transcribes with the optimisation on, byte-identical to the OPT=0 score.ServeurpersoCom commentedon Oct 16, 2026
Thanks a lot @skipzx-lab, this is exactly the kind of help I needed since I have no Intel hardware to test on. I cherry-picked your get_rows fix (ggml-org/llama.cpp#30430) into the ggml module used by my music and vocoder projects, so acestep.cpp and yue2.cpp get the full speed SYCL path right now without waiting for the upstream merge. audio.cpp should benefit too once it resyncs its ggml, since several of its decoders share one tensor between the token embedding and the lm_head.
Next on my side is the Snake fusion for SYCL, and your VAE profile shows IM2COL is worth a look too. I'll ping you when there is something to test on the B60.