Skip to content

FIR-2167 - Extend advanced Triton MAT_MUL offload coverage PR #154 - #154

Merged
akapoor3518 merged 3 commits into
masterfrom
tmu-more-shape
Aug 7, 2026
Merged

FIR-2167 - Extend advanced Triton MAT_MUL offload coverage PR #154#154
akapoor3518 merged 3 commits into
masterfrom
tmu-more-shape

Conversation

@akapoor3518

@akapoor3518 akapoor3518 commented Aug 7, 2026 •
edited by cubic-dev-ai Bot
Loading

Copy link
Copy Markdown

Summary

This PR extends the existing advanced_matmul_shape_offload path to support additional safe Triton MAT_MUL shapes.

No new deployment flag is added. The change reuses the existing advanced_matmul_shape_offload runtime option and expands the MAT_MUL shape gate for cases that already satisfy the current Triton runtime constraints.

Changes

  • Added a generic advanced MAT_MUL shape helper:
    • tsavorite_mul_mat_advanced_shape_ok()
  • Offloads additional MAT_MUL ops when:
    • result dtype is F32
    • src0/src1 can be packed to F32
    • K satisfies the Triton MAT_MUL alignment requirement
    • D2/D3 broadcast is limited to safe cases where each input dimension either matches the output dimension or is 1
  • Kept grouped-head/repeat-head 4D broadcast shapes rejected for now because simple gate relaxation caused incorrect prompt output.

Validation

Validated with advanced_matmul_shape_offload=true and triton_matmul_small_n_transpose_opt=true.

  • tinyllama-vo-5m-para.gguf

    • MAT_MUL OPU coverage increased.
    • Prompt output remained correct.

  • Tiny-Llama-v0.3-FP32-1.1B-F32.gguf

    • Prompt output remained correct:
      • My cat's name is Luna
    • MAT_MUL OPU coverage increased.
    • Remaining rejected MAT_MUL shapes are grouped-head/repeat-head 4D broadcast patterns and will be handled in a follow-up PR.

Follow-up

Grouped-head/repeat-head 4D broadcast MAT_MUL shapes such as A2=4, B2=32, D2=32 need a proper runtime packing/copyback fix. They are intentionally not enabled in this PR.

POSIX result
ed [-Wunused-but-set-variable]
516 | const char * control_message;
| ^~~~~~~~~~~~~~~
/proj/work/akapoor/llama.cpp-aug-6-sdk-4.22/llama.cpp/tools/main/main.cpp:597:27: warning: unused variable ‘skipped_tokens’ [-Wunused-variable]
597 | const int skipped_tokens = (int) embd.size() - max_embd_size;
| ^~~~~~~~~~~~~~
[100%] Linking CXX executable ../../bin/llama-cli
/proj/rel/sw/arm-gnu-toolchain-14.2.rel1-x86_64-aarch64-none-linux-gnu/bin/../lib/gcc/aarch64-none-linux-gnu/14.2.1/../../../../aarch64-none-linux-gnu/bin/ld: skipping incompatible /lib/libm.so when searching for -lm
/proj/rel/sw/arm-gnu-toolchain-14.2.rel1-x86_64-aarch64-none-linux-gnu/bin/../lib/gcc/aarch64-none-linux-gnu/14.2.1/../../../../aarch64-none-linux-gnu/bin/ld: skipping incompatible /lib/libc.so when searching for -lc
[100%] Built target llama-cli
INFO: creating tar bundle for fpga (build-fpga)
INFO: included ./tsavorite-model-deployment.yaml in FPGA package
tsi-ggml/blobs/
tsi-ggml/blobs/txe_mul_mat_tile_f32_k128.blob
tsi-ggml/blobs/txe_mul_mat_tile_f32_k64.blob
tsi-ggml/blobs/txe_mul_mat_tile_f32_k32.blob
tsi-ggml/blobs/txe_add.blob
tsi-ggml/blobs/txe_sub.blob
tsi-ggml/blobs/txe_mult.blob
tsi-ggml/blobs/txe_div.blob
tsi-ggml/blobs/txe_abs.blob
tsi-ggml/blobs/txe_neg.blob
tsi-ggml/blobs/txe_sqrt.blob
tsi-ggml/blobs/txe_sqr.blob
tsi-ggml/blobs/txe_inv.blob
tsi-ggml/blobs/txe_sin.blob
tsi-ggml/blobs/txe_silu.blob
tsi-ggml/blobs/txe_sigmoid.blob
tsi-ggml/blobs/txe_swiglu.blob
tsi-ggml/blobs/txe_rms_norm.blob
tsi-ggml/blobs/txe_soft_max.blob
tsi-ggml/blobs/txe_add_16.blob
tsi-ggml/blobs/txe_sub_16.blob
tsi-ggml/blobs/txe_mult_16.blob
tsi-ggml/blobs/txe_div_16.blob
tsi-ggml/blobs/txe_abs_16.blob
tsi-ggml/blobs/txe_neg_16.blob
tsi-ggml/blobs/txe_sqrt_16.blob
tsi-ggml/blobs/txe_sqr_16.blob
tsi-ggml/blobs/txe_inv_16.blob
tsi-ggml/blobs/txe_sin_16.blob
tsi-ggml/blobs/txe_sigmoid_16.blob
tsi-ggml/blobs/txe_triton_add/
tsi-ggml/blobs/txe_triton_add/txe_blob_0.blob
tsi-ggml/blobs/txe_silu_16.blob
tsi-ggml/blobs/txe_swiglu_16.blob
tsi-ggml/blobs/txe_rms_norm_16.blob
tsi-ggml/blobs/txe_triton_mat_mul_1x8/
tsi-ggml/blobs/txe_triton_mat_mul_1x8/txe_blob_0.blob
tsi-ggml/blobs/txe_triton_mat_mul_2x4/
tsi-ggml/blobs/txe_triton_mat_mul_2x4/txe_blob_0.blob
tsi-ggml/ggml.sh
tsi-ggml/libggml-base.so
tsi-ggml/libggml-cpu.so
tsi-ggml/libggml.so
tsi-ggml/libggml-tsavorite.so
tsi-ggml/libllama.so
tsi-ggml/llama-cli
tsi-ggml/simple-backend-tsi
tsi-ggml/tsavorite-model-deployment.yaml
[akapoor@wspd0 llama.cpp]$
[akapoor@wspd0 llama.cpp]$
[akapoor@wspd0 llama.cpp]$
[akapoor@wspd0 llama.cpp]$ build-posix/bin/llama-cli -p "My cat's name is" -m /proj/rel/sw/ggml/models/Tiny-Llama-v0.3-FP32-1.1B-F32.gguf --device tsavorite -c 12288 --temp 0.0 --n-predict 2 --repeat-penalty 1.5 -b 1024 --top-k 50 --top-p 0.9 --repeat-last-n 5 --no-warmup --no-conversation

TSI deploy yaml=tsavorite-model-deployment.yaml txe_count=1 multi_thread_enable=1 user_dram_size_gb=8 user_dram_size_bytes=8589934592 advanced_matmul_shape_offload=1 triton_matmul_small_n_transpose_opt=1
My cat's name is Luna

llama_perf_sampler_print: sampling time = 4.05 ms / 9 runs ( 0.45 ms per token, 2222.77 tokens per second)

llama_perf_context_print: load time = 108636.40 ms
llama_perf_context_print: prompt eval time = 104586.96 ms / 7 tokens (14940.99 ms per token, 0.07 tokens per second)
llama_perf_context_print: eval time = 103577.65 ms / 1 runs (103577.65 ms per token, 0.01 tokens per second)
llama_perf_context_print: total time = 212218.74 ms / 8 tokens

=== GGML Perf Summary ===
Op Target Runs TSI_KERNEL-RUN Total us Avg us
ADD OPU 88 340 102782 1167.98
MUL OPU 90 348 248551 2761.68
RMS_NORM OPU 90 90 158487 1760.97
MUL_MAT CPU 830 0 1256219 1513.52
MUL_MAT OPU 178 178 206660943 1161016.53
CONT OPU 88 0 15045 170.97
RESHAPE CPU 473 0 265 0.56
VIEW CPU 439 0 60 0.14
VIEW OPU 88 0 40 0.45
PERMUTE CPU 315 0 99 0.31
PERMUTE OPU 88 0 16 0.18
TRANSPOSE CPU 124 0 17 0.14
GET_ROWS OPU 6 0 2597 432.83
SET_ROWS CPU 330 0 515 1.56
SOFT_MAX OPU 44 0 46750 1062.50
ROPE CPU 346 0 4221 12.20
GLU OPU 44 170 382109 8684.30

OPU Profiling Results:
Profiler disabled

[akapoor@wspd0 llama.cpp]$
[akapoor@wspd0 llama.cpp]$
[akapoor@wspd0 llama.cpp]$
[akapoor@wspd0 llama.cpp]$ build-posix/bin/llama-cli -p "My cat's name is" -m /proj/rel/sw/ggml/models/tinyllama-vo-5m-para.gguf --device tsavorite -c 12288 --temp 0.0 --n-predict 2 --repeat-penalty 1.5 -b 1024 --top-k 50 --top-p 0.9 --repeat-last-n 5 --no-warmup --no-conversation

TSI deploy yaml=tsavorite-model-deployment.yaml txe_count=1 multi_thread_enable=1 user_dram_size_gb=8 user_dram_size_bytes=8589934592 advanced_matmul_shape_offload=1 triton_matmul_small_n_transpose_opt=1
My cat's name is Tim.

llama_perf_sampler_print: sampling time = 3.82 ms / 9 runs ( 0.42 ms per token, 2356.02 tokens per second)
llama_perf_context_print: load time = 1142.18 ms
llama_perf_context_print: prompt eval time = 688.71 ms / 7 tokens ( 98.39 ms per token, 10.16 tokens per second)
llama_perf_context_print: eval time = 447.21 ms / 1 runs ( 447.21 ms per token, 2.24 tokens per second)
llama_perf_context_print: total time = 1593.81 ms / 8 tokens

=== GGML Perf Summary ===
Op Target Runs TSI_KERNEL-RUN Total us Avg us
ADD OPU 32 116 25600 800.00
MUL OPU 34 124 145153 4269.21
RMS_NORM OPU 34 34 25853 760.38
MUL_MAT CPU 290 0 34124 117.67
MUL_MAT OPU 66 66 821077 12440.56
CONT OPU 32 0 1560 48.75
RESHAPE CPU 176 0 41 0.23
VIEW CPU 176 0 23 0.13
VIEW OPU 32 0 6 0.19
PERMUTE CPU 104 0 25 0.24
PERMUTE OPU 32 0 4 0.12
TRANSPOSE CPU 61 0 19 0.31
GET_ROWS OPU 6 0 25 4.17
SET_ROWS CPU 113 0 145 1.28
SOFT_MAX OPU 16 0 9346 584.12
ROPE CPU 120 0 553 4.61
GLU OPU 16 58 45383 2836.44

OPU Profiling Results:
Profiler disabled

[akapoor@wspd0 llama.cpp]$ cat ggml_op_shape_dtype_catalog.tsv
Op Decision Count Result Src0 Src1

ADD SUPPORTED 18 f32 [64,1,1,1] f32 [64,1,1,1] f32 [64,1,1,1]
ADD SUPPORTED 11 f32 [64,512,1,1] f32 [64,512,1,1] f32 [64,512,1,1]
ADD SUPPORTED 14 f32 [64,7,1,1] f32 [64,7,1,1] f32 [64,7,1,1]
CONT SUPPORTED 8 f16 [12288,4,16,1] f16 [12288,4,16,1] none [0,0,0,0]
CONT SUPPORTED 8 f16 [256,4,16,1] f16 [256,4,16,1] none [0,0,0,0]
CONT SUPPORTED 8 f32 [64,512,1,1] f32 [4,16,512,1] none [0,0,0,0]
CONT SUPPORTED 8 f32 [64,7,1,1] f32 [4,16,7,1] none [0,0,0,0]
CPY SUPPORTED 1 f16 [12288,64,1,1] f32 [12288,64,1,1] f16 [12288,64,1,1]
FLASH_ATTN_EXT REJECTED 8 f32 [4,16,1,1] f32 [4,1,16,1] f16 [4,12288,16,1]
GET_ROWS SUPPORTED 1 f32 [64,1,1,1] bf16 [64,32000,1,1] i32 [1,1,1,1]
GET_ROWS SUPPORTED 2 f32 [64,1,1,1] f32 [64,1,1,1] i32 [1,1,1,1]
GET_ROWS SUPPORTED 2 f32 [64,1,1,1] f32 [64,7,1,1] i32 [1,1,1,1]
GET_ROWS SUPPORTED 2 f32 [64,512,1,1] f32 [64,512,1,1] i32 [512,1,1,1]
GET_ROWS SUPPORTED 1 f32 [64,7,1,1] bf16 [64,32000,1,1] i32 [7,1,1,1]
GLU SUPPORTED 9 f32 [256,1,1,1] f32 [256,1,1,1] f32 [256,1,1,1]
GLU SUPPORTED 7 f32 [256,512,1,1] f32 [256,512,1,1] f32 [256,512,1,1]
GLU SUPPORTED 7 f32 [256,7,1,1] f32 [256,7,1,1] f32 [256,7,1,1]
MUL_MAT REJECTED 8 f32 [12288,512,16,1] f16 [4,12288,16,1] f32 [4,512,16,1]
MUL_MAT REJECTED 8 f32 [256,7,16,1] f16 [4,256,16,1] f32 [4,7,16,1]
MUL_MAT SUPPORTED 18 f32 [256,1,1,1] bf16 [64,256,1,1] f32 [64,1,1,1]
MUL_MAT SUPPORTED 15 f32 [256,512,1,1] bf16 [64,256,1,1] f32 [64,512,1,1]
MUL_MAT SUPPORTED 14 f32 [256,7,1,1] bf16 [64,256,1,1] f32 [64,7,1,1]
MUL_MAT SUPPORTED 2 f32 [32000,1,1,1] bf16 [64,32000,1,1] f32 [64,1,1,1]
MUL_MAT SUPPORTED 2 f32 [32000,512,1,1] bf16 [64,32000,1,1] f32 [64,512,1,1]
MUL_MAT SUPPORTED 6 f32 [4,512,16,1] f16 [12288,4,16,1] f32 [12288,512,16,1]
MUL_MAT SUPPORTED 8 f32 [4,7,16,1] f16 [256,4,16,1] f32 [256,7,16,1]
MUL_MAT SUPPORTED 9 f32 [64,1,1,1] bf16 [256,64,1,1] f32 [256,1,1,1]
MUL_MAT SUPPORTED 32 f32 [64,1,1,1] bf16 [64,64,1,1] f32 [64,1,1,1]
MUL_MAT SUPPORTED 6 f32 [64,512,1,1] bf16 [256,64,1,1] f32 [256,512,1,1]
MUL_MAT SUPPORTED 29 f32 [64,512,1,1] bf16 [64,64,1,1] f32 [64,512,1,1]
MUL_MAT SUPPORTED 7 f32 [64,7,1,1] bf16 [256,64,1,1] f32 [256,7,1,1]
MUL_MAT SUPPORTED 32 f32 [64,7,1,1] bf16 [64,64,1,1] f32 [64,7,1,1]
MUL SUPPORTED 23 f32 [64,1,1,1] f32 [64,1,1,1] f32 [64,1,1,1]
MUL SUPPORTED 13 f32 [64,512,1,1] f32 [64,512,1,1] f32 [64,1,1,1]
MUL SUPPORTED 15 f32 [64,7,1,1] f32 [64,7,1,1] f32 [64,1,1,1]
NONE SUPPORTED 8 bf16 [256,64,1,1] none [0,0,0,0] none [0,0,0,0]
NONE SUPPORTED 16 bf16 [64,256,1,1] none [0,0,0,0] none [0,0,0,0]
NONE SUPPORTED 1 bf16 [64,32000,1,1] none [0,0,0,0] none [0,0,0,0]
NONE SUPPORTED 32 bf16 [64,64,1,1] none [0,0,0,0] none [0,0,0,0]
NONE SUPPORTED 17 f32 [64,1,1,1] none [0,0,0,0] none [0,0,0,0]
RMS_NORM SUPPORTED 19 f32 [64,1,1,1] f32 [64,1,1,1] none [0,0,0,0]
RMS_NORM SUPPORTED 13 f32 [64,512,1,1] f32 [64,512,1,1] none [0,0,0,0]
RMS_NORM SUPPORTED 15 f32 [64,7,1,1] f32 [64,7,1,1] none [0,0,0,0]
ROPE SUPPORTED 16 f32 [4,16,1,1] f32 [4,16,1,1] i32 [1,1,1,1]
ROPE SUPPORTED 15 f32 [4,16,512,1] f32 [4,16,512,1] i32 [512,1,1,1]
ROPE SUPPORTED 16 f32 [4,16,7,1] f32 [4,16,7,1] i32 [7,1,1,1]
SOFT_MAX SUPPORTED 8 f32 [12288,512,16,1] f32 [12288,512,16,1] f32 [12288,512,1,1]
SOFT_MAX SUPPORTED 8 f32 [256,7,16,1] f32 [256,7,16,1] f32 [256,64,1,1]
[akapoor@wspd0 llama.cpp]$

Summary by cubic

Extends advanced_matmul_shape_offload to offload more safe Triton MAT_MUL shapes to the OPU, and centralizes size caps for consistency. No new flags; addresses FIR-2167.

  • New Features
    • Adds tsavorite_mul_mat_advanced_shape_ok() to gate offload when: result is F32; inputs pack to F32; K % 32 == 0; dims within caps (K<=8192, M<=32768, N<=4096); safe D2/D3 broadcast; and padded sizes/byte count are valid.
    • Introduces tsavorite_triton_matmul_dims_within_caps() and uses it in both the advanced path and the standard mul_mat_supported_size() checks.
    • Enables the advanced path early when advanced_matmul_shape_offload is set (with TRITON_DEBUG logging); keeps grouped-head/repeat-head 4D broadcasts disabled pending a proper fix.

Written for commit 9aa38cb. Summary will update on new commits.

Review in cubic

[akapoor@wspd0 llama.cpp]$
[akapoor@wspd0 llama.cpp]$
[akapoor@wspd0 llama.cpp]$
[akapoor@wspd0 llama.cpp]$ build-posix/bin/llama-cli -p "My cat's name is" -m /proj/rel/sw/ggml/models/gemma3:1b-it-q4_K_M --device tsavorite -c 12288 --temp 0.0 --n-predict 2 --repeat-penalty 1.5 -b 1024 --top-k 50 --top-p 0.9 --repeat-last-n 5 --no-warmup --no-conversation

TSI deploy yaml=tsavorite-model-deployment.yaml txe_count=1 multi_thread_enable=1 user_dram_size_gb=16 user_dram_size_bytes=17179869184 advanced_matmul_shape_offload=1 triton_matmul_small_n_transpose_opt=1
My cat's name is Luna and

llama_perf_sampler_print: sampling time = 28.97 ms / 9 runs ( 3.22 ms per token, 310.71 tokens per second)
llama_perf_context_print: load time = 119929.54 ms
llama_perf_context_print: prompt eval time = 116954.17 ms / 7 tokens (16707.74 ms per token, 0.06 tokens per second)
llama_perf_context_print: eval time = 114028.31 ms / 1 runs (114028.31 ms per token, 0.01 tokens per second)
llama_perf_context_print: total time = 233988.86 ms / 8 tokens

=== GGML Perf Summary ===
Op Target Runs TSI_KERNEL-RUN Total us Avg us
ADD OPU 104 404 102939 989.80
MUL OPU 314 1856 1236923 3939.25
RMS_NORM OPU 314 314 404363 1287.78
MUL_MAT CPU 813 0 1129998 1389.91
MUL_MAT OPU 262 418 228647937 872702.05
SCALE CPU 54 0 336 6.22
CONT OPU 104 0 19154 184.17
RESHAPE CPU 563 0 347 0.62
VIEW CPU 552 0 72 0.13
VIEW OPU 104 0 44 0.42
PERMUTE CPU 362 0 130 0.36
PERMUTE OPU 104 0 11 0.11
TRANSPOSE CPU 188 0 35 0.19
GET_ROWS OPU 6 0 84 14.00
SET_ROWS CPU 384 0 586 1.53
SOFT_MAX OPU 52 0 7894 151.81
ROPE CPU 204 0 3217 15.77
ROPE OPU 52 0 2089 40.17
GLU CPU 198 0 76917 388.47

OPU Profiling Results:
Profiler disabled

[akapoor@wspd0 llama.cpp]$
[akapoor@wspd0 llama.cpp]$
[akapoor@wspd0 llama.cpp]$
[akapoor@wspd0 llama.cpp]$
[akapoor@wspd0 llama.cpp]$
[akapoor@wspd0 llama.cpp]$ cat ggml_op_shape_dtype_catalog.tsv
Op Decision Count Result Src0 Src1

ADD SUPPORTED 54 f32 [1152,1,1,1] f32 [1152,1,1,1] f32 [1152,1,1,1]
ADD SUPPORTED 36 f32 [1152,512,1,1] f32 [1152,512,1,1] f32 [1152,512,1,1]
ADD SUPPORTED 50 f32 [1152,7,1,1] f32 [1152,7,1,1] f32 [1152,7,1,1]
CONT SUPPORTED 22 f16 [1024,256,1,1] f16 [1024,256,1,1] none [0,0,0,0]
CONT SUPPORTED 4 f16 [12288,256,1,1] f16 [12288,256,1,1] none [0,0,0,0]
CONT SUPPORTED 26 f16 [256,256,1,1] f16 [256,256,1,1] none [0,0,0,0]
CONT SUPPORTED 26 f32 [1024,512,1,1] f32 [256,4,512,1] none [0,0,0,0]
CONT SUPPORTED 26 f32 [1024,7,1,1] f32 [256,4,7,1] none [0,0,0,0]
CPY SUPPORTED 1 f16 [1024,64,1,1] f32 [1024,64,1,1] f16 [1024,64,1,1]
CPY SUPPORTED 1 f16 [12288,64,1,1] f32 [12288,64,1,1] f16 [12288,64,1,1]
FLASH_ATTN_EXT REJECTED 22 f32 [256,4,1,1] f32 [256,1,4,1] f16 [256,1024,1,1]
FLASH_ATTN_EXT REJECTED 4 f32 [256,4,1,1] f32 [256,1,4,1] f16 [256,12288,1,1]
GET_ROWS SUPPORTED 2 f32 [1152,1,1,1] f32 [1152,1,1,1] i32 [1,1,1,1]
GET_ROWS SUPPORTED 2 f32 [1152,1,1,1] f32 [1152,7,1,1] i32 [1,1,1,1]
GET_ROWS SUPPORTED 1 f32 [1152,1,1,1] q8_0 [1152,262144,1,1] i32 [1,1,1,1]
GET_ROWS SUPPORTED 2 f32 [1152,512,1,1] f32 [1152,512,1,1] i32 [512,1,1,1]
GET_ROWS SUPPORTED 1 f32 [1152,7,1,1] q8_0 [1152,262144,1,1] i32 [7,1,1,1]
GLU REJECTED 27 f32 [6912,1,1,1] f32 [6912,1,1,1] f32 [6912,1,1,1]
GLU REJECTED 26 f32 [6912,512,1,1] f32 [6912,512,1,1] f32 [6912,512,1,1]
GLU REJECTED 25 f32 [6912,7,1,1] f32 [6912,7,1,1] f32 [6912,7,1,1]
MUL_MAT SUPPORTED 26 f32 [1024,1,1,1] q5_0 [1152,1024,1,1] f32 [1152,1,1,1]
MUL_MAT SUPPORTED 27 f32 [1024,512,1,1] q5_0 [1152,1024,1,1] f32 [1152,512,1,1]
MUL_MAT SUPPORTED 16 f32 [1024,512,4,1] f16 [256,1024,1,1] f32 [256,512,4,1]
MUL_MAT SUPPORTED 26 f32 [1024,7,1,1] q5_0 [1152,1024,1,1] f32 [1152,7,1,1]
MUL_MAT SUPPORTED 26 f32 [1152,1,1,1] q4_K [1024,1152,1,1] f32 [1024,1,1,1]
MUL_MAT SUPPORTED 13 f32 [1152,1,1,1] q4_K [6912,1152,1,1] f32 [6912,1,1,1]
MUL_MAT SUPPORTED 14 f32 [1152,1,1,1] q6_K [6912,1152,1,1] f32 [6912,1,1,1]
MUL_MAT SUPPORTED 27 f32 [1152,512,1,1] q4_K [1024,1152,1,1] f32 [1024,512,1,1]
MUL_MAT SUPPORTED 10 f32 [1152,512,1,1] q4_K [6912,1152,1,1] f32 [6912,512,1,1]
MUL_MAT SUPPORTED 12 f32 [1152,512,1,1] q6_K [6912,1152,1,1] f32 [6912,512,1,1]
MUL_MAT SUPPORTED 26 f32 [1152,7,1,1] q4_K [1024,1152,1,1] f32 [1024,7,1,1]
MUL_MAT SUPPORTED 13 f32 [1152,7,1,1] q4_K [6912,1152,1,1] f32 [6912,7,1,1]
MUL_MAT SUPPORTED 12 f32 [1152,7,1,1] q6_K [6912,1152,1,1] f32 [6912,7,1,1]
MUL_MAT SUPPORTED 4 f32 [12288,512,4,1] f16 [256,12288,1,1] f32 [256,512,4,1]
MUL_MAT SUPPORTED 39 f32 [256,1,1,1] q5_0 [1152,256,1,1] f32 [1152,1,1,1]
MUL_MAT SUPPORTED 13 f32 [256,1,1,1] q8_0 [1152,256,1,1] f32 [1152,1,1,1]
MUL_MAT SUPPORTED 35 f32 [256,512,1,1] q5_0 [1152,256,1,1] f32 [1152,512,1,1]
MUL_MAT SUPPORTED 12 f32 [256,512,1,1] q8_0 [1152,256,1,1] f32 [1152,512,1,1]
MUL_MAT SUPPORTED 17 f32 [256,512,4,1] f16 [1024,256,1,1] f32 [1024,512,4,1]
MUL_MAT SUPPORTED 3 f32 [256,512,4,1] f16 [12288,256,1,1] f32 [12288,512,4,1]
MUL_MAT SUPPORTED 39 f32 [256,7,1,1] q5_0 [1152,256,1,1] f32 [1152,7,1,1]
MUL_MAT SUPPORTED 13 f32 [256,7,1,1] q8_0 [1152,256,1,1] f32 [1152,7,1,1]
MUL_MAT SUPPORTED 52 f32 [256,7,4,1] f16 [256,256,1,1] f32 [256,7,4,1]
MUL_MAT SUPPORTED 2 f32 [262144,1,1,1] q8_0 [1152,262144,1,1] f32 [1152,1,1,1]
MUL_MAT SUPPORTED 2 f32 [262144,512,1,1] q8_0 [1152,262144,1,1] f32 [1152,512,1,1]
MUL_MAT SUPPORTED 54 f32 [6912,1,1,1] q5_0 [1152,6912,1,1] f32 [1152,1,1,1]
MUL_MAT SUPPORTED 45 f32 [6912,512,1,1] q5_0 [1152,6912,1,1] f32 [1152,512,1,1]
MUL_MAT SUPPORTED 50 f32 [6912,7,1,1] q5_0 [1152,6912,1,1] f32 [1152,7,1,1]
MUL SUPPORTED 116 f32 [1152,1,1,1] f32 [1152,1,1,1] f32 [1152,1,1,1]
MUL SUPPORTED 73 f32 [1152,512,1,1] f32 [1152,512,1,1] f32 [1152,1,1,1]
MUL SUPPORTED 101 f32 [1152,7,1,1] f32 [1152,7,1,1] f32 [1152,1,1,1]
MUL SUPPORTED 28 f32 [256,1,1,1] f32 [256,1,1,1] f32 [256,1,1,1]
MUL SUPPORTED 20 f32 [256,1,512,1] f32 [256,1,512,1] f32 [256,1,1,1]
MUL SUPPORTED 26 f32 [256,1,7,1] f32 [256,1,7,1] f32 [256,1,1,1]
MUL SUPPORTED 26 f32 [256,4,1,1] f32 [256,4,1,1] f32 [256,1,1,1]
MUL SUPPORTED 16 f32 [256,4,512,1] f32 [256,4,512,1] f32 [256,1,1,1]
MUL SUPPORTED 26 f32 [256,4,7,1] f32 [256,4,7,1] f32 [256,1,1,1]
NONE SUPPORTED 105 f32 [1152,1,1,1] none [0,0,0,0] none [0,0,0,0]
NONE SUPPORTED 52 f32 [256,1,1,1] none [0,0,0,0] none [0,0,0,0]
NONE SUPPORTED 26 q4_K [1024,1152,1,1] none [0,0,0,0] none [0,0,0,0]
NONE SUPPORTED 13 q4_K [6912,1152,1,1] none [0,0,0,0] none [0,0,0,0]
NONE SUPPORTED 26 q5_0 [1152,1024,1,1] none [0,0,0,0] none [0,0,0,0]
NONE SUPPORTED 39 q5_0 [1152,256,1,1] none [0,0,0,0] none [0,0,0,0]
NONE SUPPORTED 52 q5_0 [1152,6912,1,1] none [0,0,0,0] none [0,0,0,0]
NONE SUPPORTED 13 q6_K [6912,1152,1,1] none [0,0,0,0] none [0,0,0,0]
NONE SUPPORTED 13 q8_0 [1152,256,1,1] none [0,0,0,0] none [0,0,0,0]
NONE SUPPORTED 1 q8_0 [1152,262144,1,1] none [0,0,0,0] none [0,0,0,0]
RMS_NORM SUPPORTED 109 f32 [1152,1,1,1] f32 [1152,1,1,1] none [0,0,0,0]
RMS_NORM SUPPORTED 73 f32 [1152,512,1,1] f32 [1152,512,1,1] none [0,0,0,0]
RMS_NORM SUPPORTED 101 f32 [1152,7,1,1] f32 [1152,7,1,1] none [0,0,0,0]
RMS_NORM SUPPORTED 26 f32 [256,1,1,1] f32 [256,1,1,1] none [0,0,0,0]
RMS_NORM SUPPORTED 17 f32 [256,1,512,1] f32 [256,1,512,1] none [0,0,0,0]
RMS_NORM SUPPORTED 26 f32 [256,1,7,1] f32 [256,1,7,1] none [0,0,0,0]
RMS_NORM SUPPORTED 26 f32 [256,4,1,1] f32 [256,4,1,1] none [0,0,0,0]
RMS_NORM SUPPORTED 17 f32 [256,4,512,1] f32 [256,4,512,1] none [0,0,0,0]
RMS_NORM SUPPORTED 26 f32 [256,4,7,1] f32 [256,4,7,1] none [0,0,0,0]
ROPE SUPPORTED 26 f32 [256,1,1,1] f32 [256,1,1,1] i32 [1,1,1,1]
ROPE SUPPORTED 26 f32 [256,1,512,1] f32 [256,1,512,1] i32 [512,1,1,1]
ROPE SUPPORTED 26 f32 [256,1,7,1] f32 [256,1,7,1] i32 [7,1,1,1]
ROPE SUPPORTED 26 f32 [256,4,1,1] f32 [256,4,1,1] i32 [1,1,1,1]
ROPE SUPPORTED 23 f32 [256,4,512,1] f32 [256,4,512,1] i32 [512,1,1,1]
ROPE SUPPORTED 26 f32 [256,4,7,1] f32 [256,4,7,1] i32 [7,1,1,1]
SCALE REJECTED 1 f32 [1152,1,1,1] f32 [1152,1,1,1] none [0,0,0,0]
SCALE REJECTED 1 f32 [1152,7,1,1] f32 [1152,7,1,1] none [0,0,0,0]
SCALE REJECTED 26 f32 [256,4,1,1] f32 [256,4,1,1] none [0,0,0,0]
SCALE REJECTED 26 f32 [256,4,512,1] f32 [256,4,512,1] none [0,0,0,0]
SCALE REJECTED 26 f32 [256,4,7,1] f32 [256,4,7,1] none [0,0,0,0]
SOFT_MAX SUPPORTED 22 f32 [1024,512,4,1] f32 [1024,512,4,1] f32 [1024,512,1,1]
SOFT_MAX SUPPORTED 4 f32 [12288,512,4,1] f32 [12288,512,4,1] f32 [12288,512,1,1]
SOFT_MAX SUPPORTED 26 f32 [256,7,4,1] f32 [256,7,4,1] f32 [256,64,1,1]

@qodo-code-review

qodo-code-review Bot commented Aug 7, 2026

Copy link
Copy Markdown

PR Summary by Qodo

Extend Triton advanced MAT_MUL offload to more safe broadcast shapes

✨ Enhancement 🕐 20-40 Minutes

Grey Divider

AI Description 

• Add a generic advanced-shape predicate to widen Triton MAT_MUL offload eligibility.
• Gate offload on F32 result, packable inputs, K alignment, and safe D2/D3 broadcasting.
• Keep known-bad grouped-head/repeat-head 4D broadcast patterns rejected to preserve correctness.
Diagram 

graph TD
  A["GGML op: MUL_MAT"] --> B(["mul_mat_supported_size()"])
  B --> C{"advanced_matmul_shape_offload?"} --> D(["tsavorite_mul_mat_advanced_shape_ok()"])
  D --> E(["triton_matmul_select_shape(M,N)"]) --> F["Triton MAT_MUL kernel"]

  subgraph Legend
    direction LR
    _fn(["Function"]) ~~~ _dec{"Decision"} ~~~ _ext["Kernel/offload"]
  end
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Explicit shape whitelist per known-good patterns
  • ➕ Lower risk of accidentally admitting a subtly-incorrect broadcast/layout case
  • ➕ Easier to reason about per-shape correctness and add regression tests per entry
  • ➖ Slower to expand coverage; needs ongoing maintenance as models vary
  • ➖ May miss safe shapes that satisfy runtime/kernel constraints
2. Enable grouped-head/repeat-head 4D broadcast via explicit pack/copyback path
  • ➕ Unlocks additional real-world matmul coverage while preserving correctness
  • ➕ Allows isolating complex broadcasting into a dedicated, testable transform
  • ➖ More code and memory traffic; higher performance variance
  • ➖ Requires careful correctness testing around strides/packing and output writeback
3. Add targeted matmul shape regression tests (golden output) for advanced gate
  • ➕ Prevents future gate relaxations from silently breaking prompt correctness
  • ➕ Documents intended safe/unsafe shape classes
  • ➖ Test infra cost; requires representative tensors/models and tolerances

Recommendation: The PR’s approach (a conservative, generic predicate enforcing packability, K alignment, and strict broadcast constraints) is a good incremental expansion that avoids known-bad 4D grouped-head/repeat-head patterns. Consider complementing this with a small set of regression tests for the newly-enabled broadcast cases, and handle the grouped-head/repeat-head 4D shapes in a follow-up with an explicit packing/copyback path rather than further relaxing the gate.

Files changed (1) +83 / -0

Enhancement (1) +83 / -0
ggml-tsavorite.cppAdd advanced MAT_MUL shape predicate and enable it in offload gate +83/-0

Add advanced MAT_MUL shape predicate and enable it in offload gate

• Introduces 'tsavorite_mul_mat_advanced_shape_ok()' to validate Triton MAT_MUL eligibility based on F32 output, input packability to F32, K-dimension alignment, and safe D2/D3 broadcasting rules. Wires the helper into 'mul_mat_supported_size()' under the existing 'advanced_matmul_shape_offload' option (with optional TRITON_DEBUG logging) to offload additional safe shapes.

ggml/src/ggml-tsavorite/ggml-tsavorite.cpp

cubic-dev-ai[bot]
cubic-dev-ai Bot reviewed Aug 7, 2026
View reviewed changes

@cubic-dev-ai cubic-dev-ai Bot left a comment •
edited
Loading

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 1 file

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread ggml/src/ggml-tsavorite/ggml-tsavorite.cpp Resolved
Comment thread ggml/src/ggml-tsavorite/ggml-tsavorite.cpp Resolved
Comment thread ggml/src/ggml-tsavorite/ggml-tsavorite.cpp Resolved
@qodo-code-review

qodo-code-review Bot commented Aug 7, 2026 •
edited
Loading

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (1) 📜 Skill insights (0)

Grey Divider


Action required

1. Advanced gate enables aborts ✓ Resolved 🐞 Bug ☼ Reliability
Description 
mul_mat_supported_size() now returns true early via tsavorite_mul_mat_advanced_shape_ok(), but
that helper does not enforce quantized block-aligned K or contiguous nb0 requirements. This can
route quantized/view tensors into tsavorite_tensor_copy_k_to_f32(), which hard-aborts on
misaligned K or non-contiguous nb0 instead of cleanly falling back.
Code

ggml/src/ggml-tsavorite/ggml-tsavorite.cpp[R3139-3142]

+    if (advanced_matmul_shape_offload && tsavorite_mul_mat_advanced_shape_ok(op)) {
+#if TRITON_DEBUG
+        const struct ggml_tensor * a_dbg = op->src[0];
+        const struct ggml_tensor * b_dbg = op->src[1];
Relevance

●●● Strong

Very close precedent: team recently added K/stride guards to avoid packing aborts; early-return
reintroduces mismatch.

PR-#152

ⓘ Recommendations generated based on similar findings in past PRs

Evidence 
The new early return accepts ops before the existing K/block/stride-aware dtype checks run, but
packing for quantized/traits->to_float tensors still aborts when those invariants are violated.

ggml/src/ggml-tsavorite/ggml-tsavorite.cpp[3136-3152]
ggml/src/ggml-tsavorite/ggml-tsavorite.cpp[3166-3172]
ggml/src/ggml-tsavorite/ggml-tsavorite.cpp[2964-2990]
ggml/src/ggml-tsavorite/ggml-tsavorite.cpp[3030-3057]
ggml/src/ggml-tsavorite/ggml-tsavorite.cpp[3886-3923]
PR-#152

Agent prompt 
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`tsavorite_mul_mat_advanced_shape_ok()` only checks `tsavorite_tensor_type_can_pack_to_f32()` and `K % 32 == 0`, then `mul_mat_supported_size()` early-returns true. This bypasses the stricter `tsavorite_tensor_type_can_pack_to_f32_k(type, K, nb0)` checks that prevent `tsavorite_tensor_copy_k_to_f32()` from aborting on quantized/view tensors.

## Issue Context
The packing path for quantized types uses `traits->to_float` and explicitly `abort()`s if `K` is not block-aligned or if `nb0` is not the expected contiguous stride. The prior `mul_mat_supported_size()` implementation guarded against these cases; the new early-return path does not.

## Fix Focus Areas
- ggml/src/ggml-tsavorite/ggml-tsavorite.cpp[3136-3152]
- ggml/src/ggml-tsavorite/ggml-tsavorite.cpp[2941-2990]
- ggml/src/ggml-tsavorite/ggml-tsavorite.cpp[3886-3947]
- ggml/src/ggml-tsavorite/ggml-tsavorite.cpp[2992-3066]

## What to change
- In `tsavorite_mul_mat_advanced_shape_ok()`, replace the generic `tsavorite_tensor_type_can_pack_to_f32(...)` checks with the existing K/stride-aware gate:
 - compute `K` and `a_nb0/b_nb0` similarly to `mul_mat_supported_size()` (use `tsavorite_tensor_nb0_or_type_size()`), then call `tsavorite_tensor_type_can_pack_to_f32_k(a->type, K, a_nb0)` and same for `b`.
- Alternatively (or additionally), remove the early-return and integrate the advanced-shape checks into the existing `mul_mat_supported_size()` flow after the full dtype/stride validation so the same guardrails apply to both paths.
- Ensure the decision is consistent with the downstream packing functions so unsupported layouts cleanly return false (fallback) rather than aborting.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

2. Advanced path skips size caps ✓ Resolved 🐞 Bug ☼ Reliability
Description 
The new early return bypasses existing sanity caps (K > 8192, M > 32768, N > 4096) in
mul_mat_supported_size(), enabling offload of arbitrarily large matmuls. Downstream
ensure_triton_full_buffers_for_device() uses tsi_alloc(...) and asserts non-null, so this can
turn large shapes into process-terminating OOM aborts.
Code

ggml/src/ggml-tsavorite/ggml-tsavorite.cpp[R3139-3152]

+    if (advanced_matmul_shape_offload && tsavorite_mul_mat_advanced_shape_ok(op)) {
+#if TRITON_DEBUG
+        const struct ggml_tensor * a_dbg = op->src[0];
+        const struct ggml_tensor * b_dbg = op->src[1];
+        fprintf(stderr,
+                "MUL_MAT_ADVANCED_SHAPE_ENABLE: a_type=%d b_type=%d op_type=%d "
+                "a=[%ld,%ld,%ld,%ld] b=[%ld,%ld,%ld,%ld] op=[%ld,%ld,%ld,%ld]\n",
+                (int)a_dbg->type, (int)b_dbg->type, (int)op->type,
+                (long)a_dbg->ne[0], (long)a_dbg->ne[1], (long)a_dbg->ne[2], (long)a_dbg->ne[3],
+                (long)b_dbg->ne[0], (long)b_dbg->ne[1], (long)b_dbg->ne[2], (long)b_dbg->ne[3],
+                (long)op->ne[0], (long)op->ne[1], (long)op->ne[2], (long)op->ne[3]);
+#endif
+        return true;
+    }
Relevance

●●● Strong

Team has accepted OOM/alloc hardening; bypassing existing size caps risks assert/abort on huge mats.

PR-#110

ⓘ Recommendations generated based on similar findings in past PRs

Evidence 
The early return occurs before the existing cap checks execute, while the allocation path later
asserts that large tsi_alloc calls succeed; skipping caps therefore increases the chance of
OOM-triggered aborts.

ggml/src/ggml-tsavorite/ggml-tsavorite.cpp[3136-3152]
ggml/src/ggml-tsavorite/ggml-tsavorite.cpp[3332-3338]
ggml/src/ggml-tsavorite/ggml-tsavorite.cpp[4854-4911]

Agent prompt 
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`mul_mat_supported_size()` now short-circuits to true when `tsavorite_mul_mat_advanced_shape_ok()` passes. That bypasses the existing broad M/N/K caps intended to prevent excessive packing/buffer allocations.

## Issue Context
The Triton multi-TXE path allocates full packed buffers using `tsi_alloc((size_t)new_M * (size_t)new_K * sizeof(float))` etc and then asserts allocation success. Without caps (or a safer allocation/fallback strategy), large matmuls can abort the process.

## Fix Focus Areas
- ggml/src/ggml-tsavorite/ggml-tsavorite.cpp[3136-3152]
- ggml/src/ggml-tsavorite/ggml-tsavorite.cpp[3332-3338]
- ggml/src/ggml-tsavorite/ggml-tsavorite.cpp[4854-4911]
- ggml/src/ggml-tsavorite/ggml-tsavorite.cpp[3886-3947]

## What to change
- Ensure the legacy caps (or equivalent resource limits) are still enforced for the advanced-shape-enabled path. Options:
 - Apply the same caps inside `tsavorite_mul_mat_advanced_shape_ok()`.
 - Or do not early-return; instead, run the existing cap logic after advanced-shape validation.
- Consider making the cap shape-aware by basing it on computed padded element counts (`M_pad*K + K*N_pad + M_pad*N_pad`) and rejecting when the packed buffers would exceed a safe threshold.
- If you intentionally want to support larger shapes, replace the hard-assert allocation behavior with a recoverable failure path that returns false (fallback) on allocation failure.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Remediation recommended

3. Extra blank line inserted 📘 Rule violation ⚙ Maintainability
Description 
The modified C++ file contains two consecutive blank lines, which will be collapsed by
clang-format (configured to keep at most 1). This means running clang-format would produce a
diff, violating the formatting requirement.
Code

ggml/src/ggml-tsavorite/ggml-tsavorite.cpp[R3152-3155]

+    }
+
+
    const struct ggml_tensor *a = op->src[0];
Relevance

●●● Strong

Repo routinely enforces clang-format style/whitespace nits; extra blank line will be fixed.

PR-#145

ⓘ Recommendations generated based on similar findings in past PRs

Evidence 
The repository .clang-format sets MaxEmptyLinesToKeep: 1, but the changed code introduces two
consecutive blank lines, so clang-format would remove one and produce formatting changes.

Rule 2281060: C++ files must be formatted with the repository root .clang-format
ggml/src/ggml-tsavorite/ggml-tsavorite.cpp[3152-3155]
.clang-format[109-113]

Agent prompt 
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The updated C++ code has two consecutive blank lines, which conflicts with the repository `.clang-format` setting `MaxEmptyLinesToKeep: 1`. This will cause `clang-format` to modify the file.

## Issue Context
Repository formatting rules require that running `clang-format` with the root `.clang-format` produces no changes.

## Fix Focus Areas
- ggml/src/ggml-tsavorite/ggml-tsavorite.cpp[3152-3155]
- .clang-format[109-113]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

4. Unchecked padding math overflow ✓ Resolved 🐞 Bug ☼ Reliability
Description 
tsavorite_mul_mat_advanced_shape_ok() computes M_pad, N_pad, and total_bytes using unchecked
signed int64_t arithmetic, which can overflow and invoke undefined behavior for large dimensions.
This can make supports_op decisions incorrect/unstable for large or malformed tensor metadata.
Code

ggml/src/ggml-tsavorite/ggml-tsavorite.cpp[R3937-3940]

+    const triton_matmul_txe_shape_t & shape = triton_matmul_select_shape(M, N);
+    const int64_t M_pad = ((M + shape.m_dim - 1) / shape.m_dim) * shape.m_dim;
+    const int64_t N_pad = ((N + shape.n_dim - 1) / shape.n_dim) * shape.n_dim;
+    const int64_t total_bytes = (M_pad * K + K * N_pad + M_pad * N_pad) * (int64_t)sizeof(float);
Relevance

●●● Strong

Overflow/UB in matmul heuristics has been fixed before using safer math; same pattern likely
accepted.

PR-#144

ⓘ Recommendations generated based on similar findings in past PRs

Evidence 
The new helper reintroduces unchecked int64_t round-up and size computations, while the same file
already provides and uses overflow-safe helpers (tsi_round_up_i64, unsigned __int128 work
estimates) to avoid this class of UB.

ggml/src/ggml-tsavorite/ggml-tsavorite.cpp[3886-3944]
ggml/src/ggml-tsavorite/ggml-tsavorite.cpp[4566-4579]
ggml/src/ggml-tsavorite/ggml-tsavorite.cpp[4608-4631]
PR-#144

Agent prompt 
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`tsavorite_mul_mat_advanced_shape_ok()` does padding and byte-size estimation with raw `int64_t` sums/products:
- `M + shape.m_dim - 1`
- `M_pad * K`, `K * N_pad`, `M_pad * N_pad`
These can overflow signed `int64_t` (UB), and the subsequent `<= 0` checks do not make the computation safe.

## Issue Context
This file already contains `tsi_round_up_i64()` implemented with `unsigned __int128` overflow detection, and other Triton MAT_MUL heuristics use `unsigned __int128` for safe compute-work estimation.

## Fix Focus Areas
- ggml/src/ggml-tsavorite/ggml-tsavorite.cpp[3886-3947]
- ggml/src/ggml-tsavorite/ggml-tsavorite.cpp[4566-4579]

## What to change
- Replace the manual padding arithmetic with `tsi_round_up_i64(M, shape.m_dim)` and `tsi_round_up_i64(N, shape.n_dim)`.
- Compute `total_bytes` using `unsigned __int128` intermediates (or checked-math) and explicitly reject if the computed size exceeds `INT64_MAX` (and/or `SIZE_MAX` since allocation APIs take `size_t`).
- Keep the existing positivity checks, but make them operate on overflow-safe results (e.g., treat `INT64_MAX` return from `tsi_round_up_i64` as reject).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Grey Divider

Context used
✅ Compliance rules (platform): 6 rules

To customize comments, go to the Qodo configuration screen, or learn more in the docs.

Qodo Logo

qodo-code-review[bot]
qodo-code-review Bot reviewed Aug 7, 2026
View reviewed changes
Comment thread ggml/src/ggml-tsavorite/ggml-tsavorite.cpp Resolved
Comment thread ggml/src/ggml-tsavorite/ggml-tsavorite.cpp Resolved
atrivedi-tsavoritesi
atrivedi-tsavoritesi reviewed Aug 7, 2026
View reviewed changes
Comment thread ggml/src/ggml-tsavorite/ggml-tsavorite.cpp Resolved
Comment thread ggml/src/ggml-tsavorite/ggml-tsavorite.cpp Resolved
cubic-dev-ai[bot]
cubic-dev-ai Bot reviewed Aug 7, 2026
View reviewed changes

@cubic-dev-ai cubic-dev-ai Bot left a comment •
edited
Loading

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 1 file (changes from recent commits).

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread ggml/src/ggml-tsavorite/ggml-tsavorite.cpp Resolved
@akapoor3518
akapoor3518 merged commit 02a54bb into master Aug 7, 2026
1 check passed
@qodo-code-review qodo-code-review Bot mentioned this pull request Aug 9, 2026
Merged
@akapoor3518 akapoor3518 changed the title @FIR-2167 - Extend advanced Triton MAT_MUL offload coverage @FIR-2172- Add runtime-controlled broadcast Triton MAT_MUL offload support Aug 9, 2026
@akapoor3518 akapoor3518 changed the title @FIR-2172- Add runtime-controlled broadcast Triton MAT_MUL offload support FIR-2167 - Extend advanced Triton MAT_MUL offload coverage PR #154 Aug 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@cubic-dev-ai cubic-dev-ai[bot] cubic-dev-ai[bot] left review comments
@mmankal mmankal Awaiting requested review from mmankal mmankal is a code owner
@dmpatra dmpatra Awaiting requested review from dmpatra dmpatra is a code owner
@mikeuhler mikeuhler Awaiting requested review from mikeuhler mikeuhler is a code owner
@dineshReddy6381 dineshReddy6381 Awaiting requested review from dineshReddy6381 dineshReddy6381 is a code owner
@gkethamallax gkethamallax Awaiting requested review from gkethamallax gkethamallax is a code owner
@Nithyanand-G Nithyanand-G Awaiting requested review from Nithyanand-G Nithyanand-G is a code owner
+2 more reviewers
@qodo-code-review qodo-code-review[bot] qodo-code-review[bot] left review comments
@atrivedi-tsavoritesi atrivedi-tsavoritesi atrivedi-tsavoritesi approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@akapoor3518 @atrivedi-tsavoritesi