Eval bug: GGML_ASSERT(ggml_can_mul_mat) in build_pooling when n_batch/n_ubatch not divisible by n_seq_max (MEAN pooling) #27175

Name and Version

version: 0.1.0-dev (build 10451, commit 10bf611)
built with GNU 16.1.1 for Linux x86_64

Operating systems

Linux

Which llama.cpp modules do you know to be affected?

libllama (core library)

Command line

I hit this through a custom C program linking libllama directly (llama_init_from_model with
n_batch=n_ubatch=512, n_seq_max=5, pooling_type=LLAMA_POOLING_TYPE_MEAN). You may see the only main file here: https://codeberg.org/nita-andrei-cristian/change/src/commit/b78a5160ab3df4e7d804593127827eb91fdb460b/src/main.c

Problem description & steps to reproduce

Problem: llama_init_from_model() aborts with GGML_ASSERT(ggml_can_mul_mat(a, b)) failed inside build_pooling (llama-graph.cpp:3593) when using LLAMA_POOLING_TYPE_MEAN on a causal model, if n_batch/n_ubatch is not evenly divisible by n_seq_max.

During graph_reserve's worst-case sizing pass, the pooling weight tensor (inp_mean) is sized using n_tokens rounded up to the nearest multiple of n_seq_max (e.g. 512 → 515 for n_seq_max=5), while the embeddings tensor it's multiplied against (result_norm) is capped at the real n_ubatch (512, not rounded). This produces two tensors with mismatched ne[0] (512 vs 515), which fails ggml_can_mul_mat.

Confirmed via gdb at the crash site:

a (result_norm, transposed) = [512, 2560, 1, 1]
b (inp_mean) = [515, 5, 1, 1]

Steps to reproduce: My actual application code that surfaced this: https://codeberg.org/nita-andrei-cristian/change/src/commit/b78a5160ab3df4e7d804593127827eb91fdb460b/src/main.c, see lines 102-137, please see the comment indicating the workaround.

Workaround: use a n_batch/n_ubatch value evenly divisible by n_seq_max (e.g. 510 instead of 512), or use LLAMA_POOLING_TYPE_LAST instead of MEAN. ** I left in the code an alternative variable you may use to make it work **

I built llama.cpp with:

cmake -B build-debug \
  -DCMAKE_BUILD_TYPE=Debug \
  -DGGML_CUDA=ON \
  -DCMAKE_CUDA_ARCHITECTURES=native \
  -DLLAMA_CURL=OFF
```*

### First Bad Commit

_No response_

### Relevant log output

<details>
<summary>Logs</summary>
<!-- Copy-pasted short logs go into the "console" area here -->

```console
(gdb) p a->ne[0]
$36 = 512
(gdb) p a->ne[1]
$37 = 2560
(gdb) p a->ne[2]
$38 = 1
(gdb) p a->ne[3]
#39 = 1
(gdb) p b->ne[0]
$40 = 515
(gdb) p b->ne[1]
$41 = 5
(gdb) p b->ne[2]
$42 = 1
(gdb) p b->ne[3]
$43 = 1
(gdb) p a->name
$44 = "result_norm (transposed) (cont)", '\000' <repeats 32 times>
(gdb) p b->name
$45 = '\000' <repeats 63 times>


llama_init_from_model: model default pooling_type is [-1], but [1] was specified

llama_context: n_seq_max             = 5
llama_context: n_ctx                 = 2560
llama_context: n_ctx_seq             = 512
llama_context: n_batch               = 512
llama_context: n_ubatch              = 512
llama_context: causal_attn           = 1
llama_context: flash_attn            = auto
llama_context: kv_unified            = false
llama_context: freq_base             = 10000000.0
llama_context: freq_scale            = 1
llama_context: n_rs_seq              = 0
llama_context: n_outputs_max         = 512
llama_context: n_outputs_max_per_seq = 1

#0 0x00007ffff3a9a01c in ?? () from /usr/lib/libc.so.6
#1 0x00007ffff3a3e5d0 in raise () from /usr/lib/libc.so.6
#2 0x00007ffff3a25685 in abort () from /usr/lib/libc.so.6
#3 0x00007ffff7d45d8b in ggml_abort (
file=0x7ffff7dec2e0 "/home/nita/dev/c/change/libs/llama.cpp/ggml/src/ggml.c", line=3282,
fmt=0x7ffff7dec017 "GGML_ASSERT(%s) failed")
at /home/nita/dev/c/change/libs/llama.cpp/ggml/src/ggml.c:271
#4 0x00007ffff7d4b578 in ggml_mul_mat (ctx=0x55555729a350, a=0x7fffa1ffae40, b=0x7fffa1ffab60)
at /home/nita/dev/c/change/libs/llama.cpp/ggml/src/ggml.c:3282
#5 0x00007ffff77bf669 in llm_graph_context::build_pooling (this=0x555555e0a9f0, cls=0x0, cls_b=0x0,
cls_out=0x0, cls_out_b=0x0, cls_norm=0x0)
at /home/nita/dev/c/change/libs/llama.cpp/src/llama-graph.cpp:3593
#6 0x00007ffff78872df in llama_model::build_graph (this=0x555556051f40, params=...)
at /home/nita/dev/c/change/libs/llama.cpp/src/llama-model.cpp:2454
#7 0x00007ffff7745c40 in llama_context::graph_reserve (this=0x555555e0de10, n_tokens=515, n_seqs=5,
n_outputs=512, mctx=0x555556073440, split_only=false, sizes=0x0)
at /home/nita/dev/c/change/libs/llama.cpp/src/llama-context.cpp:2430
#8 0x00007ffff773d685 in llama_context::sched_reserve (this=0x555555e0de10)
at /home/nita/dev/c/change/libs/llama.cpp/src/llama-context.cpp:633
#9 0x00007ffff773c2ed in llama_context::llama_context (this=0x555555e0de10, model=..., params=...)
at /home/nita/dev/c/change/libs/llama.cpp/src/llama-context.cpp:461
#10 0x00007ffff774a189 in llama_init_from_model (model=0x555556051f40, params=...)
at /home/nita/dev/c/change/libs/llama.cpp/src/llama-context.cpp:3643
#11 0x0000555555555a0b in create_embedding_context (session=0x7fffffffb640, n_tokens=20, n_sequences=5)
at /home/nita/dev/c/change/src/main.c:133
#12 0x0000555555555ba9 in compute_embedding_sesion (session=0x7fffffffb640, model=0x7fffffffb730,
sequences=0x7fffffffbb90, n_sequences=5, logit_mode=CALC_ALL_LOGITS)
at /home/nita/dev/c/change/src/main.c:153
#13 0x0000555555556396 in main () at /home/nita/dev/c/change/src/main.c:224