Sync upstream master (2026-10-28, ggml-org ac8e1694d) by benjigill · Pull Request #27 · benjigill/llama.cpp

benjigill · 2026-10-28T17:54:56Z

252 upstream commits, 797de63b2..ac8e1694d (ggml-org master, 2026-10-29), merged in three chunks: 1f4c2eeaa, 1a97e387d, ac8e1694d.

Dropped carried patches

CUDA: fix shared-memory WAR hazard in flash_attn_ext_vec ggml-org/llama.cpp#30573 FA vec WAR race fix: merged upstream under its own number.
model: keep Qwen3.5 delta-net output projection 2D ggml-org/llama.cpp#28265 qwen35 ssm_out 2D: upstream qwen: reshape output to 2d instead of 3d ggml-org/llama.cpp#30823 makes the same 2D reshape.
CUDA: fuse FFN gate/up matmuls and GLU in MMQ ggml-org/llama.cpp#28702 fused Q4_K gate/up + SwiGLU MMQ: closed upstream and replaced by the merged CUDA: fuse FFN gate/up matmuls and GLU in MMQ with interleaved gate/up rows ggml-org/llama.cpp#29948 (interleaved gate/up rows, Q4_K/Q5_K/Q6_K/Q8_0/IQ4_XS). The local alloc-dep entry now covers only the meta-backend part (no graph_optimize on the meta backend under -sm tensor); the CUDA part is upstream now.

Expected impact: the ledger records ggml-org#29948 at about -2.5% pp vs ggml-org#28702 on Q4_K (GB10), so the pp A/B below decides whether this is acceptable.

Conflicts and resolutions

Chunk 1 (1f4c2eeaa):

ggml/src/ggml-cuda/fattn-vec.cuh: took upstream (CUDA: fix shared-memory WAR hazard in flash_attn_ext_vec ggml-org/llama.cpp#30573 landed).
src/models/qwen35.cpp, qwen35moe.cpp: took upstream qwen: reshape output to 2d instead of 3d ggml-org/llama.cpp#30823; d2t (feat(qwen35): support d2t draft-vocabulary trim for MTP speculative decoding ggml-org/llama.cpp#29143 + local trim) untouched.
tools/server/server-common.cpp: kept server_prompt_checkpoint_due() next to upstream's get_server_exec_path().
tools/server/server-context.cpp: kept server: fix checkpoint handling for hybrid/recurrent models (#24055) ggml-org/llama.cpp#25592 pos_stale invalidation and added server : fix swa checkpoint invalidation logic ggml-org/llama.cpp#30666's n_tokens > n_prefix check (same set for exact state, stricter for SWA).
tools/server/tests/unit/test_completion.py: kept both sides' tests.

Chunk 2 (1a97e387d):

mmq.cu, cp-async.cuh, mmvq.cu, MMQ template instances: took upstream (only CUDA: fuse FFN gate/up matmuls and GLU in MMQ ggml-org/llama.cpp#28702 there).
mmq.cuh: took upstream, re-applied ggml-cuda: vectorize the MMQ activation tile copy to int2 ggml-org/llama.cpp#30493 unchanged (it now also covers the CUDA: fuse FFN gate/up matmuls and GLU in MMQ with interleaved gate/up rows ggml-org/llama.cpp#29948 fused path).
ggml-cuda.cu, common.cuh: dropped CUDA: fuse FFN gate/up matmuls and GLU in MMQ ggml-org/llama.cpp#28702 and the CUDA half of the alloc dep; kept CUDA: Fuse GDN alpha/beta projections ggml-org/llama.cpp#29187, CUDA: Support of GDN chunked kernel for prefill ggml-org/llama.cpp#26001 + local chunked GDN, and cuda : avoid repeated warmup after stable graph replay ggml-org/llama.cpp#29768 recapture ported to CUDA: chunk graphs to amortize graph launch ggml-org/llama.cpp#30804 chunked graphs.
unary.cu: took upstream softplus (model: DSV4.1 flash ggml-org/llama.cpp#31086).
src/llama-model.cpp, llama-model.h: kept supports_backend_sampling() next to upstream's no_alloc().
tests/CMakeLists.txt, test-llama-archs.cpp: upstream llama : add llama_arch_* query API and refactor test-llama-archs to use it ggml-org/llama.cpp#30880 layout; fork tests re-registered.
tests/test-backend-ops.cpp: dropped the CUDA: fuse FFN gate/up matmuls and GLU in MMQ ggml-org/llama.cpp#28702 tests, kept the GDN tests.

Chunk 3 (ac8e1694d): clean merge.

sync.sh verify: the only files back to upstream are the ggml-org#28702 files and fattn-vec.cuh; no fork file lost; leak check clean.

Checklist (Linux box)

CUDA build (sm_120) of master and this branch
test-backend-ops -b CUDA0 -o MUL_MAT,MUL_MAT_VEC_FUSION,GATED_DELTA_NET,GATED_DELTA_NET_AB_FUSION,GATED_DELTA_NET_CACHE_FUSION,SSM_CONV,FLASH_ATTN_EXT all OK
bench_qwen38.py A/B vs master: pp/tg and spec acceptance within noise (pp is the CUDA: fuse FFN gate/up matmuls and GLU in MMQ ggml-org/llama.cpp#28702 -> CUDA: fuse FFN gate/up matmuls and GLU in MMQ with interleaved gate/up rows ggml-org/llama.cpp#29948 check)
KLD vs master (fused gate/up kernel changed)
server smoke test with MTP + ngram-mod: coherent text, draft_n_accepted > 0
merge with "Create a merge commit" (no squash/rebase)

) * CUDA: fix KQ write-after-read race in flash_attn_ext_vec Assisted-by: ChatGPT * CUDA: apply review suggestions for KQ sync comments Co-authored-by: Johannes Gäßler <johannesg@5d6.de> --------- Co-authored-by: Johannes Gäßler <johannesg@5d6.de>

.pi/gg/SYSTEM.md repeated the full agent ruleset. Move it to .pi/gg/APPEND_SYSTEM.md and keep only the llama.cpp-specific rules, so it can be appended to a main system prompt with --append-system-prompt. Drop the .pi/SYSTEM.md symlink setup from wc2wt.sh and pr2wt.sh. Worktrees no longer need it now that the folder holds an addendum instead of a full prompt. Assisted-by: pi:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (RTX 5090 32GB, DDR5 128GB)

* tests: add beam fusion rows to MTL baseline Regenerate tests/fusion/MTL.csv with test-fusion --record. The newly added beam architecture has no baseline rows, so test-fusion fails with 2 failing rows (MUL+ADD x6, RMS_NORM+MUL x25). All other arch counts are unchanged. Assisted-by: pi:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB) * skills: add update-fusion-table skill Add a skill for regenerating a per-device test-fusion baseline. It keeps the workflow from tests/fusion/README.md and the parts that are easy to get wrong: diff the recorded file before applying it, the distinct meanings of a FAIL row, and never recording a partial run. Assisted-by: pi:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB)

…gml-org#30497) * The problem here in the repo is gguf-split can silently produce corrupted tensor output when the input GGUF file is truncated. The fix is to add tensor-data bounds checks and short-read checks in gguf-split.cpp and export-lora.cpp, and a regression test that checks truncated input is rejected and valid tensor data is preserved.# * Bug : GGUF-split can silently produce corrupted tensor output when the input GGUF file is truncated. Fix : add short-read checks in gguf-split.cpp (in copy_file_to_file in split_strategy) and export-lora.cpp (in read_tensor_data of file_input) so that truncated input is rejected.

…0687) * ui : keep markdown tables inside the chat column on phones The phone rule made the table wrapper full-bleed with -1rem margins, so the first and the last column sat under the screen edge. Keep the wrapper inside the chat column and cap it at its width, so a wide table scrolls in place instead. Assisted-by: llama-ui (ggml-org/Qwen3.8-Flash-Next:IQ4_NL) * test : show a table-only model answer in the chat message story Adds a story with an answer that is mostly a table, so the table layout can be checked at a phone width. Assisted-by: llama-ui (ggml-org/Qwen3.8-Flash-Next:IQ4_NL) * ui : indent markdown lists by 2rem 1.5rem put the marker too close to the item text. Assisted-by: llama-ui (ggml-org/Qwen3.8-Flash-Next:IQ4_NL)

…rg#30607) * qwen3 : bidirectional attention and dense_2 output projection Assisted-by: Grok * convert : read use_bidirectional_attention in base Assisted-by: Grok * Update conversion/base.py Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> --------- Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* cuda: cap Stream-K blocks at the number of k-blocks Stream-K normally launches one CUDA block per SM. For small matrices, this can exceed the total number of output-tile/k-block pairs, leaving blocks with no work and adding launch and fixup overhead. Cap the grid at that count to remove these excess blocks without changing the useful rounded K ranges or numerical results. Assisted-by: Codex * cuda: cap Stream-K blocks at the number of k iterations Cap the grid at the number of K_vram iterations across all output tiles, rounding up partial iterations. This removes extra CUDA blocks left by the k-block cap. Partial iterations can change FP32 summation grouping. MUL_MAT and MUL_MAT_ID backend tests pass on RTX 5090. * Update ggml/src/ggml-cuda/mmq.cuh --------- Co-authored-by: Johannes Gäßler <johannesg@5d6.de>

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

* ci: add quantize to big-endian tests Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * ci: remove incoherent data types from testing Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * tests: add verified hashes Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * tests: rename Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * tests: add result table Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * tests: swap hash and result column, add colours Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * tests: switch to perplexity test Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * tests: update output table Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> --------- Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

…rn` (ggml-org#25273)

…org#30795) The fused moe_reduce kernel computes dst = sum(experts[e] * weights[e]) in a single dispatch, reading experts and weights while writing dst. The graph allocator computes tensor liveness on the unfused graph, where the inputs are fully consumed by earlier nodes of the fused group, so it can legally recycle the input memory as dst. In the fused kernel this becomes a threadgroup-vs-threadgroup race that produces rare non-deterministic wrong results (observed with test-llama-archs -a kolibri on the meta backend). Reject the fusion when dst overlaps either input, same guard as the mul_mat_add fusion. The unfused MUL + ADD path is always correct. Assisted-by: pi:llama.cpp/ggml-org/GLM-5.3-Flash-GGUF (2x DGX Spark (RDMA, TP=2))

…0637)

* server : fix swa checkpoint invalidation logic Assisted-by: Claude * add testcase that fails with > n_past, but works with > n_prefix

* jinja : implement `unique` filter * cont : inline key_of * cont : fix alignment

* opencl: support i32 GET_ROWS * opencl: support i32 and f32 copy conversions

…#30741) * opencl: add cooperative-k dp4a GEMM for q4_0 * opencl: add cooperative-k dp4a GEMM for q4_k * opencl: add cooperative-k dp4a GEMM for q6_k * opencl: fix alignment

* vocab : support Carbon-14B HybridDNA tokenizer Carbon-14B auto-tags untagged DNA runs and uses N-padded tail tokens (tail=pad_n). Also strip whitespace inside <dna> tags and handle a stray </dna> like the HF tokenizer. * convert : fix special tokens and merges for older Carbon models Carbon-500M/3B/8B don't register the Qwen added tokens and ship no tokenizer.json, so take the token types, merges and special ids from the base tokenizer. * Update conversion/base.py Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update src/llama-vocab.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update src/llama-vocab.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update src/llama-vocab.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update conversion/base.py Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update conversion/base.py Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update conversion/base.py Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * address review comments * Update src/llama-vocab.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> --------- Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

…he container (ggml-org#30806) In the attachment preview, SyntaxHighlightedCode's wrapper (overflow-auto) was not the element that ended up owning horizontal overflow: the inner code box kept its content clipped at the wrapper's width, so a single line longer than the preview pane was cut off with no scrollbar (scrollWidth == clientWidth while the code's own content measured tens of thousands of px). Give the pre/code boxes their intrinsic width (width: max-content, min-width: 100%) so long-line overflow propagates to the scrolling wrapper. Verified in the running WebUI: with a 6000-char line, wrapper scrollWidth goes from 990 (clipped, unscrollable) to 50449 with a working horizontal scrollbar; short content still fills the container exactly as before (min-width: 100%). Fixes ggml-org#30796

…rg#28277) * init * wip * wire it up * make it clear that it's experimental * add props.connect_code * nits * fix windows build * server: do not advertise a connect code without the tunnel --connect-code on its own had no effect other than being served in /props, so the Web UI showed a remote access code that no tunnel was serving. Clear it when --connect is off, and say so once at startup. The teardown comment claimed that terminate() closes the child stdout. It does not: the log thread leaves fgets() only once every writer of that pipe is gone, so a descendant of llama-connect would block the join forever. Describe the real contract instead. Also restore the --reasoning-preserve default in the generated docs. * Update tools/connect/CMakeLists.txt Co-authored-by: Adrien Gallouët <angt@huggingface.co> * server: dial llama-connect on the bound port and a reachable host Pass ctx_http.port so --port 0 and the GCP override work, pick the dial host from --host (wildcards map to loopback, IPv6 is bracketed) and refuse --connect when only unix sockets are bound. --------- Co-authored-by: Pascal <admin@serveurperso.com> Co-authored-by: Adrien Gallouët <angt@huggingface.co>

* ci, docs: add WASI CI and docs * accept review feedback

Co-authored-by: Sam <sam@garagellm.tail7dd858.ts.net>

SearchInput already renders its own clear button. Hide the browser's native search cancel button to avoid showing both controls. Fixes ggml-org#28627 Signed-off-by: aiqubits <aiqubits@hotmail.com>

…micolons (ggml-org#30271)

…gml-org#30854) * hex-build: bump toolchain, enable v85 * hex-build: bump toolchain to v0.8 in the CI and build scripts * hex-build: add a little python script for swiv needed on some iot devices * hex-build: fix flake8 warnings * hex-arch: update version cap to v85 * hex-inspect: flexible TOOLS_ROOT * hex-build: update to snake_case naming * hex-build: update leftover old toolchain refs * hex-target: update build and run scripts to support ubuntu with adb transport

…gml-org#30818) * tests : make MoE router selection independent of the backend noise The router scores are in the range [0, 1), so a random draw can leave the last expert (or expert group) still selected and the first one dropped tied within the noise between the backends. They then route a token to different experts and the logits differ by 4e-3 instead of the usual 5e-7. Beam is the only arch here with a real top-k (8 experts, 2 used) and a group top-k (4 groups, 2 used), so it is the only one that hits this: 2 of 20 seeds failed on an M2 Ultra, the other MoE archs select 2 of 2 and cannot tie. Fill the selection bias with a ramp of step 2 to give every expert a score range of its own, which turns the top-k into a function of the expert index alone. The bias affects only the selection, the weights still come from the unbiased probs, so the MoE output keeps its magnitude. Beam now matches the CPU reference within 4e-7 for all of 160 seeds. Assisted-by: pi:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB) * tests : tell the CPU and the GPU of an Apple SoC apart in the device table Both devices report the same description ("Apple M2 Ultra" here), so the device column of the test table had two indistinguishable rows and a failing row did not say whether the Metal device or the host CPU produced it. Add the device name, which is also what -b takes, unless the description already is the name. Assisted-by: pi:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB)

…gexes (ggml-org#30845)

* ui : replace the sidebar search mode with a search dialog Search opens a spotlight dialog (Cmd+K or the Search entry under New chat in the sidebar) with keyboard navigation over recent chats and sidebar actions. Phones keep the search page. Sidebar actions show their shortcuts, with new ones for Settings (Cmd+,) and Models (Shift+Cmd+M). Assisted-by: Claude (Opus 5.5 Medium) * ui : ignore search dialog keys while an IME is composing Enter confirms an IME candidate and the arrow keys move through the candidates, so the search field leaves them to the IME instead of opening the highlighted result or moving the highlight. Assisted-by: Claude (Opus 5.5 Medium)

…ssage" action (ggml-org#31214) * ui: keep new chat draft when adding a system prompt Fixes ggml-org#24338 Assisted-by: Claude (Opus 5.5 Medium) * ui: remove unused initialMessage from chat form Assisted-by: Claude (Opus 5.5 Medium)

…#31237) * ui: support OGG audio attachments by converting them to WAV Fixes ggml-org#24164 Assisted-by: Claude (Opus 5.5 Medium) * ui: convert all non WAV/MP3 audio uploads to WAV Assisted-by: Claude (Opus 5.5 Medium)

…g#31248) * ui : show save and reset settings buttons only when relevant Assisted-by: Claude (Opus 5.5 Medium) * ui : move reset settings button next to reload app, pad settings caption Assisted-by: Claude (Opus 5.5 Medium)

…ors (ggml-org#30557) * chat : strip trailing whitespace from parsed reasoning and content Assisted-by: Claude Fable 5.1 * chat : stop qwen3-coder reasoning and content before template separators Assisted-by: Claude * tests : remove the Qwen parser case from the chat session tests Assisted-by: Claude * common : handle partial prefix until match at end of generation --------- Co-authored-by: Zeshawn Zahid <zzshawnn@gmail.com> Co-authored-by: Alde Rojas <hello@alde.dev>

…0283) * fix: clamp the A prefetch to end_k in the Vulkan coopmat1 int8 matmul, like the B prefetch * fix: zero the A scale for k steps past end_k in the Vulkan coopmat1 int8 matmul instead of relying on the clamped block

Assisted-by: Codex

…g#31273) * ui: build the reasoning menu from the template effort levels When the chat template reports effort levels, the menu offers Default, Off and those levels, and a pick is sent as reasoning_effort. A template that reports none keeps the token budget menu. Picking a model other than the one that produced the last answer resets the conversation effort to Default, and an effort the current menu lacks resolves to Default. * ui: wait for a cold router model before building the request A router serves the props of a loaded model only, so a request sent to a cold model was built without its template effort levels and fell back to a token budget. The request now waits for the model to load, joining a load already in flight, and a stop during the wait sends nothing. Status waiters hold a list per model so a load and the requests that wait on it each get theirs, and a props request in flight is shared with a second caller instead of answering null.

* ggml : bump version to 0.28.0 * authors : update

* server : add tool calls to /v1/realtime Assisted-by: Claude Opus 5.5 * refactor --------- Co-authored-by: Xuan Son Nguyen <son@huggingface.co>

* mtmd : add LFM3-VL and LFM3-Omni video input Co-authored-by: Xuan Son Nguyen <son@huggingface.co> Assisted-by: Claude Opus 5.5 * mtmd : reserve the soundtrack command to avoid a GCC 11 warning --------- Co-authored-by: Xuan Son Nguyen <son@huggingface.co>

* llama.cpp : bump version to 0.8.0 * authors : update

* gguf-py: fix np_roundf dropping the sign * Update requirements/requirements-convert_legacy_llama.txt Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> * requirements.txt * Update tools/server/tests/requirements.txt Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> --------- Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* sycl: work-group argmax for short TOP_K rows Used angelX to optimize SYCL/drama 3 coordination, results are as follows Rows with ncols <= max work-group size: one work-item per column, k rounds of work-group argmax. Ties take the lower column, NaN is never picked. Longer rows keep the existing paths. Arc Pro B70, oneAPI 2026.1, test-backend-ops perf (existing short-row cases), base 9b56ccf vs patch, median ratio of 6 rotating A/B/A rounds: | ncols | nrows | k | base us | patch us | speedup | |---|---|---|---|---|---| | 16 | 1024 | 1 | 19.97 | 2.88 | 6.95x | | 16 | 1024 | 2 | 23.83 | 2.90 | 8.25x | | 16 | 1024 | 8 | 66.12 | 4.66 | 14.21x | | 16 | 8192 | 1 | 110.42 | 5.96 | 18.51x | | 16 | 8192 | 2 | 130.21 | 8.82 | 14.77x | | 16 | 8192 | 8 | 420.62 | 27.27 | 15.42x | | 128 | 1024 | 1 | 20.49 | 2.94 | 6.97x | | 128 | 1024 | 2 | 25.89 | 3.45 | 7.52x | | 128 | 1024 | 8 | 101.67 | 8.95 | 11.36x | | 128 | 8192 | 1 | 112.19 | 13.29 | 8.44x | | 128 | 8192 | 2 | 141.18 | 20.60 | 6.85x | | 128 | 8192 | 8 | 643.21 | 67.60 | 9.51x | | 1024 | 1024 | 1 | 17.32 | 14.98 | 1.16x | | 1024 | 1024 | 2 | 28.77 | 24.02 | 1.20x | | 1024 | 1024 | 8 | 151.87 | 77.88 | 1.95x | | 1024 | 8192 | 1 | 156.49 | 138.78 | 1.13x | | 1024 | 8192 | 2 | 201.47 | 209.18 | 0.96x | | 1024 | 8192 | 8 | 1006.25 | 632.60 | 1.59x | The 1024 x 8192, k=2 case is 3.7% slower. Correctness: test-backend-ops -o TOP_K 533/533 and -o ARGSORT 98/98 on the patch, again on the rebase onto 3f03b17. Full test-backend-ops aborts on base and patch alike at the non-F32 SWIGLU_OAI assert (element_wise.cpp:1187, see ggml-org#30857). With only those cases excluded: 20145/20173 on base and patch. One CPY f16->q2_0 [256,2,3,4] failure was seen once on the patch, and 3/3 re-runs pass on base and patch. All other patch-run failures also fail on base. Assisted-by: Claude Opus 5.5 * made changes as suggested Assisted-by: Claude Opus 5.5

* codeowners : refresh teams and maintainer PRs link Update the CODEOWNERS team legend with the current `ggml-org` team memberships, and extend the maintainer PRs link in the README to cover all current members of the `maintainers` team. Assisted-by: pi:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB) * codeowners : list remaining backend sub-teams Add the `ggml-*` sub-teams that were missing from the team legend at the top of the file: base, cpu, et, hip, openvino and riscv. The legend now covers every sub-team of the `ggml` and `llama` maintainer groups. No ownership rules change - only the comment block is extended. Assisted-by: pi:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB) * codeowners : use ggml-hip team for HIP backend Route `/ggml/src/ggml-hip/` to `ggml-org/ggml-hip` instead of a single collaborator, so review requests reach the whole team. Assisted-by: pi:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB) * codeowners : add llama-examples team List the newly created `llama-examples` team in the team legend at the top of the file. Assisted-by: pi:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB) * codeowners : use llama-examples team for examples Collapse the per-directory rules that only listed a single collaborator into a single `/examples/` rule owned by `ggml-org/llama-examples`. Directories owned by other collaborators keep dedicated rules, since the last matching pattern takes precedence in CODEOWNERS. Assisted-by: pi:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB) * cont : update zdnn * codeowners : use llama-base team for src Route `/src/` to `ggml-org/llama-base` and list the new team in the legend at the top of the file. Rules for individual `src` files are kept, since the last matching pattern takes precedence; `/src/models/` drops its individual owner, which is already covered by `llama-models`. Assisted-by: pi:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB)

* llama : fix DSV4 compressor padding for simple batches Count actual tokens per sequence when padding compressor writes. Add a graph reuse regression test for odd-sized DeepSeek V4 and V4.1 batches. Assisted-by: Codex * tests : cover multiple sequences in DSV4 padding regression Exercise one and two sequences at different positions while keeping a fixed ubatch shape. Assisted-by: Codex * llama : remove simple batch splitting for DSV4 Use the existing sequence splitting to keep n_seq_tokens consistent with the compressor padding calculation. Extend the graph reuse regression to 6-token batches. Assisted-by: Codex

…8141) * mxfp4 sve 256 impl completed * working with pretty good perf improvement * review comments addressed * review comments for SVE 256 check * trailing white spaces removed --------- Co-authored-by: Anubhav Srivastava <anubhav.srivastava@fujitsu.com>

…ml-org#27472) * q6_k GEMM working * cleanup * review comments * review comment addressed - removed pg16_8 * incorporated the review comments - replaced switch with if and other points * marking q8_k_blocklen UNUSEDto disable warning for non-i8mm cases

100 upstream commits, 797de63..1f4c2ee (chunk 1 of 3). Carried ggml-org#30573 landed under its own number; upstream ggml-org#30823 makes the same qwen35 2D ssm_out reshape as our ggml-org#28265 carry. Both are dropped from the carried list. Conflicts: - ggml/src/ggml-cuda/fattn-vec.cuh: took upstream; the whole fork delta was the ggml-org#30573 cherry-pick, which landed upstream. - src/models/qwen35.cpp, src/models/qwen35moe.cpp: took upstream ggml-org#30823's 2D final reshape (same change as our ggml-org#28265); d2t draft vocab (ggml-org#29143) and the local embedded d2t trim are untouched. - tools/server/server-common.cpp: kept our server_prompt_checkpoint_due() (local periodic checkpoints) next to upstream's get_server_exec_path() (ggml-org#28277). - tools/server/server-context.cpp: kept our ggml-org#25592 pos_stale invalidation and added upstream ggml-org#30666's n_tokens > n_prefix test to the same erase check. Same set for exact (recurrent) state; stricter for SWA range checkpoints, which is the upstream fix. - tools/server/tests/unit/test_completion.py: kept our ggml-org#28992 prompt-cache tests and upstream's ggml-org#30666 checkpoint tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…erged upstream) (local) ggml-org#30573 landed under its own number. ggml-org#28265 is superseded by upstream ggml-org#30823, which makes the same 2D reshape of the qwen35/qwen35moe ssm_out input. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

100 upstream commits, 1f4c2ee..1a97e38 (chunk 2 of 3). Upstream merged ggml-org#29948 (fused FFN gate/up + GLU in MMQ, interleaved rows), the successor to our carried ggml-org#28702, which upstream closed. ggml-org#28702 is dropped in favour of ggml-org#29948; the meta-backend part of the local gate-up alloc dep stays, because upstream's meta backend still has no graph_optimize. Conflicts: - ggml/src/ggml-cuda/mmq.cu, plus auto-merged cp-async.cuh, mmvq.cu and template-instances/{generate_cu_files.py,mmq-instance-q4_k.cu}: took upstream; the whole fork delta was ggml-org#28702. - ggml/src/ggml-cuda/mmq.cuh: took upstream (ggml-org#29948, ggml-org#30837) and re-applied ggml-org#30493 (int2 activation tile copy) unchanged; the fused ggml-org#29948 path uses the same tile copy. - ggml/src/ggml-cuda/ggml-cuda.cu, common.cuh: dropped the ggml-org#28702 matcher/dispatch/helpers and the CUDA graph_optimize half of the local alloc dep (upstream matches both gate/up orders); kept ggml-org#29187 GDN alpha/beta fusion (default off), ggml-org#26001 + local chunked GDN, and ggml-org#29768 graph recapture, ported to ggml-org#30804's chunked graphs (graph->instances); the replayed flag is still cleared after a direct evaluation, as before. - ggml/src/ggml-cuda/unary.cu: took upstream's softplus (ggml-org#31086); the ggml-org#29187 helper in unary.cuh is unchanged. - src/llama-model.cpp, src/llama-model.h: kept supports_backend_sampling() (local mirrored d2t sampling) next to upstream's no_alloc() (ggml-org#31141). - tests/CMakeLists.txt: took upstream's ggml-org#30880 layout and labels; re-registered test-mtp-ubatch-sync, test-ngram-mod and test-backend-sampler-tensor. - tests/test-backend-ops.cpp: dropped the ggml-org#28702 tests (MUL_MAT_Q_GATE_UP_SWIGLU_DOWN, broadcast_weights) for upstream's ggml-org#29948 cases; kept the GDN chunked/snapshot/strided_v and AB-fusion tests. - tests/test-llama-archs.cpp: took upstream's ggml-org#30880 kv_writer, kept the mtp fixture flag for ggml-org#26827's test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

ggml-org#29948) (local) Upstream closed ggml-org#28702 and merged its successor ggml-org#29948. The local alloc-dep entry now covers only the meta-backend part, for ggml-org#29948 under -sm tensor. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

invaliddayta and others added 30 commits October 19, 2026 13:25

ggml-cpu: vectorize ggml_v_silu and ggml_v_expf (ggml-org#30791)

42b7003

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

CUDA: fix warning for nonexistent extra semicolon (ggml-org#30774)

4866db3

cann : refactor #ifdef blocks to avoid unreachable code after `retu…

f3fb2fc

…rn` (ggml-org#25273)

moe-cache : replace MoE LRU cache with a 50/50 SLRU cache (ggml-org#3…

33d6135

…0637)

vendor : update cpp-httplib to 0.62.0 (ggml-org#30785)

7951f60

server : fix swa checkpoint invalidation logic (ggml-org#30666)

f174553

* server : fix swa checkpoint invalidation logic Assisted-by: Claude * add testcase that fails with > n_past, but works with > n_prefix

jinja : implement unique filter (ggml-org#30819)

0ce8cd1

* jinja : implement `unique` filter * cont : inline key_of * cont : fix alignment

opencl: add get_rows for i32, cpy for i32 and f32 (ggml-org#30756)

6fa4f6c

* opencl: support i32 GET_ROWS * opencl: support i32 and f32 copy conversions

opencl: add cooperative-k dp4a gemm for q4_0, q4_k and q6_k (ggml-org…

1b29621

…#30741) * opencl: add cooperative-k dp4a GEMM for q4_0 * opencl: add cooperative-k dp4a GEMM for q4_k * opencl: add cooperative-k dp4a GEMM for q6_k * opencl: fix alignment

ci : re-enable type/lint-check for skills (ggml-org#30713)

3ffb629

ci, docs: add WASI CI and docs (ggml-org#30754)

921257f

* ci, docs: add WASI CI and docs * accept review feedback

mtmd : match the transformers Qwen-VL resize (ggml-org#30736)

f711c8f

sycl : fix get_rows on reordered quantized weights (ggml-org#30430)

3bb63cb

Co-authored-by: Sam <sam@garagellm.tail7dd858.ts.net>

ui: hide the native clear button in SearchInput (ggml-org#30719)

1d88cb0

SearchInput already renders its own clear button. Hide the browser's native search cancel button to avoid showing both controls. Fixes ggml-org#28627 Signed-off-by: aiqubits <aiqubits@hotmail.com>

sycl: fix compiler warnings - missing prototypes, dead code, extra se…

dd002b9

…micolons (ggml-org#30271)

vocab : add \p{M} to lookaheads in Tekken and GPT-4o pre-tokenizer re…

d8d9297

…gexes (ggml-org#30845)

allozaur and others added 27 commits October 28, 2026 08:53

cuda : add top-k MoE fusion for 384 experts (ggml-org#31288)

a7af149

Assisted-by: Codex

tests : adjust constants to reduce irrelevant warnings (ggml-org#31254)

f1a7dea

ggml : bump version to v.28.0 (ggml/1676)

ec941ee

* ggml : bump version to 0.28.0 * authors : update

sync : ggml

5d7793d

server : support tool calls in /v1/realtime (ggml-org#31287)

d1eb0de

* server : add tool calls to /v1/realtime Assisted-by: Claude Opus 5.5 * refactor --------- Co-authored-by: Xuan Son Nguyen <son@huggingface.co>

llama.cpp : bump version to 0.8.0 (ggml-org#31309)

8642541

* llama.cpp : bump version to 0.8.0 * authors : update

cuda: fuse MoE (MUL_MAT_ID) gate/up and GLU in MMQ (ggml-org#31188)

f5222f2

ci(s390x): switch to Ubuntu 26.04 runners (ggml-org#31317)

ff0d396

Merge upstream/master into fork (ggml-org ac8e169)

e058fc9

benjigill merged commit 39fc591 into master Oct 28, 2026

benjigill deleted the sync/upstream-2026-10-28 branch October 28, 2026 21:40

benjigill mentioned this pull request Oct 28, 2026

scout : record dropped/adopted PRs from the 2026-10-28 sync #28

Open

Navigation Menu

Repository navigation

Sync upstream master (2026-10-28, ggml-org ac8e1694d) - #27

Sync upstream master (2026-10-28, ggml-org ac8e1694d)#27
benjigill merged 257 commits into
masterfrom
sync/upstream-2026-10-28

benjigill commented Oct 28, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

Conversation

benjigill commented Oct 28, 2026

Dropped carried patches

Conflicts and resolutions

Checklist (Linux box)

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants