v0.4.9

Changelog

• Updated the llama.cpp base from c06f84160 to d327bf1f0 and ggml to 0.27.0. Inherited changes include Flint loading/conversion and Jinja hexadecimal escapes, Kolibri-1, SenseNova U1, Nemotron embedding/vision updates, Granite Switch and H2O-Lightning decision models, Soprano TTS and Voxtral realtime audio; multi-GPU MoE caching, LiLiCorr and probabilistic DFlash2 drafting, speculative logprobs, live draft state in disk slot saves, chat-session APIs, server TTS/realtime transcription, and backend, conversion, and Web UI updates. Bee's RPC protocol is now 9.0.0 because the upstream TOP_K_MASK insertion shifts fork operation numbers; matching Bee RPC builds are required.
• Moved structured KVarN mid-prompt checkpoints to the nonzero descriptor-group boundary at or before each user-message start (PR #186). Retained seam anchors sit less than one group before their user boundary; checkpoint spacing and per-slot FIFO limits still apply, so this is not a bound on total follow-up replay.
• Extended Bee's transactional slot restore to disk files, preparing target, owned draft, speculative continuation, and prompt/checkpoint metadata before publication. Malformed or incompatible appendices leave the live slot unchanged, shared draft KV is never restored independently, and legacy target-only files with an owned draft require full prompt reevaluation. Added llama_state_seq_prepare_file for file-backed restore preparation.
• Fixed historical prompt-cache selection and accounting. MTP checkpoints stay within the restored continuation boundary, RAM candidates without a usable checkpoint must fit recurrent rollback limits, and cached/planned token counts and cumulative cache metrics follow the actual committed boundary after descriptor-aligned rollback. Ordinary body-backed historical truncation and explicit reduced-tail coverage remain unchanged.
• Fixed Qwen4Exp QSA precision-tail masking. Sparse selection now applies to both cache-body and exact-tail attention, while the ordinary mask remains available when needed for tail planning. Added a numerical graph regression.

macOS:

• macOS Apple Silicon

Linux:

• Ubuntu x64 CPU
• Ubuntu arm64 CPU
• Ubuntu x64 CUDA 12.4
• Ubuntu x64 CUDA 13.3
• Ubuntu x64 Vulkan
• Ubuntu x64 ROCm 7.2
• Ubuntu x64 SYCL

Windows:

• Windows x64 CPU
• Windows x64 Vulkan
• Windows x64 SYCL
• Windows x64 CUDA 12.4 - DLLs
• Windows x64 CUDA 13.3 - DLLs
• Windows x64 HIP

Docker:

• CPU: docker pull ghcr.io/anbeeld/beellama.cpp:server-cpu-v0.4.9
• CUDA: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda-v0.4.9
• CUDA 12: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda12-v0.4.9
• CUDA 13.3: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda13-v0.4.9
• ROCm: docker pull ghcr.io/anbeeld/beellama.cpp:server-rocm-v0.4.9
• Vulkan: docker pull ghcr.io/anbeeld/beellama.cpp:server-vulkan-v0.4.9
• SYCL: docker pull ghcr.io/anbeeld/beellama.cpp:server-sycl-v0.4.9

Browse all container images

v0.4.9 Preview

Warning

Rolling preview build from v0.4.9 at b7902d18d6aa. Assets and moving Docker tags are replaced by newer successful branch builds. Last updated: 27/10/2026 01:32.

Changelog

• Updated the llama.cpp base from c06f84160 to d327bf1f0 and ggml to 0.27.0. Inherited changes include Flint loading/conversion and Jinja hexadecimal escapes, Kolibri-1, SenseNova U1, Nemotron embedding/vision updates, Granite Switch and H2O-Lightning decision models, Soprano TTS and Voxtral realtime audio; multi-GPU MoE caching, LiLiCorr and probabilistic DFlash2 drafting, speculative logprobs, live draft state in disk slot saves, chat-session APIs, server TTS/realtime transcription, and backend, conversion, and Web UI updates. Bee's RPC protocol is now 9.0.0 because the upstream TOP_K_MASK insertion shifts fork operation numbers; matching Bee RPC builds are required.
• Moved structured KVarN mid-prompt checkpoints to the nonzero descriptor-group boundary at or before each user-message start (PR #186). Retained seam anchors sit less than one group before their user boundary; checkpoint spacing and per-slot FIFO limits still apply, so this is not a bound on total follow-up replay.
• Extended Bee's transactional slot restore to disk files, preparing target, owned draft, speculative continuation, and prompt/checkpoint metadata before publication. Malformed or incompatible appendices leave the live slot unchanged, shared draft KV is never restored independently, and legacy target-only files with an owned draft require full prompt reevaluation. Added llama_state_seq_prepare_file for file-backed restore preparation.
• Fixed historical prompt-cache selection and accounting. MTP checkpoints stay within the restored continuation boundary, RAM candidates without a usable checkpoint must fit recurrent rollback limits, and cached/planned token counts and cumulative cache metrics follow the actual committed boundary after descriptor-aligned rollback. Ordinary body-backed historical truncation and explicit reduced-tail coverage remain unchanged.
• Fixed Qwen4Exp QSA precision-tail masking. Sparse selection now applies to both cache-body and exact-tail attention, while the ordinary mask remains available when needed for tail planning. Added a numerical graph regression.

macOS:

• macOS Apple Silicon

Linux:

• Ubuntu x64 CPU
• Ubuntu arm64 CPU
• Ubuntu x64 CUDA 12.4
• Ubuntu x64 CUDA 13.3
• Ubuntu x64 Vulkan
• Ubuntu x64 ROCm 7.2
• Ubuntu x64 SYCL

Windows:

• Windows x64 CPU
• Windows x64 Vulkan
• Windows x64 SYCL
• Windows x64 CUDA 12.4 - DLLs
• Windows x64 CUDA 13.3 - DLLs
• Windows x64 HIP

Docker:

• CPU: docker pull ghcr.io/anbeeld/beellama.cpp:server-cpu-preview-v0.4.9
• CUDA: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda-preview-v0.4.9
• CUDA 12: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda12-preview-v0.4.9
• CUDA 13.3: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda13-preview-v0.4.9
• ROCm: docker pull ghcr.io/anbeeld/beellama.cpp:server-rocm-preview-v0.4.9
• Vulkan: docker pull ghcr.io/anbeeld/beellama.cpp:server-vulkan-preview-v0.4.9
• SYCL: docker pull ghcr.io/anbeeld/beellama.cpp:server-sycl-preview-v0.4.9

Browse all container images

v0.4.8

Changelog

• Updated the llama.cpp base from 465e49b9c to c06f84160 and ggml to 0.26.0. Inherited changes include GLM-5.3-Flash, Maple, HrmText, Ling 3.0 VL, and decision-model support; expanded MTP/DFlash, probabilistic drafting, sparse FlashAttention, multimodal embeddings, /v1/systemone, extended batch APIs, imatrix calibration, and backend/server/Web UI fixes. Session/sequence-state formats are now 11/4, the backend API is 3, and RPC is 7.0.0. Legacy mmap/mlock/direct-I/O switches are replaced by --load-mode.
• Added PrismML Bonsai 2 folded-weight support. Validated prism.hadamard.* metadata drives activation-side sign/rotation transforms and inverse token-embedding transforms, including Qwen3.5 MTP; graph validation rejects folded weights consumed without their required transform instead of silently producing incorrect output. Added group-128 PQ2_0 and PTQ1_0 weight formats with CPU and CUDA quantization, dequantization, row lookup, matrix-vector/matrix-matrix kernels, GGUF/Python type plumbing, and llama-quantize options. Legacy group-128 files mislabeled as Q2_0 receive migration guidance. Fixed non-MSVC block declarations and GCC metadata-reader linking, with C ABI and ternary round-trip regressions (issue #184).
• Added an AVX2 CPU vec-dot kernel for serialized group-64 Q2_0 weights against Q8_0, replacing the scalar x86 fallback for CPU expert layers. This weight format remains distinct from Bee's cache-facing GGML_TYPE_Q2_0S.
• Enabled Qwen4Exp CUDA tensor splitting with shared MTP sidecars. HyperConnection initialization stays in the first layer's graph split, shared drafts omit unused trunk tensors, draft mixer and projected-state shapes are corrected, and mirrored output projections and padded empty logits keep backend sampling compatible with multi-GPU placement.
• Fixed single-head tensor splits for Gemma KV caches. Whole-head ownership survives reshape, view, cache rotation, and output projection, and trailing host cache views close Meta subgraphs correctly, preventing tensor-split crashes. Added numerical and graph-boundary regressions.
• Improved multi-stream KVarN decode routing and stores. Native-attention admission uses each stream's query width rather than the flattened batch size, qualified one-row multi-stream SWA decode consumes compressed records directly on CPU and CUDA, and other backends or wider SWA query batches retain materialization. CUDA descriptors use stream-local rings and read indices with global stage addressing, in-flight tails resolve the correct staging slots, and independent headwide decode stores run in parallel after validating actual stream ownership, with a serial fallback for shared destinations. Regression coverage includes wrapped rings, exact tails, and hinted stores.
• Fixed self-contained KVarN state for per-sequence and single-sequence fixed stage layouts. Saves include every sealed compressed record independently of the F16 rollback window, preserving both fresh/cross-slot continuation and suffix rollback that reopens a completed group. The compact read-plan path and KVarN state format are unchanged; regressions cover restore into fresh or overwritten sequences, hard memory clears, and rollback replay.
• Preserved historical slot checkpoints across disk save/restore and server restarts. An optional versioned appendix stores target, owned draft, and speculative checkpoint state, retaining Bee's atomic restore transaction and cross-slot reuse instead of probing compatibility by mutating live draft state. Blob lengths and metadata are validated before admission, malformed or truncated appendices are ignored without accepting partial checkpoint lists, incomplete writes report failure, UTF-8 paths remain supported, and save/restore byte counts include the appendix. Added F16/KVarN hybrid and owned-MTP checkpoint regressions.
• Windows release builds now tolerate taskkill races when compiler children exit during shutdown, preserving the rolling CUDA build cache.

macOS:

• macOS Apple Silicon

Linux:

• Ubuntu x64 CPU
• Ubuntu arm64 CPU
• Ubuntu x64 CUDA 12.4
• Ubuntu x64 CUDA 13.3
• Ubuntu x64 Vulkan
• Ubuntu x64 ROCm 7.2
• Ubuntu x64 SYCL

Windows:

• Windows x64 CPU
• Windows x64 Vulkan
• Windows x64 SYCL
• Windows x64 CUDA 12.4 - DLLs
• Windows x64 CUDA 13.3 - DLLs
• Windows x64 HIP

Docker:

• CPU: docker pull ghcr.io/anbeeld/beellama.cpp:server-cpu-v0.4.8
• CUDA: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda-v0.4.8
• CUDA 12: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda12-v0.4.8
• CUDA 13.3: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda13-v0.4.8
• ROCm: docker pull ghcr.io/anbeeld/beellama.cpp:server-rocm-v0.4.8
• Vulkan: docker pull ghcr.io/anbeeld/beellama.cpp:server-vulkan-v0.4.8
• SYCL: docker pull ghcr.io/anbeeld/beellama.cpp:server-sycl-v0.4.8

Browse all container images

v0.4.8 Preview

Warning

Rolling preview build from v0.4.8 at d73c05590f90. Assets and moving Docker tags are replaced by newer successful branch builds. Last updated: 14/10/2026 14:54.

Changelog

• Updated the llama.cpp base from 465e49b9c to c06f84160 and ggml to 0.26.0. Inherited changes include GLM-5.3-Flash, Maple, HrmText, Ling 3.0 VL, and decision-model support; expanded MTP/DFlash, probabilistic drafting, sparse FlashAttention, multimodal embeddings, /v1/systemone, extended batch APIs, imatrix calibration, and backend/server/Web UI fixes. Session/sequence-state formats are now 11/4, the backend API is 3, and RPC is 7.0.0. Legacy mmap/mlock/direct-I/O switches are replaced by --load-mode.
• Added PrismML Bonsai 2 folded-weight support. Validated prism.hadamard.* metadata drives activation-side sign/rotation transforms and inverse token-embedding transforms, including Qwen3.5 MTP; graph validation rejects folded weights consumed without their required transform instead of silently producing incorrect output. Added group-128 PQ2_0 and PTQ1_0 weight formats with CPU and CUDA quantization, dequantization, row lookup, matrix-vector/matrix-matrix kernels, GGUF/Python type plumbing, and llama-quantize options. Legacy group-128 files mislabeled as Q2_0 receive migration guidance. Fixed non-MSVC block declarations and GCC metadata-reader linking, with C ABI and ternary round-trip regressions (issue #184).
• Added an AVX2 CPU vec-dot kernel for serialized group-64 Q2_0 weights against Q8_0, replacing the scalar x86 fallback for CPU expert layers. This weight format remains distinct from Bee's cache-facing GGML_TYPE_Q2_0S.
• Enabled Qwen4Exp CUDA tensor splitting with shared MTP sidecars. HyperConnection initialization stays in the first layer's graph split, shared drafts omit unused trunk tensors, draft mixer and projected-state shapes are corrected, and mirrored output projections and padded empty logits keep backend sampling compatible with multi-GPU placement.
• Fixed single-head tensor splits for Gemma KV caches. Whole-head ownership survives reshape, view, cache rotation, and output projection, and trailing host cache views close Meta subgraphs correctly, preventing tensor-split crashes. Added numerical and graph-boundary regressions.
• Improved multi-stream KVarN decode routing and stores. Native-attention admission uses each stream's query width rather than the flattened batch size, qualified one-row multi-stream SWA decode consumes compressed records directly on CPU and CUDA, and other backends or wider SWA query batches retain materialization. CUDA descriptors use stream-local rings and read indices with global stage addressing, in-flight tails resolve the correct staging slots, and independent headwide decode stores run in parallel after validating actual stream ownership, with a serial fallback for shared destinations. Regression coverage includes wrapped rings, exact tails, and hinted stores.
• Fixed self-contained KVarN state for per-sequence and single-sequence fixed stage layouts. Saves include every sealed compressed record independently of the F16 rollback window, preserving both fresh/cross-slot continuation and suffix rollback that reopens a completed group. The compact read-plan path and KVarN state format are unchanged; regressions cover restore into fresh or overwritten sequences, hard memory clears, and rollback replay.
• Preserved historical slot checkpoints across disk save/restore and server restarts. An optional versioned appendix stores target, owned draft, and speculative checkpoint state, retaining Bee's atomic restore transaction and cross-slot reuse instead of probing compatibility by mutating live draft state. Blob lengths and metadata are validated before admission, malformed or truncated appendices are ignored without accepting partial checkpoint lists, incomplete writes report failure, UTF-8 paths remain supported, and save/restore byte counts include the appendix. Added F16/KVarN hybrid and owned-MTP checkpoint regressions.
• Windows release builds now tolerate taskkill races when compiler children exit during shutdown, preserving the rolling CUDA build cache.

macOS:

• macOS Apple Silicon

Linux:

• Ubuntu x64 CPU
• Ubuntu arm64 CPU
• Ubuntu x64 CUDA 12.4
• Ubuntu x64 CUDA 13.3
• Ubuntu x64 Vulkan
• Ubuntu x64 ROCm 7.2
• Ubuntu x64 SYCL

Windows:

• Windows x64 CPU
• Windows x64 Vulkan
• Windows x64 SYCL
• Windows x64 CUDA 12.4 - DLLs
• Windows x64 CUDA 13.3 - DLLs
• Windows x64 HIP

Docker:

• CPU: docker pull ghcr.io/anbeeld/beellama.cpp:server-cpu-preview-v0.4.8
• CUDA: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda-preview-v0.4.8
• CUDA 12: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda12-preview-v0.4.8
• CUDA 13.3: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda13-preview-v0.4.8
• ROCm: docker pull ghcr.io/anbeeld/beellama.cpp:server-rocm-preview-v0.4.8
• Vulkan: docker pull ghcr.io/anbeeld/beellama.cpp:server-vulkan-preview-v0.4.8
• SYCL: docker pull ghcr.io/anbeeld/beellama.cpp:server-sycl-preview-v0.4.8

Browse all container images

v0.4.7

Changelog

• Added true rectangular KVarN storage and attention for 64-dimensional K/V heads, including 64x128 K records, 128x64 V records, WHT64 transforms, target and owned draft caches, full-context and iSWA routing, and F16/BF16 precision tails. Existing D128, D256, and D512 record layouts and state formats remain compatible, while unsupported backend placements continue to fail closed.
• Optimized CUDA D64 KVarN with block-parallel quantization, batched workspace staging and commit, grouped materialization, split portable/GQA attention, and generated split-MMA decode kernels. Decode consumes compressed records directly at every KV length; prompt batches above the native rotated-query limit use transient tiled materialization. Packed K2/K4 decoding now handles odd indirect and SWA offsets without slowing aligned pairs.
• Optimized CUDA D128 KVarN decode for low GQA with exact GQA2/GQA4 geometries, an eight-warp split-128 route with a measured shallow-context crossover, cooperative staging of complete packed records, and shared-word unpacking of adjacent 3-, 5-, and 6-bit values, raising occupancy and halving deep-context split/combine work.
• Improved HIP/ROCm KVarN correctness and performance. Qualified gfx1100 D256 prompt prefill uses F32-accumulator WMMA, larger prompt batches publish complete softmax metadata, and portable attention sizes scratch storage for the full head dimension. RDNA4 D256+ remains on the portable route until its F32 tiles qualify. HIP also gains wave32 top-k handling, compact MoE MMQ tiles, quantized vec-dot and dequant-matvec kernels, strict dequant controls, and a fix for the top-k radix upper boundary; the unsafe APU host-mapped placement is disabled.
• Added independent physical ubatch sizing for model-backed drafts with --spec-draft-ubatch-size (-ubd), defaulting to 128 or, for parallel DFlash/DSpark, the capacity needed for all slots' noise blocks. Explicit -ubd values remain unchanged; draft contexts inherit the target -b logical capacity without changing target -ub. Owned draft KVarN contexts also gain --spec-draft-kvarn-window-chunk, defaulting to 2048, so their transient prefill workspace can be bounded independently of the target context.
• Added shared Qwen4Exp MTP sidecars. They borrow the target embedding and output projection while retaining their own draft block and output mixer; legacy self-contained sidecars remain supported, and conversion maps the MTP mixer into the shared layout.
• Improved server prompt-cache and checkpoint restoration. Long unstructured prompts gain periodic stable anchors, retained cache bodies can be restored transactionally without unnecessary clearing and rebuilding, and response timings now separate slot selection, RAM-cache work, and checkpoint prepare, commit, and restore phases.
• Preserved input marking through chat-template tokenization so special-token text supplied by users or tools is treated as content rather than template control syntax. The same protection applies to text-only, multimodal, continuation, and token-counting paths, fixing output stalls caused by stray message tags.
• Fixed compact precision-tail updates when a batch reuses a tail slot. Only the globally last write is committed across tokens and sequence owners, avoiding duplicate-destination races on CPU and GPU backends.
• Fixed tensor-split speculative graph reuse after Meta context growth. Resetting the rotating graph context now recreates every reserved subgraph slot, preventing stale graph pointers when speculative topology changes. Attention planning uses the query-token dimension, and the original four-argument KVarN planner symbol remains available for binary compatibility.
• Fixed Qwen4Exp QSA serving on KVarN caches. Index-mirroring hybrid memories keep one attention row per physical cache cell, KVarN-only stage slots no longer reach the plain index cache, and state save/restore selects stage rows with the mapping the store actually uses.
• Enabled direct-record CUDA attention for qualified owned DFlash1/DFlash2 non-causal KVarN draft caches, including supported full-context, SWA, and exact-tail layouts. Admission checks the actual layer backend and final attention operation; unqualified shapes and backends retain materialization or fail closed. Multi-stream SWA remains materialized after a native target-token mismatch, while tested single-stream and unified multi-slot routes matched the materialized oracle. Persistent cache and checkpoint formats are unchanged.
• Made CUDA exact-tail attention materialize unsupported KVarN body layouts through the standard fallback instead of aborting, and made CUDA graph and dequant optimizations honor each device's controls rather than a single global decision.
• Fixed llama-perplexity KLD processing to honor --batch-size directly, so compressed-logit working memory scales with the requested batch instead of a vocabulary-derived row limit. Baseline and candidate comparisons should use matching --batch-size and --ubatch-size values.
• Fixed a first-decode abort from stale scheduler hash entries after graph reserve and tensor probing: inserting into the hash set now invalidates the reset state, so the next reset always clears it.

macOS:

• macOS Apple Silicon

Linux:

• Ubuntu x64 CPU
• Ubuntu arm64 CPU
• Ubuntu x64 CUDA 12.4
• Ubuntu x64 CUDA 13.3
• Ubuntu x64 Vulkan
• Ubuntu x64 ROCm 7.2
• Ubuntu x64 SYCL

Windows:

• Windows x64 CPU
• Windows x64 Vulkan
• Windows x64 SYCL
• Windows x64 CUDA 12.4 - DLLs
• Windows x64 CUDA 13.3 - DLLs
• Windows x64 HIP

Docker:

• CPU: docker pull ghcr.io/anbeeld/beellama.cpp:server-cpu-v0.4.7
• CUDA: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda-v0.4.7
• CUDA 12: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda12-v0.4.7
• CUDA 13.3: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda13-v0.4.7
• ROCm: docker pull ghcr.io/anbeeld/beellama.cpp:server-rocm-v0.4.7
• Vulkan: docker pull ghcr.io/anbeeld/beellama.cpp:server-vulkan-v0.4.7
• SYCL: docker pull ghcr.io/anbeeld/beellama.cpp:server-sycl-v0.4.7

Browse all container images

v0.4.6

Changelog

• Updated the llama.cpp base from 57291f264 (b10689) to 465e49b9c (b10830) and ggml to 0.23.0. Notable inherited changes include sparse FlashAttention for DeepSeek V4 and GLM, fused DFlash encoder injection, concurrent CUDA streams for multi-GPU splits, CUDA MoE reduction and FlashAttention improvements, faster KV-cell history lookup and state restoration, new Tencent Hunyuan 4 and Spark 2.5 model support, expanded DeepSeek V4 vision and Nemotron support, broader OpenCL/Metal/SYCL/Vulkan/Hexagon backend coverage, and upstream server, conversion, RPC, multimodal, and Web UI updates through that merge point.
• Restored property-safe CUDA graph reuse for tensor splits. Projected Meta graphs now keep stable nonzero identities for standard and KVarN execution, while snapshots of reachable tensor metadata, graph roots, use counts, and operation parameters force rebuilding when executable properties change. Consumer synchronization and generation-scoped projected storage preserve asynchronous tensor lifetimes instead of relying on the broad UID-zero capture suppression introduced with sharded precision tails.
• Restored the default CUDA KVarN prefill materialization window to 65,536 tokens, avoiding an additional floating-point partial-softmax merge through 64K contexts. Memory-constrained concurrent workloads can select a smaller transient window with GGML_KVARN_WINDOW_CHUNK without changing persistent cache capacity.

macOS:

• macOS Apple Silicon

Linux:

• Ubuntu x64 CPU
• Ubuntu arm64 CPU
• Ubuntu x64 CUDA 12.4
• Ubuntu x64 CUDA 13.3
• Ubuntu x64 Vulkan
• Ubuntu x64 ROCm 7.2
• Ubuntu x64 SYCL

Windows:

• Windows x64 CPU
• Windows x64 Vulkan
• Windows x64 SYCL
• Windows x64 CUDA 12.4 - DLLs
• Windows x64 CUDA 13.3 - DLLs
• Windows x64 HIP

Docker:

• CPU: docker pull ghcr.io/anbeeld/beellama.cpp:server-cpu-v0.4.6
• CUDA: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda-v0.4.6
• CUDA 12: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda12-v0.4.6
• CUDA 13.3: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda13-v0.4.6
• ROCm: docker pull ghcr.io/anbeeld/beellama.cpp:server-rocm-v0.4.6
• Vulkan: docker pull ghcr.io/anbeeld/beellama.cpp:server-vulkan-v0.4.6
• SYCL: docker pull ghcr.io/anbeeld/beellama.cpp:server-sycl-v0.4.6

Browse all container images

v0.4.5

Changelog

• Updated the llama.cpp base from 6fdd0ac89 to 57291f264 (b10689). Notable inherited changes include Qwen3.8 Flash Next (qwen4exp) target support with HyperConnections, QSA sparse attention, PLE n-gram embeddings, GGUF metadata and tensor mappings, conversion, multi-GPU placement, and graph lifecycle integration, plus --kv-unified-per-slot context limits, OpenVINO 2026.3.1, and upstream server, Web UI, conversion, model, and backend updates through that merge point. The public lazy-loading API now uses llama_lazy_mode and llama_model_params::lazy_mode instead of llama_tensor_read_lazy and tensor_read_lazy.
• Added owned draft KVarN caches through --spec-draft-type-k/v for draft-simple, EAGLE3, audited Qwen MTP (Qwen3.5/Qwen3.6 dense and MoE), DFlash1/DFlash2, and non-MLA DSpark. All six KVarN widths are selectable with the intrinsic 128-token exact suffix and no draft-tail option, target and draft settings stay isolated, capability checks use the actual MTP cache layers, DSV4/MLA DSpark fails closed because its latent cache is not dense K/V, and unsafe draft-memory fit probes fail with guidance to use -fit off rather than silently omit draft allocations. DFlash attention semantics resolve from the draft model's metadata before context construction; non-causal blocks use materialized K/V with the validated native exact-tail merge, avoiding a context-by-ubatch F32 score matrix.
• Extended KVarN to shared Gemma 4 MTP and multi-stream SWA target caches. Gemma 4 MTP reads the target cache's KVarN records and exact-tail rows (configure target --cache-type-k/v kvarn*; explicit draft selections fail closed), the position-addressed SWA ring encodes an independent KV stream per parallel slot with per-stream staging, records, and store/materialization routing on CPU, CUDA, and Vulkan, multi-stream SWA uses materialized attention while single-stream keeps native attention, and server checkpoint, restore, and rollback no longer treat shared-cache MTP drafts as owning draft state.
• Added standalone Qwen3.8 MTP sidecars. The draft loader follows the configured draft path, device, and offload settings; the Qwen4Exp MTP graph consumes target hidden states and applies the output HyperConnection mixer, and the converter exports complete sidecars.
• Extended standard quantized caches, KVarN, and F16/BF16 precision tails to Qwen3.8 QSA. Sparse selection composes with generic attention, native KVarN routing, and tail handling, with a validated mirrored index-cache slot layout. Prompt checkpoints preserve transactional hybrid attention/index state, rebuild mirrored precision-tail slots, and restore PLE recurrent rows into the selected destination.
• Fixed Windows CUDA tensor-parallel speculative decoding for MTP, DFlash1, and DFlash2, covering meta-graph headroom, GQA head partitions, mirrored softmax inputs, CPU scheduler boundaries, Windows graph-projection stack size, DFlash1 backend-sampling fallback, and DFlash2 vocabulary-logit gathering.
• Optimized CUDA KVarN decode and speculative verification: wider descriptor reduction with division-free single-stream scan, masked multi-query batches through eight rows with skipped empty splits and K/V reuse across query tiles, occupancy-based 64/128-token geometry, packed fragment loads, half-width quantization axes, and chunked windowed materialization. Fast tiles use padded MMA query/probability storage, record-boundary checks, synchronized shared-memory reductions, and query-count-independent split partitioning. Unsupported shapes retain descriptor-native MMA fallback, and GGML_KVARN_DEBUG_ROUTES reports the selected wide MMA path (#132).
• Improved KVarN structured cache allocation. Single-slot decode carries its planned stage assignment into cache mutation instead of rescanning the full context per token, and general stage-liveness scans use a flat sequence/group matrix instead of per-record tree allocations. Unified-cache planning reserves complete monotonic physical segments for interleaved sequences, preventing record-tile aliasing.
• Fixed historical KVarN SWA checkpoint restoration: partial SWA checkpoints own their ring records so later prefill cannot evict live anchors, incomplete legacy SWA partial frames and shared-stream ring replacement fail closed, and repeated historical-prefix reuse no longer reprocesses the whole prefix (issue #140).
• Fixed DFlash image position validation and SWA capacity (issue #140). Overlapping image positions and the logical gaps after them are preserved for feature injection. Draft generation and rollback use the target's next position rather than the image-expanded token count, preventing the post-image draft-acceptance and decode-speed collapse. Owned, unshared DFlash SWA caches start compact and grow on demand while preserving state, up to the full-context limit, so image feature rows can exceed their logical position extent without forcing full-capacity startup allocation. This applies to ordinary and KVarN caches; explicit --swa-full remains available.
• Reset per-sequence MTP hidden-state carry-over before processing a prompt from position zero, while preserving restored carry for nonzero continuations and the final prefill state needed by the first draft.
• Corrected compact-tail and KVarN suffix-rollback capability reporting so legal speculative rollback no longer triggers unnecessary checkpoints; tail-state serialization scans once per stream, MTP draft rollback follows the draft token axis with widened-removal and full-clear recovery, KV-tail placement follows the realized cache device, and projector thread counts reach every compatible scheduler backend.
• Declined the KV precision tail when no device advertises a native KV-tail attention entry point: the context warns and resolves the request to zero instead of paying the several-times-slower generic tail route, and LLAMA_KV_TAIL_ALLOW_GENERIC=1 keeps it anyway.
• Fixed Metal compilation for standard Q2 cache quantization and explicit F16/BF16 conversions; embedded Metal assembly now carries a content digest so sccache cannot serve stale kernels.
• Added native Vulkan FlashAttention decoding for standard q2_0, q2_1, q3_0, q3_1, q6_0, and q6_1 K/V caches, removing the CPU fallback that collapsed decode throughput.
• Updated release and validation tooling: ROCm runtime images bypass registry build-cache import/export, shared CUDA KVarN kernels are deduplicated with a binary-equivalence verification script, and regression coverage now exercises draft-cache validation, Qwen3.8 MTP conversion and state reuse, DFlash image/SWA growth, and historical checkpoints.

macOS:

• macOS Apple Silicon

Linux:

• Ubuntu x64 CPU
• Ubuntu arm64 CPU
• Ubuntu x64 CUDA 12.4
• Ubuntu x64 CUDA 13.3
• Ubuntu x64 Vulkan
• Ubuntu x64 ROCm 7.2
• Ubuntu x64 SYCL

Windows:

• Windows x64 CPU
• Windows x64 Vulkan
• Windows x64 SYCL
• Windows x64 CUDA 12.4 - DLLs
• Windows x64 CUDA 13.3 - DLLs
• Windows x64 HIP

Docker:

• CPU: docker pull ghcr.io/anbeeld/beellama.cpp:server-cpu-v0.4.5
• CUDA: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda-v0.4.5
• CUDA 12: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda12-v0.4.5
• CUDA 13.3: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda13-v0.4.5
• ROCm: docker pull ghcr.io/anbeeld/beellama.cpp:server-rocm-v0.4.5
• Vulkan: docker pull ghcr.io/anbeeld/beellama.cpp:server-vulkan-v0.4.5
• SYCL: docker pull ghcr.io/anbeeld/beellama.cpp:server-sycl-v0.4.5

Browse all container images

v0.4.4

Changelog

• Updated the llama.cpp base from 74ce15741 to 6fdd0ac89 and ggml from 0.19.0 to 0.22.0. Notable inherited changes include Qwen3.8 DFlash2 with local convolution, candidate selection, M-RoPE, converter/GGUF support, speculative p_min/n_min, and wide-row CUDA top-k; Granite Switch/SWA, Muse Glimmer, Kimi K3, MiniMax Text, BailingMoE3, Nanbeige4.2-3B, Pocket TTS, expanded MTP, DFlash, and DSpark coverage, and multi-output backend sampling; media-aware server state, Web UI tools and navigation, stronger tool isolation, automatic iGPU-safe loading, Metal kernel splitting, Vulkan TQ2_0 and Lightning Indexer support, asynchronous RPC APIs, and Apple RDMA transport. Bee keeps adaptive draft-max on DFlash1 while DFlash2 uses its fixed trained block and selector confidence.
• Optimized native KVarN SWA attention on CPU and CUDA. Fully covered non-SWA windows now use native exact storage, and CUDA routing retains split geometry and device capabilities without model- or GPU-specific policy.
• Hardened KV and speculative state handling with transactional target/draft restore, selective checkpoint validation, KVarN stage ownership and generation tracking, restored bulk-workspace routing, complete DSA+iSWA lifecycle forwarding, head-aligned multi-GPU placement, and correct partial sharded-buffer memset.
• Fixed MTP multi-ubatch synchronization and extended loop detection to pathological visible output while retaining force-close behavior for hidden reasoning loops.
• Updated release packaging to CUDA 13.3, source-built Web UI assets, non-development version metadata, fetched LLVM OpenMP on Windows, and matching HIP runtime DLLs. Release metadata now validates the source version, Windows package checks fail closed on required runtime files, preview and stable concurrency are isolated, and reruns reconcile stale assets.

macOS:

• macOS Apple Silicon

Linux:

• Ubuntu x64 CPU
• Ubuntu arm64 CPU
• Ubuntu x64 CUDA 12.4
• Ubuntu x64 CUDA 13.3
• Ubuntu x64 Vulkan
• Ubuntu x64 ROCm 7.2
• Ubuntu x64 SYCL

Windows:

• Windows x64 CPU
• Windows x64 Vulkan
• Windows x64 SYCL
• Windows x64 CUDA 12.4 - DLLs
• Windows x64 CUDA 13.3 - DLLs
• Windows x64 HIP

Docker:

• CPU: docker pull ghcr.io/anbeeld/beellama.cpp:server-cpu-v0.4.4
• CUDA: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda-v0.4.4
• CUDA 12: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda12-v0.4.4
• CUDA 13.3: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda13-v0.4.4
• ROCm: docker pull ghcr.io/anbeeld/beellama.cpp:server-rocm-v0.4.4
• Vulkan: docker pull ghcr.io/anbeeld/beellama.cpp:server-vulkan-v0.4.4
• SYCL: docker pull ghcr.io/anbeeld/beellama.cpp:server-sycl-v0.4.4

Browse all container images

v0.4.3

Changelog

• Updated the llama.cpp base through upstream commit 74ce15741. Notable inherited changes include Qwen3-TTS, DeepSeek V4 and DSpark, MTP support for GLM-4.7-Flash, GLM-5.2, Qwen3-Next, and DeepSeek V3.2, router LRU scheduling, initial Docker tool isolation, working-directory and filesystem tools in the server and Web UI, speculative metrics, and broad CUDA, Metal, Vulkan, SYCL, WebGPU, multimodal, conversion, and UI updates. ggml is now 0.19.0 and the RPC protocol is 5.0.1.
• Fixed HIP/ROCm KVarN routing. Precision tails remain on native attention instead of allocating a context-sized F16 materialization, unsupported AMD matrix shapes fall back to portable direct-record attention, and split/vector decode is selected only for supported AMD routes. Capability reporting, route telemetry, memory accounting, and Vulkan RoPE/offset checks now match the implemented kernels.
• Made KVarN and precision-tail memory fitting exact and bounded. The CLI, fit probes, and final context share one immutable tail request; invalid group specifications fail context creation; Vulkan uses backend workspace planners during fit; and final no-allocation validation restarts fit from pristine inputs or fails deterministically. Vulkan also gains a native one-softmax route for standard quantized bodies with F16/BF16 history and current segments.
• Reworked prompt-cache reuse across standard, recurrent, and KVarN state. Live and RAM candidates share one safe-prefix planner and transactional target/draft/speculative restore; failures leave live state unchanged. Self-contained sequence-selective checkpoints preserve precision tails and repeatable RAM restores. Durable KVarN reuse remains descriptor-aligned, while standard/recurrent batching and explicit GPU-fit behavior remain upstream-compatible. Unified KVarN slots now borrow shared capacity instead of using fixed stripes, and full state remaps live stage rows across slot counts; compact sparse reads and batched checkpoint transfers improve multi-slot --cache-ram prefill throughput.
• Fixed CUDA KVarN attention at both ends of the hardware range. Pre-Turing portable attention now passes architecture-neutral capability checks and exports the softmax metadata required to merge exact tails correctly. Split and vector decode share a once-per-device dynamic shared-memory opt-in, preventing configuration failures and extending the vector combine path beyond the default 48 KiB limit. KVarN MMA warning fixes keep fatal-warning builds clean.
• Restored status.args and status.preset in router /v1/models responses for upstream-compatible clients while continuing to strip sensitive options.

macOS:

• macOS Apple Silicon

Linux:

• Ubuntu x64 CPU
• Ubuntu arm64 CPU
• Ubuntu x64 CUDA 12.4
• Ubuntu x64 CUDA 13.1
• Ubuntu x64 Vulkan
• Ubuntu x64 ROCm 7.2
• Ubuntu x64 SYCL

Windows:

• Windows x64 CPU
• Windows x64 Vulkan
• Windows x64 SYCL
• Windows x64 CUDA 12.4 - DLLs
• Windows x64 CUDA 13.1 - DLLs
• Windows x64 HIP

Docker:

• CPU: docker pull ghcr.io/anbeeld/beellama.cpp:server-cpu-v0.4.3
• CUDA: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda-v0.4.3
• CUDA 12: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda12-v0.4.3
• CUDA 13: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda13-v0.4.3
• ROCm: docker pull ghcr.io/anbeeld/beellama.cpp:server-rocm-v0.4.3
• Vulkan: docker pull ghcr.io/anbeeld/beellama.cpp:server-vulkan-v0.4.3
• SYCL: docker pull ghcr.io/anbeeld/beellama.cpp:server-sycl-v0.4.3

Browse all container images

v0.4.2

Changelog

• Updated the llama.cpp base through upstream commit 6ba5ef247. Notable inherited changes include MiniMax-M3 sparse attention and vision, GLM-5.2-Vision and its indexer, Laguna, Nanbeige4.2, MiMo-V2.5 audio, DeepSeek4 fused operators, BitNet conversion, gpt-oss EAGLE3-v3, server MCP stdio, "reasoning_effort": "none", multi-end-sequence reasoning budgets, draft-sidecar discovery, broad backend and Web UI updates, and the unified --load-mode (-lm) selector. GGML_OP_COUNT is now 105 and the RPC protocol version is 4.0.5.
• Enabled bounded-memory native KVarN attention on portable-only CUDA devices, including the pre-Turing route needed by issue #112. A versioned backend capability record now distinguishes portable body-plus-F16/BF16-tail support from specialized MMA, split/vector decode, original-V support, and per-route query limits. Portable CUDA prompt batches consume rotated compressed records directly instead of re-entering classic full-score attention; Turing-and-newer route selection is unchanged. CI compiles explicit CUDA 12.4 Maxwell/Pascal/Volta targets and CUDA 13.1 current targets, while pre-Turing runtime qualification remains pending matching hardware.
• Added experimental --split-mode tensor support for standard quantized and KVarN caches with F16/BF16 precision tails. Cache components split only at complete-head boundaries, dynamic meta graphs and auxiliary speculative contexts follow the target topology safely, invalid weights fail during model creation, and invalid cache layouts fail during context creation. Layer split is unchanged; physical multi-GPU peer transfer, NCCL, and heterogeneous rollback still require validation.
• Added CUDA support for serialized ternary, group-64 Q2_0 model weights across dequantization, GET_ROWS, MMVQ, and MMQ, including corrected template declarations. This remains distinct from Bee's cache-facing GGML_TYPE_Q2_0S.
• Fixed CUDA matrix-kernel selection for padded or offset right-hand-side tensors. MMVF, MMF, fused, and expert paths now reject addresses or strides that are unsafe for their float2 loads instead of asserting in the kernel.
• Fixed KVarN correctness during unaligned eager stores, multi-turn suffix removal, compact-tail reprocessing, and deep SWA rollback. Reopened groups retain the correct F16 source, compressed-body tails avoid unnecessary reprocessing, and record-safe rollback boundaries prevent stale rows.
• Restored context creation for cacheless, noncausal architectures after KV-tail type resolution; requesting a precision tail for a model without cache memory now fails explicitly.
• Fixed release and test infrastructure: Windows CPU and Vulkan jobs resolve Visual Studio through vswhere, CPU packages require OpenMP, release ccache can seed from the nearest ancestor version branch, LLAMA_BUILD_TESTS=ON builds compile after the upstream reasoning-budget API change, and release regressions run without ignored local files.

macOS:

• macOS Apple Silicon

Linux:

• Ubuntu x64 CPU
• Ubuntu arm64 CPU
• Ubuntu x64 CUDA 12.4
• Ubuntu x64 CUDA 13.1
• Ubuntu x64 Vulkan
• Ubuntu x64 ROCm 7.2
• Ubuntu x64 SYCL

Windows:

• Windows x64 CPU
• Windows x64 Vulkan
• Windows x64 SYCL
• Windows x64 CUDA 12.4 - DLLs
• Windows x64 CUDA 13.1 - DLLs
• Windows x64 HIP

Docker:

• CPU: docker pull ghcr.io/anbeeld/beellama.cpp:server-cpu-v0.4.2
• CUDA: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda-v0.4.2
• CUDA 12: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda12-v0.4.2
• CUDA 13: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda13-v0.4.2
• ROCm: docker pull ghcr.io/anbeeld/beellama.cpp:server-rocm-v0.4.2
• Vulkan: docker pull ghcr.io/anbeeld/beellama.cpp:server-vulkan-v0.4.2
• SYCL: docker pull ghcr.io/anbeeld/beellama.cpp:server-sycl-v0.4.2

Browse all container images