github llama.cpp · 1h ago · inference

llama.cpp b10581 adds DSpark support for bailinmoe3

from English

The llama.cpp project has released build b10581, which introduces support for the DSpark backend to enable the bailinmoe3 model.

The release includes binaries for macOS (Apple Silicon and Intel), Linux (Ubuntu x64, arm64, s390x), Windows (x64 and arm64), and Android (arm64).
GPU acceleration is available via CUDA 12.4/13.3, Vulkan, OpenCL Adreno, ROCm 7.14, OpenVINO, and SYCL across supported platforms.
Bailinmoe3 support specifically utilizes the DSpark backend as detailed in pull request #27508.
macOS Apple Silicon with KleidiAI and Ubuntu x64 with ROCm 7.14 are currently disabled for this build.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

github llama.cpp · 2h ago · 1 view

ggml optimizes concat op by replacing per-element memcpy with row-level memcpy

The ggml library has optimized its concatenation operation by replacing per-element memory copies with row-level memcpy operations. This change is part of the b10578 release of llama.cpp, which includes various platform-specific binaries and updates to the underlying computation graph.

github llama.cpp · 3h ago · 1 view

llama.cpp b10576 adds Q2_K reordered MMVQ and ESIMD kernels for SYCL

The llama.cpp project has released build b10576, which re-adds Q2_K reordered MMVQ and ESIMD kernels for SYCL support. This change effectively reverts a previous commit that had removed these specific kernel implementations.

github llama.cpp · 3h ago · 1 view

llama.cpp b10577 fixes draft-mtp with embeddings

The llama.cpp project released build b10577, which includes a fix for draft-mtp when used with embeddings. This update addresses issues #26352 and #27299.

github llama.cpp · 7h ago · 1 view

mtmd adds WebP support via ffmpeg

The mtmd component in llama.cpp now supports the WebP image format through ffmpeg integration.

github llama.cpp · 12h ago · 2 views

llama.cpp b10568 uses ggml_rope_set_offset() for DeepSeek 2

The llama.cpp project released build b10568, which updates the model handling to use the `ggml_rope_set_offset()` function. This change is partially applied to support DeepSeek 2.