github llama.cpp · 1h ago · inference

llama.cpp b10576 adds Q2_K reordered MMVQ and ESIMD kernels for SYCL

from English

The llama.cpp project has released build b10576, which re-adds Q2_K reordered MMVQ and ESIMD kernels for SYCL support. This change effectively reverts a previous commit that had removed these specific kernel implementations.

The release includes binaries for macOS (Apple Silicon and Intel), Linux (Ubuntu x64, arm64, s390x), Windows, and Android.
GPU acceleration is supported via Vulkan, ROCm 7.14, OpenVINO, CUDA 12/13, and SYCL across multiple platforms.
macOS KleidiAI support remains disabled in this build.

This update restores specific quantization kernels for SYCL devices while providing comprehensive binaries for various operating systems and hardware accelerators.

Importance 1/3 Trust 1/3 llama.cpp Hugging Face Inference efficiency

Read original

github llama.cpp · just now Live

ggml optimizes concat op by replacing per-element memcpy with row-level memcpy

The ggml library has optimized its concatenation operation by replacing per-element memory copies with row-level memcpy operations. This change is part of the b10578 release of llama.cpp, which includes various platform-specific binaries and updates to the underlying computation graph.

github llama.cpp · 1h ago · 1 view Live

ggml optimizes concat op by replacing per-element memcpy with row-level memcpy

llama.cpp b10577 fixes draft-mtp with embeddings

llama.cpp b10568 uses ggml_rope_set_offset() for DeepSeek 2

llama.cpp v0.2.0 release with ggml 0.21.0 and Kleidiai SME2 support

llama.cpp b10541 adds --mmproj-device argument for multimodal projects