github llama.cpp · 5h ago · inference

llama.cpp b11042 adds OpenCL A8 Q6_K binary kernel

from English

The llama.cpp project has released version b11042, which introduces a new OpenCL binary kernel for A8 Q6_K non-MoE models. This update also includes fixes for layout compatibility within the OpenCL backend.

Adds the `kernel_gemm_noshuffle_q6_k_f32_32b_trans_ila_a8_bin` OpenCL kernel.
Resolves layout compatibility issues in the OpenCL implementation.
Provides binaries for macOS (Apple Silicon and Intel), iOS, Linux (CPU, Vulkan, CUDA 12/13, ROCm, OpenVINO, SYCL), Android, Windows (CPU, Vulkan, CUDA 12/13, ROCm, OpenVINO, SYCL), and openEuler.

This release expands hardware support for specific quantization formats on OpenCL devices while maintaining broad compatibility across major operating systems and GPU architectures.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency Local inference

Read original

github llama.cpp · just now · 1 view Live

llama.cpp b11046 adds OpenCL support for flash_attn_f32_f16_bin kernel

The llama.cpp project released version b11046, which introduces OpenCL support for the `flash_attn_f32_f16_bin` kernel. This update also includes guarded prefill functionality for Flash Attention on OpenCL devices.

github llama.cpp · 9h ago · 1 view

llama.cpp b11039 adds SWA pattern support to model-saver and loaders

The llama.cpp project released build b11039, which updates the model-saver to write sliding window attention (SWA) patterns and MLA SWA geometry for all models using SWA. The change ensures that the `sliding_window_pattern` is written as a per-layer flag rather than being collapsed to a scalar, preserving exact data representation.

github llama.cpp · 1d ago · 6 views

llama.cpp b11024 updates OpenVINO to 2026.4 and fixes warnings

The llama.cpp project released build b11024, which primarily updates the OpenVINO integration to version 2026.4.

github llama.cpp · 1d ago · 2 views

llama.cpp b11025 extends Nemotron MTP support

The llama.cpp project has released build b11025, which includes an extension of Nemotron Multi-Token Prediction (MTP) support.

github llama.cpp · 2d ago · 4 views

llama.cpp b11019 fixes GGUF data alignment for embedded files and adds LoRA loading

The llama.cpp project released version b11019, addressing a critical bug where GGUF files embedded at non-aligned offsets returned incorrect tensor data. The update aligns the GGUF data section relative to the file start rather than offset 0, ensuring correct loading of embedded models.