github llama.cpp · 1h ago · inference

llama.cpp b11147 adds A8 Q6_K non-MoE dp4a OpenCL kernel

from English

The llama.cpp project has released version b11147, which introduces a new binary kernel for OpenCL. This update specifically adds support for the A8 Q6_K non-MoE dp4a configuration.

Adds an A8 Q6_K non-MoE dp4a binary kernel for OpenCL.
Provides binaries for macOS (Apple Silicon and Intel), iOS, Linux (CPU, Vulkan, CUDA, ROCm, OpenVINO, SYCL, Snapdragon), Android, Windows (CPU, Adreno, CUDA, Vulkan, OpenVINO, SYCL, ROCm), and openEuler.

This release enables users to run llama.cpp on a wide range of hardware platforms with the latest kernel optimizations.

Importance 1/3 Trust 1/3 llama.cpp Hugging Face Inference efficiency

Read original

github llama.cpp · 6h ago · 12 views

llama.cpp b11132 adds Gemma 4 DSpark draft backbone support

The llama.cpp project released version b11132, introducing support for the Gemma 4 DSpark draft backbone. This update enables GGUF conversion and runtime capabilities for full-attention and SWA Gemma 4 drafts, including tied output weights and boolean backbone metadata.

github llama.cpp · 11h ago · 12 views

llama.cpp b11122 extends MMVQ GLU fusion and adds new SYCL fusions

The llama.cpp project has released version b11122, which includes significant updates to the SYCL backend. The release extends the MMVQ GLU fusion to support mixed quantization types and introduces new fusions for rms_norm+scale and ssm_conv+silu operations.

github llama.cpp · 1d ago · 13 views

llama.cpp b11111 adds Intel Xe flash attention optimization kernels

The llama.cpp project released build b11111, which introduces Vulkan-based Intel Xe flash attention optimization kernels for split k paths on Xe-LPG Plus, Xe2, and Xe3 architectures.

github llama.cpp · 1d ago · 11 views

llama.cpp adds DFlash support for HunyuanOCR and fixes draft conversion

The llama.cpp project has added support for DFlash speculative decoding with the HunyuanOCR model by exposing layer input tensors in the target graph. This change allows the draft model to read residual streams, enabling image requests to run successfully with a draft acceptance rate of approximately 0.5 and byte-identical OCR output compared to non-speculative runs.

github llama.cpp · 1d ago · 14 views

llama.cpp b11101 release allows repeated find_package calls for llama

The llama.cpp project has released version b11101, which includes a build system fix allowing repeated find_package calls for the llama library.