github llama.cpp · 2d ago · inference

llama.cpp b11017 skips unneeded MoE work in mul_mm coopmat1 path

from English

The llama.cpp project released build b11017, which includes a Vulkan optimization to skip unnecessary Mixture of Experts (MoE) work in the mul_mm coopmat1 path.

This update provides pre-built binaries for macOS (Apple Silicon and Intel), Linux (CPU, Vulkan, CUDA 12/13, ROCm, OpenVINO, SYCL), Windows (CPU, OpenCL Adreno, CUDA 12/13, Vulkan, OpenVINO, SYCL, ROCm), Android, and openEuler.

The release also includes the llama.cpp UI.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

github llama.cpp · just now · 1 view Live

llama.cpp b11046 adds OpenCL support for flash_attn_f32_f16_bin kernel

The llama.cpp project released version b11046, which introduces OpenCL support for the `flash_attn_f32_f16_bin` kernel. This update also includes guarded prefill functionality for Flash Attention on OpenCL devices.

github llama.cpp · 1h ago · 1 view Live

llama.cpp b11044 updates hexagon im2col and releases binaries

The llama.cpp project has released version b11044, which includes an update to the hexagon backend for improved image-to-column (im2col) operations. The release provides pre-built binaries for macOS, Linux, Windows, Android, and openEuler across CPU, GPU, and other hardware accelerators.

github llama.cpp · 2h ago · 1 view

llama.cpp b11045 adds ROLL op support to Hexagon backend

The llama.cpp project has released version b11045, which introduces support for the ROLL operation within the Hexagon backend.

github llama.cpp · 3h ago · 1 view

llama.cpp b11043 allows HMX flash-attention with head_dim=72

The llama.cpp project released build b11043, which includes a change to the HMX flash-attention implementation. This update enables support for head dimensions that are not multiples of 64, specifically allowing configurations like SigLIP's head_dim=72.

github llama.cpp · 5h ago · 1 view

llama.cpp b11042 adds OpenCL A8 Q6_K binary kernel

The llama.cpp project has released version b11042, which introduces a new OpenCL binary kernel for A8 Q6_K non-MoE models. This update also includes fixes for layout compatibility within the OpenCL backend.