github llama.cpp · 3h ago · inference

llama.cpp b11043 allows HMX flash-attention with head_dim=72

from English

The llama.cpp project released build b11043, which includes a change to the HMX flash-attention implementation. This update enables support for head dimensions that are not multiples of 64, specifically allowing configurations like SigLIP's head_dim=72.

The HMX flash-attention kernel now operates on DK/DV rounded up to 64 with zero-filled tail lanes.
Pre-built binaries are available for macOS (Apple Silicon and Intel), iOS, Linux (CPU, Vulkan, CUDA, ROCm, OpenVINO, SYCL), Windows (CPU, OpenCL, CUDA, Vulkan, OpenVINO, SYCL, ROCm), Android, and openEuler.

This change allows users to run models with non-standard head dimensions on supported hardware without compatibility issues.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

github llama.cpp · just now · 1 view Live

llama.cpp b11046 adds OpenCL support for flash_attn_f32_f16_bin kernel

The llama.cpp project released version b11046, which introduces OpenCL support for the `flash_attn_f32_f16_bin` kernel. This update also includes guarded prefill functionality for Flash Attention on OpenCL devices.

github llama.cpp · 1h ago · 1 view Live

llama.cpp b11044 updates hexagon im2col and releases binaries

The llama.cpp project has released version b11044, which includes an update to the hexagon backend for improved image-to-column (im2col) operations. The release provides pre-built binaries for macOS, Linux, Windows, Android, and openEuler across CPU, GPU, and other hardware accelerators.

github llama.cpp · 2h ago · 1 view

llama.cpp b11045 adds ROLL op support to Hexagon backend

The llama.cpp project has released version b11045, which introduces support for the ROLL operation within the Hexagon backend.

github llama.cpp · 5h ago · 1 view

llama.cpp b11042 adds OpenCL A8 Q6_K binary kernel

The llama.cpp project has released version b11042, which introduces a new OpenCL binary kernel for A8 Q6_K non-MoE models. This update also includes fixes for layout compatibility within the OpenCL backend.

github llama.cpp · 9h ago · 1 view

llama.cpp b11039 adds SWA pattern support to model-saver and loaders

The llama.cpp project released build b11039, which updates the model-saver to write sliding window attention (SWA) patterns and MLA SWA geometry for all models using SWA. The change ensures that the `sliding_window_pattern` is written as a per-layer flag rather than being collapsed to a scalar, preserving exact data representation.