github llama.cpp · 2d ago · inference

llama.cpp b11232 enables tiled flash attention on x86

from English

The llama.cpp project released build b11232, which includes updates to the ggml-cpu backend. The release focuses on enabling tiled flash attention for non-vector-multiple head dimensions on x86 architectures.

Enabled tiled flash attention for non-vector-multiple head dims on x86.
Added AVX2 support for masked loading and storing in simd_gemm_ukernel_tail.
Fixed FA softcap handling for padded KV tiles.

This update provides binaries for macOS, Linux, Windows, Android, and openEuler across CPU, GPU, and NPU backends.

Importance 1/3 Trust 1/3 llama.cpp Hugging Face Inference efficiency

Read original

github llama.cpp · 11h ago · 2 views

llama.cpp adds GLM-5.3-Flash support with k-pool and MTP fixes

The llama.cpp project has added initial support for the GLM-5.3-Flash (GLM5-Next) model architecture, including migration to hybrid-index memory and early Multi-Token Prediction (MTP) capabilities.

github llama.cpp · 20h ago · 2 views

llama.cpp b11266 loads F32 A matrix 2 at a time for Vulkan on Intel

The llama.cpp project released build b11266, which includes a Vulkan optimization that loads F32 A matrices two elements at a time when they are 2-aligned. This change addresses performance issues on Intel hardware where loading floats one by one is inefficient, and it also corrects a missing validation for `a_offset` alignment in the original patch.

github llama.cpp · 22h ago · 1 view

llama.cpp b11265 fixes Vulkan MOE matmul tile selection

The llama.cpp b11265 release includes a fix for the Vulkan backend's handling of Mixture of Experts (MoE) models. The update corrects how `mat_mul_id` selects matrix multiplication tiles, addressing a performance issue where workers were left idle during dispatch.

github llama.cpp · 2d ago · 4 views

llama.cpp b11238 adds ggml_pad_ext for left padding and skips DFlash2 taps

The llama.cpp project released build b11238, introducing a unified left-padding implementation via the new `ggml_pad_ext` function. This change consolidates previous methods used by audio encoders like Parakeet, LFM2-Audio, Granite Speech, and Gemma 4, ensuring bit-identical embeddings for Gemma 4 while simplifying backend support.

github llama.cpp · 2d ago · 3 views

llama.cpp b11228 adds left and circular padding support in Metal GGML_OP_PAD

The llama.cpp b11228 release introduces support for left and circular padding in the Metal backend's GGML_OP_PAD operation, aligning it with CPU, CUDA, and Vulkan implementations. This change fixes right padding issues for permuted sources and ensures consistent behavior across different hardware backends.