github llama.cpp · 2d ago · inference

llama.cpp b11228 adds left and circular padding support in Metal GGML_OP_PAD

from English

The llama.cpp b11228 release introduces support for left and circular padding in the Metal backend's GGML_OP_PAD operation, aligning it with CPU, CUDA, and Vulkan implementations. This change fixes right padding issues for permuted sources and ensures consistent behavior across different hardware backends.

The implementation shifts source coordinates by left paddings and wraps them around for circular padding modes.
A test case has been added to cover the new functionality.
The f32_4 kernel is dropped because it is slower and fails two padding cases when enabled.
The code uses a function constant for the circular pad variant to compile the kernel once and specialize it per pipeline.

The release provides binaries for macOS, iOS, Linux, Windows, Android, and openEuler across various architectures and backend configurations including CUDA, Vulkan, ROCm, and OpenVINO.

Importance 1/3 Trust 1/3 llama.cpp Hugging Face Inference efficiency

Read original

github llama.cpp · 10h ago · 2 views

llama.cpp adds GLM-5.3-Flash support with k-pool and MTP fixes

The llama.cpp project has added initial support for the GLM-5.3-Flash (GLM5-Next) model architecture, including migration to hybrid-index memory and early Multi-Token Prediction (MTP) capabilities.

github llama.cpp · 19h ago · 2 views

llama.cpp b11266 loads F32 A matrix 2 at a time for Vulkan on Intel

The llama.cpp project released build b11266, which includes a Vulkan optimization that loads F32 A matrices two elements at a time when they are 2-aligned. This change addresses performance issues on Intel hardware where loading floats one by one is inefficient, and it also corrects a missing validation for `a_offset` alignment in the original patch.

github llama.cpp · 21h ago · 1 view

llama.cpp b11265 fixes Vulkan MOE matmul tile selection

The llama.cpp b11265 release includes a fix for the Vulkan backend's handling of Mixture of Experts (MoE) models. The update corrects how `mat_mul_id` selects matrix multiplication tiles, addressing a performance issue where workers were left idle during dispatch.

github llama.cpp · 2d ago · 4 views

llama.cpp b11238 adds ggml_pad_ext for left padding and skips DFlash2 taps

The llama.cpp project released build b11238, introducing a unified left-padding implementation via the new `ggml_pad_ext` function. This change consolidates previous methods used by audio encoders like Parakeet, LFM2-Audio, Granite Speech, and Gemma 4, ensuring bit-identical embeddings for Gemma 4 while simplifying backend support.

github llama.cpp · 2d ago · 5 views

llama.cpp b11232 enables tiled flash attention on x86

The llama.cpp project released build b11232, which includes updates to the ggml-cpu backend. The release focuses on enabling tiled flash attention for non-vector-multiple head dimensions on x86 architectures.