github llama.cpp · 2d ago · inference

llama.cpp b11238 adds ggml_pad_ext for left padding and skips DFlash2 taps

from English

The llama.cpp project released build b11238, introducing a unified left-padding implementation via the new `ggml_pad_ext` function. This change consolidates previous methods used by audio encoders like Parakeet, LFM2-Audio, Granite Speech, and Gemma 4, ensuring bit-identical embeddings for Gemma 4 while simplifying backend support.

The `ggml_pad_ext` node handles left padding in a single operation now that all backends support it, replacing complex roll or zero-fill concatenation logic.
The update includes an optimization to skip DFlash2 taps that only read padding data, as terms at or past block_size are zero.
Pre-built binaries are available for macOS (Apple Silicon and Intel), iOS, Linux (CPU, Vulkan, CUDA 12/13, ROCm, OpenVINO, SYCL, Snapdragon), Android, Windows, and openEuler.

This release improves consistency across different model architectures by standardizing padding operations and provides updated binaries for a wide range of hardware platforms.

Importance 1/3 Trust 1/3 llama.cpp Hugging Face Inference efficiency

Read original

github llama.cpp · 10h ago · 2 views

llama.cpp adds GLM-5.3-Flash support with k-pool and MTP fixes

The llama.cpp project has added initial support for the GLM-5.3-Flash (GLM5-Next) model architecture, including migration to hybrid-index memory and early Multi-Token Prediction (MTP) capabilities.

github llama.cpp · 19h ago · 2 views

llama.cpp b11266 loads F32 A matrix 2 at a time for Vulkan on Intel

The llama.cpp project released build b11266, which includes a Vulkan optimization that loads F32 A matrices two elements at a time when they are 2-aligned. This change addresses performance issues on Intel hardware where loading floats one by one is inefficient, and it also corrects a missing validation for `a_offset` alignment in the original patch.

github llama.cpp · 21h ago · 1 view

llama.cpp b11265 fixes Vulkan MOE matmul tile selection

The llama.cpp b11265 release includes a fix for the Vulkan backend's handling of Mixture of Experts (MoE) models. The update corrects how `mat_mul_id` selects matrix multiplication tiles, addressing a performance issue where workers were left idle during dispatch.

github llama.cpp · 2d ago · 5 views

llama.cpp b11232 enables tiled flash attention on x86

The llama.cpp project released build b11232, which includes updates to the ggml-cpu backend. The release focuses on enabling tiled flash attention for non-vector-multiple head dimensions on x86 architectures.

github llama.cpp · 2d ago · 3 views

llama.cpp b11228 adds left and circular padding support in Metal GGML_OP_PAD

The llama.cpp b11228 release introduces support for left and circular padding in the Metal backend's GGML_OP_PAD operation, aligning it with CPU, CUDA, and Vulkan implementations. This change fixes right padding issues for permuted sources and ensures consistent behavior across different hardware backends.