github llama.cpp · 4d ago · inference

llama.cpp b11216 adds SYCL FWHT kernels for block widths above 512

from English

The llama.cpp b11216 release introduces Fast Walsh-Hadamard Transform (FWHT) kernels in the SYCL backend that support block widths greater than 512. Previously, larger widths fell back to a dense GEMM with O(n^2) complexity; this change enables O(n log n) performance for these sizes.

The new `fwht_kernel_wide` implementation runs one row per work-group and uses local memory and registers to optimize butterfly operations.
It supports block widths of 1024, 2048, 4096, and 8192 via the Kronecker/Paley construction.
The release includes binaries for macOS, Linux, Windows, Android, and openEuler across CPU, GPU, and NPU backends.
Verification was performed using a standalone harness on an Intel oneAPI DPC++ 2026.1 SYCL CPU device, confirming correctness against a reference implementation.

This update improves computational efficiency for large block sizes in SYCL environments while maintaining compatibility with existing hardware configurations.

Importance 1/3 Trust 1/3 llama.cpp Hugging Face Inference efficiency

Read original

github llama.cpp · 19h ago · 4 views

llama.cpp b11309 optimizes Hexagon ALLREDUCE with safe scatter mode

The llama.cpp project released version b11309, which includes a key optimization for the Hexagon backend. This update adds support for safe scatter mode to the ALLREDUCE operation, aiming to improve performance and stability on Qualcomm Snapdragon devices.

github llama.cpp · 1d ago · 8 views

llama.cpp adds GLM-5.3-Flash support with k-pool and MTP fixes

The llama.cpp project has added initial support for the GLM-5.3-Flash (GLM5-Next) model architecture, including migration to hybrid-index memory and early Multi-Token Prediction (MTP) capabilities.

github llama.cpp · 2d ago · 4 views

llama.cpp b11266 loads F32 A matrix 2 at a time for Vulkan on Intel

The llama.cpp project released build b11266, which includes a Vulkan optimization that loads F32 A matrices two elements at a time when they are 2-aligned. This change addresses performance issues on Intel hardware where loading floats one by one is inefficient, and it also corrects a missing validation for `a_offset` alignment in the original patch.

github llama.cpp · 2d ago · 1 view

llama.cpp b11265 fixes Vulkan MOE matmul tile selection

The llama.cpp b11265 release includes a fix for the Vulkan backend's handling of Mixture of Experts (MoE) models. The update corrects how `mat_mul_id` selects matrix multiplication tiles, addressing a performance issue where workers were left idle during dispatch.

github llama.cpp · 3d ago · 4 views

llama.cpp b11238 adds ggml_pad_ext for left padding and skips DFlash2 taps

The llama.cpp project released build b11238, introducing a unified left-padding implementation via the new `ggml_pad_ext` function. This change consolidates previous methods used by audio encoders like Parakeet, LFM2-Audio, Granite Speech, and Gemma 4, ensuring bit-identical embeddings for Gemma 4 while simplifying backend support.