github llama.cpp · 4d ago · inference

llama.cpp b10829 fixes GDN normalization to match flash-linear-attention

from English

The llama.cpp project released build b10829, which corrects the gated delta net (GDN) query/key normalization implementation. The change switches the normalization method from using `max` to `rsqrt`, aligning with the definition used by flash-linear-attention.

GDN q/k normalization now uses `l2norm` as defined by flash-linear-attention: `x * rsqrt(sum(x*x) + eps)`.
Previously, llama.cpp used `ggml_l2_norm`, which applied a clamp via `max(sqrt(sum(x*x)), eps)` instead of placing epsilon inside the root.
This fix addresses a misalignment that also affected Hugging Face's transformers library for Qwen3-Next before being corrected in pull request #40842.
The update ensures consistency with reference implementations where epsilon is included within the square root calculation.

This correction ensures that models using GDN layers, such as those based on flash-linear-attention, behave identically to their reference implementations.

Importance 1/3 Trust 1/3 llama.cpp Hugging Face Inference efficiency

Read original

github llama.cpp · 3h ago · 1 view

llama.cpp b10888 adds Vulkan command-buffer debug labels for GPU profilers

The llama.cpp project has released build b10888, which includes a new feature to add command-buffer debug labels for GPU profilers in the Vulkan backend.

github llama.cpp · 4h ago · 1 view

llama.cpp b10886 adds Q1_0 vector intrinsic support for s390x

The llama.cpp project released build b10886, which introduces Q1_0 vector intrinsic support for the s390x architecture. This update includes the implementation of `ggml_vec_dot_q1_0_q8_0` and updates documentation to reflect the new capability.

github llama.cpp · 18h ago · 1 view

llama.cpp b10883 fixes Vulkan matrix multiplication and restores Ampere optimizations

The llama.cpp project released build b10883, which includes significant updates to the Vulkan backend. The primary change involves using spec constants for matrix-matrix multiplication A-type operations to improve shader compilation and performance.

github llama.cpp · 1d ago · 2 views

llama.cpp b10870 adds f16 B-type matmul pipelines and warp tile tuning for Intel coopmat1

The llama.cpp project released version b10870, introducing Vulkan support for f16 B-type matrix multiplication pipelines and warp tile size tuning specifically for Intel coopmat1 hardware.

github llama.cpp · 1d ago · 2 views

llama.cpp b10871 adds dedicated iq4_xs mat-vec shader for Vulkan

The llama.cpp project released build b10871, introducing a dedicated mul_mat_vec_iq4_xs shader for the dmmv path in Vulkan. This change replaces the previous generic fallback implementation to improve performance on RDNA4 hardware.