github llama.cpp · 2h ago · inference

llama.cpp b10982 adds Vulkan sparse Flash Attention support for DSV4/GLM

from English

The llama.cpp project has released build b10982, which introduces support for sparse Flash Attention via the Vulkan backend. This update specifically targets DSV4 and GLM models, allowing them to leverage sparse attention mechanisms on compatible GPU hardware.

Adds sparse Flash Attention support for DSV4/GLM in the Vulkan implementation.
Includes tuned implementation logic and added tests.
Fixes nondeterministic atomicAdd behavior.
Adds cm2 decode vector support and f16vec4 binding for decode vectors.
Simplifies logic and improves variable name consistency.

The release provides precompiled binaries for macOS, Linux, Windows, Android, and openEuler across CPU, CUDA, ROCm, OpenVINO, SYCL, and Vulkan backends.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency Local inference

Read original

github llama.cpp · 3h ago · 2 views

llama.cpp b10981 fixes stateful decode for Gemma-4 and optimizes GPU MoE inference

The llama.cpp b10981 release addresses critical correctness issues in the OpenVINO backend's stateful decoding path, specifically fixing support for models like Gemma-4 that use per-layer head sizes. It also introduces optimizations for GPU Mixture of Experts (MoE) inference and improves performance through KV state relayouts.

github llama.cpp · 21h ago · 8 views

llama.cpp b10981 fixes stateful decode for Gemma-4 and optimizes GPU MoE inference

llama.cpp 0.4.1 adds Maple 20B-A1B, Tencent Hy 4, Spark2.5 support

llama.cpp b10951 moves llama_n_rs_seq before llama_decode to avoid unnecessary calls

llama.cpp b10930 allows model downloads at the model limit

llama.cpp b10926 release handles unsupported tq1_0 quants gracefully