github llama.cpp · 1d ago · inference

llama.cpp b10951 moves llama_n_rs_seq before llama_decode to avoid unnecessary calls

from English

The llama.cpp project released version b10951, which includes a code optimization in the decoding logic. The `llama_n_rs_seq` function call is now executed before `llama_decode`, allowing the system to return directly if the check passes.

This change eliminates the need for setting the `res` variable and using a goto statement when the early return condition is met.
The primary motivation is to avoid invoking `llama_decode` when it is not required, improving efficiency.
Binaries are available for macOS (Apple Silicon and Intel), iOS, Linux (CPU, Vulkan, ROCm, OpenVINO, SYCL), Android, Windows (CPU, CUDA 12/13, Vulkan, OpenVINO, SYCL, ROCm), and openEuler.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency Local inference

Read original

github llama.cpp · 2h ago · 2 views

llama.cpp b10982 adds Vulkan sparse Flash Attention support for DSV4/GLM

The llama.cpp project has released build b10982, which introduces support for sparse Flash Attention via the Vulkan backend. This update specifically targets DSV4 and GLM models, allowing them to leverage sparse attention mechanisms on compatible GPU hardware.

github llama.cpp · 3h ago · 2 views

llama.cpp b10981 fixes stateful decode for Gemma-4 and optimizes GPU MoE inference

The llama.cpp b10981 release addresses critical correctness issues in the OpenVINO backend's stateful decoding path, specifically fixing support for models like Gemma-4 that use per-layer head sizes. It also introduces optimizations for GPU Mixture of Experts (MoE) inference and improves performance through KV state relayouts.

github llama.cpp · 21h ago · 8 views

llama.cpp b10982 adds Vulkan sparse Flash Attention support for DSV4/GLM

llama.cpp b10981 fixes stateful decode for Gemma-4 and optimizes GPU MoE inference

llama.cpp 0.4.1 adds Maple 20B-A1B, Tencent Hy 4, Spark2.5 support

llama.cpp b10930 allows model downloads at the model limit

llama.cpp b10926 release handles unsupported tq1_0 quants gracefully