github llama.cpp · 2h ago · inference

llama.cpp b10262 adds GATED_LINEAR_ATTN to Vulkan backend

from English

The llama.cpp project released build b10262, which introduces support for the GATED_LINEAR_ATTN operation in its Vulkan backend. This update also includes documentation updates for Vulkan operations and removes an unused GLA spec constant.

Implemented GATED_LINEAR_ATTN op for Vulkan (#25601)
Updated Vulkan ops documentation
Removed unused GLA spec constant

This release provides binaries for macOS, Linux, Windows, Android, and iOS across CPU, CUDA, ROCm, OpenVINO, SYCL, and Vulkan backends.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

media Hugging Face Forums · 1h ago Live

Open-source Decay-Gated O(N) Causal Linear Attention with fused Triton kernel

A developer has open-sourced a Decay-Gated O(N) Causal Linear Attention architecture featuring fused Triton/CUDA state accumulation kernels designed to bypass quadratic MHA bottlenecks.

github llama.cpp · 4h ago

llama.cpp b10259 allows tensor reshape during load

The llama.cpp project has released build b10259, which introduces the ability to allow the reshape of tensors during the loading process.

github llama.cpp · 10h ago

llama.cpp b10256 parallelizes the SYCL non-contiguous concat kernel

The llama.cpp b10256 release includes a change to parallelize the non-contiguous concatenation kernel for SYCL devices. The update modifies the launch geometry from a single-lane work-group to one utilizing `SYCL_CONCAT_BLOCK_SIZE`.

github llama.cpp · 10h ago

llama.cpp b10254 adds DeepSeek V4 Flash 0731 template and updates V4 templates

The llama.cpp project released version b10254, introducing a new chat template for the DeepSeek V4 Flash 0731 model alongside updates to existing DeepSeek V4 templates.

github llama.cpp · 11h ago · 1 view

llama.cpp b10255 extends oneDNN SDPA to non-FP16 KV caches

The llama.cpp b10255 release extends the oneDNN SDPA path to support non-FP16 key-value (KV) caches, including Q4_0–Q8_0 quantized formats and FP32. This update allows the fused systolic kernel to run identically to the native FP16 path by dequantizing or converting K/V tensors to dense FP16 on-device before processing.