github llama.cpp · 1h ago · inference

llama.cpp b11424 fixes Vulkan Flash Attention shmem write out of bounds

from English

The llama.cpp project has released version b11424, which includes a fix for a shared memory write-out-of-bounds error in the Vulkan backend when using Flash Attention.

This update provides precompiled binaries for macOS (Apple Silicon and Intel), Linux (CPU, CUDA, ROCm, OpenVINO, SYCL, Snapdragon), Windows (CPU, CUDA, Vulkan, OpenVINO, SYCL, ROCm), Android, and iOS. The release also includes the llama.cpp UI.

The build for KleidiAI on macOS Apple Silicon is currently disabled in this version.

Importance 1/3 Trust 1/3 llama.cpp Hugging Face Inference efficiency

Read original

github llama.cpp · 4h ago · 4 views

llama.cpp b11413 adds Vulkan sparse flash attention for quantized K/V

The llama.cpp b11413 release introduces a new Vulkan implementation of sparse flash attention that supports quantized Key and Value tensors. This update addresses performance issues in the previous compaction method, which was too costly for decoding tasks with large context windows.

github llama.cpp · 8h ago · 20 views

llama.cpp b11413 adds Vulkan sparse flash attention for quantized K/V

llama.cpp b11405 tiles the lightning indexer over keys and tokens for 4 heads

llama.cpp b11403 uses MMVF for thin f16/bf16 mul_mat at small batch size

llama.cpp b11390 fixes CUDA MMQ memory fault

llama.cpp b11375 fixes graph reallocation aborts under GGML_SCHED_NO_REALLOC