github llama.cpp · just now · inference

llama.cpp b11433 adds Hexagon pool op support

from English

The llama.cpp project released version b11433, which introduces support for pooling operations on the Hexagon NPU. This update includes implementations for both 2D and 1D pooling layers.

Added pool_2d and pool_1d support for Hexagon backends.
Optimized HTP pooling boundaries and DMA pipelining.
Rewrote the DMA pipeline and added pool chunking support.
Removed vectorized scalar paths and simplified the chunk solver.

This release provides binaries for macOS, Linux, Windows, Android, and openEuler across CPU, GPU, and NPU architectures.

Importance 1/3 Trust 1/3 llama.cpp Hugging Face Inference efficiency

Read original

github llama.cpp · 9h ago · 4 views

llama.cpp b11424 fixes Vulkan Flash Attention shmem write out of bounds

The llama.cpp project has released version b11424, which includes a fix for a shared memory write-out-of-bounds error in the Vulkan backend when using Flash Attention.

github llama.cpp · 13h ago · 16 views

llama.cpp b11413 adds Vulkan sparse flash attention for quantized K/V

The llama.cpp b11413 release introduces a new Vulkan implementation of sparse flash attention that supports quantized Key and Value tensors. This update addresses performance issues in the previous compaction method, which was too costly for decoding tasks with large context windows.

github llama.cpp · 16h ago · 23 views

llama.cpp b11424 fixes Vulkan Flash Attention shmem write out of bounds

llama.cpp b11413 adds Vulkan sparse flash attention for quantized K/V

llama.cpp b11405 tiles the lightning indexer over keys and tokens for 4 heads

llama.cpp b11403 uses MMVF for thin f16/bf16 mul_mat at small batch size

llama.cpp b11390 fixes CUDA MMQ memory fault