github llama.cpp · 9h ago · inference

llama.cpp b10759 avoids KleidiAI buffer type init on dispatch

from English

The llama.cpp project released build b10759, which includes a fix to avoid KleidiAI buffer type initialization on dispatch.

The release provides binaries for macOS (Apple Silicon and Intel), iOS, Linux (Ubuntu x64, arm64, s390x), Windows (CPU, CUDA 12/13, Vulkan, OpenCL, ROCm, SYCL, OpenVINO), Android, and openEuler.
KleidiAI support is explicitly disabled for macOS Apple Silicon in this build.
Previews for Windows arm64 with CUDA 13 are included alongside standard CUDA 12.4 and 13.3 releases.

Importance 1/3 Trust 1/3 llama.cpp Hugging Face Inference efficiency

Read original

github llama.cpp · just now Live

llama.cpp b10766 adds DeepSeek 4 vision support

The llama.cpp project has released build b10766, which includes a fix to correctly support input vision for the DeepSeek 4 model.

github llama.cpp · 1h ago · 1 view Live

llama.cpp b10762 adds DeepSeek-V4-Flash-Vision-Exp support

The llama.cpp project has released build b10762, which introduces support for the DeepSeek-V4-Flash-Vision-Exp model. This update also includes handling of minimum and maximum token counts from the command line interface.

github llama.cpp · 9h ago · 1 view

llama.cpp b10758 adds MUL_MAT and MUL_MAT_ID fusion for Hexagon

The llama.cpp project released build b10758, introducing MUL_MAT and MUL_MAT_ID fusion capabilities for the Hexagon DSP. This update also includes several fixes to improve stability and performance on Qualcomm hardware.

github llama.cpp · 11h ago · 1 view

llama.cpp b10756 restricts Vulkan shader_bfloat16 extension request

The llama.cpp project released build b10756, which modifies Vulkan backend behavior to only request the VK_KHR_shader_bfloat16 extension when it is supported by the hardware.

github llama.cpp · 11h ago · 1 view

llama.cpp b10757 improves Vulkan IQ3_S mat-vec performance for larger batch sizes

The llama.cpp project released build b10757, which includes a specific optimization for the Vulkan backend. This update handles larger batch sizes greater than 4 efficiently for IQ3_S matrix-vector multiplication when NUM_COLS is greater than 4.