github llama.cpp · 1h ago · inference

llama.cpp b10206 enforces same K and V cache types for DeepSeek V4 and MLA models

from English

The llama.cpp project has released version b10206, which introduces specific changes to how key-value caches are handled in attention mechanisms. The update ensures that the K and V cache types remain consistent for DeepSeek V4 models.

Enforces identical K and V cache types for DeepSeek V4 architectures.
Enables Flash Attention (FA) when the V cache is quantized.
Applies the same K and V cache type enforcement to other MLA models.

This release provides binaries for macOS, Linux, Windows, Android, and openEuler across various hardware backends including CPU, CUDA, ROCm, Vulkan, OpenVINO, SYCL, and OpenCL.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency Local inference

Read original

github llama.cpp · 3h ago

llama.cpp b10202 release with SYCL RMS_NORM fusion

The llama.cpp project has released version b10202, introducing a SYCL optimization that fuses the RMS_NORM and MUL operations.

github llama.cpp · 4h ago · 1 view

llama.cpp b10201 improves flash attention for quantized KV at long contexts

The llama.cpp project has released version b10201, which includes a key improvement to the ggml-webgpu backend. This update enhances flash attention performance when handling quantized key-value caches during long context generation.

github llama.cpp · 19h ago · 1 view

llama.cpp b10198 adds Vulkan support for quantized concat

The llama.cpp project has released build b10198, which introduces support for quantized concatenation operations within the Vulkan backend.

github llama.cpp · 2d ago · 1 view

llama.cpp b10180 adds contiguous fast path and 32-bit index math for SYCL unary ops

The llama.cpp project has released build b10180, introducing performance optimizations for the SYCL backend. The update implements a contiguous fast path and switches elementwise index math to use fastdiv with 32-bit integers.

github llama.cpp · 4d ago · 3 views

llama.cpp b10155 adds MiMo-V2.5 audio support

The llama.cpp project has released build b10155, which introduces support for the MiMo-V2.5 audio input model. This update includes a GGUF converter for MiMo audio and implements the necessary C++ backend logic to handle RVQ-based audio models.