github llama.cpp · just now · inference

llama.cpp b10907 fixes MTP context kv cache allocation for deepseek2, glm4moe, cohere2moe

from English

The llama.cpp project released version b10907, which includes a fix for Multi-Token Prediction (MTP) context key-value cache allocation. This update addresses issues affecting the deepseek2, glm4moe, and cohere2moe architectures.

Fixes MTP context kv cache allocation for deepseek2, glm4moe, and cohere2moe models.
Adds inverse architecture gating and comprehensive architecture testing for MTP layer filtering.
Slims the NextN filter comment and drops test-llama-archs changes.

The release provides binaries for macOS (Apple Silicon and Intel), iOS, Linux (CPU, Vulkan, ROCm, OpenVINO, SYCL), Android, Windows (CPU, CUDA, OpenCL, Vulkan, OpenVINO, SYCL, ROCm), and openEuler.

Importance 1/3 Trust 1/3 llama.cpp Hugging Face Inference efficiency

Read original

github llama.cpp · just now Live

llama.cpp b10908 fixes idle threads in Metal iq mul_mv kernels

The llama.cpp project released version b10908, which includes a fix for idle threads in the remaining iq mul_mv kernels for ne00 < 1024. This change generalizes the row split logic to six specific kernels: iq1_s, iq1_m, iq2_xxs, iq2_xs, iq2_s, and iq3_s.

github llama.cpp · 1h ago · 1 view Live

llama.cpp b10908 fixes idle threads in Metal iq mul_mv kernels

llama.cpp b10905 tunes Flash Attention for AMD RDNA4 and revises stream-k logic

llama.cpp b10888 adds Vulkan command-buffer debug labels for GPU profilers

llama.cpp b10886 adds Q1_0 vector intrinsic support for s390x

llama.cpp b10883 fixes Vulkan matrix multiplication and restores Ampere optimizations