github llama.cpp · 1d ago · inference

llama.cpp b10068 rotates injected K/V cache for DFlash

from English

The llama.cpp project released version b10068, which introduces a change to the DFlash model implementation. The update modifies src/models/dflash.cpp to rotate the injected key/value (K/V) cache when K/V quantization is used.

This release provides binaries for macOS (Apple Silicon and Intel), iOS, Linux (CPU, Vulkan, ROCm, OpenVINO, SYCL), Android, Windows (CPU, OpenCL, CUDA 12/13, Vulkan, OpenVINO, SYCL, HIP), and openEuler.

The update ensures correct cache handling for DFlash models under quantization conditions.

Importance 1/3 Trust 1/3 llama.cpp Hugging Face Inference efficiency

Read original

github llama.cpp · 1d ago

llama.cpp b10067 excludes DeepSeek-V4 routing table from quantization

The llama.cpp project released version b10067, which includes a fix for the `llama-quantize` tool to correctly handle DeepSeek-V4 models. The update prevents the quantizer from attempting to convert the `ffn_gate_tid2eid` tensor, an i32 token-to-expert index table, into a float type.

github llama.cpp · 2d ago

llama.cpp b10067 excludes DeepSeek-V4 routing table from quantization

llama.cpp b10066 adds OpenCL kernel_gemm_moe_q6_k_f32_ns support

llama.cpp b10064 release with OpenCL transpose optimization

llama.cpp b10059 sets ggml-blas default hadamard mul_mat to cpu routine

llama.cpp b10057 fixes SYCL Q5_K kernels and releases binaries