github llama.cpp · 1d ago · inference

llama.cpp b10868 adds IQ type handling for MoE

from English

The llama.cpp project has released build b10868, which introduces IQ type handling for Mixture of Experts (MoE) models.

Adds support for IQ quantization types within MoE architectures via pull request #28476.
Provides binaries for macOS (Apple Silicon and Intel), Linux (CPU, Vulkan, ROCm, OpenVINO, SYCL), Windows (CPU, CUDA 12/13, Vulkan, OpenCL, ROCm, SYCL), Android, and openEuler.
Includes iOS XCFramework support and a standalone UI package.

This update enables users to run MoE models with specific IQ quantization types across a wide range of hardware platforms.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

media MarkTechPost · just now Live

DeepSeek releases DeepSeek-V4.1-Flash with 1M context and FP4 KV cache

DeepSeek AI has released DeepSeek-V4.1-Flash, a multimodal Mixture-of-Experts model featuring a 1-million-token context window and an FP4 KV cache that reduces global cache footprint to 890 bytes per token. The model utilizes a causal encoder-decoder architecture and Compressed Sparse Attention 2 (CSA2) to significantly lower memory requirements while maintaining performance.

github llama.cpp · 1h ago · 1 view Live

DeepSeek releases DeepSeek-V4.1-Flash with 1M context and FP4 KV cache

llama.cpp b10889 release avoids V cache allocation for indexer

llama.cpp b10888 adds Vulkan command-buffer debug labels for GPU profilers

llama.cpp b10886 adds Q1_0 vector intrinsic support for s390x

llama.cpp b10883 fixes Vulkan matrix multiplication and restores Ampere optimizations