github vLLM · 2d ago · inference

v0.29.0rc5 changes default prefix_cache_retention_interval to dense for Mamba

from English

The v0.29.0rc5 release updates the default value of the `prefix_cache_retention_interval` parameter to `dense` specifically for Mamba models.

The change targets the Mamba architecture within the core system.
It modifies the caching behavior by setting the retention interval to dense.

Importance 1/3 Trust 2/3 vLLM Inference efficiency

Read original

media MarkTechPost · just now Live

DeepSeek releases DeepSeek-V4.1-Flash with 1M context and FP4 KV cache

DeepSeek AI has released DeepSeek-V4.1-Flash, a multimodal Mixture-of-Experts model featuring a 1-million-token context window and an FP4 KV cache that reduces global cache footprint to 890 bytes per token. The model utilizes a causal encoder-decoder architecture and Compressed Sparse Attention 2 (CSA2) to significantly lower memory requirements while maintaining performance.

github llama.cpp · 1h ago · 1 view Live

DeepSeek releases DeepSeek-V4.1-Flash with 1M context and FP4 KV cache

llama.cpp b10889 release avoids V cache allocation for indexer

llama.cpp b10888 adds Vulkan command-buffer debug labels for GPU profilers

llama.cpp b10886 adds Q1_0 vector intrinsic support for s390x

llama.cpp b10883 fixes Vulkan matrix multiplication and restores Ampere optimizations