github llama.cpp · 3d ago · inference

llama.cpp b10833 adds Vulkan RMS norm fusion for ~4% speedup on gemma4

from English

The llama.cpp project released version b10833, introducing Vulkan backend optimizations that fuse specific normalization operations to improve inference performance.

Support for RMS_NORM + MUL + ADD (+ MUL) and RMS_NORM + VIEW + SET_ROWS fusion in the Vulkan backend.
Extension of ROPE + VIEW + SET_ROWS to support IMROPE.
The author reports a performance gain of approximately 4% on gemma4 models on their system.

This update provides users with faster inference speeds for compatible models running on Vulkan-compatible hardware.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

media MarkTechPost · just now Live

DeepSeek releases DeepSeek-V4.1-Flash with 1M context and FP4 KV cache

DeepSeek AI has released DeepSeek-V4.1-Flash, a multimodal Mixture-of-Experts model featuring a 1-million-token context window and an FP4 KV cache that reduces global cache footprint to 890 bytes per token. The model utilizes a causal encoder-decoder architecture and Compressed Sparse Attention 2 (CSA2) to significantly lower memory requirements while maintaining performance.

github llama.cpp · 1h ago · 1 view Live

DeepSeek releases DeepSeek-V4.1-Flash with 1M context and FP4 KV cache

llama.cpp b10889 release avoids V cache allocation for indexer

llama.cpp b10888 adds Vulkan command-buffer debug labels for GPU profilers

llama.cpp b10886 adds Q1_0 vector intrinsic support for s390x

llama.cpp b10883 fixes Vulkan matrix multiplication and restores Ampere optimizations