github llama.cpp · 21h ago · inference

llama.cpp 0.4.1 adds Maple 20B-A1B, Tencent Hy 4, Spark2.5 support

from English

The llama.cpp project released version 0.4.1, introducing support for the Maple 20B-A1B ternary Mixture of Experts architecture on CPU, as well as preview support for Tencent Hy 4 and Spark2.5 models.

Added Maple 20B-A1B ternary MoE architecture (CPU) support.
Added Tencent Hy 4 (hy_v4) preview architecture support.
Added Spark2.5 model support.
Updated ggml to v0.24.0, expanding backend coverage and robustness with a new precision-control API.
Improved JSON schema handling by refactoring it with a common_schema internal representation.
Split specialized chat parsers into common/parsers and added structured JSONL logging via --log-jsonl.
Fixed MTP context KV cache allocation for DeepSeek2, GLM-MoE, and related models.
Fixed GDN normalization from max to rsqrt for affected Qwen/Kimi/GLM models.

This release enables running new model architectures and improves stability through corrected normalization, cache allocation, and server child-process management.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency Local inference

Read original

github llama.cpp · 2h ago · 2 views

llama.cpp b10982 adds Vulkan sparse Flash Attention support for DSV4/GLM

The llama.cpp project has released build b10982, which introduces support for sparse Flash Attention via the Vulkan backend. This update specifically targets DSV4 and GLM models, allowing them to leverage sparse attention mechanisms on compatible GPU hardware.

github llama.cpp · 3h ago · 2 views

llama.cpp b10981 fixes stateful decode for Gemma-4 and optimizes GPU MoE inference

The llama.cpp b10981 release addresses critical correctness issues in the OpenVINO backend's stateful decoding path, specifically fixing support for models like Gemma-4 that use per-layer head sizes. It also introduces optimizations for GPU Mixture of Experts (MoE) inference and improves performance through KV state relayouts.

github llama.cpp · 1d ago · 2 views

llama.cpp b10982 adds Vulkan sparse Flash Attention support for DSV4/GLM

llama.cpp b10981 fixes stateful decode for Gemma-4 and optimizes GPU MoE inference

llama.cpp b10951 moves llama_n_rs_seq before llama_decode to avoid unnecessary calls

llama.cpp b10930 allows model downloads at the model limit

llama.cpp b10926 release handles unsupported tq1_0 quants gracefully