github llama.cpp · just now · inference

llama.cpp b10238 adds MTP support for Qwen3-Next

from English

The llama.cpp project released build b10238, which introduces Multi-Token Prediction (MTP) support for the Qwen3-Next model family. This update includes specific implementation changes to handle MTP layers and fixes related to Python type checking within the codebase.

Added MTP support for Qwen3-Next models in src/models/qwen3next.cpp.
Fixed computation of num_mtp directly from MTP layers and defined opt_num_mtp_layers in _QwenMtpMixin.
Resolved Python type-checking issues and updated gguf-py/gguf/constants.py.
Provided binaries for macOS (Apple Silicon/Intel), Linux (CPU, Vulkan, ROCm, OpenVINO, SYCL), Android, Windows (CPU, CUDA 12/13, Vulkan, OpenCL, OpenVINO, SYCL, HIP), and openEuler.

This release enables users to run Qwen3-Next models with MTP capabilities on a wide range of hardware architectures and operating systems.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency Local inference

Read original

github llama.cpp · 2h ago · 1 view

llama.cpp b10237 adds MTP support for DeepSeek V3.2

The llama.cpp project released build b10237, which introduces Multi-Token Prediction (MTP) support for the DeepSeek V3.2 model.

github llama.cpp · 14h ago · 7 views

llama.cpp b10232 implements DeepSeek V4 hyper-connections in Metal

The llama.cpp project released build b10232, which introduces support for DeepSeek V4 hyper-connections via the Metal backend. This update adds implementation of GGML_OP_DSV4_HC_COMB, GGML_OP_DSV4_HC_PRE, and GGML_OP_DSV4_HC_POST operations.

github llama.cpp · 2d ago · 9 views

llama.cpp b10219 persists reasoning_content in chat history

The llama.cpp project released version b10219, which includes a fix for the CLI to persist reasoning content within the chat history. Previously, while `llama-cli` collected reasoning from the stream for display, it only stored assistant content in messages, preventing the `--reasoning-preserve` flag from re-injecting prior thoughts on later turns.

github llama.cpp · 3d ago · 8 views

llama.cpp b10213 adds support for rotated kv cache quantization

The llama.cpp project has released version b10213, introducing support for rotated key-value cache quantization. This update is available across a wide range of platforms and hardware accelerators.

github llama.cpp · 3d ago · 3 views

llama.cpp b10212 loads MTP tensors only when used

The llama.cpp project released version b10212, which includes a change to load Multi-Token Prediction (MTP) tensors only if they are actually used. This update ensures that MTP tensors are skipped during loading for remaining models that support the feature but do not require it.