github llama.cpp · 1h ago · inference

llama.cpp b10298 adds mtmd chunk save/load function

from English

The llama.cpp project has released build b10298, which introduces a new chunk save and load function for the mtmd module. This update also includes code cleanup and additional tests.

Added chunk save/load functionality to the mtmd module.
Included minor code nits and test additions.
Renamed internal constants from _MAX to _COUNT.

The release provides binaries for macOS, Linux, Windows, Android, and openEuler across various hardware backends including CPU, CUDA, Vulkan, ROCm, OpenVINO, SYCL, and HIP.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency Local inference

Read original

github llama.cpp · 1d ago

llama.cpp b10282 adds spec-decode counters to /metrics endpoint

The llama.cpp project has released version b10282, which introduces new monitoring capabilities for speculative decoding. The server now includes spec-decode counters in its /metrics endpoint, allowing users to track performance metrics related to this feature.

github llama.cpp · 2d ago

llama.cpp b10259 allows tensor reshape during load

The llama.cpp project has released build b10259, which introduces the ability to allow the reshape of tensors during the loading process.

github llama.cpp · 3d ago · 2 views

llama.cpp b10254 adds DeepSeek V4 Flash 0731 template and updates V4 templates

The llama.cpp project released version b10254, introducing a new chat template for the DeepSeek V4 Flash 0731 model alongside updates to existing DeepSeek V4 templates.

github llama.cpp · 3d ago

llama.cpp b10251 adds MTP support for GLM-4.7-Flash

The llama.cpp project has released build b10251, which introduces Multi-Token Prediction (MTP) support for the GLM-4.7-Flash model.

github llama.cpp · 4d ago

llama.cpp b10238 adds MTP support for Qwen3-Next

The llama.cpp project released build b10238, which introduces Multi-Token Prediction (MTP) support for the Qwen3-Next model family. This update includes specific implementation changes to handle MTP layers and fixes related to Python type checking within the codebase.