github llama.cpp · 1h ago · inference

llama.cpp b10603 adds MTP support for GLM-4.5-Air

from English

The llama.cpp project has released build b10603, which introduces Multi-Token Prediction (MTP) support for the GLM-4.5-Air model.

Adds MTP capability for GLM-4.5-Air via pull request #26534.
Provides binaries for macOS (Apple Silicon and Intel), iOS, Linux (CPU, Vulkan, ROCm, OpenVINO, SYCL), Android, Windows (CPU, CUDA 12/13, Vulkan, OpenCL, OpenVINO, SYCL, ROCm), and openEuler.

This release enables users to run GLM-4.5-Air with MTP on a wide range of hardware architectures.

Importance 1/3 Trust 1/3 llama.cpp Hugging Face Inference efficiency

Read original

github llama.cpp · 1d ago · 1 view

ggml optimizes concat op by replacing per-element memcpy with row-level memcpy

The ggml library has optimized its concatenation operation by replacing per-element memory copies with row-level memcpy operations. This change is part of the b10578 release of llama.cpp, which includes various platform-specific binaries and updates to the underlying computation graph.

github llama.cpp · 1d ago · 3 views

llama.cpp b10576 adds Q2_K reordered MMVQ and ESIMD kernels for SYCL

The llama.cpp project has released build b10576, which re-adds Q2_K reordered MMVQ and ESIMD kernels for SYCL support. This change effectively reverts a previous commit that had removed these specific kernel implementations.

github llama.cpp · 2d ago · 2 views

llama.cpp b10577 fixes draft-mtp with embeddings

The llama.cpp project released build b10577, which includes a fix for draft-mtp when used with embeddings. This update addresses issues #26352 and #27299.

github llama.cpp · 2d ago · 5 views

llama.cpp b10568 uses ggml_rope_set_offset() for DeepSeek 2

The llama.cpp project released build b10568, which updates the model handling to use the `ggml_rope_set_offset()` function. This change is partially applied to support DeepSeek 2.

github llama.cpp · 2d ago · 11 views

llama.cpp v0.2.0 release with ggml 0.21.0 and Kleidiai SME2 support

The llama.cpp project has released version 0.2.0, which includes a synchronization with the ggml library bumped to version 0.21.0. This update introduces several backend improvements and new features across various hardware platforms.