github llama.cpp · 1h ago · inference

llama.cpp b10605 flattens mamba2 projections to dispatch GEMM

from English

The llama.cpp project released build b10605, which includes a key optimization for the Mamba2 architecture. The update flattens the input and output projections of Mamba2 models to dispatch General Matrix Multiplication (GEMM) operations instead of General Matrix Vector (GEMV) operations.

Flattens mamba2 in/out projections to enable GMM dispatching rather than GEMV.
Removes redundant output reshape operations within the Mamba2 implementation.

This release provides binaries for macOS, Linux, Windows, Android, and openEuler across various hardware backends including CPU, CUDA, ROCm, Vulkan, and OpenVINO.

Importance 1/3 Trust 1/3 llama.cpp Hugging Face Inference efficiency

Read original

github llama.cpp · 15h ago · 8 views

llama.cpp b10603 adds MTP support for GLM-4.5-Air

The llama.cpp project has released build b10603, which introduces Multi-Token Prediction (MTP) support for the GLM-4.5-Air model.

github llama.cpp · 2d ago · 2 views

ggml optimizes concat op by replacing per-element memcpy with row-level memcpy

The ggml library has optimized its concatenation operation by replacing per-element memory copies with row-level memcpy operations. This change is part of the b10578 release of llama.cpp, which includes various platform-specific binaries and updates to the underlying computation graph.

github llama.cpp · 2d ago · 3 views

llama.cpp b10603 adds MTP support for GLM-4.5-Air

ggml optimizes concat op by replacing per-element memcpy with row-level memcpy

llama.cpp b10576 adds Q2_K reordered MMVQ and ESIMD kernels for SYCL

llama.cpp b10577 fixes draft-mtp with embeddings

llama.cpp b10568 uses ggml_rope_set_offset() for DeepSeek 2