github llama.cpp · 1h ago · inference

llama.cpp b10577 fixes draft-mtp with embeddings

from English

The llama.cpp project released build b10577, which includes a fix for draft-mtp when used with embeddings. This update addresses issues #26352 and #27299.

Fixes draft-mtp compatibility with embeddings in the common module.
Provides binaries for macOS (Apple Silicon and Intel), Linux (CPU, Vulkan, OpenVINO, SYCL), Windows (CPU, CUDA 12/13, Vulkan, OpenCL, ROCm), Android, and openEuler.

This release ensures that draft-mtp functions correctly with embedding inputs across supported platforms.

Importance 1/3 Trust 1/3 llama.cpp Hugging Face Inference efficiency

Read original

github llama.cpp · just now Live

ggml optimizes concat op by replacing per-element memcpy with row-level memcpy

The ggml library has optimized its concatenation operation by replacing per-element memory copies with row-level memcpy operations. This change is part of the b10578 release of llama.cpp, which includes various platform-specific binaries and updates to the underlying computation graph.

github llama.cpp · 1h ago · 1 view Live

ggml optimizes concat op by replacing per-element memcpy with row-level memcpy

llama.cpp b10576 adds Q2_K reordered MMVQ and ESIMD kernels for SYCL

llama.cpp b10568 uses ggml_rope_set_offset() for DeepSeek 2

llama.cpp v0.2.0 release with ggml 0.21.0 and Kleidiai SME2 support

llama.cpp b10541 adds --mmproj-device argument for multimodal projects