github llama.cpp · just now · inference

llama.cpp b10444 adds MTP assistant model support and drops eagle3

from English

The llama.cpp project has released version b10444, introducing support for loading Multi-Token Prediction (MTP) assistant models via the `--models-dir` flag. This update also includes strict prefix checking for MTP models in presets and removes support for the eagle3 draft type.

common: support --models-dir loading MTP assistant models
common: preset: check for MTP models with strict prefix
handle other draft types
drop eagle3

The release provides binaries for macOS (Apple Silicon and Intel), Linux (Ubuntu x64, arm64, s390x, Vulkan, OpenVINO, SYCL), Android, Windows (CPU, CUDA 12/13, Vulkan, OpenVINO, SYCL, ROCm), and openEuler.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency Local inference

Read original

github llama.cpp · 4h ago

llama.cpp b10442 adds Vulkan SHMEM_STRIDE_PAD for Intel Xe and fixes OOB read

The llama.cpp project released version b10442, introducing specific Vulkan optimizations for Intel Xe hardware while addressing memory safety issues in matrix multiplication routines.

github llama.cpp · 1d ago · 2 views

llama.cpp b10434 passes reasoning_effort to templates

The llama.cpp project released build b10434, which adds support for passing the `reasoning_effort` parameter through to Jinja templates. This change ensures that OpenAI Chat Completions `reasoning_effort` values are stored and made available during generation.

github llama.cpp · 2d ago

llama.cpp b10442 adds Vulkan SHMEM_STRIDE_PAD for Intel Xe and fixes OOB read

llama.cpp b10434 passes reasoning_effort to templates

llama.cpp b10418 adds SYCL host pinned memory support

llama.cpp OpenVINO backend adds Qwen3.5 support and memory optimizations

llama.cpp b10414 adds TQ2_0 support and optimizes mul_mv kernel