github llama.cpp · 1h ago · inference

llama.cpp b10447 redesigns yield_to_queue thread model

from English

The llama.cpp project has released build b10447, which includes a re-design of the `yield_to_queue` thread model. This change involves running `common_speculative_process` in the worker and swapping the worker-to-main thread design.

The release provides binaries for macOS (Apple Silicon and Intel), Linux (Ubuntu with CPU, Vulkan, OpenVINO, SYCL, and disabled ROCm), Android, Windows (CPU, OpenCL, CUDA 12/13, Vulkan, OpenVINO, SYCL, and ROCm 7.14), and openEuler (with some builds disabled).

This update is available for download across the supported platforms.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency Local inference

Read original

github llama.cpp · 3h ago · 1 view

llama.cpp b10444 adds MTP assistant model support and drops eagle3

The llama.cpp project has released version b10444, introducing support for loading Multi-Token Prediction (MTP) assistant models via the `--models-dir` flag. This update also includes strict prefix checking for MTP models in presets and removes support for the eagle3 draft type.

github llama.cpp · 6h ago

llama.cpp b10442 adds Vulkan SHMEM_STRIDE_PAD for Intel Xe and fixes OOB read

The llama.cpp project released version b10442, introducing specific Vulkan optimizations for Intel Xe hardware while addressing memory safety issues in matrix multiplication routines.

github llama.cpp · 1d ago · 3 views

llama.cpp b10434 passes reasoning_effort to templates

The llama.cpp project released build b10434, which adds support for passing the `reasoning_effort` parameter through to Jinja templates. This change ensures that OpenAI Chat Completions `reasoning_effort` values are stored and made available during generation.

github llama.cpp · 2d ago

llama.cpp b10418 adds SYCL host pinned memory support

The llama.cpp project released version b10418, introducing support for host pinned memory to improve SYCL Host-to-Device memory access. This update addresses thread-safety issues within the backend implementation.

github llama.cpp · 2d ago · 4 views

llama.cpp OpenVINO backend adds Qwen3.5 support and memory optimizations

The llama.cpp project has merged significant updates to its OpenVINO backend, primarily enabling support for the Qwen3.5 model family and introducing several memory optimization techniques.