github llama.cpp · 1h ago · inference

llama.cpp b10434 passes reasoning_effort to templates

from English

The llama.cpp project released build b10434, which adds support for passing the `reasoning_effort` parameter through to Jinja templates. This change ensures that OpenAI Chat Completions `reasoning_effort` values are stored and made available during generation.

The server now correctly reads `reasoning_effort` from the request body.
The Responses API conversion logic handles mapping `reasoning.effort` to `reasoning_effort`.
The parameter is exposed in `generation_params` and positioned next to `enable_thinking`.
Context is passed through the analyze function to mirror `preserve_reasoning` behavior.

This update allows users to control reasoning effort levels directly via template inputs, facilitating more granular configuration of model inference parameters.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency Local inference

Read original

github llama.cpp · 22h ago

llama.cpp b10418 adds SYCL host pinned memory support

The llama.cpp project released version b10418, introducing support for host pinned memory to improve SYCL Host-to-Device memory access. This update addresses thread-safety issues within the backend implementation.

github llama.cpp · 22h ago

llama.cpp OpenVINO backend adds Qwen3.5 support and memory optimizations

The llama.cpp project has merged significant updates to its OpenVINO backend, primarily enabling support for the Qwen3.5 model family and introducing several memory optimization techniques.

github llama.cpp · 1d ago · 1 view

llama.cpp b10414 adds TQ2_0 support and optimizes mul_mv kernel

The llama.cpp project released build b10414, introducing support for the GGML_TYPE_TQ2_0 ternary quantization type in the Metal backend. This update also includes optimizations to the contiguous tensor (cont) mul_mv kernel.

github llama.cpp · 4d ago · 15 views

llama.cpp b10354 fixes CPU affinity mask on Android

The llama.cpp project has released version b10354, which includes a fix for the CPU affinity mask being ignored on Android devices.

github llama.cpp · 4d ago · 15 views

llama.cpp b10344 adds MTP support for Nemotron models

The llama.cpp project has released build b10344, which introduces Multi-Token Prediction (MTP) support specifically for the Nemotron and Nemotron Nano models.