github llama.cpp · 1h ago · inference

llama.cpp b10763 enables preserve_reasoning by default

from English

The llama.cpp project released build b10763, which changes the default behavior of the `preserve_reasoning` chat template keyword argument. The server now logs the effective state of this kwarg and issues a warning when it is enabled by default on templates that support it.

The `--chat-template-kwargs` method for setting this option is deprecated in favor of explicit flags like `--reasoning-preserve` or `--no-reasoning-preserve`.
Pre-built binaries are available for macOS (Apple Silicon and Intel), iOS, Linux (CPU, Vulkan, ROCm 7.14, OpenVINO, SYCL), Android, Windows (CPU, CUDA 12/13, Vulkan, OpenVINO, SYCL, ROCm 7.14), and openEuler.
The release includes UI binaries for all supported platforms.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency Local inference

Read original

github llama.cpp · 8h ago · 1 view

llama.cpp b10760 fixes Qwen3-tts-0.6b inference stability

The llama.cpp project released build b10760, which includes specific fixes for the Qwen3-tts-0.6B model to prevent numerical instability during inference.

github llama.cpp · 23h ago · 1 view

llama.cpp b10751 fuses MoE weighted expert reduction for CUDA

The llama.cpp project released build b10751, which introduces a fused kernel for the MoE (Mixture of Experts) weighted expert reduction on CUDA. This optimization replaces the previous baseline that ran two physical kernels with a single weighted-reduction kernel to reduce intermediate global-memory traffic.

github llama.cpp · 1d ago · 1 view

llama.cpp b10760 fixes Qwen3-tts-0.6b inference stability

llama.cpp b10751 fuses MoE weighted expert reduction for CUDA

llama.cpp b10750 optimizes KV cache n-gram lookup for 4.9% throughput gain

llama.cpp b10742 adds fa-vec tunings for A18 Pro

llama.cpp b10739 adds fa-vec tuning for M2 Max