github llama.cpp · 1h ago · inference

ggml-org releases llama.cpp b10774 with finetune KV cache fix

from English

The ggml-org project has released version b10774 of llama.cpp, which includes a fix for the no KV cache issue in the finetune module.

The training component now correctly handles KV cache usage during fine-tuning operations.
Binaries are provided for macOS (Apple Silicon and Intel), Linux (Ubuntu with CPU, Vulkan, ROCm, OpenVINO, SYCL), Android, Windows (CPU, CUDA 12/13, Vulkan, OpenCL, OpenVINO, SYCL, ROCm), and openEuler.
An iOS XCFramework and a standalone UI package are also included in the release assets.

This update ensures that fine-tuning workflows function correctly by addressing the previously broken key-value cache handling.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency Local inference

Read original

github llama.cpp · 9h ago · 5 views

llama.cpp b10763 enables preserve_reasoning by default

The llama.cpp project released build b10763, which changes the default behavior of the `preserve_reasoning` chat template keyword argument. The server now logs the effective state of this kwarg and issues a warning when it is enabled by default on templates that support it.

github llama.cpp · 16h ago · 1 view

llama.cpp b10760 fixes Qwen3-tts-0.6b inference stability

The llama.cpp project released build b10760, which includes specific fixes for the Qwen3-tts-0.6B model to prevent numerical instability during inference.

github llama.cpp · 1d ago · 2 views

llama.cpp b10751 fuses MoE weighted expert reduction for CUDA

The llama.cpp project released build b10751, which introduces a fused kernel for the MoE (Mixture of Experts) weighted expert reduction on CUDA. This optimization replaces the previous baseline that ran two physical kernels with a single weighted-reduction kernel to reduce intermediate global-memory traffic.

github llama.cpp · 1d ago · 2 views

llama.cpp b10750 optimizes KV cache n-gram lookup for 4.9% throughput gain

The llama.cpp project released build b10750, which refactors the key-value cell management to improve sequence position indexing and n-gram history lookups.

github llama.cpp · 1d ago · 2 views

llama.cpp b10742 adds fa-vec tunings for A18 Pro

The llama.cpp project has released version b10742, which includes a new feature adding fast vector (fa-vec) tunings specifically for the Apple A18 Pro chip found in the MacBook Neo.