github llama.cpp · 2h ago · inference

llama.cpp b10816 adds M3 fa-vec tunings and new binaries

from English

The llama.cpp project has released version b10816, which includes updates to the Metal backend for Apple Silicon devices. The release focuses on adding remaining fast-vector tunings specifically for the M3 chip architecture.

Adds M3 entries to the fa_vec_tuned_table.
Includes q4_0, q4_1, q5_0, and q5_1 quantizations in ggml-metal-tuning.
Provides binaries for macOS (Apple Silicon and Intel), iOS, Linux, Windows, Android, and openEuler.
Supports various backends including CPU, Vulkan, ROCm 10.0, OpenVINO, SYCL, CUDA 12/13, and OpenCL Adreno.

This update ensures optimized performance for M3 hardware while expanding the availability of pre-built binaries across multiple operating systems and GPU architectures.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency Local inference

Read original

github llama.cpp · 16h ago · 1 view

llama.cpp b10796 adds n_expert_used_max to fix Nemotron-3-Puzzle loading

The llama.cpp project released build b10796, which includes a change to the model loader to support models with varying numbers of experts per layer.

github llama.cpp · 1d ago · 3 views

llama.cpp b10791 adds OpenCL quant lm_head and decode GEMV optimizations

The llama.cpp project released version b10791, introducing significant OpenCL optimizations for quantized language model heads and decoding GEMV operations. This update also includes medium-batch GEMM improvements relevant to speculative decoding and multi-token prediction.

github llama.cpp · 2d ago · 3 views

ggml-org releases llama.cpp b10774 with finetune KV cache fix

The ggml-org project has released version b10774 of llama.cpp, which includes a fix for the no KV cache issue in the finetune module.

github llama.cpp · 2d ago · 9 views

llama.cpp b10763 enables preserve_reasoning by default

The llama.cpp project released build b10763, which changes the default behavior of the `preserve_reasoning` chat template keyword argument. The server now logs the effective state of this kwarg and issues a warning when it is enabled by default on templates that support it.

github llama.cpp · 2d ago · 3 views

llama.cpp b10760 fixes Qwen3-tts-0.6b inference stability

The llama.cpp project released build b10760, which includes specific fixes for the Qwen3-tts-0.6B model to prevent numerical instability during inference.