github llama.cpp · 3h ago · inference

llama.cpp b10822 embeds UI assets with CMake and simplifies cross-compilation

from English

The llama.cpp project released build b10822, which modifies the build system to embed user interface assets directly using CMake. This change removes the need for a build-time C++ helper and an external gzip dependency, thereby simplifying the cross-compilation process.

The generated C++ code is kept in templates to maintain readability while preserving fully embedded UI assets.
Pre-built binaries are available for macOS (Apple Silicon and Intel), iOS, Linux (Ubuntu with CPU, Vulkan, ROCm 10.0, OpenVINO, SYCL), Android, Windows (CPU, CUDA 12/13, Vulkan, OpenCL, ROCm 10.0, OpenVINO, SYCL), and openEuler.
macOS Apple Silicon builds with KleidiAI are disabled in this release.

This update streamlines the build infrastructure for developers while providing comprehensive platform support for the latest llama.cpp version.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency Local inference

Read original

github llama.cpp · 2d ago · 3 views

llama.cpp b10816 adds M3 fa-vec tunings and new binaries

The llama.cpp project has released version b10816, which includes updates to the Metal backend for Apple Silicon devices. The release focuses on adding remaining fast-vector tunings specifically for the M3 chip architecture.

github llama.cpp · 2d ago · 5 views

llama.cpp b10796 adds n_expert_used_max to fix Nemotron-3-Puzzle loading

The llama.cpp project released build b10796, which includes a change to the model loader to support models with varying numbers of experts per layer.

github llama.cpp · 3d ago · 9 views

llama.cpp b10791 adds OpenCL quant lm_head and decode GEMV optimizations

The llama.cpp project released version b10791, introducing significant OpenCL optimizations for quantized language model heads and decoding GEMV operations. This update also includes medium-batch GEMM improvements relevant to speculative decoding and multi-token prediction.

github llama.cpp · 4d ago · 6 views

ggml-org releases llama.cpp b10774 with finetune KV cache fix

The ggml-org project has released version b10774 of llama.cpp, which includes a fix for the no KV cache issue in the finetune module.

github llama.cpp · 4d ago · 16 views

llama.cpp b10763 enables preserve_reasoning by default

The llama.cpp project released build b10763, which changes the default behavior of the `preserve_reasoning` chat template keyword argument. The server now logs the effective state of this kwarg and issues a warning when it is enabled by default on templates that support it.