github llama.cpp · 2d ago · inference

llama.cpp b11081 release with configurable tensor data stdev and improved test tools

from English

The llama.cpp project has released build b11081, which includes several updates to the `test-llama-archs` testing infrastructure. Key changes involve making tensor data standard deviation configurable, expanding usage examples, and adding support for architecture regex patterns.

The `test-llama-archs` tool now allows configurable tensor data standard deviation and fails on unknown arguments with logged usage.
Recurrent state rollback tests report NMSE for logits mismatches and use NMSE for rollback checks.
Synthetic fixtures zero DSA indexer score projection, and invalid tests are disabled.
The release provides binaries for macOS (Apple Silicon/Intel), iOS, Linux (CPU/Vulkan/CUDA/ROCm/OpenVINO/SYCL), Android, Windows (CPU/OpenCL/CUDA/Vulkan/OpenVINO/SYCL/ROCm), and openEuler.

This update ensures broader hardware support across multiple platforms while improving the reliability and configurability of internal architecture testing.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency Local inference

Read original

github llama.cpp · just now · 1 view Live

llama.cpp b11122 extends MMVQ GLU fusion and adds new SYCL fusions

The llama.cpp project has released version b11122, which includes significant updates to the SYCL backend. The release extends the MMVQ GLU fusion to support mixed quantization types and introduces new fusions for rms_norm+scale and ssm_conv+silu operations.

github llama.cpp · 1h ago · 11 views Live

llama.cpp b11120 hides Vulkan internal symbols to fix state destruction

The llama.cpp project released version b11120, which addresses a critical bug in the Vulkan backend by hiding internal symbols. This change prevents duplicate-dlopen state destruction issues that occurred when internal symbols were exported.

github llama.cpp · 6h ago · 13 views

llama.cpp b11118 introduces direct-mapped DMA cache for HVX FA mask handling

The llama.cpp project has released build b11118, which includes a new hex-dma feature introducing a direct-mapped DMA cache better suited for Hexagon Vector eXtended (HVX) FA mask handling.

github llama.cpp · 17h ago · 9 views

llama.cpp adds DFlash support for HunyuanOCR and fixes draft conversion

The llama.cpp project has added support for DFlash speculative decoding with the HunyuanOCR model by exposing layer input tensors in the target graph. This change allows the draft model to read residual streams, enabling image requests to run successfully with a draft acceptance rate of approximately 0.5 and byte-identical OCR output compared to non-speculative runs.

github llama.cpp · 18h ago · 13 views

llama.cpp b11104 adds support for binding llama-server to multiple addresses

The llama.cpp project released version b11104, introducing a new feature that allows the llama-server component to bind to multiple network addresses simultaneously.