github llama.cpp · 6h ago · inference

llama.cpp b11118 introduces direct-mapped DMA cache for HVX FA mask handling

from English

The llama.cpp project has released build b11118, which includes a new hex-dma feature introducing a direct-mapped DMA cache better suited for Hexagon Vector eXtended (HVX) FA mask handling.

This release provides binaries and frameworks across multiple platforms and hardware backends:

macOS Apple Silicon (arm64), macOS Intel (x64), and iOS XCFramework
Linux Ubuntu x64, arm64, s390x, and various GPU/CPU backends including CUDA 12/13, ROCm 10.0, OpenVINO, SYCL, and Vulkan
Windows x64 and arm64 with CPU, OpenCL Adreno, CUDA 12/13, Vulkan, OpenVINO, SYCL, and ROCm 10.0 support
Android arm64 for CPU and Snapdragon (CPU, Adreno GPU, Hexagon NPU)
Linux arm64 for Snapdragon devices including CPU, Adreno GPU, and Hexagon NPU
A standalone UI build is also available.

The update enables users to run llama.cpp on a wide range of architectures, particularly improving support for Qualcomm Snapdragon processors with Hexagon NPUs through the new direct-mapped DMA cache.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency Local inference

Read original

github llama.cpp · 1h ago · 10 views Live

llama.cpp b11120 hides Vulkan internal symbols to fix state destruction

The llama.cpp project released version b11120, which addresses a critical bug in the Vulkan backend by hiding internal symbols. This change prevents duplicate-dlopen state destruction issues that occurred when internal symbols were exported.

github llama.cpp · 17h ago · 9 views

llama.cpp adds DFlash support for HunyuanOCR and fixes draft conversion

The llama.cpp project has added support for DFlash speculative decoding with the HunyuanOCR model by exposing layer input tensors in the target graph. This change allows the draft model to read residual streams, enabling image requests to run successfully with a draft acceptance rate of approximately 0.5 and byte-identical OCR output compared to non-speculative runs.

github llama.cpp · 17h ago · 13 views

llama.cpp b11104 adds support for binding llama-server to multiple addresses

The llama.cpp project released version b11104, introducing a new feature that allows the llama-server component to bind to multiple network addresses simultaneously.

github llama.cpp · 19h ago · 15 views

llama.cpp b11102 adds MiMo-V2.6 support

The llama.cpp project released build b11102, which introduces support for converting MiMo-V2.6 models.

lab Hugging Face Blog · 20h ago · 10 views

Transformers adds GGUF support via ggml kernels for local inference

Hugging Face's Transformers library now supports loading GGUF quantized models, leveraging the underlying ggml kernels to bring performance close to llama.cpp. This integration allows developers to run quantized checkpoints directly within the standard PyTorch-based API on supported hardware.