github llama.cpp · 20h ago · inference

llama.cpp b11101 release allows repeated find_package calls for llama

from English

The llama.cpp project has released version b11101, which includes a build system fix allowing repeated find_package calls for the llama library.

cmake: allow repeated find_package calls for llama (PR #29228)
macOS/iOS binaries provided for Apple Silicon (arm64), Intel (x64), and iOS XCFramework; KleidiAI support is disabled.
Linux binaries available for Ubuntu x64, arm64, and s390x across CPU, Vulkan, CUDA 12/13, ROCm 10.0, OpenVINO, SYCL, and Snapdragon backends.
Android arm64 binaries released for CPU and Snapdragon (CPU, Adreno GPU, Hexagon NPU).
Windows binaries provided for x64 and arm64 CPUs, OpenCL Adreno, CUDA 12/13, Vulkan, OpenVINO, SYCL, and ROCm 10.0.
openEuler support is largely disabled, with only specific 310p and 910b (ACL Graph) configurations listed.

This release provides updated binaries for a wide range of hardware accelerators and operating systems, alongside the cmake fix.

Importance 1/3 Trust 1/3 llama.cpp Hugging Face Inference efficiency

Read original

github llama.cpp · just now · 1 view Live

llama.cpp b11122 extends MMVQ GLU fusion and adds new SYCL fusions

The llama.cpp project has released version b11122, which includes significant updates to the SYCL backend. The release extends the MMVQ GLU fusion to support mixed quantization types and introduces new fusions for rms_norm+scale and ssm_conv+silu operations.

github llama.cpp · 14h ago · 11 views

llama.cpp b11111 adds Intel Xe flash attention optimization kernels

The llama.cpp project released build b11111, which introduces Vulkan-based Intel Xe flash attention optimization kernels for split k paths on Xe-LPG Plus, Xe2, and Xe3 architectures.

github llama.cpp · 17h ago · 9 views

llama.cpp adds DFlash support for HunyuanOCR and fixes draft conversion

The llama.cpp project has added support for DFlash speculative decoding with the HunyuanOCR model by exposing layer input tensors in the target graph. This change allows the draft model to read residual streams, enabling image requests to run successfully with a draft acceptance rate of approximately 0.5 and byte-identical OCR output compared to non-speculative runs.

lab Hugging Face Blog · 21h ago · 10 views

Transformers adds GGUF support via ggml kernels for local inference

Hugging Face's Transformers library now supports loading GGUF quantized models, leveraging the underlying ggml kernels to bring performance close to llama.cpp. This integration allows developers to run quantized checkpoints directly within the standard PyTorch-based API on supported hardware.

github llama.cpp · 1d ago · 11 views

llama.cpp b11097 adds A8 Q4_0 non-MoE dp4a binary kernel

The llama.cpp project has released version b11097, which introduces an OpenCL binary kernel for the A8 Q4_0 non-MoE dp4a format.