github llama.cpp · 2h ago · inference

llama.cpp b11045 adds ROLL op support to Hexagon backend

from English

The llama.cpp project has released version b11045, which introduces support for the ROLL operation within the Hexagon backend.

The release includes binaries for macOS (Apple Silicon and Intel), Linux (Ubuntu with CPU, Vulkan, CUDA 12/13, ROCm 10.0, OpenVINO, and SYCL), Windows (CPU, OpenCL Adreno, CUDA 12/13, Vulkan, OpenVINO, SYCL, and ROCm 10.0), Android arm64, and openEuler.
An iOS XCFramework and a standalone UI build are also provided.
KleidiAI support for macOS Apple Silicon is currently disabled in this release.

Importance 1/3 Trust 1/3 llama.cpp Hugging Face Inference efficiency

Read original

github llama.cpp · just now · 1 view Live

llama.cpp b11046 adds OpenCL support for flash_attn_f32_f16_bin kernel

The llama.cpp project released version b11046, which introduces OpenCL support for the `flash_attn_f32_f16_bin` kernel. This update also includes guarded prefill functionality for Flash Attention on OpenCL devices.

github llama.cpp · 1h ago · 1 view Live

llama.cpp b11044 updates hexagon im2col and releases binaries

The llama.cpp project has released version b11044, which includes an update to the hexagon backend for improved image-to-column (im2col) operations. The release provides pre-built binaries for macOS, Linux, Windows, Android, and openEuler across CPU, GPU, and other hardware accelerators.

github llama.cpp · 12h ago · 1 view

llama.cpp b11035 adds IQ3_S MMQ matmul kernels for Vulkan

The llama.cpp project has released version b11035, introducing support for IQ3_S MMQ matrix multiplication kernels in its Vulkan backend.

github llama.cpp · 1d ago · 5 views

llama.cpp b11026 skips gate_up_exps for Qwen35MoE

The llama.cpp project released build b11026, which introduces a change to skip gate_up_exps when the TENSOR_SKIP flag is set. This modification is specifically required for Qwen35MoE models where MTP tensors are fused but not loaded.

github llama.cpp · 2d ago · 4 views

llama.cpp b11019 fixes GGUF data alignment for embedded files and adds LoRA loading

The llama.cpp project released version b11019, addressing a critical bug where GGUF files embedded at non-aligned offsets returned incorrect tensor data. The update aligns the GGUF data section relative to the file start rather than offset 0, ensuring correct loading of embedded models.