github llama.cpp · 1d ago · inference

llama.cpp b11026 skips gate_up_exps for Qwen35MoE

from English

The llama.cpp project released build b11026, which introduces a change to skip gate_up_exps when the TENSOR_SKIP flag is set. This modification is specifically required for Qwen35MoE models where MTP tensors are fused but not loaded.

The release provides binaries and frameworks for macOS (Apple Silicon and Intel), iOS, Linux (CPU, Vulkan, CUDA 12/13, ROCm 10.0, OpenVINO, SYCL), Android, Windows (CPU, OpenCL, CUDA 12/13, Vulkan, OpenVINO, SYCL, ROCm 10.0), and openEuler.

This update ensures compatibility with specific Qwen35MoE configurations by handling fused but unloaded MTP tensors correctly.

Importance 1/3 Trust 1/3 llama.cpp Hugging Face Inference efficiency

Read original

github llama.cpp · just now · 1 view Live

llama.cpp b11046 adds OpenCL support for flash_attn_f32_f16_bin kernel

The llama.cpp project released version b11046, which introduces OpenCL support for the `flash_attn_f32_f16_bin` kernel. This update also includes guarded prefill functionality for Flash Attention on OpenCL devices.

github llama.cpp · 1h ago · 1 view Live

llama.cpp b11044 updates hexagon im2col and releases binaries

The llama.cpp project has released version b11044, which includes an update to the hexagon backend for improved image-to-column (im2col) operations. The release provides pre-built binaries for macOS, Linux, Windows, Android, and openEuler across CPU, GPU, and other hardware accelerators.

github llama.cpp · 2h ago · 1 view

llama.cpp b11045 adds ROLL op support to Hexagon backend

The llama.cpp project has released version b11045, which introduces support for the ROLL operation within the Hexagon backend.

github llama.cpp · 12h ago · 1 view

llama.cpp b11035 adds IQ3_S MMQ matmul kernels for Vulkan

The llama.cpp project has released version b11035, introducing support for IQ3_S MMQ matrix multiplication kernels in its Vulkan backend.

github llama.cpp · 2d ago · 4 views

llama.cpp b11019 fixes GGUF data alignment for embedded files and adds LoRA loading

The llama.cpp project released version b11019, addressing a critical bug where GGUF files embedded at non-aligned offsets returned incorrect tensor data. The update aligns the GGUF data section relative to the file start rather than offset 0, ensuring correct loading of embedded models.