github llama.cpp · 1h ago · inference

llama.cpp b11044 updates hexagon im2col and releases binaries

from English

The llama.cpp project has released version b11044, which includes an update to the hexagon backend for improved image-to-column (im2col) operations. The release provides pre-built binaries for macOS, Linux, Windows, Android, and openEuler across CPU, GPU, and other hardware accelerators.

ggml-hexagon now accepts 1D and padded IM2COL ops.
The pure-DDR IM2COL kernel is made 2D-aware.
The IM2COL DMA patch-embed fast path is extended to 1D.
A blocked-staging general IM2COL DMA kernel is added.

Importance 1/3 Trust 1/3 llama.cpp Hugging Face Inference efficiency

Read original

github llama.cpp · just now · 1 view Live

llama.cpp b11046 adds OpenCL support for flash_attn_f32_f16_bin kernel

The llama.cpp project released version b11046, which introduces OpenCL support for the `flash_attn_f32_f16_bin` kernel. This update also includes guarded prefill functionality for Flash Attention on OpenCL devices.

github llama.cpp · 2h ago · 1 view

llama.cpp b11045 adds ROLL op support to Hexagon backend

The llama.cpp project has released version b11045, which introduces support for the ROLL operation within the Hexagon backend.

github llama.cpp · 12h ago · 1 view

llama.cpp b11035 adds IQ3_S MMQ matmul kernels for Vulkan

The llama.cpp project has released version b11035, introducing support for IQ3_S MMQ matrix multiplication kernels in its Vulkan backend.

github llama.cpp · 1d ago · 5 views

llama.cpp b11026 skips gate_up_exps for Qwen35MoE

The llama.cpp project released build b11026, which introduces a change to skip gate_up_exps when the TENSOR_SKIP flag is set. This modification is specifically required for Qwen35MoE models where MTP tensors are fused but not loaded.

github llama.cpp · 2d ago · 4 views

llama.cpp b11019 fixes GGUF data alignment for embedded files and adds LoRA loading

The llama.cpp project released version b11019, addressing a critical bug where GGUF files embedded at non-aligned offsets returned incorrect tensor data. The update aligns the GGUF data section relative to the file start rather than offset 0, ensuring correct loading of embedded models.