github llama.cpp · just now · inference

llama.cpp b11333 adds WebGPU bfloat16 support for MUL_MAT operations

from English

The llama.cpp project has released build b11333, which introduces bfloat16 support for the MUL_MAT, MUL_MAT_ID, and GET_ROWS operations within its WebGPU backend.

This update is included in precompiled binaries for macOS (Apple Silicon and Intel), iOS, Linux (Ubuntu x64, arm64, s390x), Windows, Android, and openEuler. The release provides builds for various hardware accelerators including CUDA 12 and 13, Vulkan, ROCm 10.0, OpenVINO, SYCL, and Snapdragon NPUs.

The addition of bfloat16 precision to these specific WebGPU matrix operations allows for more efficient computation on compatible GPUs.

Importance 1/3 Trust 1/3 llama.cpp Hugging Face Inference efficiency

Read original

github llama.cpp · just now · 1 view Live

llama.cpp b11335 routes sm70 to Turing MMVQ table for Volta performance

The llama.cpp project released version b11335, which includes a CUDA optimization for Volta GPUs. The update routes the sm_70 architecture to the existing Turing MMVQ nwarps table instead of falling through to the generic configuration.

github llama.cpp · 23h ago · 5 views

llama.cpp b11309 optimizes Hexagon ALLREDUCE with safe scatter mode

The llama.cpp project released version b11309, which includes a key optimization for the Hexagon backend. This update adds support for safe scatter mode to the ALLREDUCE operation, aiming to improve performance and stability on Qualcomm Snapdragon devices.

github llama.cpp · 2d ago · 9 views

llama.cpp adds GLM-5.3-Flash support with k-pool and MTP fixes

The llama.cpp project has added initial support for the GLM-5.3-Flash (GLM5-Next) model architecture, including migration to hybrid-index memory and early Multi-Token Prediction (MTP) capabilities.

github llama.cpp · 2d ago · 5 views

llama.cpp b11266 loads F32 A matrix 2 at a time for Vulkan on Intel

The llama.cpp project released build b11266, which includes a Vulkan optimization that loads F32 A matrices two elements at a time when they are 2-aligned. This change addresses performance issues on Intel hardware where loading floats one by one is inefficient, and it also corrects a missing validation for `a_offset` alignment in the original patch.

github llama.cpp · 2d ago · 2 views

llama.cpp b11265 fixes Vulkan MOE matmul tile selection

The llama.cpp b11265 release includes a fix for the Vulkan backend's handling of Mixture of Experts (MoE) models. The update corrects how `mat_mul_id` selects matrix multiplication tiles, addressing a performance issue where workers were left idle during dispatch.