github llama.cpp · just now · inference

llama.cpp b11338 adds shared strided DMA copy for CPY and CONCAT

from English

The llama.cpp project released build b11338, introducing a shared strided Direct Memory Access (DMA) copy mechanism for the CPY and CONCAT operations. This update enables any-dimension CONCAT via DMA on Hexagon architectures.

Implemented shared strided DMA copy for CPY and CONCAT operations.
Enabled any-dimension CONCAT via DMA.
Removed broken CONCAT_DMA_MIN_ROW logic that was incompatible with 64-bit DMA.
Added missing dma_queue_flush() calls and additional guards for unsupported conditions.

This release provides binaries for macOS, Linux, Windows, Android, and openEuler across CPU, GPU, and NPU backends.

Importance 1/3 Trust 1/3 llama.cpp Hugging Face Inference efficiency

Read original

github llama.cpp · 2h ago · 1 view

llama.cpp b11335 routes sm70 to Turing MMVQ table for Volta performance

The llama.cpp project released version b11335, which includes a CUDA optimization for Volta GPUs. The update routes the sm_70 architecture to the existing Turing MMVQ nwarps table instead of falling through to the generic configuration.

github llama.cpp · 3h ago · 1 view

llama.cpp b11333 adds WebGPU bfloat16 support for MUL_MAT operations

The llama.cpp project has released build b11333, which introduces bfloat16 support for the MUL_MAT, MUL_MAT_ID, and GET_ROWS operations within its WebGPU backend.

github llama.cpp · 1d ago · 5 views

llama.cpp b11309 optimizes Hexagon ALLREDUCE with safe scatter mode

The llama.cpp project released version b11309, which includes a key optimization for the Hexagon backend. This update adds support for safe scatter mode to the ALLREDUCE operation, aiming to improve performance and stability on Qualcomm Snapdragon devices.

github llama.cpp · 2d ago · 9 views

llama.cpp adds GLM-5.3-Flash support with k-pool and MTP fixes

The llama.cpp project has added initial support for the GLM-5.3-Flash (GLM5-Next) model architecture, including migration to hybrid-index memory and early Multi-Token Prediction (MTP) capabilities.

github llama.cpp · 2d ago · 5 views

llama.cpp b11266 loads F32 A matrix 2 at a time for Vulkan on Intel

The llama.cpp project released build b11266, which includes a Vulkan optimization that loads F32 A matrices two elements at a time when they are 2-aligned. This change addresses performance issues on Intel hardware where loading floats one by one is inefficient, and it also corrects a missing validation for `a_offset` alignment in the original patch.