github llama.cpp · 1d ago · inference

llama.cpp b10693 adds Hexagon NPU device discovery and on-demand session support

from English

The llama.cpp project released build b10693, which introduces support for Qualcomm Hexagon NPUs. This update enables runtime discovery of available NPU cores and allows for the creation of inference sessions on demand.

Added lazy session allocation and cleanup interfaces for Hexagon devices.
Implemented runtime discovery of available NPU cores.
Configured early rejection of non-existing devices during initialization.
Provided binaries for macOS, Linux, Windows, Android, and openEuler across CPU, Vulkan, ROCm, OpenVINO, SYCL, and CUDA backends.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

github llama.cpp · just now · 1 view Live

llama.cpp b10721 fixes WebGPU crash and adds Windows CUDA 13.4 support

The llama.cpp project has released build b10721, which includes a critical fix for the WebGPU backend and expands hardware support on Windows.

github llama.cpp · 2h ago

llama.cpp b10718 extends MOE fusion to specdec and multi-token

The llama.cpp project released version b10718, which includes a significant update to its CUDA backend. The most notable change is the extension of Mixture of Experts (MOE) fusion to support speculative decoding (specdec), addressing previous limitations where MOE glu fusion and topk-router fusion were restricted to processing only one token at a time.

github llama.cpp · 4h ago · 3 views

llama.cpp b10721 fixes WebGPU crash and adds Windows CUDA 13.4 support

llama.cpp b10718 extends MOE fusion to specdec and multi-token

llama.cpp b10717 release enhances SYCL memory info retrieval

llama.cpp b10715 fuses DFlash encoder into KV cache injection

pipecat-ai releases phonellm-alpha-1 with GPT 5.6 Terra performance