github llama.cpp · 8h ago · inference

llama.cpp b11280 release reduces tensor allreduce sync with pinned host buffers

from English

The llama.cpp project has released version b11280, which includes a change to reduce tensor allreduce synchronization by utilizing pinned host buffers.

This update is available across multiple platforms and backends, including macOS (Apple Silicon and Intel), Linux (CPU, Vulkan, CUDA 12/13, ROCm 10.0, OpenVINO, SYCL, and Snapdragon), Windows (CPU, OpenCL, CUDA 12/13, Vulkan, OpenVINO, SYCL, and ROCm 10.0), Android, and iOS.

The release also provides binaries for the llama.cpp UI.

Importance 1/3 Trust 1/3 llama.cpp Hardware & chips Inference efficiency

Read original

media Hugging Face Forums · just now Live

ThermaCompute AI releases CrashLens for local GPU crash log triage

ThermaCompute AI has released CrashLens, a free, local tool designed to help engineers investigate GPU crashes by analyzing log files. The tool identifies known failure patterns within the logs and suggests specific next steps for investigation.

media Hugging Face Forums · 18h ago · 1 view

AI Hardware Fit connects model VRAM requirements to GPU selection

The creator of AI Hardware Fit has released a free, open-source browser tool designed to help users determine if a local model fits on their GPU and compare suitable hardware options. The tool addresses the complexity of piecing together model files, quantization, context length, and runtime support from disparate sources.

github llama.cpp · 4d ago · 8 views

llama.cpp b11203 adds F16 input support to CUDA FWHT

The llama.cpp project released version b11203, which includes a change to the CUDA Fast Walsh-Hadamard Transform (FWHT) to accept F16 inputs. Previously, the CUDA FWHT only accepted F32 inputs; this update makes the source type a template parameter so the kernel reads an F16 source directly without requiring a converted copy.

github llama.cpp · 5d ago · 21 views

llama.cpp fixes MTT S5000 operator failures and enables CUB paths

The llama.cpp project has released a fix for MUSA (MTT S5000) operator failures and build issues, specifically targeting the PH1 architecture. This update resolves critical bugs that caused incorrect results or timeouts on this hardware.

github llama.cpp · 6d ago · 22 views

llama.cpp b11160 adds int8 coopmat1 matmul for AMD RDNA3/4 and IQ4_XS support

The llama.cpp project released version b11160, introducing an int8 cooperative matrix (coopmat1) implementation for Vulkan on AMD RDNA3 and RDNA4 architectures. This update includes a dedicated shader that probes coopmat values directly to improve performance.