github llama.cpp · 1h ago · inference

llama.cpp b11382 adds f16 support to WebGPU fill/set_rows

from English

The llama.cpp project has released build b11382, which includes a key update to its WebGPU backend. This release adds 16-bit floating-point (f16) support specifically for the `fill` and `set_rows` operations within the WebGPU implementation.

The source code change is tracked in pull request #29897 on the ggml-org/llama.cpp repository.
Binaries are provided for macOS (Apple Silicon and Intel), iOS, Linux (CPU, Vulkan, CUDA 12/13, ROCm, OpenVINO, SYCL, Snapdragon), Windows (CPU, OpenCL, CUDA, Vulkan, OpenVINO, SYCL, ROCm), Android, and openEuler.
The release also includes the standard llama.cpp UI binary.

Importance 1/3 Trust 1/3 llama.cpp Hardware & chips Inference efficiency

Read original

github llama.cpp · 20h ago · 10 views

llama.cpp b11370 fuses shared experts into MMVQ

The llama.cpp project has released build b11370, which includes a CUDA optimization to fuse shared experts into the Mixture of MoE with Vector Quantization (MMVQ) architecture. This change is part of pull request #29184 and also addresses buffer null checks and argument handling for stride_col_dst.

github llama.cpp · 2d ago · 6 views

llama.cpp b11335 routes sm70 to Turing MMVQ table for Volta performance

The llama.cpp project released version b11335, which includes a CUDA optimization for Volta GPUs. The update routes the sm_70 architecture to the existing Turing MMVQ nwarps table instead of falling through to the generic configuration.

media Hugging Face Forums · 3d ago · 6 views

ThermaCompute AI releases CrashLens for local GPU crash log triage

ThermaCompute AI has released CrashLens, a free, local tool designed to help engineers investigate GPU crashes by analyzing log files. The tool identifies known failure patterns within the logs and suggests specific next steps for investigation.

github llama.cpp · 4d ago · 8 views

llama.cpp b11280 release reduces tensor allreduce sync with pinned host buffers

The llama.cpp project has released version b11280, which includes a change to reduce tensor allreduce synchronization by utilizing pinned host buffers.

media Hugging Face Forums · 4d ago · 5 views

AI Hardware Fit connects model VRAM requirements to GPU selection

The creator of AI Hardware Fit has released a free, open-source browser tool designed to help users determine if a local model fits on their GPU and compare suitable hardware options. The tool addresses the complexity of piecing together model files, quantization, context length, and runtime support from disparate sources.