github llama.cpp · just now · inference

llama.cpp b11335 routes sm70 to Turing MMVQ table for Volta performance

from English

The llama.cpp project released version b11335, which includes a CUDA optimization for Volta GPUs. The update routes the sm_70 architecture to the existing Turing MMVQ nwarps table instead of falling through to the generic configuration.

Volta (sm_70) now uses the TURING tuning for K-quant batch-1 decode, changing the warp count from 4 to 2.
Benchmarks on a Tesla V100 with Qwen3.8-27B show a +3.17% throughput increase (+1.091 t/s) with bit-identical perplexity.
The change is derived from the anyei/llamacpp-v100 fork and applies to both device and host table selectors.

This adjustment improves inference speed on Volta hardware without affecting model accuracy.

Importance 1/3 Trust 1/3 llama.cpp Hugging Face Hardware & chips Inference efficiency

Read original

github llama.cpp · 7d ago · 22 views

llama.cpp fixes MTT S5000 operator failures and enables CUB paths

The llama.cpp project has released a fix for MUSA (MTT S5000) operator failures and build issues, specifically targeting the PH1 architecture. This update resolves critical bugs that caused incorrect results or timeouts on this hardware.

github llama.cpp · 7d ago · 33 views

llama.cpp b11160 adds int8 coopmat1 matmul for AMD RDNA3/4 and IQ4_XS support

The llama.cpp project released version b11160, introducing an int8 cooperative matrix (coopmat1) implementation for Vulkan on AMD RDNA3 and RDNA4 architectures. This update includes a dedicated shader that probes coopmat values directly to improve performance.

media Hugging Face Forums · 37d ago · 40 views

EdgeFirst publishes Rust pipeline validating YOLO models on 20 edge platforms

EdgeFirst has published the EdgeFirst Model Zoo on Hugging Face, featuring YOLOv5, YOLOv8, YOLO11, and YOLO26 models validated across 20 distinct edge platforms. The validation relies on a single Rust pipeline that handles zero-copy DMA pre-processing, model decoding, and post-processing directly on the target hardware.

github llama.cpp · 59d ago · 38 views

llama.cpp b10255 extends oneDNN SDPA to non-FP16 KV caches

The llama.cpp b10255 release extends the oneDNN SDPA path to support non-FP16 key-value (KV) caches, including Q4_0–Q8_0 quantized formats and FP32. This update allows the fused systolic kernel to run identically to the native FP16 path by dequantizing or converting K/V tensors to dense FP16 on-device before processing.

media r/LocalLLaMA · 88d ago · 33 views

llama.cpp uses hipBLAS for dense prefill on gfx900 GPUs

A pull request to the llama.cpp repository introduces a change that utilizes hipBLAS for dense prefill operations on AMD gfx900 architecture GPUs. This modification is specifically targeted at legacy Vega GPU hardware, including models like the Radeon RX Vega 56/64 and Radeon Pro Vega series.