github llama.cpp · just now · inference

llama.cpp fixes MTT S5000 operator failures and enables CUB paths

from English

The llama.cpp project has released a fix for MUSA (MTT S5000) operator failures and build issues, specifically targeting the PH1 architecture. This update resolves critical bugs that caused incorrect results or timeouts on this hardware.

Fixes FlashAttention performance by correcting 16-byte copy width derivation for sm_70+ targets.
Enables CUB paths on MUSA, lifting limits on ARGSORT and TOP_K operations.
Resolves MMQ defects causing quantized matmul failures by switching to the cuBLAS/dequant path.
Fixes fused TOPK_MOE kernel hangs by ensuring block barriers are reachable for all warps.
Enables GATED_DELTA_NET operator support, significantly improving prefill and decode throughput.

These changes restore full functionality and improve performance for MTT S5000 users running models like Qwen3.8-27B.

Importance 1/3 Trust 1/3 llama.cpp Hugging Face Hardware & chips Inference efficiency

Read original

github llama.cpp · 20h ago · 6 views

llama.cpp b11160 adds int8 coopmat1 matmul for AMD RDNA3/4 and IQ4_XS support

The llama.cpp project released version b11160, introducing an int8 cooperative matrix (coopmat1) implementation for Vulkan on AMD RDNA3 and RDNA4 architectures. This update includes a dedicated shader that probes coopmat values directly to improve performance.

media Hugging Face Forums · 30d ago · 39 views

EdgeFirst publishes Rust pipeline validating YOLO models on 20 edge platforms

EdgeFirst has published the EdgeFirst Model Zoo on Hugging Face, featuring YOLOv5, YOLOv8, YOLO11, and YOLO26 models validated across 20 distinct edge platforms. The validation relies on a single Rust pipeline that handles zero-copy DMA pre-processing, model decoding, and post-processing directly on the target hardware.

github llama.cpp · 52d ago · 35 views

llama.cpp b10255 extends oneDNN SDPA to non-FP16 KV caches

The llama.cpp b10255 release extends the oneDNN SDPA path to support non-FP16 key-value (KV) caches, including Q4_0–Q8_0 quantized formats and FP32. This update allows the fused systolic kernel to run identically to the native FP16 path by dequantizing or converting K/V tensors to dense FP16 on-device before processing.

media r/LocalLLaMA · 82d ago · 29 views

llama.cpp uses hipBLAS for dense prefill on gfx900 GPUs

A pull request to the llama.cpp repository introduces a change that utilizes hipBLAS for dense prefill operations on AMD gfx900 architecture GPUs. This modification is specifically targeted at legacy Vega GPU hardware, including models like the Radeon RX Vega 56/64 and Radeon Pro Vega series.

github llama.cpp · 1d ago · 2 views

llama.cpp b11158 enables KHR cooperative matrix support for Adreno GPUs

The llama.cpp project released build b11158, which includes a tune of KHR cooperative matrix support specifically for Adreno GPUs via the Vulkan backend.