github llama.cpp · 2h ago · inference

llama.cpp b11157 adds conv3d with implicit GEMM for CUDA

from English

The llama.cpp project has released version b11157, introducing support for 3D convolution operations using implicit GEMM on CUDA. This update refines the implementation to handle empty kernels and is available across a wide range of platforms including macOS, Linux, Windows, Android, and iOS.

Adds conv3d with implicit GEMM for CUDA backends.
Refines conv3d implicit GEMM logic to properly handle empty kernels.
Provides binaries for CPU, Vulkan, ROCm, OpenVINO, SYCL, and Snapdragon on Linux and Windows.
Includes builds for macOS Apple Silicon (arm64), macOS Intel (x64), and iOS XCFramework.
Offers Android arm64 builds with support for CPU, Adreno GPU, and Hexagon NPU.

This release expands the library's hardware acceleration capabilities by enabling 3D convolution operations on NVIDIA GPUs.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

github llama.cpp · 2h ago · 1 view

llama.cpp b11156 adds Ling 3.0 VL support via BailingMoeV3

The llama.cpp project has released build b11156, which introduces support for the Ling 3.0 VL model. This update folds Ling 3.0 VL into the BailingMoeV3 architecture and ensures shared NORM rope lists remain intact when gating BailingMoe3 on mrope sections.

github llama.cpp · 12h ago · 3 views

llama.cpp v0.5.0 adds HRM-Text support, CUDA conv2d acceleration, and multi-address binding

The llama.cpp project has released version 0.5.0, focusing on backend performance, broader model coverage, and more robust server operation. This update introduces support for new architectures like HRM-Text (DFM Mimir 1B) and MiMo-V2.6, while significantly enhancing computational efficiency through CUDA and Metal optimizations.

lab Microsoft Research Blog · 15h ago · 15 views

Microsoft Research shows offloading physical AI inference improves robot performance and battery life

Microsoft Research challenges the assumption that physical AI inference must run exclusively on onboard robot GPUs, demonstrating that offloading to edge or cloud infrastructure significantly enhances mobile manipulation workloads. Their systematic study of robotics workloads reveals that relying on onboard compute limits performance, battery life, and scalability.

github llama.cpp · 15h ago · 11 views

llama.cpp b11147 adds A8 Q6_K non-MoE dp4a OpenCL kernel

The llama.cpp project has released version b11147, which introduces a new binary kernel for OpenCL. This update specifically adds support for the A8 Q6_K non-MoE dp4a configuration.

github llama.cpp · 17h ago · 9 views

llama.cpp b11140 enables sparse-fa for dsv4 prefill on CUDA

The llama.cpp project released version b11140, which includes a CUDA optimization enabling sparse-fa for dsv4 prefill. This update addresses performance issues in the flash attention mask scanning process by templating kernels to allow compile-time loop unrolling.