github llama.cpp · just now · inference

llama.cpp b10739 adds fa-vec tuning for M2 Max

from English

The llama.cpp project has released version b10739, which includes a specific performance optimization for Apple's M2 Max hardware. This update introduces fast attention vector (fa-vec) tuning tailored to the M2 Max's 30 GPU cores.

Added fa-vec tuning rows for M2 Max collected with ggml-metal-tuning on f16 and q8_0 dtypes.
Provided binaries for macOS Apple Silicon, Intel Macs, iOS, Linux (CPU, Vulkan, ROCm, OpenVINO, SYCL), Windows (CPU, CUDA 12/13, Vulkan, OpenVINO, SYCL, ROCm), Android, and openEuler.

This release ensures optimized inference performance on the latest Apple silicon while maintaining broad support across various operating systems and hardware accelerators.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency Local inference

Read original

lab Hugging Face Blog · just now · 1 view Live

Hugging Face releases 207 WebGPU kernels and a browser benchmarking tool

Hugging Face has released @huggingface/kernels, a library for loading and running optimized WebGPU kernels from the Hugging Face Hub, alongside an initial collection of 207 kernels. The release also introduces Fleet, an in-browser GPU benchmarking suite that crowdsources performance and correctness evidence from users' devices.

github llama.cpp · 4h ago · 1 view

Hugging Face releases 207 WebGPU kernels and a browser benchmarking tool

llama.cpp b10734 enables Metal 4.0 tensor API on M5+/A19+

ExLlamaV3 adds CPU offload for MoE experts and new GLM-5.3-Flash, Qwen3.8-Flash models

llama.cpp b10724 optimizes kv-cache state restore to reduce copy count from 1.3M to 224

llama.cpp b10721 fixes WebGPU crash and adds Windows CUDA 13.4 support