github llama.cpp · just now · inference

llama.cpp b10673 adds fa-vec tunings for Apple M4

from English

The llama.cpp project released build b10673, which includes specific performance optimizations for Apple M4 hardware. This update introduces fa_vec_tuned_table records into the ggml-metal-tuning module to improve Metal backend efficiency.

Adds tuned lookup tables for F16, Q4_0, Q4_1, Q5_0, Q5_1, and Q8_0 quantizations on M4 chips with 10 GPU cores.
Provides binaries for macOS Apple Silicon (arm64), macOS Intel (x64), and iOS XCFramework.
Includes Linux builds for Ubuntu x64 and arm64 across CPU, Vulkan, ROCm 7.14, OpenVINO, and SYCL backends.
Offers Windows binaries for CPU, OpenCL Adreno, CUDA 12/13, Vulkan, OpenVINO, SYCL, and ROCm 7.14.
Adds Android arm64 (CPU) support and UI binaries.

This release ensures that users on Apple M4 devices can leverage optimized Metal kernels for faster inference.

Importance 1/3 Trust 1/3 llama.cpp Hugging Face Inference efficiency

Read original

github llama.cpp · 2h ago · 2 views

llama.cpp b10670 routes quantized KV decode to TILE on Xe2 (BMG)

The llama.cpp project released version b10670, which includes a specific optimization for Intel's Xe2 architecture. The update routes the quantized key-value (KV) decode operation to use the TILE backend exclusively for BMG hardware, while keeping the VEC backend on other architectures until they are validated.

github llama.cpp · 2h ago · 2 views

llama.cpp b10670 routes quantized KV decode to TILE on Xe2 (BMG)

llama.cpp b10672 adds Whisper.cpp support, Qwen3.5 NPU enablement, and OpenVINO 2026.3.1 update

llama.cpp adds fa-vec tunings for Apple M3 Max, M5 and M5 Pro

llama.cpp b10665 adds DSpark support for Nemotron3.5

llama.cpp adds Qwen3.8-Flash-Next (qwen4exp) architecture support