github llama.cpp · 1d ago · inference

llama.cpp b10696 adds fa-vec tuning for M3 Pro

from English

The llama.cpp project has released build b10696, which includes a pull request adding fast vector (fa-vec) tuning specifically for the Apple M3 Pro chip.

This update addresses issue #27668 to improve performance on this specific hardware configuration. The release provides binaries for macOS, iOS, Linux, Windows, Android, and openEuler across various backends including CPU, CUDA, ROCm, Vulkan, OpenVINO, and SYCL.

The fa-vec tuning is intended to optimize inference speed or efficiency for users running llama.cpp on devices equipped with the M3 Pro processor.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

github llama.cpp · just now · 1 view Live

llama.cpp b10721 fixes WebGPU crash and adds Windows CUDA 13.4 support

The llama.cpp project has released build b10721, which includes a critical fix for the WebGPU backend and expands hardware support on Windows.

github llama.cpp · 2h ago

llama.cpp b10718 extends MOE fusion to specdec and multi-token

The llama.cpp project released version b10718, which includes a significant update to its CUDA backend. The most notable change is the extension of Mixture of Experts (MOE) fusion to support speculative decoding (specdec), addressing previous limitations where MOE glu fusion and topk-router fusion were restricted to processing only one token at a time.

github llama.cpp · 4h ago · 3 views

llama.cpp b10721 fixes WebGPU crash and adds Windows CUDA 13.4 support

llama.cpp b10718 extends MOE fusion to specdec and multi-token

llama.cpp b10717 release enhances SYCL memory info retrieval

llama.cpp b10715 fuses DFlash encoder into KV cache injection

pipecat-ai releases phonellm-alpha-1 with GPT 5.6 Terra performance