github llama.cpp · 1h ago · inference

llama.cpp adds fa-vec tunings for Apple M3 Max, M5 and M5 Pro

from English

The llama.cpp project has added fast attention vector (fa-vec) tuning records for Apple M3 Max, M5, and M5 Pro GPUs to improve Metal performance.

Tunings for the Apple M5 were generated on an M5 machine using f16 and q8_0 data types.
Records for the Apple M5 Pro cover F16, Q4_0, and Q8_0 formats across 20 GPU cores.
Support for the M3 Max includes F16 and Q8_0 configurations on a MacBook Pro with 64GB in low power mode.

These updates extend Metal acceleration capabilities to newer Apple silicon architectures.

Importance 1/3 Trust 1/3 llama.cpp Hugging Face Inference efficiency

Read original

github llama.cpp · 14h ago · 3 views

llama.cpp b10665 adds DSpark support for Nemotron3.5

The llama.cpp project released build b10665, which introduces DSpark support for the Nemotron3.5 model.

github llama.cpp · 17h ago · 7 views

llama.cpp adds Qwen3.8-Flash-Next (qwen4exp) architecture support

The llama.cpp project has added support for the Qwen3.8-Flash-Next (qwen4exp) model architecture, implementing its specific components including hyper-connections, gated delta nets, Mixture of Experts, and PLE n-gram hash embeddings.

github llama.cpp · 1d ago · 8 views

llama.cpp b10653 adds TENSOR_READ_LAZY for model loading

The llama.cpp project has released build b10653, introducing the TENSOR_READ_LAZY feature to its model loader. This change includes the addition of the TENSOR_GET_ROW_LAZY function and a corresponding --tensor-read-lazy command-line flag.

github llama.cpp · 1d ago · 7 views

llama.cpp b10655 adds LIGHTNING_INDEXER support for Deepseek V4 ops on Vulkan

The llama.cpp project released build b10655, which introduces support for the LIGHTNING_INDEXER operation within the Vulkan backend to enable Deepseek V4 operations.

github llama.cpp · 2d ago · 6 views

llama.cpp b10639 fixes Vulkan warptile warp size assumption

The llama.cpp project released build b10639, which includes a fix for the Vulkan backend where warptiles previously assumed warp sizes were less than or equal to 64. This change clamps the warp size to work around issues with larger warps.