github llama.cpp · just now · inference

llama.cpp b10677 fixes Vulkan view-alias dependency bug

from English

The llama.cpp b10677 release addresses a critical bug in the Vulkan backend where missing view-alias dependencies caused the graph optimizer to incorrectly reorder nodes.

The fix ensures that `is_src_of` correctly treats two views of one tensor as dependent, preventing silent errors during greedy decoding and speculative decoding.
Nodes with operations like NONE, RESHAPE, TRANSPOSE, VIEW, or PERMUTE are no longer treated as aliasing dependencies, restoring optimizer freedom without compromising correctness.
The code compiles by making the lambda parameter const and capturing `is_empty` in `is_src_of`, resolving visibility issues with non-const pointers.
This change specifically fixes recurrent state issues for models like Qwen3.8 on AMD and NVIDIA Vulkan hardware, while CUDA remains unaffected.

The release resolves issue #27805 and provides binaries for macOS, Linux, Windows, Android, and openEuler across CPU, GPU, and various accelerator backends.

Importance 1/3 Trust 1/3 llama.cpp Hugging Face Inference efficiency

Read original

github llama.cpp · 2h ago · 1 view

llama.cpp b10676 fixes conv_transpose_2d for multiple batches in ggml and Metal

The llama.cpp release b10676 corrects a bug in `conv_transpose_2d` where only the first batch was computed, leaving subsequent batches as zero. This fix applies to both the core ggml library and the Metal backend implementation.

github llama.cpp · 4h ago · 1 view

llama.cpp b10673 adds fa-vec tunings for Apple M4

The llama.cpp project released build b10673, which includes specific performance optimizations for Apple M4 hardware. This update introduces fa_vec_tuned_table records into the ggml-metal-tuning module to improve Metal backend efficiency.

github llama.cpp · 5h ago · 2 views

llama.cpp b10670 routes quantized KV decode to TILE on Xe2 (BMG)

The llama.cpp project released version b10670, which includes a specific optimization for Intel's Xe2 architecture. The update routes the quantized key-value (KV) decode operation to use the TILE backend exclusively for BMG hardware, while keeping the VEC backend on other architectures until they are validated.

github llama.cpp · 6h ago · 2 views

llama.cpp b10672 adds Whisper.cpp support, Qwen3.5 NPU enablement, and OpenVINO 2026.3.1 update

The llama.cpp project released build b10672, which updates the OpenVINO backend to version 2026.3.1 and adds support for Whisper.cpp models on the NPU.

github llama.cpp · 7h ago · 1 view

llama.cpp adds fa-vec tunings for Apple M3 Max, M5 and M5 Pro

The llama.cpp project has added fast attention vector (fa-vec) tuning records for Apple M3 Max, M5, and M5 Pro GPUs to improve Metal performance.