github llama.cpp · just now · inference

llama.cpp b10672 adds Whisper.cpp support, Qwen3.5 NPU enablement, and OpenVINO 2026.3.1 update

from English

The llama.cpp project released build b10672, which updates the OpenVINO backend to version 2026.3.1 and adds support for Whisper.cpp models on the NPU.

The OpenVINO backend now fuses IM2COL + MatMul convolutions into a single OpenVINO convolution operation.
Qwen3.5 models are enabled for execution on the NPU via the updated backend.
New operations including RELU, POOL_2D, QUICK_GEGLU, and ROLL are supported in the OpenVINO backend.
Static shape handling was corrected to fix dynamic slot dimension propagation and token count independence issues.
Chunked prefill logic was fixed to prevent padded tokens from being folded into the recurrent cache permanently.
A new environment variable GGML_OPENVINO_NPU_COMPILE_CONFIG allows users to configure NPU compilation parameters, such as optimization-level=3.

This update improves inference performance and model compatibility for OpenVINO users, particularly those targeting Intel NPUs.

Importance 1/3 Trust 1/3 llama.cpp Hugging Face Inference efficiency

Read original

github llama.cpp · just now · 1 view Live

llama.cpp b10670 routes quantized KV decode to TILE on Xe2 (BMG)

The llama.cpp project released version b10670, which includes a specific optimization for Intel's Xe2 architecture. The update routes the quantized key-value (KV) decode operation to use the TILE backend exclusively for BMG hardware, while keeping the VEC backend on other architectures until they are validated.

github llama.cpp · 2h ago · 1 view

llama.cpp b10670 routes quantized KV decode to TILE on Xe2 (BMG)

llama.cpp adds fa-vec tunings for Apple M3 Max, M5 and M5 Pro

llama.cpp b10665 adds DSpark support for Nemotron3.5

llama.cpp adds Qwen3.8-Flash-Next (qwen4exp) architecture support

llama.cpp b10653 adds TENSOR_READ_LAZY for model loading