github llama.cpp · 1h ago · inference

llama.cpp b10085 fixes Qwen3-VL grounding by aligning position embedding interpolation

from English

The llama.cpp project released build b10085, which includes a fix for the Qwen3-VL vision model's position embedding interpolation to resolve grounding coordinate mismatches.

The change switches the interpolation method from the default bilinear+antialias (align_corners=False) to align_corners=True, matching the transformers reference implementation.
This corrects scaling issues in grounding coordinates that previously grew with image size and varied per-axis for non-square images.
Binaries are available for macOS (Apple Silicon and Intel), iOS, Linux (CPU, Vulkan, ROCm, OpenVINO, SYCL), Android, Windows (CPU, CUDA 12/13, Vulkan, OpenCL, OpenVINO, SYCL, HIP), and openEuler.

Importance 1/3 Trust 1/3 llama.cpp Hugging Face Inference efficiency

Read original

github llama.cpp · 2d ago · 1 view

llama.cpp b10069 adds OpenCL broadcast support for Adreno MUL_MAT

The llama.cpp project released build b10069, which includes specific OpenCL updates for Qualcomm Adreno GPUs. The primary change involves supporting broadcast operations for the Adreno MUL_MAT kernel and honoring view offsets for the Q8_0 MUL_MAT variant to enable multi-stream functionality in llama-server.

github llama.cpp · 4d ago · 1 view

llama.cpp b10068 rotates injected K/V cache for DFlash

The llama.cpp project released version b10068, which introduces a change to the DFlash model implementation. The update modifies src/models/dflash.cpp to rotate the injected key/value (K/V) cache when K/V quantization is used.

github llama.cpp · 4d ago · 1 view

llama.cpp b10067 excludes DeepSeek-V4 routing table from quantization

The llama.cpp project released version b10067, which includes a fix for the `llama-quantize` tool to correctly handle DeepSeek-V4 models. The update prevents the quantizer from attempting to convert the `ffn_gate_tid2eid` tensor, an i32 token-to-expert index table, into a float type.

github llama.cpp · 5d ago

llama.cpp b10066 adds OpenCL kernel_gemm_moe_q6_k_f32_ns support

The llama.cpp project has released version b10066, introducing support for the `kernel_gemm_moe_q6_k_f32_ns` kernel via OpenCL. This update enables the loading and usage of this specific binary kernel library within the framework.

github llama.cpp · 5d ago · 5 views

llama.cpp b10064 release with OpenCL transpose optimization

The llama.cpp project has released version b10064, which includes a change to the OpenCL backend. This update modifies the q4_K quantization format by transposing noshuffle scales to enable coalesced reads.