github llama.cpp · 1h ago · inference

llama.cpp b11009 fixes split state for fused QKV in gemma4 and qwen35

from English

The llama.cpp project released build b11009, which addresses critical issues with split state and granularity calculations for fused QKV attention layers.

The model code now correctly calculates split states for attn_qkv based on n_head * n_embd_head_k, resolving compatibility issues with gemma4 models using the --fuse-qkv flag where n_embd is 5376 but Q is 8192.
Support for fused full attention layers has been added for qwen35 and qwen35moe architectures.
A TODO tag [TAG_SPLIT_QGATE_QWEN] was added to track further work on QGate splitting for Qwen models.

This release ensures that specific model configurations, particularly those involving fused QKV operations in gemma4 and qwen35, function correctly without state calculation errors.

Importance 1/3 Trust 1/3 llama.cpp Hugging Face Inference efficiency

Read original

github llama.cpp · 3h ago · 1 view

llama.cpp b11006 adds Hexagon support for K-Quants Q4_K and Q6_K

The llama.cpp project released version b11006, introducing support for Qualcomm Hexagon processors to handle K-Quantization formats. This update implements specific kernels for Q4_K and Q6_K quantization levels on the Hexagon architecture.

github llama.cpp · 4h ago · 2 views

llama.cpp b11005 accepts zeroed rope probe to reduce graph splits

The llama.cpp project released version b11005, which includes a fix for the Hexagon backend to accept zeroed rope probes in supports_op.

github llama.cpp · 5h ago · 2 views

llama.cpp adds support for HrmTextForCausalLM (DFM Mimir 1B)

The llama.cpp project has added support for the HrmTextForCausalLM architecture, specifically enabling inference for the DFM Mimir 1B model. This implementation introduces a new GGUF writer and loader to handle the model's unique structure, which runs two transformer stacks in an alternating cycle over the same token stream.

github llama.cpp · 7h ago · 1 view

llama.cpp b11001 fixes int16 transpose corruption on RISC-V

The llama.cpp project released build b11001, which includes a critical fix for the `permute_transpose_impl` function handling int16 data on RISC-V architectures.

github llama.cpp · 13h ago · 1 view

llama.cpp b10995 makes Vulkan MUL_MAT_ID BN/2 tail unconditional

The llama.cpp project released build b10995, which modifies the Vulkan backend's handling of the MUL_MAT_ID operation. Specifically, the change makes the BN/2 tail unconditional by using BN/2 as the default for BNover2 and as the disabled fallback for BNover4, while removing the enable gate from that branch.