github llama.cpp · just now · inference

llama.cpp b10922 adds OpenCL A8 Q4_K binary kernel

from English

The llama.cpp project released version b10922, introducing a new OpenCL binary kernel for A8 Q4_K non-MoE matrix multiplications. This update also includes fixes for layout compatibility and renames binary kernel selection helpers within the codebase.

Added the `kernel_gemm_noshuffle_q4_k_f32_32b_trans_ila_a8_bin` OpenCL kernel.
Fixed layout compatibility issues in the OpenCL backend.
Renamed binary kernel selection helpers for clarity.

This release provides updated binaries for macOS, Linux, Windows, Android, and iOS across various hardware backends including CPU, CUDA, Vulkan, ROCm, and SYCL.

Importance 1/3 Trust 1/3 llama.cpp Hugging Face Inference efficiency

Read original

github llama.cpp · 22h ago · 1 view

llama.cpp b10907 fixes MTP context kv cache allocation for deepseek2, glm4moe, cohere2moe

The llama.cpp project released version b10907, which includes a fix for Multi-Token Prediction (MTP) context key-value cache allocation. This update addresses issues affecting the deepseek2, glm4moe, and cohere2moe architectures.

github llama.cpp · 22h ago · 1 view

llama.cpp b10908 fixes idle threads in Metal iq mul_mv kernels

The llama.cpp project released version b10908, which includes a fix for idle threads in the remaining iq mul_mv kernels for ne00 < 1024. This change generalizes the row split logic to six specific kernels: iq1_s, iq1_m, iq2_xxs, iq2_xs, iq2_s, and iq3_s.

github llama.cpp · 23h ago · 1 view

llama.cpp b10907 fixes MTP context kv cache allocation for deepseek2, glm4moe, cohere2moe

llama.cpp b10908 fixes idle threads in Metal iq mul_mv kernels

llama.cpp b10905 tunes Flash Attention for AMD RDNA4 and revises stream-k logic

llama.cpp b10888 adds Vulkan command-buffer debug labels for GPU profilers

llama.cpp b10886 adds Q1_0 vector intrinsic support for s390x