github llama.cpp · 17h ago · inference

llama.cpp b10069 adds OpenCL broadcast support for Adreno MUL_MAT

from English

The llama.cpp project released build b10069, which includes specific OpenCL updates for Qualcomm Adreno GPUs. The primary change involves supporting broadcast operations for the Adreno MUL_MAT kernel and honoring view offsets for the Q8_0 MUL_MAT variant to enable multi-stream functionality in llama-server.

Added broadcast support for Adreno GEMM/GEMV noshuffle kernels.
Ensured view offsets are correctly handled for Adreno noshuffle GEMM/GEMV operations.
Implemented general GEMM/GEMV broadcast support within the OpenCL backend.
Removed unnecessary tests and comments from the codebase.

This update improves compatibility with multi-stream workloads on Adreno hardware by fixing critical kernel handling issues.

Importance 1/3 Trust 1/3 llama.cpp Hugging Face Inference efficiency

Read original

github llama.cpp · 2d ago · 1 view

llama.cpp b10068 rotates injected K/V cache for DFlash

The llama.cpp project released version b10068, which introduces a change to the DFlash model implementation. The update modifies src/models/dflash.cpp to rotate the injected key/value (K/V) cache when K/V quantization is used.

github llama.cpp · 2d ago

llama.cpp b10067 excludes DeepSeek-V4 routing table from quantization

The llama.cpp project released version b10067, which includes a fix for the `llama-quantize` tool to correctly handle DeepSeek-V4 models. The update prevents the quantizer from attempting to convert the `ffn_gate_tid2eid` tensor, an i32 token-to-expert index table, into a float type.

github llama.cpp · 3d ago

llama.cpp b10068 rotates injected K/V cache for DFlash

llama.cpp b10067 excludes DeepSeek-V4 routing table from quantization

llama.cpp b10066 adds OpenCL kernel_gemm_moe_q6_k_f32_ns support

llama.cpp b10064 release with OpenCL transpose optimization

llama.cpp b10059 sets ggml-blas default hadamard mul_mat to cpu routine