github llama.cpp · 2d ago · inference

llama.cpp enables xmem GEMM and bypasses tiled f32 on Adreno GPUs

from English

The llama.cpp project has updated its OpenCL backend to improve matrix multiplication performance on two generations of Adreno GPUs. The changes primarily target the X2E generation by defaulting the xmem F16xF32 GEMM path and optimizing handling for the A7X generation.

Defaulting the Adreno xmem F16xF32 GEMM to enabled for X2E devices, which previously required an opt-in environment variable.
Bypassing the tiled f32 GEMM on the Adreno A7X compiler by routing batched operations to a per-row kernel to avoid register spilling issues.
The xmem optimization provides approximately 25% prefill speedup on gpt-oss-20b for Adreno X2-90 devices, while the A7X bypass yields about 9% improvement on gemma-3n-E4B.

These optimizations address specific compiler inefficiencies in older and newer Adreno silicon, reducing GPU time spent in slow matmul kernels during model prefill phases.

Importance 1/3 Trust 1/3 llama.cpp Hugging Face Inference efficiency Local inference

Read original

github llama.cpp · 2h ago

llama.cpp b10718 extends MOE fusion to specdec and multi-token

The llama.cpp project released version b10718, which includes a significant update to its CUDA backend. The most notable change is the extension of Mixture of Experts (MOE) fusion to support speculative decoding (specdec), addressing previous limitations where MOE glu fusion and topk-router fusion were restricted to processing only one token at a time.

github llama.cpp · 3d ago · 9 views

llama.cpp b10670 routes quantized KV decode to TILE on Xe2 (BMG)

The llama.cpp project released version b10670, which includes a specific optimization for Intel's Xe2 architecture. The update routes the quantized key-value (KV) decode operation to use the TILE backend exclusively for BMG hardware, while keeping the VEC backend on other architectures until they are validated.

github llama.cpp · 4d ago · 23 views

llama.cpp b10718 extends MOE fusion to specdec and multi-token

llama.cpp b10670 routes quantized KV decode to TILE on Xe2 (BMG)

llama.cpp adds Qwen3.8-Flash-Next (qwen4exp) architecture support

llama.cpp b10653 adds TENSOR_READ_LAZY for model loading

llama.cpp b10568 uses ggml_rope_set_offset() for DeepSeek 2