github llama.cpp · 1h ago · inference

llama.cpp b10322 coalesces ssm_conv window loads for ~1.85x speedup

from English

The llama.cpp project released build b10322, which includes a performance optimization that coalesces SSM_CONV window loads in the SYCL backend.

On an Arc Pro B70, interleaved A/B tests showed a 1.85x to 1.87x speedup for specific tensor shapes, with negligible change for n_t=1.
Benchmarks on Qwen35 27B Q4_K - Medium models running ssm_conv in 48 of 64 blocks indicated a +2.2% improvement in prompt processing time.
The release provides binaries for macOS (Apple Silicon and Intel), iOS, Linux (CPU, Vulkan, ROCm, OpenVINO, SYCL), Android, Windows (CPU, CUDA, Vulkan, OpenVINO, SYCL, HIP), and openEuler.

This update improves inference throughput for workloads utilizing SSM_CONV operations on compatible hardware.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

github llama.cpp · just now Live

llama.cpp b10326 accounts for vocoder pass in timings

The llama.cpp project released build b10326, which modifies how timing measurements are calculated by including the vocoder pass.

github llama.cpp · just now Live

llama.cpp b10321 fixes NORM/RMS_NORM for partial simdgroup row lengths

The llama.cpp project released version b10321, which includes a critical fix for GGML_OP_NORM and GGML_OP_RMS_NORM operations on Metal. The update resolves an issue where threadgroups sized to leave a partial SIMD group resulted in incorrect mean and variance calculations due to dropped partial sums.

github llama.cpp · 3h ago

llama.cpp b10326 accounts for vocoder pass in timings

llama.cpp b10321 fixes NORM/RMS_NORM for partial simdgroup row lengths

llama.cpp b10313 adds LRU scheduler to the server

llama.cpp b10311 fixes Qwen3-TTS text stream duplication

llama.cpp b10312 release with server fix for busy models