github llama.cpp · 1h ago · inference

llama.cpp b10326 accounts for vocoder pass in timings

from English

The llama.cpp project released build b10326, which modifies how timing measurements are calculated by including the vocoder pass.

The update ensures that `get_output` runs the waveform work deferred by the pipeline, measuring it from a single trailing window to a full pass depending on the model.
This change keeps the reported total time and the audio-to-process ratio accurate.
Binaries are available for macOS (Apple Silicon and Intel), iOS, Linux (CPU, Vulkan, ROCm, OpenVINO, SYCL), Android, Windows (CPU, OpenCL, CUDA 12/13, Vulkan, OpenVINO, SYCL, HIP), and openEuler.

The modification improves the honesty of performance metrics by ensuring that audio processing overhead is correctly accounted for in the reported timings.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

github llama.cpp · 1h ago Live

llama.cpp b10321 fixes NORM/RMS_NORM for partial simdgroup row lengths

The llama.cpp project released version b10321, which includes a critical fix for GGML_OP_NORM and GGML_OP_RMS_NORM operations on Metal. The update resolves an issue where threadgroups sized to leave a partial SIMD group resulted in incorrect mean and variance calculations due to dropped partial sums.

github llama.cpp · 2h ago

llama.cpp b10321 fixes NORM/RMS_NORM for partial simdgroup row lengths

llama.cpp b10322 coalesces ssm_conv window loads for ~1.85x speedup

llama.cpp b10313 adds LRU scheduler to the server

llama.cpp b10311 fixes Qwen3-TTS text stream duplication

llama.cpp b10312 release with server fix for busy models