github llama.cpp · 1h ago · inference

llama.cpp b10311 fixes Qwen3-TTS text stream duplication

from English

The llama.cpp project released build b10311, which includes a fix for the Qwen3-TTS pipeline. The update resolves an issue where the model read the input utterance twice during generation.

Fixes a bug in the non-streaming prefill overlay that caused the text stream to be fed again during Qwen3-TTS generation.
Corrects the trailing text hidden state to collapse to a single tts_pad row, matching the prefill layout.
Provides binaries for macOS (Apple Silicon and Intel), iOS, Linux (CPU, Vulkan, ROCm, OpenVINO, SYCL), Android, Windows (CPU, CUDA 12/13, Vulkan, OpenCL, ROCm, SYCL), and openEuler.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

github llama.cpp · just now Live

llama.cpp b10313 adds LRU scheduler to the server

The llama.cpp project has released version b10313, which introduces a Least Recently Used (LRU) scheduler for its server component. This update includes handling for request coalescing and fixes for streaming cases.

github llama.cpp · 1h ago Live

llama.cpp b10312 release with server fix for busy models

The llama.cpp project has released version b10312, which includes a specific update to the server component. The key change addresses an issue where busy models were being evicted from the router.

github llama.cpp · 4h ago

llama.cpp b10307 fixes SYCL NVFP4 parsing

The llama.cpp project released version b10307, which includes a fix for the SYCL GPU code to correctly parse UE4M3 scaling factors used in NVFP4 quantization.

github llama.cpp · 5h ago

llama.cpp b10305 adds SYCL support for DSv4 OPs

The llama.cpp project has released build b10305, which introduces support for specific DSv4 operations within the SYCL backend. This update includes the implementation of LIGHTNING_INDEXER, DSV4_HC_COMB, DSV4_HC_POST, and DSV4_HC_PRE operators.

github llama.cpp · 5h ago

llama.cpp b10306 adds SYCL GLU flat path and consolidates kernels

The llama.cpp b10306 release introduces performance optimizations for the SYCL backend, specifically targeting Gated Linear Unit (GLU) operations. The update adds a contiguous fast path for fused GLU ops and consolidates previously distinct kernels to share a common launcher.