github llama.cpp · 8h ago · inference

llama.cpp b10760 fixes Qwen3-tts-0.6b inference stability

from English

The llama.cpp project released build b10760, which includes specific fixes for the Qwen3-tts-0.6B model to prevent numerical instability during inference.

The mtmd loader now treats the qwen3-tts code predictor proj_in as optional, aligning with the reference implementation where this tensor is omitted.
The code predictor ffn_down layer is kept in F32 precision because its activations peak around 1.5e5, exceeding the F16 ceiling and causing NaN errors.

These changes ensure that the graph correctly handles missing weights and prevents overflow in high-activation layers for this specific checkpoint.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency Local inference

Read original

github llama.cpp · 1h ago · 1 view Live

llama.cpp b10763 enables preserve_reasoning by default

The llama.cpp project released build b10763, which changes the default behavior of the `preserve_reasoning` chat template keyword argument. The server now logs the effective state of this kwarg and issues a warning when it is enabled by default on templates that support it.

github llama.cpp · 23h ago · 1 view

llama.cpp b10751 fuses MoE weighted expert reduction for CUDA

The llama.cpp project released build b10751, which introduces a fused kernel for the MoE (Mixture of Experts) weighted expert reduction on CUDA. This optimization replaces the previous baseline that ran two physical kernels with a single weighted-reduction kernel to reduce intermediate global-memory traffic.

github llama.cpp · 1d ago · 1 view

llama.cpp b10763 enables preserve_reasoning by default

llama.cpp b10751 fuses MoE weighted expert reduction for CUDA

llama.cpp b10750 optimizes KV cache n-gram lookup for 4.9% throughput gain

llama.cpp b10742 adds fa-vec tunings for A18 Pro

llama.cpp b10739 adds fa-vec tuning for M2 Max