github llama.cpp · 20h ago · inference

llama.cpp b10701 fixes DFlash2 NVFP4 speculative decoding

from English

The llama.cpp project released build b10701, which includes a fix for DFlash2 NVFP4 draft models. The update addresses an issue where these models produced almost no accepted speculative tokens.

Passes missing NVFP4 scales to attention operations in pull request #28000.
Ensures Q, K, V, and output projection scales are correctly passed to corresponding graph operations.

This change restores proper functionality for DFlash2 NVFP4 models by correcting the scale propagation required for speculative decoding.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

github llama.cpp · just now · 1 view Live

llama.cpp b10721 fixes WebGPU crash and adds Windows CUDA 13.4 support

The llama.cpp project has released build b10721, which includes a critical fix for the WebGPU backend and expands hardware support on Windows.

github llama.cpp · 2h ago

llama.cpp b10718 extends MOE fusion to specdec and multi-token

The llama.cpp project released version b10718, which includes a significant update to its CUDA backend. The most notable change is the extension of Mixture of Experts (MOE) fusion to support speculative decoding (specdec), addressing previous limitations where MOE glu fusion and topk-router fusion were restricted to processing only one token at a time.

github llama.cpp · 4h ago · 3 views

llama.cpp b10721 fixes WebGPU crash and adds Windows CUDA 13.4 support

llama.cpp b10718 extends MOE fusion to specdec and multi-token

llama.cpp b10717 release enhances SYCL memory info retrieval

llama.cpp b10715 fuses DFlash encoder into KV cache injection

pipecat-ai releases phonellm-alpha-1 with GPT 5.6 Terra performance