github llama.cpp · 20h ago · inference

llama.cpp b11307 preserves batch order for speculative decoding

from English

The llama.cpp project released build b11307, which fixes layer input ordering to preserve the original batch sequence during speculative decoding. This change ensures that token order is correctly maintained for unmasked NextN embeddings and compatible with tensor splits.

Restores original row order after synchronization by copying microbatch tensors from offset zero.
Uses original-token mapping for unmasked NextN rows, even when layer-input capture is disabled.
Fixes WebGPU reservation and OpenVINO hidden-state capture issues.
Validates across 256 CPU/CUDA/tensor configurations with Qwen3.8-27B completing MT-Bench at concurrency 16.

The update ensures correctness in speculative decoding workflows by preventing order corruption in layer inputs and embeddings.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency Local inference

Read original

github llama.cpp · 22h ago · 1 view

llama.cpp b11301 fixes integer overflow in ggml/gguf

The llama.cpp project has released build b11301, which includes a fix for an integer overflow issue within the ggml and gguf components. This update addresses validation logic to prevent potential errors when handling zero-element tensors.

github llama.cpp · 23h ago · 3 views

llama.cpp b11302 fixes ThreadSanitizer data race in sparse indexer

The llama.cpp project released build b11302, which addresses a critical concurrency issue identified by ThreadSanitizer in the CI pipeline. The update resolves a data race within the sparse attention mechanism where multiple CPU threads were incorrectly writing to the same memory elements.

github llama.cpp · 23h ago · 2 views

llama.cpp b11301 fixes integer overflow in ggml/gguf

llama.cpp b11302 fixes ThreadSanitizer data race in sparse indexer

llama.cpp b11298 adds dflash support to convert and feature extraction

llama.cpp b11284 fixes OpenVINO GET_ROWS on quantized weight views

llama.cpp adds GLM-5.3-Flash support with k-pool and MTP fixes