github llama.cpp · 1h ago · inference

llama.cpp b10182 moves suppress_tokens handling to common/sampling

from English

The llama.cpp project has released build b10182, which includes a structural change to how token suppression is managed. The primary modification involves moving the `suppress_tokens` handling logic into the `common/sampling` module.

The `has_logit_bias` feature has been removed from the codebase.
The release addresses security issues related to the previous implementation.
Binaries are provided for macOS (Apple Silicon and Intel), iOS, Linux (CPU, Vulkan, ROCm, OpenVINO, SYCL), Android, Windows (CPU, CUDA 12/13, Vulkan, OpenCL, ROCm, OpenVINO, SYCL, HIP), and openEuler.

This update consolidates sampling logic and resolves specific security concerns within the token suppression mechanism.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

github llama.cpp · 2h ago · 1 view

llama.cpp b10180 adds contiguous fast path and 32-bit index math for SYCL unary ops

The llama.cpp project has released build b10180, introducing performance optimizations for the SYCL backend. The update implements a contiguous fast path and switches elementwise index math to use fastdiv with 32-bit integers.

github llama.cpp · 3h ago · 1 view

llama.cpp b10181 disables MMQ on devices with less than 48 KiB shared memory

The llama.cpp project released build b10181, which includes a fix in ggml-cuda to disable Matrix Multiplication Quantization (MMQ) on GPUs with less than 48 KiB of per-block shared memory. Previously, MMQ configurations assumed a minimum of 48 KiB, causing fatal errors on devices like NVIDIA Pascal GPUs and Moore Threads MTT S70 where the tile size exceeded available memory.

media Hugging Face Forums · 4h ago

Jason Van Pham documents Gemma 4 multi-turn greedy degeneration under letter-level constraints

Jason Van Pham and co-engineer Grok (xAI) published a diagnostic report detailing a near-vanilla multi-turn failure mode in the Hydrodynamic Swarm harness when using a custom Gemma 4 instruct path. The study identifies that combining greedy decoding with long token budgets and letter-level constraints triggers classical text degeneration, including repetition and orthographic thrash.

media Hugging Face Forums · 5h ago

Hugging Face fixes DynamicCache to prevent wrong output in generate()

A bug in Hugging Face's `generate()` function causes incorrect outputs when `past_key_values` is an externally-built `DynamicCache`. This issue affects KV cache compression methods like SnapKV, H2O, and KIVI that reconstruct caches from prefill data. The root cause is the failure to propagate positional context, leading to a 10.7pp accuracy loss on Qwen3-VL-8B.

github llama.cpp · 6h ago · 1 view

llama.cpp b10178 adds trace logging for slot similarity checking

The llama.cpp project released build b10178, which introduces trace logging for slot similarity checking within the server component. This update adds detailed logging in `server-context.cpp` during prompt cache slot selection, specifically capturing skip reasons and similarity calculation details.