github llama.cpp · 1h ago · inference

llama.cpp b10241 fixes CUDA SMEM data-races in block_reduce

from English

The llama.cpp project released version b10241, which addresses a critical bug in the CUDA backend. The update resolves data-races that occurred when reusing shared memory (SMEM) during block reductions.

Fixes missing resync after reading from SMEM in block_reduce to prevent data-races.
Implements double-buffering for single-row softmax and normalization operations.
Adds explanatory comments and memory barriers specifically for multi-warp scenarios.
Provides binaries for macOS, Linux, Windows, Android, and iOS across CPU, CUDA, Vulkan, ROCm, and other backends.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

media Hugging Face Forums · 4h ago · 1 view

Quesma publishes curated map of 218 tools and papers for LLM token cost efficiency

Quesma has released an open, curated list containing 218 entries that track where tokens are spent in LLM and agent workflows. The resource covers monitoring, semantic caching, cheap local models, context engineering, and cost governance.

github llama.cpp · 7h ago

llama.cpp b10238 adds MTP support for Qwen3-Next

The llama.cpp project released build b10238, which introduces Multi-Token Prediction (MTP) support for the Qwen3-Next model family. This update includes specific implementation changes to handle MTP layers and fixes related to Python type checking within the codebase.

github llama.cpp · 8h ago · 3 views

llama.cpp b10237 adds MTP support for DeepSeek V3.2

The llama.cpp project released build b10237, which introduces Multi-Token Prediction (MTP) support for the DeepSeek V3.2 model.

media Hugging Face Forums · 10h ago · 2 views

Qxern-v6 uses latent tokens and AST sidecar for faster, accurate code transfer

The Qxern-v6 system enables two LLMs to communicate via 32 compressed latent tokens while preserving exact symbol accuracy through an adaptive deterministic Abstract Syntax Tree (AST) sidecar. Built by a 15-year-old developer, the architecture uses Qwen2.5-Coder-1.5B to compress code and a frozen Qwen3.5-0.8B decoder to interpret it without seeing raw text.

github llama.cpp · 11h ago · 6 views

llama.cpp b10236 implements Metal DSv4 Lightning Indexer

The llama.cpp b10236 release introduces a Metal implementation of the DSv4 Lightning Indexer, specifically supporting 128-dimensional inputs with F32 queries and weights alongside F16 keys and masks.