github llama.cpp · 1h ago · inference

llama.cpp b10924 fixes router state command parsing to prevent stuck downloads

from English

The llama.cpp project released build b10924, which addresses a bug in the server's router where child state commands were not framed as whole lines. This issue occurred because the logger emitted trailing color reset escape sequences without newlines on stderr, causing them to be glued to the next command read from stdout via a shared pipe.

The fix writes commands with a leading newline to ensure they always start at a line boundary, preventing the router from misinterpreting state commands as log lines.
This resolves an issue where finished downloads remained stuck in the downloading state due to failed line prefix checks.
The release includes binaries for macOS (Apple Silicon and Intel), iOS, Linux (CPU, Vulkan, ROCm, OpenVINO, SYCL), Android, Windows (CPU, CUDA 12/13, Vulkan, OpenVINO, SYCL, ROCm), and openEuler.

This update ensures reliable download completion handling in the llama.cpp server environment.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

github llama.cpp · just now · 1 view Live

llama.cpp b10921 aligns WebGPU tensor bindings to block size

The llama.cpp project has released build b10921, which includes a fix for WebGPU by aligning tensor bindings to the type block size. This change ensures that block quantized views receive a valid element offset in the shader by walking the binding offset back until the distance to the tensor is a whole number of blocks.

github llama.cpp · just now · 1 view Live

llama.cpp b10922 adds OpenCL A8 Q4_K binary kernel

The llama.cpp project released version b10922, introducing a new OpenCL binary kernel for A8 Q4_K non-MoE matrix multiplications. This update also includes fixes for layout compatibility and renames binary kernel selection helpers within the codebase.

media Latent Space · 3h ago · 1 view

DeepSeek releases v4.1-Flash with 763B-P8B-D16B architecture for efficient inference

DeepSeek has launched DeepSeek v4.1-Flash, a new open-weight flagship model featuring a novel causal encoder-decoder architecture designed to maximize inference efficiency and reduce costs.

github llama.cpp · 5h ago · 1 view

hexagon adds multi-device row-split support for model parallelism

The hexagon backend introduces support for splitting models across multiple devices using a row-split strategy. This change enables work distribution in fused kernels and updates device configuration to handle device groups.

media Hugging Face Forums · 7h ago · 1 view

Proposal for vendor-neutral AI runtime with memory-budget-aware execution

A proposal on the Hugging Face forums outlines a common GPU execution and memory-management layer designed to decouple application code from specific hardware vendors like NVIDIA, AMD, and Intel. The goal is to allow users to specify a memory budget rather than manually configuring backend-specific flags, device maps, or offloading strategies.