github llama.cpp · 1h ago · inference

llama.cpp b10290 adds ggml_build_forward_order to fix audio graph execution

from English

The llama.cpp project released build b10290, introducing a new `ggml_build_forward_order` function to the ggml library. This change addresses a bug in the mtmd audio graph where using `ggml_build_forward_expand` as a pure ordering hint caused unselected branches to execute with stale inputs.

The previous method forced computation on unselected branches, leading to `get_rows` bound assertion failures on CPU due to stale `inp_code0` data.
The new function inserts nodes without the compute flag, restoring it only when the branch is actually selected.
Build b10290 provides binaries for macOS (Apple Silicon and Intel), Linux (CPU, Vulkan, ROCm, OpenVINO, SYCL), Android, Windows (CPU, CUDA 12/13, Vulkan, OpenCL, OpenVINO, SYCL, HIP), and openEuler.
An iOS XCFramework and a standalone UI build are also included in the release.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

github llama.cpp · 4h ago · 1 view

llama.cpp b10289 hardens file_glob_search and fixes Windows path handling

The llama.cpp b10289 release hardens the server's file_glob_search directory walk to prevent infinite loops on Windows junctions and improves error reporting for unreadable directories.

media Hugging Face Forums · 6h ago

QuantCheck warns against assuming final checkpoint is best for 4-bit quantization

A new tool called QuantCheck highlights that the final pretraining checkpoint may not be the optimal choice for 4-bit quantization, as benchmarks can remain flat while fragility increases. The author observed this pattern on Pythia-160m, where the final checkpoint suffered approximately four times more quality damage than an earlier checkpoint of equal quality.

github llama.cpp · 11h ago

llama.cpp b10285 adds multi-row batching for deepseek-ocr

The llama.cpp project has released build b10285, which introduces support for multi-row batching in the mtmd module specifically for deepseek-ocr models. This update allows the system to weave deepseek-ocr rows in a single operation rather than processing them individually.

github llama.cpp · 13h ago

llama.cpp b10282 adds spec-decode counters to /metrics endpoint

The llama.cpp project has released version b10282, which introduces new monitoring capabilities for speculative decoding. The server now includes spec-decode counters in its /metrics endpoint, allowing users to track performance metrics related to this feature.

github llama.cpp · 1d ago

llama.cpp b10275 decodes Windows OEM output to UTF-8 in built-in tools

The llama.cpp project released build b10275, which fixes a character encoding issue on Windows where accented characters were silently lost. The update ensures that text from child processes writing in the OEM code page is correctly decoded to UTF-8 before reaching the JSON layer.