github llama.cpp · 1h ago · inference

llama.cpp b10291 fixes Vulkan submission batching and adds DeviceLost debug tools

from English

The llama.cpp project has released version b10291, which includes critical fixes for the Vulkan backend alongside standard platform updates. The primary technical changes address submission batching size issues and introduce new diagnostic capabilities for driver errors.

Fixes a bug where the submission threshold was applied too late in the Vulkan pipeline.
Adds debug tooling to provide more detailed information when diagnosing DeviceLost driver errors.
Replaces abort calls with throw exceptions and implements logging macros for better error handling.
Cleans up circular dependencies within the codebase.

This release improves stability for users running llama.cpp on Vulkan-enabled hardware by resolving specific submission timing issues and offering better visibility into crash causes.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

github llama.cpp · just now Live

llama.cpp b10293 adds ROCm CI for gfx1151 and fixes RDNA3.5 inference

The llama.cpp project released build b10293, which includes onboarding AMD ROCm continuous integration for the gfx1151 architecture and resolves correctness issues on integrated RDNA3.5 GPUs.

github llama.cpp · 8h ago

llama.cpp b10290 adds ggml_build_forward_order to fix audio graph execution

The llama.cpp project released build b10290, introducing a new `ggml_build_forward_order` function to the ggml library. This change addresses a bug in the mtmd audio graph where using `ggml_build_forward_expand` as a pure ordering hint caused unselected branches to execute with stale inputs.

github llama.cpp · 11h ago · 1 view

llama.cpp b10289 hardens file_glob_search and fixes Windows path handling

The llama.cpp b10289 release hardens the server's file_glob_search directory walk to prevent infinite loops on Windows junctions and improves error reporting for unreadable directories.

media Hugging Face Forums · 13h ago · 1 view

QuantCheck warns against assuming final checkpoint is best for 4-bit quantization

A new tool called QuantCheck highlights that the final pretraining checkpoint may not be the optimal choice for 4-bit quantization, as benchmarks can remain flat while fragility increases. The author observed this pattern on Pythia-160m, where the final checkpoint suffered approximately four times more quality damage than an earlier checkpoint of equal quality.

github llama.cpp · 18h ago

llama.cpp b10285 adds multi-row batching for deepseek-ocr

The llama.cpp project has released build b10285, which introduces support for multi-row batching in the mtmd module specifically for deepseek-ocr models. This update allows the system to weave deepseek-ocr rows in a single operation rather than processing them individually.