github llama.cpp · 1h ago · inference

llama.cpp b10148 fixes draft sidecar resolution and log verbosity

from English

The llama.cpp project released build b10148, addressing issues with speculative decoding configuration and logging. The update ensures that explicit command-line arguments for draft models take precedence over automatic sidecar resolution.

Explicit --model-draft flags now disable automatic sidecar resolution, allowing manual CLI configuration to override default behavior.
Sidecar selection logic was updated to anchor on the primary model's tag, enabling exact matches or closest quantization fallbacks.
Speculative loading logs were promoted from trace to info level, making draft model and MTP context visibility consistent with other system logs.

These changes improve reliability for users relying on manual draft model specifications and provide clearer operational logging.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

media Hugging Face Forums · just now Live

TraceML adds Hugging Face Trainer callback to diagnose data-loading bottlenecks

The author has integrated a diagnostic tool into TraceML, an open-source PyTorch observability project, to help users distinguish between GPU compute-bound and input-bound training performance issues. This new feature allows developers to quickly identify whether low GPU utilization is caused by slow data loading or actual model computation limits.

github llama.cpp · 2h ago · 1 view

Aaron Teo adds -lm mlock argument to llama.cpp

Aaron Teo has submitted a change to the llama.cpp repository that introduces a new `-lm mlock` command-line argument. This addition allows users to explicitly lock memory in regions where the code performs `mlock` operations but does not use `mmap`.

github llama.cpp · 6h ago · 2 views

llama.cpp b10144 fixes stream routes for model names with slashes

The llama.cpp b10144 release addresses critical issues in the server and UI regarding resumable streams, specifically fixing handling of model names containing slashes. It also resolves bugs where requests were not properly cancelled or resumed during model loading.

github llama.cpp · 11h ago

llama.cpp b10142 adds preliminary MiniMax-M3 vision support

The llama.cpp project released build b10142, which introduces preliminary vision support for the MiniMax-M3 model. This update implements a text-only port that reuses existing components from MiniMax-M2, including GQA with per-head QK-norm and partial rotary, DeepSeek-V3 style experts, and swigluoai activation.

github llama.cpp · 3d ago · 1 view

llama.cpp b10107 fixes Windows crash and provides binaries for multiple platforms

The llama.cpp project has released version b10107, which includes a fix for a Windows crash that occurred when the op_poll feature was enabled.