media Hugging Face Forums · 1h ago · open_models

TraceML adds Hugging Face Trainer callback to diagnose data-loading bottlenecks

from English

The author has integrated a diagnostic tool into TraceML, an open-source PyTorch observability project, to help users distinguish between GPU compute-bound and input-bound training performance issues. This new feature allows developers to quickly identify whether low GPU utilization is caused by slow data loading or actual model computation limits.

The integration adds a `TraceMLTrainerCallback` to the Hugging Face `Trainer`, requiring minimal setup changes.
It separates input wait time from step work to provide a practical verdict on the bottleneck type.
A Colab example with ResNet-50 demonstrated that adjusting DataLoader settings changed the diagnosis from input-bound to compute-bound and improved speed by 1.83×.

This tool helps users avoid optimizing the wrong component by clarifying the specific constraint affecting their training runs.

Importance 1/3 Trust 1/3 Hugging Face Forums Inference efficiency Local inference

Read original

github llama.cpp · 3h ago · 1 view

Aaron Teo adds -lm mlock argument to llama.cpp

Aaron Teo has submitted a change to the llama.cpp repository that introduces a new `-lm mlock` command-line argument. This addition allows users to explicitly lock memory in regions where the code performs `mlock` operations but does not use `mmap`.

github llama.cpp · 4d ago

llama.cpp b10093 fixes DeepSeek v4 template and adds drop_reasoning flag

The llama.cpp project has released build b10093, which includes specific updates to support DeepSeek v4 models. The release addresses template handling and introduces new configuration flags for this model family.

github llama.cpp · 5d ago · 1 view

llama.cpp b10089 adds full k-quant and i-quant support to CUDA GET_ROWS

The llama.cpp b10089 release extends CUDA GET_ROWS operations to support k-quants, i-quants, and mxfp4 quantization formats. This change ensures that all quantized GGML types can now take the direct device path for embedding lookups.

github llama.cpp · 6d ago

llama.cpp b10076 vectorizes get_rows with int4 copy for 27% speedup

The llama.cpp project released build b10076, which introduces a performance optimization to the CUDA backend by vectorizing the `get_rows` operation using int4 copies. This change hoists row-invariant work out of the per-element loop and adds a vectorized path that copies 16 bytes per thread for contiguous same-type data.

media Hugging Face Forums · 9d ago · 3 views

zhongkaifu releases TensorSharp, a native .NET LLM inference engine for GGUF models

zhongkaifu has released TensorSharp, an open-source local LLM inference engine built entirely in C# that supports Windows, macOS, and Linux. The engine handles GGUF models from Unsloth, including Gemma 4 and Qwen 3.6 with multimodal capabilities, while providing APIs compatible with OpenAI and Ollama.