github llama.cpp · 2h ago · inference

llama.cpp b10180 adds contiguous fast path and 32-bit index math for SYCL unary ops

from English

The llama.cpp project has released build b10180, introducing performance optimizations for the SYCL backend. The update implements a contiguous fast path and switches elementwise index math to use fastdiv with 32-bit integers.

SYCL: contiguous fast path + 32-bit index math for unary elementwise ops
SYCL: use fastdiv for elementwise index math

This release provides binaries for macOS, Linux, Windows, Android, and openEuler across CPU, GPU, and various accelerator backends.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency Local inference

Read original

github llama.cpp · 2d ago · 3 views

llama.cpp b10155 adds MiMo-V2.5 audio support

The llama.cpp project has released build b10155, which introduces support for the MiMo-V2.5 audio input model. This update includes a GGUF converter for MiMo audio and implements the necessary C++ backend logic to handle RVQ-based audio models.

github llama.cpp · 2d ago · 2 views

llama.cpp b10150 adjusts ggml offloading logic and fixes dsv4 graph

The llama.cpp project has released build b10150, which includes adjustments to the ggml logic for offloading operations to the weight's backend. This update also addresses fixes for the llama dsv4 graph.

media Hugging Face Forums · 2d ago · 2 views

TraceML adds Hugging Face Trainer callback to diagnose data-loading bottlenecks

The author has integrated a diagnostic tool into TraceML, an open-source PyTorch observability project, to help users distinguish between GPU compute-bound and input-bound training performance issues. This new feature allows developers to quickly identify whether low GPU utilization is caused by slow data loading or actual model computation limits.

github llama.cpp · 2d ago · 6 views

Aaron Teo adds -lm mlock argument to llama.cpp

Aaron Teo has submitted a change to the llama.cpp repository that introduces a new `-lm mlock` command-line argument. This addition allows users to explicitly lock memory in regions where the code performs `mlock` operations but does not use `mmap`.

github llama.cpp · 6d ago · 1 view

llama.cpp b10093 fixes DeepSeek v4 template and adds drop_reasoning flag

The llama.cpp project has released build b10093, which includes specific updates to support DeepSeek v4 models. The release addresses template handling and introduces new configuration flags for this model family.