github llama.cpp · 1h ago · inference

llama.cpp b10198 adds Vulkan support for quantized concat

from English

The llama.cpp project has released build b10198, which introduces support for quantized concatenation operations within the Vulkan backend.

The primary technical update is the addition of Vulkan support for quantized concat, implemented in pull request #25684.
macOS Apple Silicon builds with KleidiAI are explicitly disabled in this release.
Binaries are provided for multiple platforms including macOS (Apple Silicon and Intel), Linux (Ubuntu x64, arm64, s390x), Windows (CPU, CUDA 12/13, Vulkan, ROCm, OpenVINO, SYCL, HIP), Android, and openEuler.
A standalone UI binary is also included in the release package.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency Local inference

Read original

github llama.cpp · 1d ago · 1 view

llama.cpp b10180 adds contiguous fast path and 32-bit index math for SYCL unary ops

The llama.cpp project has released build b10180, introducing performance optimizations for the SYCL backend. The update implements a contiguous fast path and switches elementwise index math to use fastdiv with 32-bit integers.

github llama.cpp · 3d ago · 3 views

llama.cpp b10155 adds MiMo-V2.5 audio support

The llama.cpp project has released build b10155, which introduces support for the MiMo-V2.5 audio input model. This update includes a GGUF converter for MiMo audio and implements the necessary C++ backend logic to handle RVQ-based audio models.

github llama.cpp · 3d ago · 2 views

llama.cpp b10150 adjusts ggml offloading logic and fixes dsv4 graph

The llama.cpp project has released build b10150, which includes adjustments to the ggml logic for offloading operations to the weight's backend. This update also addresses fixes for the llama dsv4 graph.

media Hugging Face Forums · 3d ago · 2 views

TraceML adds Hugging Face Trainer callback to diagnose data-loading bottlenecks

The author has integrated a diagnostic tool into TraceML, an open-source PyTorch observability project, to help users distinguish between GPU compute-bound and input-bound training performance issues. This new feature allows developers to quickly identify whether low GPU utilization is caused by slow data loading or actual model computation limits.

github llama.cpp · 3d ago · 6 views

Aaron Teo adds -lm mlock argument to llama.cpp

Aaron Teo has submitted a change to the llama.cpp repository that introduces a new `-lm mlock` command-line argument. This addition allows users to explicitly lock memory in regions where the code performs `mlock` operations but does not use `mmap`.