github llama.cpp · 9h ago · inference

llama.cpp b11320 adds LLM-jp-4.1 Harmony dialect handler

from English

The llama.cpp project released build b11320, which includes a new dialect handler for the LLM-jp-4.1 model.

The update adds support for the LLM-jp-4.1 Harmony dialect to address compatibility issues with its tokenizer and parallel tool call formatting.
The release provides binaries for macOS (Apple Silicon and Intel), iOS, Linux (CPU, Vulkan, CUDA 12/13, ROCm, OpenVINO, SYCL, Snapdragon), Android, Windows, and openEuler.
A standalone UI package is also included in the distribution.

Importance 1/3 Trust 1/3 llama.cpp Local inference

llama.cpp b11307 preserves batch order for speculative decoding

The llama.cpp project released build b11307, which fixes layer input ordering to preserve the original batch sequence during speculative decoding. This change ensures that token order is correctly maintained for unmasked NextN embeddings and compatible with tensor splits.

github llama.cpp · 22h ago · 1 view

llama.cpp b11301 fixes integer overflow in ggml/gguf

The llama.cpp project has released build b11301, which includes a fix for an integer overflow issue within the ggml and gguf components. This update addresses validation logic to prevent potential errors when handling zero-element tensors.

github llama.cpp · 23h ago · 3 views

llama.cpp b11302 fixes ThreadSanitizer data race in sparse indexer

The llama.cpp project released build b11302, which addresses a critical concurrency issue identified by ThreadSanitizer in the CI pipeline. The update resolves a data race within the sparse attention mechanism where multiple CPU threads were incorrectly writing to the same memory elements.

github llama.cpp · 23h ago · 2 views

llama.cpp b11298 adds dflash support to convert and feature extraction

The llama.cpp project has released build b11298, which introduces support for the dflash format. This update enables both conversion of models to this format and subsequent feature extraction capabilities.

github llama.cpp · 1d ago · 3 views

llama.cpp b11284 fixes OpenVINO GET_ROWS on quantized weight views

The llama.cpp project released build b11284, which includes a fix for the OpenVINO backend to correctly handle GET_ROWS operations on quantized weight views.