github llama.cpp · 1h ago · inference

llama.cpp b10307 fixes SYCL NVFP4 parsing

from English

The llama.cpp project released version b10307, which includes a fix for the SYCL GPU code to correctly parse UE4M3 scaling factors used in NVFP4 quantization.

The update corrects the conversion of scale values from signed E4M3 to unsigned, as NVFP4 stores these factors in single UE4M3 bytes.
Binaries are available for macOS (Apple Silicon and Intel), Linux (CPU, Vulkan, ROCm 7.2, OpenVINO, SYCL FP32/FP16), Android, Windows (CPU, OpenCL Adreno, CUDA 12/13, Vulkan, OpenVINO, SYCL, HIP), and openEuler.
A UI build is also provided for general use.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency Local inference

Read original

github llama.cpp · 16h ago

llama.cpp b10298 adds mtmd chunk save/load function

The llama.cpp project has released build b10298, which introduces a new chunk save and load function for the mtmd module. This update also includes code cleanup and additional tests.

github llama.cpp · 2d ago · 1 view

llama.cpp b10282 adds spec-decode counters to /metrics endpoint

The llama.cpp project has released version b10282, which introduces new monitoring capabilities for speculative decoding. The server now includes spec-decode counters in its /metrics endpoint, allowing users to track performance metrics related to this feature.

github llama.cpp · 3d ago

llama.cpp b10298 adds mtmd chunk save/load function

llama.cpp b10282 adds spec-decode counters to /metrics endpoint

llama.cpp b10259 allows tensor reshape during load

llama.cpp b10254 adds DeepSeek V4 Flash 0731 template and updates V4 templates

llama.cpp b10251 adds MTP support for GLM-4.7-Flash