github llama.cpp · 1h ago · inference

llama.cpp b10549 enables tensor split for LFM2/LFM2MOE

from English

The llama.cpp project has released version b10549, which primarily introduces support for tensor splitting in LFM2 and LFM2MOE models.

The update includes binaries for macOS (Apple Silicon and Intel), Linux (Ubuntu x64, arm64, s390x with CPU, Vulkan, OpenVINO, and SYCL backends), Windows (CPU, CUDA 12/13, Vulkan, OpenCL, ROCm), and Android.
iOS support is provided via an XCFramework.
openEuler builds are available for x86 and aarch64 architectures using ACL Graph.

This release provides updated binaries across multiple platforms to facilitate running LFM2/LFM2MOE models with the new tensor split capability.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

github vLLM · 4h ago · 3 views

vLLM v0.28.0rc2 adds DFlash2 Spec Decode with local convolution and candidate selector

The vLLM project released version 0.28.0rc2, which includes the DFlash2 speculative decoding feature.

github llama.cpp · 6h ago · 2 views

llama.cpp b10539 fixes Vulkan FA MMQ overflow and adds Windows CUDA 13 support

The llama.cpp project has released version b10539, which addresses a critical numerical stability issue in the Vulkan backend while expanding hardware support for Windows users.

github llama.cpp · 7h ago · 1 view

llama.cpp b10541 adds --mmproj-device argument for multimodal projects

The llama.cpp project released build b10541, introducing the `--mmproj-device` command-line argument to allow users to specify the device backend for loading multimodal projection files.

github llama.cpp · 7h ago · 1 view

llama.cpp b10545 fixes Metal matmul out-of-bounds reads for K not multiple of 32

The llama.cpp project released build b10545, which addresses a critical bug in the Metal backend's tensor API matrix multiplication kernel. The update prevents undefined behavior and corrupted results when processing tensors where the K dimension is not a multiple of 32.

github llama.cpp · 8h ago · 1 view

llama.cpp b10534 adds per-HW CUDA switch points for faster decode

The llama.cpp project released build b10534, which introduces hardware-specific switch points to tune the crossover between vector and tensor-core paths during quantized decoding on CUDA.