github llama.cpp · 1h ago · inference

llama.cpp b10207 adds SYCL support for missed types in cpy

from English

The llama.cpp project has released version b10207, which includes a pull request to add SYCL support for previously unsupported data types in the cpy operation.

The update addresses missing type support within the cpy functionality via PR #26005.
Binary distributions are provided for macOS (Apple Silicon and Intel), Linux (CPU, Vulkan, ROCm 7.2, OpenVINO, SYCL FP32/FP16), Windows (CPU, CUDA 12/13, Vulkan, OpenVINO, SYCL, HIP), Android, and openEuler.
macOS Apple Silicon with KleidiAI is currently disabled in this release.

This release expands hardware compatibility for llama.cpp by enabling SYCL execution on additional data types.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency Local inference

Read original

github llama.cpp · 3h ago

llama.cpp b10206 enforces same K and V cache types for DeepSeek V4 and MLA models

The llama.cpp project has released version b10206, which introduces specific changes to how key-value caches are handled in attention mechanisms. The update ensures that the K and V cache types remain consistent for DeepSeek V4 models.

github llama.cpp · 5h ago

llama.cpp b10202 release with SYCL RMS_NORM fusion

The llama.cpp project has released version b10202, introducing a SYCL optimization that fuses the RMS_NORM and MUL operations.

github llama.cpp · 7h ago · 1 view

llama.cpp b10201 improves flash attention for quantized KV at long contexts

The llama.cpp project has released version b10201, which includes a key improvement to the ggml-webgpu backend. This update enhances flash attention performance when handling quantized key-value caches during long context generation.

github llama.cpp · 21h ago · 1 view

llama.cpp b10198 adds Vulkan support for quantized concat

The llama.cpp project has released build b10198, which introduces support for quantized concatenation operations within the Vulkan backend.

github llama.cpp · 2d ago · 1 view

llama.cpp b10180 adds contiguous fast path and 32-bit index math for SYCL unary ops

The llama.cpp project has released build b10180, introducing performance optimizations for the SYCL backend. The update implements a contiguous fast path and switches elementwise index math to use fastdiv with 32-bit integers.