github llama.cpp · 1h ago · inference

llama.cpp b10202 release with SYCL RMS_NORM fusion

from English

The llama.cpp project has released version b10202, introducing a SYCL optimization that fuses the RMS_NORM and MUL operations.

This update provides pre-built binaries for macOS (Apple Silicon and Intel), iOS, Linux (Ubuntu x64, arm64, s390x), Windows, and Android. It supports various hardware backends including CUDA 12/13, Vulkan, ROCm 7.2, OpenVINO, SYCL FP32/FP16, HIP, and OpenCL Adreno.

The release also includes the standard llama.cpp UI and notes that KleidiAI support for macOS Apple Silicon is currently disabled.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency Local inference

Read original

github llama.cpp · 3h ago · 1 view

llama.cpp b10201 improves flash attention for quantized KV at long contexts

llama.cpp b10198 adds Vulkan support for quantized concat

llama.cpp b10180 adds contiguous fast path and 32-bit index math for SYCL unary ops

llama.cpp b10155 adds MiMo-V2.5 audio support

llama.cpp b10150 adjusts ggml offloading logic and fixes dsv4 graph