github llama.cpp · just now · inference

llama.cpp b10738 adds SYCL host-pinned memory limit

from English

The llama.cpp project has released version b10738, providing binaries for macOS, Linux, Windows, Android, and openEuler across CPU, GPU, and various accelerator backends.

The release includes a specific update to SYCL support that limits maximum allocation memory to 2GB for host-pinned memory.
macOS Apple Silicon builds with KleidiAI are disabled in this version.
Windows arm64 CUDA 13 support is included as a preview.
ROCm 7.14 and OpenVINO 2026.3.1 binaries are available for Ubuntu and Windows.

This update allows users to access the latest llama.cpp functionality on supported hardware, with specific attention to memory constraints in SYCL implementations.

Importance 1/3 Trust 1/3 llama.cpp Hugging Face Inference efficiency

Read original

lab Hugging Face Blog · just now · 1 view Live

Hugging Face releases 207 WebGPU kernels and a browser benchmarking tool

Hugging Face has released @huggingface/kernels, a library for loading and running optimized WebGPU kernels from the Hugging Face Hub, alongside an initial collection of 207 kernels. The release also introduces Fleet, an in-browser GPU benchmarking suite that crowdsources performance and correctness evidence from users' devices.

github llama.cpp · 2h ago · 11 views

llama.cpp b10737 fixes qwen4exp state save/restore and disables -sm tensor

The llama.cpp project released build b10737, which addresses critical bugs in the qwen4exp architecture support. The update resolves issues with sequence state persistence and prevents CUDA aborts during model loading.

github llama.cpp · 11h ago · 2 views

llama.cpp b10730 optimizes Qwen4exp prompt processing by summing indexer heads by slices

The llama.cpp project released build b10730, which includes optimizations for the Qwen4exp model architecture. The primary change involves summing indexer heads by slices instead of using a transpose and sum_rows operation, reducing memory copy overhead.

github llama.cpp · 19h ago · 3 views

llama.cpp b10727 adds concat support for quantized types

The llama.cpp project has released build b10727, which introduces concatenation support for quantized data types. This update is assisted by DeepSeek-V4-Flash-0731 and includes binaries for macOS, Linux, Windows, Android, and openEuler across various hardware backends including CPU, CUDA, ROCm, and Vulkan.

github llama.cpp · 21h ago · 6 views

llama.cpp b10726 speeds up large batch IQ model prompt processing with AVX2

The llama.cpp project released version b10726, which introduces significant performance optimizations for processing large batch sizes in IQ quantized models. The primary update leverages AVX2 instructions to accelerate grid IQ quants through batched GEMM operations.