github llama.cpp · 2h ago · inference

llama.cpp b10737 fixes qwen4exp state save/restore and disables -sm tensor

from English

The llama.cpp project released build b10737, which addresses critical bugs in the qwen4exp architecture support. The update resolves issues with sequence state persistence and prevents CUDA aborts during model loading.

Fixes kvu NaN collapse and restores ext.x/ext.y indexer cache data during save/restore round-trips for qwen4exp models.
Corrects per_layer_token_embd row count derivation and rejects null pointer dereferences in PLE recurrent cache layers.
Disables the -sm tensor flag for qwen4exp, which previously caused NaN logits and crashes on real devices.
Renames seq_set to seq_get_all to resolve function overloading conflicts and adds tests verifying state integrity.

These changes ensure that qwen4exp models can be loaded and saved without data corruption or runtime errors, particularly when using 2D mrope content or PLE layers.

Importance 1/3 Trust 1/3 llama.cpp Hugging Face Inference efficiency

Read original

github llama.cpp · just now · 1 view Live

llama.cpp b10738 adds SYCL host-pinned memory limit

The llama.cpp project has released version b10738, providing binaries for macOS, Linux, Windows, Android, and openEuler across CPU, GPU, and various accelerator backends.

lab Hugging Face Blog · just now · 1 view Live

Hugging Face releases 207 WebGPU kernels and a browser benchmarking tool

Hugging Face has released @huggingface/kernels, a library for loading and running optimized WebGPU kernels from the Hugging Face Hub, alongside an initial collection of 207 kernels. The release also introduces Fleet, an in-browser GPU benchmarking suite that crowdsources performance and correctness evidence from users' devices.

github llama.cpp · 10h ago · 1 view

llama.cpp b10730 optimizes Qwen4exp prompt processing by summing indexer heads by slices

The llama.cpp project released build b10730, which includes optimizations for the Qwen4exp model architecture. The primary change involves summing indexer heads by slices instead of using a transpose and sum_rows operation, reducing memory copy overhead.

github llama.cpp · 19h ago · 3 views

llama.cpp b10727 adds concat support for quantized types

The llama.cpp project has released build b10727, which introduces concatenation support for quantized data types. This update is assisted by DeepSeek-V4-Flash-0731 and includes binaries for macOS, Linux, Windows, Android, and openEuler across various hardware backends including CPU, CUDA, ROCm, and Vulkan.

github llama.cpp · 21h ago · 6 views

llama.cpp b10726 speeds up large batch IQ model prompt processing with AVX2

The llama.cpp project released version b10726, which introduces significant performance optimizations for processing large batch sizes in IQ quantized models. The primary update leverages AVX2 instructions to accelerate grid IQ quants through batched GEMM operations.