github llama.cpp · 1h ago · inference

llama.cpp b10159 adds FWHT kernel for Metal backend

from English

The llama.cpp project released build b10159, which introduces a Fast Walsh-Hadamard Transform (FWHT) kernel for the Metal backend. This update includes work-in-progress implementation details along with shape guards and code formatting adjustments.

Added FWHT kernel support for the Metal backend in ggml-metal.
Included fixes for narrowing issues and minor style corrections.
Provided binaries for macOS, iOS, Linux, Windows, Android, and openEuler across CPU, GPU (CUDA, Vulkan, ROCm, SYCL, OpenVINO), and Apple Silicon architectures.

The release makes the new Metal kernel available to users on compatible platforms.

Importance 1/3 Trust 1/3 llama.cpp Hugging Face Inference efficiency

Read original

github llama.cpp · 3h ago · 2 views

llama.cpp b10158 adds eagle3-v3 support for gpt-oss model

The llama.cpp project has released build b10158, which introduces support for the eagle3-v3 architecture within the gpt-oss model.

github llama.cpp · 5d ago · 2 views

llama.cpp b10098 updates Hexagon activation ops

The llama.cpp project released version b10098, which includes an update to the Hexagon activation operations. This release provides binaries for macOS, Linux, Android, Windows, and openEuler across various CPU and GPU backends.

lab Hugging Face Blog · 5d ago · 1 view

Diffusers adds native Nunchaku Lite 4-bit inference support

Hugging Face Diffusers now supports loading Nunchaku Lite checkpoints natively via `from_pretrained()`, eliminating the need for a separate inference engine or local CUDA compilation. This integration utilizes the SVDQuant method to run transformer layers with 4-bit weights and activations (W4A4), significantly reducing memory usage while improving inference speed.

github llama.cpp · 6d ago · 1 view

llama.cpp b10089 adds full k-quant and i-quant support to CUDA GET_ROWS

The llama.cpp b10089 release extends CUDA GET_ROWS operations to support k-quants, i-quants, and mxfp4 quantization formats. This change ensures that all quantized GGML types can now take the direct device path for embedding lookups.

github llama.cpp · 6d ago

llama.cpp b10085 fixes Qwen3-VL grounding by aligning position embedding interpolation

The llama.cpp project released build b10085, which includes a fix for the Qwen3-VL vision model's position embedding interpolation to resolve grounding coordinate mismatches.