github llama.cpp · just now · inference

llama.cpp b10158 adds eagle3-v3 support for gpt-oss model

from English

The llama.cpp project has released build b10158, which introduces support for the eagle3-v3 architecture within the gpt-oss model.

This update is available across multiple platforms and hardware configurations. Downloads include binaries for macOS (Apple Silicon and Intel), Linux (Ubuntu with CPU, Vulkan, ROCm 7.2, OpenVINO, and SYCL backends), Windows (CPU, CUDA 12/13, Vulkan, OpenVINO, SYCL, and HIP), Android, and openEuler.

The release provides pre-built binaries for the main application and its UI, enabling users to run the updated model on a wide variety of systems without compiling from source.

Importance 1/3 Trust 1/3 llama.cpp Hugging Face Inference efficiency

Read original

github llama.cpp · 5d ago · 2 views

llama.cpp b10098 updates Hexagon activation ops

The llama.cpp project released version b10098, which includes an update to the Hexagon activation operations. This release provides binaries for macOS, Linux, Android, Windows, and openEuler across various CPU and GPU backends.

lab Hugging Face Blog · 5d ago · 1 view

Diffusers adds native Nunchaku Lite 4-bit inference support

Hugging Face Diffusers now supports loading Nunchaku Lite checkpoints natively via `from_pretrained()`, eliminating the need for a separate inference engine or local CUDA compilation. This integration utilizes the SVDQuant method to run transformer layers with 4-bit weights and activations (W4A4), significantly reducing memory usage while improving inference speed.

github llama.cpp · 6d ago · 1 view

llama.cpp b10089 adds full k-quant and i-quant support to CUDA GET_ROWS

The llama.cpp b10089 release extends CUDA GET_ROWS operations to support k-quants, i-quants, and mxfp4 quantization formats. This change ensures that all quantized GGML types can now take the direct device path for embedding lookups.

github llama.cpp · 6d ago

llama.cpp b10085 fixes Qwen3-VL grounding by aligning position embedding interpolation

The llama.cpp project released build b10085, which includes a fix for the Qwen3-VL vision model's position embedding interpolation to resolve grounding coordinate mismatches.

github llama.cpp · 8d ago · 2 views

llama.cpp b10069 adds OpenCL broadcast support for Adreno MUL_MAT

The llama.cpp project released build b10069, which includes specific OpenCL updates for Qualcomm Adreno GPUs. The primary change involves supporting broadcast operations for the Adreno MUL_MAT kernel and honoring view offsets for the Q8_0 MUL_MAT variant to enable multi-stream functionality in llama-server.