github llama.cpp · 2h ago · inference

llama.cpp b10813 adds Adreno xmem SDPA path and fixes numerical errors

from English

The llama.cpp project released build b10813, which introduces an OpenCL implementation for Adreno GPUs using the xmem SDPA (Scaled Dot-Product Attention) path. This update also addresses numerical errors in GQA and masked attention mechanisms.

Added the Adreno xmem SDPA path for OpenCL acceleration.
Removed the Adreno-specific queue profiling override.
Fixed numerical errors in GQA and masked attention.
Introduced the GGML_OPENCL_XMEM_SDPA environment variable to control the feature.

The release provides binaries for macOS, Linux, Windows, Android, and openEuler across CPU, GPU, and specialized hardware backends.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

github llama.cpp · 2h ago · 1 view

llama.cpp b10814 extends OpenCL elementwise ops and data movement coverage

The llama.cpp b10814 release expands OpenCL backend support by adding nine new elementwise unary operations and improving data-movement kernels.

github llama.cpp · 2h ago · 1 view

llama.cpp b10816 adds M3 fa-vec tunings and new binaries

The llama.cpp project has released version b10816, which includes updates to the Metal backend for Apple Silicon devices. The release focuses on adding remaining fast-vector tunings specifically for the M3 chip architecture.

github llama.cpp · 2h ago · 1 view

llama.cpp 0.4.0 adds Qwen3.8-Flash-Next and Nemotron-3-Puzzle support

llama.cpp version 0.4.0 introduces initial architecture support for the Qwen3.8-Flash-Next and NVIDIA Nemotron-3-Puzzle-75B-A9B models, alongside a bump of the underlying ggml library to version 0.23.0.

github llama.cpp · 8h ago · 1 view

llama.cpp b10798 makes build info output stream configurable

The llama.cpp project released version b10798, introducing a change to make the build information output stream configurable. This update allows `llama_print_build_info` to write to a caller-provided FILE* instead of hardcoding stderr.

github llama.cpp · 16h ago · 1 view

llama.cpp b10796 adds n_expert_used_max to fix Nemotron-3-Puzzle loading

The llama.cpp project released build b10796, which includes a change to the model loader to support models with varying numbers of experts per layer.