github llama.cpp · 2h ago · inference

llama.cpp b10814 extends OpenCL elementwise ops and data movement coverage

from English

The llama.cpp b10814 release expands OpenCL backend support by adding nine new elementwise unary operations and improving data-movement kernels.

Added extended elementwise unary ops including sgn, step, elu, hardswish, hardsigmoid, floor, ceil, round, and trunc to prevent CPU fallbacks.
Implemented a contiguous f32 copy kernel that dispatches over the whole device for linear moves, optimizing performance for tensors with few long rows.
Extended CONCAT support to all easy-copy types (f16, bf16, i8, i16, i32, i64) by keying kernels on element size rather than specific type.

These changes improve OpenCL coverage and efficiency for non-CPU backends across macOS, Linux, Windows, and Android.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

github llama.cpp · 2h ago · 1 view

llama.cpp b10813 adds Adreno xmem SDPA path and fixes numerical errors

The llama.cpp project released build b10813, which introduces an OpenCL implementation for Adreno GPUs using the xmem SDPA (Scaled Dot-Product Attention) path. This update also addresses numerical errors in GQA and masked attention mechanisms.

github llama.cpp · 2h ago · 1 view

llama.cpp b10816 adds M3 fa-vec tunings and new binaries

The llama.cpp project has released version b10816, which includes updates to the Metal backend for Apple Silicon devices. The release focuses on adding remaining fast-vector tunings specifically for the M3 chip architecture.

github llama.cpp · 2h ago · 1 view

llama.cpp 0.4.0 adds Qwen3.8-Flash-Next and Nemotron-3-Puzzle support

llama.cpp version 0.4.0 introduces initial architecture support for the Qwen3.8-Flash-Next and NVIDIA Nemotron-3-Puzzle-75B-A9B models, alongside a bump of the underlying ggml library to version 0.23.0.

github llama.cpp · 8h ago · 1 view

llama.cpp b10798 makes build info output stream configurable

The llama.cpp project released version b10798, introducing a change to make the build information output stream configurable. This update allows `llama_print_build_info` to write to a caller-provided FILE* instead of hardcoding stderr.

github llama.cpp · 16h ago · 1 view

llama.cpp b10796 adds n_expert_used_max to fix Nemotron-3-Puzzle loading

The llama.cpp project released build b10796, which includes a change to the model loader to support models with varying numbers of experts per layer.