The llama.cpp project released version b9968, introducing OpenCL optimizations for Qualcomm Adreno GPUs. The update adds int8 dp4 dense matrix multiplication and Mixture of Experts (MoE) GEMM capabilities to the OpenCL backend.

  • Added int8 dp4 dense and MoE GEMM support for Adreno GPUs via OpenCL.
  • Refactored the OpenCL codebase as part of the update.
  • macOS Apple Silicon with KleidiAI is disabled in this release.
  • Binaries are provided for macOS (Apple Silicon and Intel), Linux, Android, Windows, and openEuler across CPU, Vulkan, CUDA, ROCm, OpenVINO, SYCL, and HIP backends.

This release provides updated binaries for various platforms while enabling specific performance optimizations for Adreno mobile GPUs.