The llama.cpp project released build b10058, which introduces support for the Q2_0 quantization format within its Vulkan backend. This update also includes binary distributions for various platforms including macOS, Linux, Windows, and Android.
- Vulkan backend now supports Q2_0 quantization, with performance improvements achieved by doubling rows per workgroup in mat-vec-mul tests.
- macOS Apple Silicon builds with KleidiAI enabled are disabled in this release.
- New binaries are available for Ubuntu (CPU, Vulkan, ROCm 7.2, OpenVINO, SYCL), Windows (CPU, CUDA 12/13, Vulkan, OpenVINO, SYCL, HIP), and Android arm64.
- The macOS UI framework is updated to version b10058.
This release allows users to leverage lower-bit quantization on Vulkan-compatible hardware while providing updated binaries across major operating systems.