The llama.cpp project released version b9969, addressing a critical stability issue in the Vulkan backend. The update routes large matrix multiplications to medium tiles on Adreno GPUs, resolving crashes that occurred with q4_0 quantized networks and longer prompt sizes due to insufficient shared memory.

  • Fixes llama-cli breaking for longer prompts by optimizing shared memory usage in Vulkan matmuls.
  • Removes unused Adreno device code and updates the small pipeline matmul implementation.
  • Disables KleidiAI support for macOS Apple Silicon builds.
  • Provides binaries for Ubuntu (CPU, Vulkan, ROCm 7.2, OpenVINO, SYCL), Windows (CPU, CUDA 12/13, Vulkan, HIP, SYCL), macOS, iOS, Android, and openEuler.

This release ensures stable inference on Adreno devices with larger context windows and maintains compatibility across a wide range of hardware backends.