The llama.cpp project has released version b10042, which introduces support for enabling CUDA graphs on Volta and Turing architecture GPUs.

  • The release includes binaries for macOS (Apple Silicon and Intel), Linux (Ubuntu x64, arm64, s390x), Android (arm64), Windows (CPU, OpenCL, Vulkan, ROCm, SYCL, HIP), and openEuler.
  • CUDA support is provided via CUDA 12.4 and CUDA 13.3 DLLs for Windows x64.
  • macOS Apple Silicon builds with KleidiAI are explicitly disabled in this version.
  • OpenEuler support includes x86 and aarch64 variants, with ACL Graph enabled for the 910b variant.

This update allows users on older NVIDIA GPU architectures to utilize CUDA graphs for potential performance improvements.