The llama.cpp project released build b9974, which includes a critical fix for CUDA builds. The update prevents fatal crashes that occurred when the library queried GPU memory information on devices with zero or limited available memory.

  • Fixes a crash in cudaMemGetInfo() by making the error non-fatal and assigning zero total/free memory to the affected device.
  • Prevents the fit algorithm from attempting to place layers on devices with no free memory, avoiding hard crashes during startup or when using '-dev none'.
  • macOS Apple Silicon (arm64) builds are available, while KleidiAI support for this platform is disabled.
  • Linux binaries provided for Ubuntu x64 and arm64 (CPU), Vulkan, ROCm 7.2, OpenVINO, and SYCL FP32/FP16.
  • Android arm64 (CPU) builds are included.
  • Windows binaries cover CPU, OpenCL Adreno, CUDA 12/13, Vulkan, OpenVINO, SYCL, and HIP.
  • openEuler support is partially disabled, with specific builds for x86 and aarch64 architectures available.

This change ensures that llama.cpp remains stable on systems where GPU memory resources are exhausted or unavailable, allowing the software to gracefully handle such conditions rather than terminating unexpectedly.