The llama.cpp project released version b10594, which includes a fix to skip the device_info loop when log verbosity is not set to LOG_LEVEL_TRACE. Previously, this loop iterated over discovered devices to query memory counts, requiring the creation of GPU contexts that resulted in a 550 MB VRAM allocation even when the data was discarded.

  • The change prevents unnecessary GPU resource usage by checking the verbosity level before iterating.
  • Binaries are available for macOS (Apple Silicon and Intel), Linux (CPU, Vulkan, ROCm, OpenVINO, SYCL), Windows (CPU, CUDA 12/13, Vulkan, OpenVINO, SYCL, ROCm), Android, and iOS.
  • The release also provides UI binaries and attestations for verification.

This update helps users who do not want to use any GPU resources avoid the overhead of creating contexts and allocating VRAM when device information is not needed.