The llama.cpp project has released version b9909, which introduces timing and progress information to the /responses API stream. This update is available across multiple platforms including macOS, Linux, Windows, Android, and openEuler.
- The primary functional change is the addition of timings and progress details in the streaming output for the /responses API endpoint.
- Pre-built binaries are provided for various architectures such as Apple Silicon, Intel x64, ARM64, and s390x.
- GPU support includes CUDA 12.4 and 13.3, Vulkan, ROCm 7.2, OpenVINO, SYCL, and HIP.
- KleidiAI support for macOS Apple Silicon is disabled in this release.
This update allows users to monitor the progress of API responses more effectively.