The llama.cpp project released version b9895, which addresses a critical memory safety issue within its speculative decoding component. The update resolves an out-of-bounds read error that occurred in the ngram-map during prompt shrinking operations.
- Fixes out-of-bounds read in ngram-map on prompt shrink.
- Corrects ngram-map cleanup cutoff after prompt shrink.
- Provides binaries for macOS (Apple Silicon and Intel), iOS, Linux (CPU, Vulkan, ROCm, OpenVINO, SYCL), Android, Windows (CPU, CUDA 12/13, Vulkan, OpenCL, OpenVINO, SYCL, HIP), and openEuler.
This release ensures stability for users relying on speculative decoding by preventing potential crashes or undefined behavior associated with the previous ngram-map handling.