The llama.cpp project released version b9910, which addresses a critical inconsistency between draft model fitting and loading in the server component. This update includes a fix for the progress callback throttling issue during draft model loading and refactors speculative initialization.
- Unifies draft/multi-token prediction parameter initialization, model, and context loading.
- Moves speculative initialization logic to speculative.cpp.
- Changes server_context_impl model_dft and ctx_dft to use raw pointers.
- Disables macOS Apple Silicon KleidiAI support in this release.
- Provides binaries for macOS (Apple Silicon and Intel), iOS, Linux (CPU, Vulkan, ROCm, OpenVINO, SYCL), Android, Windows (CPU, OpenCL, CUDA 12/13, Vulkan, OpenVINO, SYCL, HIP), and openEuler.
This release ensures stable server operation when using speculative decoding by correcting the underlying initialization and loading mechanisms.