The llama.cpp project released version b9910, which addresses a critical inconsistency between draft model fitting and loading in the server component. This update includes a fix for the progress callback throttling issue during draft model loading and refactors speculative initialization.

  • Unifies draft/multi-token prediction parameter initialization, model, and context loading.
  • Moves speculative initialization logic to speculative.cpp.
  • Changes server_context_impl model_dft and ctx_dft to use raw pointers.
  • Disables macOS Apple Silicon KleidiAI support in this release.
  • Provides binaries for macOS (Apple Silicon and Intel), iOS, Linux (CPU, Vulkan, ROCm, OpenVINO, SYCL), Android, Windows (CPU, OpenCL, CUDA 12/13, Vulkan, OpenVINO, SYCL, HIP), and openEuler.

This release ensures stable server operation when using speculative decoding by correcting the underlying initialization and loading mechanisms.