The llama.cpp project released version b9948, which includes a key optimization to the ggml library for CUDA. The update modifies `ggml_top_k()` and `ggml_argsort()` to process data in smaller chunks, thereby reducing temporary buffer memory usage.

  • Implemented chunked processing in both CUDA CUB top-k and argsort implementations.
  • Allocated `tmp_dst` only once before the loop to further optimize memory.
  • Replaced ternary operators with min/max functions for code clarity.
  • Released binaries for macOS (Apple Silicon and Intel), Linux, Windows, Android, and openEuler across CPU, CUDA, Vulkan, ROCm, OpenVINO, SYCL, and HIP backends.

This release provides users with updated builds for various platforms while addressing memory efficiency in specific CUDA operations.