The llama.cpp project released version b10032, which introduces a new CUDA implementation for the GGML_OP_LIGHTNING_INDEXER operation. This update includes both generic vector and WMMA kernels to handle indexing tasks on NVIDIA hardware.
- The CUDA lightning indexer uses generic vector and WMMA kernels.
- Template parameters WARPS_PER_BLOCK and K_VECS_PER_BLOCK were added to reduce constant duplication.
- MMA architecture requirements were relaxed to support Turing GPUs.
- Alignment checks for Q and K tensors were added to the implementation.
- The function ggml_cuda_op_lightning_indexer was renamed to ggml_cuda_lightning_indexer.
This release provides binaries for macOS, Linux, Windows, Android, and openEuler across CPU, CUDA, Vulkan, ROCm, OpenVINO, SYCL, and other backends.