The llama.cpp project released version b9901, which includes specific updates to the SYCL backend to enhance performance on Intel hardware. The release provides binaries for macOS, Linux, Windows, Android, and openEuler across CPU, CUDA, ROCm, Vulkan, and other backends.
- Sets K_QUANTS_PER_ITERATION to 1 on the DMMV path to improve GPU utilization.
- Fixes the ggml_sycl_supports_reorder_dmmv gate to enable the reordered feature.
- Adds missing types to ggml_backend_sycl_buffer_init_tensor to ensure proper allocation.
- Replaces QK_WARP_SIZE with WARP_SIZE for QK_5 in SYCL operations.
These changes result in a significant boost to tokens per second (tg t/s) on the Intel B70 GPU.