The llama.cpp b10208 release introduces SYCL support for oneMKL GEMM flash attention, enabling XMX hardware acceleration for prompt processing. This change significantly improves inference speed on Intel Arc GPUs by utilizing MKL kernels instead of the default TILE path.

  • Performance benchmarks show a 1.97x speedup for Gemma-4-26B (1473 t/s vs 746 t/s) and a 1.85x speedup for Qwen3.6-27B (609 t/s vs 330 t/s) on B70/Battlemage hardware.
  • The implementation fixes interleaved destination layout bugs in the normalize kernel that previously corrupted attention outputs for most models.
  • MKL flash attention is now enabled for all KV cache types (F16, BF16, F32, and quantized), removing previous restrictions to only quantized caches.
  • New environment variables allow users to enable debugging, disable the feature, or control activation thresholds via GGML_SYCL_ENABLE_MKL_FA.

This update provides substantial latency reductions for multi-token prefill operations on compatible Intel hardware while maintaining perplexity parity with standard implementations.