The llama.cpp b10016 release introduces SYCL support for Flash Attention using the XMX engine via the oneDNN graph API. This update is primarily targeted at Intel Battlemage (Xe2) hardware, narrowing the scope of the feature to this architecture while other architectures fall back to existing kernels.

  • F16 Flash Attention provides a 1.21x prefill speedup for Qwen3.6-27b-Q8_0 at sequence length 512 and a 4.26x speedup at 80k.
  • Benchmarks on Meta-Llama-3.1-8B-Instruct show a 32% increase in prefill performance with only a 0.11% difference in perplexity.
  • The implementation includes a Battlemage gate, multi-device synchronization for stream handling, and fixes for multi-GPU sync issues.

This change enables faster inference on compatible Intel GPUs by leveraging hardware-specific acceleration capabilities.