The llama.cpp project released version b9928, introducing new VTCM layouts and improved pipelines for the hexagon backend. These changes specifically target performance enhancements for MUL_MAT, MUL_MAT_ID, and FLASH_ATTN_EXT operations.

  • Refactored kernel parameter computation to use a common layout builder.
  • Added explicit compiler barriers to make hmx functions more robust.
  • Implemented a more generic VTCM layout builder for matrix multiplication and flash-attn kernels.
  • Unrolled inner kernels and replaced intrinsics with inline assembly to avoid compiler issues.
  • Optimized DMA pipelines, prefetching, and synchronization fallback paths.
  • Added hybrid polling to the workpool and improved queue wakeup mechanisms.

The release provides binaries for macOS (Apple Silicon and Intel), iOS, Linux (Ubuntu x64, arm64, s390x), Android, Windows, and openEuler across various hardware backends including CPU, CUDA, ROCm, Vulkan, OpenVINO, SYCL, and HIP.