The llama.cpp project released build b11059, which introduces significant updates to the Metal backend for Apple Silicon hardware. The primary change is the addition of F16 input support to the Fast Walsh-Hadamard Transform (FWHT) kernel, allowing it to read F16 sources directly without requiring a converted copy.

  • The Metal FWHT kernel now accepts F16 input by making the source type a template parameter, while F32 instantiations remain unchanged.
  • A branchless butterfly select in the FWHT SIMD group kernel replaces ternary operators, yielding a 3.0% performance gain (1285.0 us vs 1324.6 us) on an M5 Pro for large Hadamard matmuls.
  • Dispatch predicates were moved to ggml-metal-common to fix the macos-latest-arm64 build and ensure consistency between supports_op and the dispatch logic.
  • The release includes binaries for macOS, iOS, Linux, Windows, Android, and openEuler across CPU, CUDA, ROCm, Vulkan, OpenVINO, and SYCL backends.

This update improves efficiency on Apple Silicon by reducing memory overhead and latency for F16 operations in the Metal backend.