The llama.cpp b10858 release introduces Vulkan kernel fusion for adjacent UNARY activation (GELU, SIGMOID, SILU, SOFTPLUS) and MUL operations. This optimization recovers a 10-18% regression in prompt-processing speed by eliminating redundant memory accesses and runtime branching.

  • Fused pipelines handle GELU, SIGMOID, SILU, and SOFTPLUS with MUL, using specialized SPIR-V shaders for f32 and f16.
  • The graph optimizer now schedules fused pairs even when separated by zero-compute nodes like VIEW or RESHAPE, extending support to patterns found in models like Gemma4.
  • Support for OP-on-B fusion allows activations on the smaller MUL operand, enabling efficient gating in shared-expert architectures.
  • Backend tests cover new layouts including 'gate' and 'view_mid' to validate correctness across various tensor shapes.

The change significantly improves inference latency for models relying heavily on activation gating mechanisms by reducing computational overhead in the Vulkan backend.