The llama.cpp project released version b10099, which includes a significant update to the CUDA backend for improved NVFP4 W4A4 activation quantization.

  • Fused per-channel amax and quantization kernels for better performance.
  • Added support for nvfp4x4 intrinsics when available on NVIDIA hardware.
  • Optimized scale-search with intrinsics and restored per-block amax calculations.
  • Implemented 32-byte loads and pointer arithmetic optimizations to reduce overhead.
  • Guarded builtin_align(32) struct usage specifically to NVIDIA, noting HIP compatibility issues.

This release provides updated binaries for macOS, Linux, Windows, Android, and openEuler across CPU, CUDA, Vulkan, ROCm, OpenVINO, SYCL, and other backends.