A pull request to llama.cpp introduces fused MMVQ (Multi-Mode Vector Quantization) post-scale operations for NVFP4 quantization on CUDA, specifically targeting batch size 1.
The implementation includes dense MMVQ fusion and restricts scale-fusion to NVFP4 to prevent performance regressions in GEMV-heavy configurations. Benchmarks on B4500, DGX Spark, and B6000 hardware show speedups for qwen35moe 35B.A3B NVFP4 ranging from 1.02x to 1.08x compared to the master branch.
The changes also involve refactoring host-side fusion logic, merging single-lane mm-fusion helpers, and reordering bias/scale additions to adhere to related PRs.