DFlash, a speculative decoding method using block diffusion from z-lab, has been merged into llama.cpp (PR #22105). The author benchmarked the new feature on an NVIDIA RTX PRO 6000 Blackwell with Qwen 3.6 27B, finding it significantly faster than previous methods like MTP.

  • DFlash fills a block of up to 15 tokens in a single pass rather than drafting them one by one.
  • At 36K context, DFlash achieved 273.04 tok/s compared to 61.47 tok/s for the baseline, a 4.44x speedup.
  • Quality tests on MATH-500 showed minimal degradation (86% vs 87% for the base model).
  • The method incurs approximately 5GB of additional VRAM overhead for the drafter weights and buffers.
  • Speedups increase with context length, reaching 4.44x at 36K, whereas baseline performance decays as context grows.

This approach offers substantial performance gains for local AI usage, particularly in low-concurrency scenarios where the diffusion passes do not compete heavily for compute resources.