Hugging Face Diffusers now supports loading Nunchaku Lite checkpoints natively via `from_pretrained()`, eliminating the need for a separate inference engine or local CUDA compilation. This integration utilizes the SVDQuant method to run transformer layers with 4-bit weights and activations (W4A4), significantly reducing memory usage while improving inference speed.

  • NVFP4 kernels require NVIDIA Blackwell GPUs, while INT4 variants support earlier generations.
  • The system uses `svdq_w4a4` for attention/MLP projections and `awq_w4a16` for precision-sensitive normalization layers.
  • Benchmarks on an RTX PRO 6000 show up to 50% VRAM reduction and roughly 30% latency improvement over BF16 baselines.
  • Combining with `torch.compile` increases the speedup to 1.8x, reducing end-to-end latency to 1.68 seconds for 1024x1024 generation.
  • The diffuse-compressor toolkit allows users to quantize and publish new architectures as standard Diffusers repositories.

This update enables users to leverage low-precision inference directly within the standard Diffusers workflow, making high-performance diffusion models more accessible without custom pipeline implementations.