The kernels team at Together AI has optimized the ThunderKittens library to leverage new features in the NVIDIA Vera Rubin NVL72 platform, specifically targeting improved performance for NVFP4 and FP8 matrix multiplications.

  • Vera Rubin tensor cores consume operands twice as fast with a 64-byte K step compared to Blackwell's 32 bytes.
  • Tensor memory increases to 576 columns, providing an additional 32 KiB accessible via the .exclusive qualifier.
  • Shared memory is dynamically expandable to 328 KiB, and a B-side collector buffer allows operand reuse.
  • The tcgen05.commit.sync_restrict instruction enables early release of A tiles to decouple loads from computation.
  • Integrating these features pushes NVFP4 performance over 22 PFLOPS, achieving parity with cuBLAS and CuTE DSL.

These optimizations address the bottleneck where tensor cores consume data faster than previous Blackwell kernels could supply it, enabling competitive throughput on the new architecture.