Researchers from Nvidia have introduced FusionRelight, a method for portrait relighting that combines physically plausible illumination transfer with identity preservation and compact real-time inference. The approach utilizes Hybrid Domain Knowledge Fusion (HDKF), a training framework that distills physics, reflectance, and realism priors from synthetic, One-Light-at-a-Time (OLAT), and in-the-wild data into a student model.

  • HDKF learns complementary priors by distilling source-routed supervision from teacher labels and degraded student inputs.
  • The framework is trained with pixel-aligned RGB, albedo, and normal supervision to provide a substrate for physically grounded relighting.
  • On a held-out OLAT benchmark, the method achieves the best MSE, PSNR, and SSIM among evaluated methods while remaining competitive in LPIPS.
  • The distilled model runs in real time at 512x512 resolution, reaching 11.89 ms on an RTX 2060 and 1.82 ms on an RTX 4090.

This work addresses the trade-off between fine detail synthesis in diffusion methods and the poor transfer of physically grounded relighting to unconstrained cameras by providing a deterministic, fast solution for live video creation.