The author has created Nemotron 3.5 Lightning-Omni by attaching pre-trained projectors from NVIDIA’s Nemotron-3-Nano-Omni to the text-only Nemotron 3.5 Lightning model, enabling image and audio understanding without any additional training.
This approach works because both models share the exact backbone geometry (nemotron_h, hidden 2688, 52 layers). The donor Omni’s projectors translate C-RADIO vision features and Parakeet audio features into the embedding space Lightning already uses. Key findings include:
- Image understanding matches the donor model exactly on text and shape reading.
- Audio performance shows a 7.9 percent word error rate compared to the donor's 2.6 percent, with errors limited to rare proper nouns.
- Video support is experimental; while functional, it often reports black screens due to potential drift in the temporal patch pathway.
- Decode speed remains unchanged at 60-69 tok/s on a DGX Spark.
The release serves as an existence proof that perception capabilities can be detached and attached to any future checkpoint sharing the nemotron_h geometry within the architecture family.