The author addresses silent failures in popular GGUF versions of Nemotron-3-Nano-Omni-30B, where audio and video inputs were ignored due to incomplete projector files and missing inference graphs. To fix this, a unified mmproj file containing the vision tower, full Parakeet/FastConformer audio encoder, and temporal video embedder has been released alongside a specialized llama.cpp fork.
- The new mmproj combines the C-RADIO vision tower, 24-layer FastConformer audio encoder, and temporal video patch embeddings into a single file.
- A dedicated llama.cpp fork implements the necessary audio and video inference graphs, including support for EVS token pruning.
- Verification confirms the video graph matches NVIDIA’s PyTorch reference with 0.0019 percent relative L2 error and exact EVS parity.
- The model is available in nine quantizations on Hugging Face, while audio and video currently require the forked llama.cpp binaries.
This update allows users to process image, audio, and video inputs simultaneously in a single pass, with future upstream releases planned to integrate the audio projector layout.