NVIDIA has released Nemotron-Labs-Audex-30B-A3B (Audex), a unified audio-text large language model built on the Nemotron-Cascade-2-30B-A3B backbone. The model utilizes a single Transformer decoder to process audio and text tokens uniformly, enabling strong multimodal fusion without sacrificing the reasoning or agentic capabilities of its text-only predecessor.
- Audex is trained on 157.4B audio tokens and 320.5B text tokens using multi-stage supervised training.
- The architecture treats text tokens and quantized audio output tokens uniformly during generation.
- Training includes text-only Cascade RL and multi-domain on-policy distillation.
- The model achieves state-of-the-art performance in audio understanding, speech recognition, translation, text-to-speech, audio generation, and speech-to-speech generation.
- Checkpoints are released to facilitate open research.
The release aims to provide a unified solution for audio intelligence tasks while maintaining the high-level reasoning and knowledge retention of the underlying text model.