NVIDIA's NeMo team has released Molt, an open-source agentic reinforcement learning framework designed to reduce the complexity of algorithm iteration for researchers. The codebase is approximately 8.6K lines of RL code, significantly smaller than comparable frameworks like verl (62K lines) and slime (25K lines).

  • Molt composes Ray for placement, vLLM for rollout, and NVIDIA AutoModel with FSDP2 for training without forking upstream components.
  • Agents are implemented as standard Python modules using either a Gymnasium-aligned Env or stock OpenAI/Anthropic SDKs via a loopback server.
  • The framework enforces three correctness invariants: token identity, policy-version semantics, and forward consistency, including rollout routing replay for mixture-of-experts models.
  • It supports scaling from dense 4B models to 700B MoE models and ships with Slurm scripts and prebuilt containers for deployment on H100/H200 hardware.

Molt aims to provide research infrastructure that allows researchers to hold the entire codebase in their head, facilitating easier modification of estimators and pipeline stages.