Mistral has released Magistral Small and Magistral Medium, its first reasoning models built using a ground-up reinforcement learning (RL) pipeline without distillation from prior models. Magistral Medium is trained on top of Mistral Medium 3 using only RL, while Magistral Small (24B) includes cold-start data from Magistral Medium and is open-sourced under the Apache 2.0 license.
- Magistral Medium achieves a nearly 50% boost in AIME-24 (pass@1) through pure RL training.
- The team developed an asynchronous infrastructure for fast, continuous online RL that updates generators frequently without interruption.
- A strategy was introduced to force the model's chain-of-thought and final response into the user's language.
- Experiments show that RL on text data maintains or improves multimodal understanding, instruction following, and function calling capabilities.
The release provides open-source weights for Magistral Small and details a scalable GRPO-based framework that eliminates the need for a critic model and KL divergence penalties.