The Agon framework enables reinforcement learning from verifiable rewards by making two competing models grade each other's reasoning processes during training. Instead of relying on final answers or external reward models, the system uses a mechanism where one model drafts a solution while the other solves it in alternating roles, rewarding the pair for out-solving their rival.
- Both models are optimized simultaneously, facing a progressively stronger opponent that single-model RL cannot provide.
- The approach requires no process labels and functions without a separate reward model.
- At inference, the trained pair deploys as a two-stage cascade where one drafts and the other answers after reading the draft.
- On the hard split of DeepMath with Qwen3, Agon doubles GRPO's pass@1 score.
- The method yields roughly eight times the gain of an untrained Mixture-of-Agents pass over the same base model.
- Performance improvements replicate across competitive programming code and different model families including Qwen3.5 and Gemma 4.
This method allows models to improve reasoning implicitly by competing against each other, with future work aiming to enable latent space interaction.