Agon enables competitive cross-model RL with implicit rival grading of reasoning

The Agon framework introduces a reinforcement learning method where two competing models act as each other's graders, addressing the limitation that standard verifiable reward RL only grades final answers. In this setup, one model drafts a solution while the other reads it and solves simultaneously, with rewards given for out-solving the rival.

• Both models are optimized to progressively out-reason a strengthening opponent without requiring process labels or a separate reward model.
• At inference, the pair deploys as a two-stage cascade where one model drafts and the other answers after reading the draft.
• On the hard split of DeepMath with Qwen3, Agon doubles GRPO's pass@1, achieving roughly eight times the gain of an untrained Mixture-of-Agents approach.
• The ordering effect replicates on competitive-programming code and across model families including Qwen3.5 and Gemma 4.

This approach allows reasoning to be judged implicitly during training, providing a progressively stronger rival that single-model RL cannot offer.