Researchers present CANON (Consensus-ANchored self-distillatiON), a label-free training method that converts the consensus signal from sampling multiple solutions into dense, token-level supervision. For each unlabeled prompt, the method extracts the majority answer and uses it to condition a frozen snapshot of the model, which then supervises the model on its own rollouts at every token.
- CANON improves pass@1 by up to 12 points on mathematical and scientific reasoning benchmarks.
- It outperforms label-free reinforcement learning by 6 points while using only a seventh of the compute.
- The method approaches performance levels of teacher models conditioned on gold solutions.
- Trained on pooled unlabeled data, it transfers to held-out benchmarks, matching methods that use gold labels.
The authors consider this important because the improvements are not merely distribution sharpening; after training, the model solves problems it previously never solved in 32 attempts, and its majority vote itself becomes more accurate.