Anchored Self-Play improves code repair by anchoring generator-fixer self-play with reference data

Researchers propose Anchored Self-Play (ASP), a method to scale supervision for language model code repair by using a single model trained via reinforcement learning to generate bugs and fix them. The team introduces BugSourceBench, a benchmark spanning realistic bug sources including human-written code and LM-generated code.

• Standard generator-fixer self-play drifts toward difficult but unrealistic bugs, improving performance on synthetic data while degrading on human-authored ones.
• ASP anchors the process with a small reference set by adding a code-embedding similarity reward for generation and mixing reference bugs into fixer training.
• Across bug sources, ASP achieves the best fix rates, improving average fix rate over standard self-play by +24% relative / +7.0 pp absolute.

The authors consider this important because it addresses the limitation of limited code repair data by creating an automatic curriculum that generalizes better to realistic bugs from both humans and LMs.