AdaPrefix-GRPO adapts reference prefixes to double GRPO accuracy on hard math

The authors propose AdaPrefix-GRPO, a method that dynamically adjusts the length of a correct solution prefix during Group Relative Policy Optimization (GRPO) training. By treating prefix length as a feedback-controlled knob, the system maintains a success rate near 50% to maximize gradient signal before withdrawing assistance entirely.

• The approach prevents GRPO from stalling on hard problems where group-relative advantages vanish due to zero successes.
• It increases accuracy by more than 2x for a 0.6B model and 1.6x for Qwen3-1.7B on hard math benchmarks at matched training FLOPs.
• The method roughly halves trace length while requiring only data preparation changes and a loss mask on prefix tokens.

This technique allows models to learn from frontier examples more effectively by ensuring the difficulty is continuously adjusted to match the model's current capability.