AdaPrefix-GRPO adapts trace prefixes to double GRPO accuracy on hard math

AdaPrefix-GRPO addresses the issue of vanishing gradients in Group Relative Policy Optimization (GRPO) by dynamically adjusting the length of correct solution prefixes prepended to training problems. By maintaining a success rate near 50%, the method ensures the gradient signal remains strong, eventually withdrawing assistance so the deployed model solves problems unaided.

• On hard math tasks with matched training FLOPs, AdaPrefix-GRPO more than doubles GRPO accuracy for a 0.6B model (2.1x gain).
• It achieves a 1.6x improvement on Qwen3-1.7B and a 1.7x improvement on AIME benchmarks.
• The approach roughly halves the trace length while implementing only data preparation changes and a loss mask on prefix tokens.

The method is particularly effective for smaller models, offering significant accuracy gains without requiring modifications to the underlying trainer.