SCOPE-RL optimizes reasoning paths before and after success

Researchers introduce SCOPE-RL, a two-stage framework that densifies reinforcement learning rewards for large language models while retaining the GRPO update mechanism.

The approach combines Adaptive Scaffolded RL, which adds prefix-decomposed verifiable rewards on sub-question chains before success, with Quality-Aware Process RL to refine correct trajectories after success. On Qwen3-8B-Instruct trained on DAPO-Math and Big-Math, SCOPE-RL improves average accuracy by up to 11.2 percentage points and reduces reasoning tokens by up to 27.1% compared to outcome-only GRPO.

These gains hold under GSPO and on Qwen3-0.6B-Instruct, indicating that reward-signal densification is complementary to policy-update-level RLVR advances.