A controlled study of 18 runs on 4B and 8B scale language and vision-language models finds that Group Relative Policy Optimization (GRPO) does not credibly improve success rates over a strong supervised baseline on mastered tasks. Moderate to high learning rates make performance worse, while the method only helps when the sampled policy already succeeds more often than the greedy one.
- No configuration improved success rates across variations in learning rate, KL weight, seed, initialization, and clipping.
- The null result holds under paired testing with 25 evaluation seeds and 6 training seeds.
- GRPO increases success by 22 points on tasks where the reward is reachable by sampling.
- Middle learning rates degrade attention and MLP blocks, while high rates cause collapse without traceable group localization.
- At 4B, effective rank in late layers tracks capability; at 8B, this coupling disappears.
The failure indicates that GRPO is ineffective for agents that have already mastered the task, as it mostly reshapes existing behavior rather than adding new skill.