Researchers propose Agent Explorative Policy Optimization (AXPO) to address the "Thinking-Acting Gap" in vision-language models that require external tools. This gap causes standard RL methods like GRPO to attempt tool use on only ~30% of rollouts, with high failure rates suppressing learning signals.

  • AXPO fixes the thinking prefix and resamples the tool call for all-wrong subgroups, paired with uncertainty-based prefix selection.
  • Evaluated across nine multimodal benchmarks and three scales of Qwen3-VL-Thinking.
  • SFT+AXPO outperforms SFT+GRPO by +1.8pp Pass@1 and +1.8pp Pass@4 at 8B on average.
  • An 8B model using SFT+AXPO surpasses a 32B Base model on Pass@4 with four times fewer parameters.

This approach effectively resolves the structural asymmetry between internal reasoning and tool use, enabling more efficient and accurate agentic reasoning.