OpAgent achieves 71.6% success rate on WebArena via online RL and modular framework

Researchers propose OpAgent, an operator agent for web navigation that uses online reinforcement learning to overcome distributional shifts in real-world websites. The system combines hierarchical multi-task fine-tuning with a hybrid reward mechanism to optimize policy through direct interaction.

• Hierarchical Multi-Task Fine-tuning curates datasets of Planning, Acting, and Grounding primitives to establish strong instruction-following capabilities.
• Online Agentic RL utilizes a WebJudge for outcome assessment and a Rule-based Decision Tree for progress rewards to mitigate credit assignment challenges.
• The modular OpAgent framework orchestrates a Planner, Grounder, Reflector, and Summarizer to enable robust error recovery and self-correction.

The RL-enhanced model achieves a 38.1% success rate on WebArena, while the full OpAgent framework elevates performance to a state-of-the-art success rate of 71.6%.