The authors introduce WebMRE, an offline benchmark of 541 tasks and 5,293 steps derived from successful WebArena trajectories, designed to provide a deterministic protocol for scoring web agent checkpoints without environment drift. This framework pairs human-oriented guide sentences with grounded actions to study the mutual reinforcement effect between them.
- Jointly decoding a guide lifts element selection by 0.9 and 0.2 points for Qwen3.5-4B, and by 1.7 and 2.2 points for Qwen3.5-9B.
- Mediation analysis confirms the guide acts as a causal channel: forcing the gold guide as a decoding prefix lifts action accuracy from .422 to .684.
- Fine-tuned models outperform GPT-5.5, Claude Opus 4.8, and Gemini 3.5 Flash in zero-shot offline metrics.
The study demonstrates that guides carry instruction meaning rather than just label strings, offering a reproducible method for evaluating training phenomena in web agents.