Benchmark · agentic
WebArena
WebArena tests whether an autonomous web agent can complete real, multi-step tasks on a suite of realistic, self-hosted websites — online shopping, a forum, and a content-management system (CMS). Its metric is task success rate: the percentage of tasks the agent finishes correctly.
Read more
- Example
- A representative task: "find the cheapest item in a category and add two of them to the cart" — the agent must reach the goal by browsing, clicking, and typing across the live site.
- Scoring
- The metric is task success rate — completed tasks divided by total tasks, reported as a percentage.
- Verification
- Each task ships with a programmatic checker that verifies the outcome automatically, either by matching an expected answer or by inspecting the website's resulting state, so results are machine-graded rather than human-judged.
- Why it matters
- It shows whether LLM agents can actually operate real websites end to end — a harder, more practical test than single-step or synthetic web tasks — making it a key yardstick for agentic web automation.
Worked example
Task
You are an autonomous web agent in WebArena's self-hosted GitLab environment: you observe the page's accessibility tree and, each step, emit one action from the WebArena action space (e.g.
click [id], type [id] [text] [enter], goto [url], stop [answer]). Intent: create a new issue titled Bug: login button unresponsive in the a11yproject/a11yproject.com repository and assign it to yourself.Solution
goto [http://gitlab.webarena/a11yproject/a11yproject.com/-/issues/new]
type [1023] [Bug: login button unresponsive] [0]
click [1187] # open the Assignee dropdown
click [1203] # choose "Assign to me"
click [1250] # click the "Create issue" button
stop [N/A]Walkthrough
The trajectory opens the new-issue form, types the exact title, self-assigns via the assignee menu, and submits;
stop [N/A] ends the episode because the task mutates state instead of returning a value. WebArena's program_html evaluator reloads the created issue and checks the DOM functionally — title equals the given string and assignee equals the logged-in user — so near-misses score 0.| Date | Model | Score | Source |
|---|---|---|---|
| 2026-07-31 | Qwen-UI-Agent | 73.6% | Qwen-UI-Agent sets state of the art on mobile-use benchmarks |
| 2026-02-24 | GUI-Owl-1.5 | 48.4% | GUI-Owl-1.5 introduces multi-platform fundamental GUI agents |
| 2026-01-12 | ColorBrowserAgent | 71.2% | ColorBrowserAgent introduces adaptive knowledge evolution for robust long-horizon web automation |
| 2025-10-22 | Surfer 2 | 69.6% | Surfer 2 achieves state-of-the-art cross-platform computer use via visual observation |
| 2025-01-23 | CUA | 58.1% | OpenAI introduces Computer-Using Agent (CUA) powering Operator research preview |
| 2024-10-17 | AgentOccam | 9.8% | AgentOccam aligns observation and action spaces to boost LLM web agent performance |
| 2024-08-08 | AWA 1.5 | 57.14% | AWA 1.5 achieves 57.14% on WebArena benchmark |
| 2023-10-05 | SteP | 33.5% | SteP uses stacked LLM policies to improve web task performance |
| 2023-07-25 | GPT-4-based agent | 14.41% | WebArena: realistic web environment for autonomous agents |