Benchmark · agentic
BrowseComp
BrowseComp is OpenAI's benchmark for AI web-browsing agents, testing whether they can track down hard-to-find facts scattered across the internet. Results are reported as the percentage of questions answered correctly.
Read more
- Example
- A typical item asks a question whose short factual answer — a name, date, or number — is buried deep online and takes persistent, multi-step searching across many pages to locate.
- Scoring
- The metric is accuracy: the percentage of questions where the agent's answer matches the single known correct answer.
- Verification
- An answer is accepted when it matches the predetermined ground-truth answer, exact-match style; questions are built to be hard to find but easy to verify once the answer is given.
- Why it matters
- It probes a skill plain chatbots lack — deep, persistent web research over many steps — so it stays a demanding test of agentic browsing even for strong models.
Worked example
Task
BrowseComp item: 'Name a feature film released between 2010 and 2015 that (1) won the Academy Award for Best Picture, (2) dramatizes a single covert rescue operation, (3) was directed by the same person who plays its lead role, and (4) reaches its climax at an airport in Tehran. Give only the film's title.'
Solution
The constraints intersect at one film — a Best Picture winner from 2010–2015 that is a director-led covert-rescue drama climaxing at a Tehran airport. Final answer: Argo (2012), directed by and starring Ben Affleck.
Walkthrough
Each clue prunes the candidate set — Best Picture winners in 2010–2015, then those that are director-starred rescue dramas, then the one climaxing at Tehran's airport — leaving only Argo, which Ben Affleck both directed and starred in. BrowseComp grades a short free-text answer against the reference by exact/model-checked match, so 'Argo' scores correct.