WebArena

WebArena is an academic benchmark released in 2023 by researchers at Carnegie Mellon University for evaluating autonomous web agents on realistic, end-to-end tasks. The benchmark provides a self-hosted environment with functional clones of popular sites – an e-commerce store (OneStopShop), a content management system (GitLab), a forum (Reddit), a collaborative workspace, and a maps service – and asks agents to complete 812 human-written tasks spanning product search, code review, navigation, and information retrieval. Because the sites are self-hosted, evaluation is reproducible and does not depend on the live internet.

The benchmark works by giving an agent a natural-language goal (for example, "What is the total cost of my latest order?") and a starting URL, then measuring whether the agent successfully completes the task within a fixed step budget. Agents interact through a browser interface and are scored on functional correctness rather than on intermediate actions. When WebArena was published, frontier LLMs scored roughly 14 percent on its tasks – an order of magnitude below human performance at 78 percent – which made it one of the most cited reference points for the limits of agent capability. By 2025, top systems have pushed past 50 percent, but the gap to humans remains real.

For AI builders evaluating browser-agent products or computer-use systems, WebArena scores are useful as a relative signal: a vendor claiming agentic capabilities should be able to point to credible numbers on WebArena, VisualWebArena, or a comparable benchmark. The benchmark's limitation is that its sites are static and self-hosted, which makes it easier than the open web. For end-to-end realism, builders typically pair WebArena results with results on Online-Mind2Web or live-site evaluations such as ClawBench.