serp.fast

ClawBench

Open source benchmark evaluating AI browser agents on 153 everyday tasks across 144 live websites, with request interception and full behavioral trace capture.

Nathan Kessler
By Nathan KesslerUpdated

Each tool is evaluated against our methodology using public docs, vendor demos, and hands-on testing.

Benchmarks are how you separate marketing claims from measured reality. Instead of trusting vendor-reported numbers, benchmarks run the same tasks against every system under a shared methodology and publish the results. For AI product builders picking an agentic extraction or search stack, a trustworthy benchmark is a strong input to the build-vs-buy decision – and a fast way to spot when a category is still too immature to rely on.

Visit ClawBench →

Features

✓JS Rendering
✓Structured Output
✓Open Source
✓Self-Hosted Option
Pricing:Free

Editorial assessment

A rare benchmark run on real production websites rather than sandboxed environments. Five layers of behavioral data – session replay, screenshots, HTTP traffic, reasoning traces, and browser actions – make failure analysis tractable, while a request interceptor blocks irreversible actions like payments and bookings before they fire. Ships as `pip install clawbench-eval` with an interactive leaderboard and trace viewer at claw-bench.com. The current numbers are sobering: Claude Sonnet 4.6 tops at 33.3%, GLM-5 trails at 24.2%, and no model exceeds 50% in any category. Finance and academic tasks are easier; travel and dev tasks are much harder. If you're building or picking an agentic extraction stack, this is the honest scoreboard to test against.
See full ClawBench alternatives comparison →

Frequently asked questions

Is ClawBench free to use?

Yes. ClawBench is open source and free, with no paid tier or subscription. You install the evaluation harness with pip install clawbench-eval and run it against your own agent. The leaderboard and an interactive trace viewer are public at claw-bench.com. Because it is a benchmark rather than a hosted product, your only real cost is the model API usage you spend running the tasks.

Is ClawBench open source and can I self-host it?

Yes on both counts. ClawBench is open source, and the evaluation harness installs locally through pip install clawbench-eval, so you run it on your own infrastructure instead of calling a hosted service. That lets you score in-house agents privately, inspect the task definitions, and reproduce results without sending your runs to a third party.

What does ClawBench actually measure?

ClawBench measures whether AI browser agents can finish everyday online tasks on real, live websites rather than sandboxed clones. The corpus covers 153 tasks across 144 sites, such as booking travel, ordering food, and applying for jobs. It captures five layers of behavioral data per run: session replay, screenshots, HTTP traffic, reasoning traces, and browser actions. A request interceptor blocks irreversible actions like payments before they fire.

How do AI agents score on ClawBench?

Low, which is the point. The top system, Claude Sonnet 4.6, reaches 33.3 percent, and no model clears 50 percent in any category. Finance and academic tasks tend to be easier, while travel and developer tasks are much harder. The leaderboard is updated as newer models and harnesses are submitted, so check claw-bench.com for current standings rather than relying on a fixed figure.

How does ClawBench compare to Browser Use?

They solve different problems. Browser Use is an agent framework you build browser automation with, whereas ClawBench is the scoreboard you test such agents against. They work together. You would run a Browser Use agent through the ClawBench harness to see how it handles live sites. Stagehand and Skyvern are the closest agent-framework alternatives if you are choosing a stack. ClawBench itself has no real substitute for evaluating on real production websites.

Who should use ClawBench?

Teams building or selecting an agentic web-extraction or automation stack. If you are deciding between agent frameworks or model backends for browser tasks, ClawBench gives you a reproducible test on real sites instead of vendor demos. It is less useful if you only need a finished scraper, since it is an evaluation tool, not a data API. Pair it with a framework like Browser Use, Stagehand, or Skyvern to generate the runs you score.

Weekly briefing – tool launches, legal shifts, market data.

Visit

ClawBench

Visit →