BrowseComp

BrowseComp is a benchmark of 1,266 short-answer questions, open-sourced by OpenAI in April 2025, that measures whether an agent can find facts that are hard to locate but trivial to verify once found. Each question is written backwards from a target entity: a human author picks an object, then assembles a set of properties that collectively identify only that object, producing a question whose answer is a name or a number but whose evidence is scattered across pages that no single query will surface.

The backwards construction is what makes the benchmark work. Because the answer is short and unambiguous, grading is cheap and reliable, which is where earlier browsing benchmarks struggled. Because the identifying properties are deliberately obscure and spread out, a model without web access scores close to zero and a model with a single search call does little better, so what the benchmark actually measures is persistence: how many query reformulations and page fetches an agent will spend before it either finds the entity or gives up. OpenAI reported its Deep Research model at roughly 51.5 percent with extensive test-time compute, and also reported that Deep Research solved 16 percent of tasks on all 64 trials while failing 14 percent on every trial, which says the question set has a wide internal difficulty spread rather than a uniform bar.

For someone choosing a search API or a research agent, BrowseComp is the closest public proxy for deep-research quality, and several search vendors now publish scores against it. Exa has self-reported results for its agent tiers on the full question set with per-query costs attached, and Parallel has published results for its search API inside a shared deep-research harness with a capped tool-call budget. That cost-per-query annotation is the part worth reading: a system that reaches a given accuracy in 25 tool calls and one that reaches it in 200 are not the same product, and BrowseComp accuracy alone will not distinguish them.

Scores have moved fast. As of July 2026, BenchLM.ai lists top reported BrowseComp results in the high 80s and low 90s, with the leading entries clustered within about two points of each other, which it reads as the benchmark approaching saturation for frontier systems. Public leaderboards state the caveat plainly, and it is the one to carry into any vendor conversation: Steel.dev's BrowseComp leaderboard notes that it mixes base-model, model-with-browsing, and full research-agent reports, and that model-only and tool-augmented rows are directional unless the source setups match. Two numbers on the same leaderboard are often not comparable. BrowseComp-Plus, presented at ACL 2026, was built specifically to separate retriever quality from agent quality by running the same reasoning-heavy queries against a fixed corpus of roughly 100,000 human-verified documents rather than the live web.