What is Artificial Analysis Search Index?
Independent leaderboard from Artificial Analysis that runs one fixed agent over many search APIs and scores each on answer quality, cost per task, and time per task.
Our verdict
Artificial Analysis, better known for its model leaderboards, launched this index on August 18, 2026 to answer a question search vendors usually answer about themselves: which API helps an agent give better answers? The method holds everything else still. The candidate model is always GPT-5.6 Luna (medium). Third-party APIs run inside Stirrup, AA's open-source agent harness, with web_search and web_fetch tools and a 25-turn budget; OpenAI Web Search is the one exception and runs through OpenAI's own tool. The score is an equal-weighted mean of three evals: DeepSearchQA (all 900 tasks, mean F1), a hard 200-question BrowseComp subset, and 600 private AA-Omniscience questions.
All the numbers below are AA's. At launch it tested 11 configurations from 7 providers, led by Parallel Search (advanced) at 75, Exa at 74 and Firecrawl at 73, against 33 for the model with no search. The live board (data as of October 19, 2026) covers 27 configurations from 14 providers. Exa Search (auto, dynamic highlights) and Perplexity Search (medium) both show 80, and Octen is fastest at 14.5 seconds per task. Cost per task adds the provider's list-price search spend to the model's token spend, which is closer to your real bill than a per-query price.
For a team picking a search API, this is the most direct public comparison available. Two caveats apply. Every result comes from one model in one harness, so a different model or prompt could reorder the middle of the table. And vendors now cite their rank in marketing, so check the live leaderboard rather than a screenshot.
Categories:
Benchmarks are how you separate marketing claims from measured reality. Instead of trusting vendor-reported numbers, benchmarks run the same tasks against every system under a shared methodology and publish the results. For AI product builders picking an agentic extraction or search stack, a trustworthy benchmark is a strong input to the build-vs-buy decision – and a fast way to spot when a category is still too immature to rely on.
