Search API latency for AI agents: how to build a latency budget

A search call inside an agent turn is not one number. Published third-party runs put it anywhere between roughly 400 milliseconds and 13.6 seconds, and most of that spread comes from which endpoint you called rather than which logo is on the invoice. Proxyway's January 2026 search API report measured index-backed APIs near 400ms at p50 with very little variance, and found four of the real-time SERP APIs it tested crossing 5 seconds at p95. AIMultiple's agentic search benchmark, which scores providers on 100 real-world queries and roughly 4,000 retrieved results using an LLM judge with partial human verification, reports a range from Brave Search at 669ms to Parallel Search Pro at 13.6 seconds.

Neither of those is a budget. A budget is what remains after you subtract the planner model call, the content fetch, the extraction step and the synthesis call from whatever deadline a human will actually tolerate. Both reports are written for people running batch jobs, where a 13-second call is a throughput question you solve with concurrency. If you ship an interactive agent, a 13-second call is a product failure, and it is a product failure that recurs.

The number that matters is buried in a single AIMultiple bullet: across five consecutive search calls, total wait ranged from 3 seconds to 68 seconds. That is the shape of the problem. Agent loops do not make one search call, they make several, often in sequence, and every call carries its own tail. This guide is about converting the public latency data into a budget you can hold a design to.

Where the seconds go in one agent turn

Before you argue about vendors, account for the whole turn. A ReAct-style agent answering a research question typically spends time in five places:

Planner model call. Time to first token plus enough generation to emit a tool call. Hundreds of milliseconds on a fast model, longer on a reasoning model.

The search call itself. The number everyone benchmarks, and usually not the largest line item.

Content fetch. Only if snippets are not enough. This is a second network round trip per URL, and it fans out.

Extraction. Turning HTML into something the model can read without burning 40,000 tokens on navigation chrome.

Synthesis model call. Streaming helps here, because time to first token is what the user perceives, not total generation time.

Steps two through four repeat once per loop iteration. That is why per-call latency compounds: an agent that searches, reads three pages, then searches again has already spent two search calls, three fetches, and three extractions before it writes a word.

Two consequences follow. First, shaving 200ms off your search vendor is worth less than removing one loop iteration. Second, the fetch and extraction steps are frequently larger than the search step, and almost nobody benchmarks them together.

Median is the wrong number once calls run in sequence

Both public reports lead with a central-tendency figure. AIMultiple's five-call range (3 to 68 seconds) is the per-call mean multiplied by five, which understates the real distribution because it assumes every call lands at the middle.

Chains do not work that way. If a call exceeds its p95 five percent of the time, the probability that a five-call chain contains at least one such call is 1 minus 0.95^5, about 23 percent. Roughly one turn in four is decided by a tail event rather than a typical one. Inverting it gives the design rule: to deliver a p95 on a five-call chain you need each call to clear its share of the budget about 99 percent of the time, since 0.99^5 is 0.951. Budget against per-call p99, not per-call p50.

This changes vendor selection outright. Consider two providers:

Vendor B wins every median comparison and loses the product. Proxyway's finding that index-backed APIs showed minimal latency variance while real-time APIs showed "much bigger spread" is therefore the most actionable sentence in that report, and it is not the sentence anyone quotes.

What the published numbers say, and who measured them

Two independent runs, different methodologies, no overlap in conclusions for several vendors. Both are worth reading; neither is a substitute for your own measurement.

Four caveats before you cite any of this.

Vendor claims are marketing until someone independent reproduces them. Brave's sub-600ms p90 figure and Tavily's 90ms ultra-fast figure are both self-published and neither has been audited by a third party as far as I can find. Treat them as the vendor's best case on its easiest queries.

The two runs disagree by an order of magnitude on the same vendor. Proxyway places Perplexity near 400ms at p50; AIMultiple places it above 11 seconds. The likeliest explanation is that they called different products, an index lookup versus an answer-synthesis endpoint. Exa shows a smaller version of the same effect. The lesson is that endpoint and mode are the variable, not brand.

Query shape drives the result. Proxyway used randomly generated short queries such as "how tall are dogs". Real agent queries are longer, rarer, and more likely to miss a cache. Expect your own p50 to sit above any published p50.

AIMultiple's page is a rolling comparison. Check the run date on the page before you quote the figures, and note that its scores come from an LLM judge with roughly ten percent human verification rather than from a deterministic metric.

For the pricing and capability side of the same vendor set, see AI search APIs compared; for choosing among the categories in the first place, how to choose a search API for your AI product.

Budgeting backwards from a user-facing deadline

Start with a deadline, not a vendor. Jakob Nielsen's response-time limits, which trace back to Miller's 1968 work, are still the usual anchor: about one second keeps a user in uninterrupted flow, and about ten seconds is the outer limit of held attention. For a streaming agent you get some relief, because time to first token is what the user perceives, but the retrieval phase happens before the first token and is fully exposed.

Here is an illustrative allocation for a four-second retrieval phase inside an interactive turn. These are budget targets to design against, not measurements.

Two rules make this hold. Every allocation is a p99 target for that step, per the arithmetic above. And the reserve is not optional padding; it is the line item that absorbs the one call in four that goes long. A budget with no reserve is a budget that is wrong 23 percent of the time.

If the allocation does not close, the productive lever is almost never "find a vendor 200ms faster". It is to remove a loop iteration, move a step off the interactive path, or accept a partial answer.

Fan-out, parallel calls, and partial results

Running independent queries concurrently converts a sum into a maximum, and it is usually the biggest single reduction available to an agent loop that has not already done it. Three sequential 800ms calls cost 2.4 seconds; the same three concurrently cost roughly 800ms plus connection setup.

Parallelism is not free, though. The maximum of N samples sits further into the tail than any one sample: with three concurrent calls, the chance that at least one lands beyond its p95 is about 14 percent, and the fan-out finishes only when the slowest returns. Two mitigations are worth building.

Return on first k of n. Fire five queries, synthesize as soon as three have returned, cancel the rest. Your latency becomes the third-order statistic instead of the fifth, which sits well short of the tail, and for most research questions the last two results rarely change the answer.

Move anything non-interactive off the critical path. Cache pre-warming and scheduled refreshes belong in a batch lane. ValueSERP is an instructive example here: its batch endpoint accepts jobs of up to 15,000 requests and is asynchronous by design, which makes it a poor interactive dependency and a good offline one. Deciding which of your queries are genuinely user-blocking is usually worth more than any vendor swap. Throughput on that batch lane is a different engineering problem with different answers, covered in fast web scraping.

Timeouts, hedged requests, and a fallback that works

Three practices, in order of how often teams get them wrong.

Set the timeout at the budget, not at your patience. A 30-second timeout on a call you budgeted 900ms for does nothing except guarantee that the failure mode is a hang. Set it slightly above your own observed p95 and treat a breach as a routing decision rather than an exception to log and swallow.

Hedge, do not just retry. Dean and Barroso's "The Tail at Scale" (CACM, 2013) describes the pattern: when a request passes its p95 without returning, issue a duplicate to a second provider and take whichever answers first. You pay for a small percentage of extra calls and recover most of the tail. A plain retry after a timeout is strictly worse, because you have already spent the timeout before you start over.

Make the fallback a different failure domain. This is the part that is usually wrong. Many SERP APIs ultimately resolve the same upstream engine, so when that engine rate-limits or slows down, your primary and your fallback degrade together and you have bought nothing. A fallback is only real if it queries a different index. Brave Search API runs its own independent index; Exa runs its own embeddings index; a Google-derived SERP API such as Serper.dev does not, whatever its own uptime looks like. Pair across index families, and route a small share of live traffic through the fallback on a schedule, because a path you have never exercised tends to fail the first time you need it.

What rendering and extraction add on top of search

The search call is often the smaller half of retrieval. What happens next depends on a design decision most teams make implicitly.

If the search response already contains content the model can use, you are done in one round trip. Tavily, Linkup and Exa's contents mode all return ranked, deduplicated, LLM-ready text in the search response. If the response contains links only, which is what a classic SERP API such as Serper.dev or ValueSERP returns, you now owe a fetch per URL plus an extraction step per page, and both sit on the critical path.

That second round trip is expensive in a way benchmarks rarely capture. Jina AI's Reader turns a URL into markdown in one call, which is the cheap version. The expensive version is a full browser: a cloud session has to start, navigate, wait for network idle, and only then hand you a DOM, and that startup cost lands before any content exists. If your agent needs a browser, budget for it explicitly rather than discovering it in production; browser infrastructure for AI agents covers the tradeoffs.

The practical rule: choosing a search API that returns usable content is a larger latency win than choosing the fastest search API that returns links. A 300ms advantage at the search step does not survive a 1.2-second fetch-and-extract round trip that the other design avoided entirely.

Latency against recall

The intuitive assumption is that a slower provider is doing more work and handing back better results. AIMultiple's run does not support it. Parallel Search Pro, at 13.6 seconds the slowest provider measured, scored 14.21 on that benchmark's agent score. Brave Search, at 669ms the fastest, scored 14.89. The whole scored field sat between 12.28 and 14.89, a narrower band than the twenty-fold latency spread underneath it.

The caveats on that comparison are heavy: one run, 100 queries, judged largely by a model, on a query mix chosen by someone else. It does not establish that deep-research tiers are pointless; those tiers exist because multi-hop questions genuinely need multiple retrieval rounds, and a single-shot relevance score is a poor test of that. What it does establish is that you should not assume you are buying accuracy when you pay in seconds. Measure the lift on your own task before you spend the budget.

Nobody currently publishes latency and accuracy on the same axes for this category. Vals AI's Web Search Index, updated mid-July 2026, scores how much web search lifts model accuracy on legal and finance research tasks but does not report per-provider latency. Proxyway reports latency and does not score answer quality. Joining the two is your job, on your own workload.

Instrumenting your own p95 rather than trusting anyone's chart

Every number above is someone else's query mix on someone else's day. Treat published benchmarks as a shortlist filter and then measure. A workable minimum:

Log per call, not per turn. Vendor, endpoint, mode, query length, time to first byte, total elapsed, result count, retry and hedge counts. Time to first byte separates the vendor's processing from network and payload size, which matters when you are deciding whether to switch regions or switch vendors.

Aggregate per turn. Your user experiences the turn, so the turn is what needs a p95. A dashboard of healthy per-call percentiles alongside a slow product is the classic symptom of aggregating at the wrong level.

Track p99 divided by p50 as a health metric. The ratio measures tail risk directly, and it is what actually degrades when a vendor is having a bad week. A rising ratio with a flat median is the early warning that a median-only dashboard will miss entirely.

Segment by query class. Short factual lookups and long multi-clause research queries have different latency distributions. An overall percentile that mixes them tells you about your traffic mix rather than about your vendor.

Re-measure after every vendor mode change. These vendors ship new speed tiers frequently, and a mode rename can move your p95 without any change on your side.

The short version

Budget the turn, not the call. Take the published numbers as a shortlist filter, noting that Proxyway's January 2026 run and AIMultiple's agentic search benchmark disagree by an order of magnitude on at least one vendor because they called different endpoints. Size every step against p99 rather than p50, because a five-call chain hits at least one p95 event about 23 percent of the time. Fan out where queries are independent and synthesize on the first k of n. Set the timeout at your budget line, hedge past it, and make the fallback a genuinely different index. Prefer an API that returns usable content over one that returns links and a second round trip. Then measure your own percentiles, segmented by query class, and stop trusting the chart, including this one.