Multi-Hop Retrieval

Multi-hop retrieval is answering a question that requires chaining evidence across two or more documents, where the result of each retrieval step determines what to search for next. The canonical shape is a bridge question: "who directed the film that won Best Picture the year X was born" cannot be answered by any single document, because the first hop returns a birth year, the second returns a film, and only the third returns the director. No amount of ranking quality on the original question text will surface the final document, because the terms that would retrieve it are not in the question.

Single-shot vector retrieval fails here for a structural reason rather than a quality reason. Embedding the user's question and pulling the top k nearest chunks assumes the answer is semantically close to the question, which holds for lookup questions and breaks for chained ones. The intermediate entity, the bridge, is what connects the hops, and it does not exist in the query until the first retrieval has already run. Widening k does not fix it; it fills the context with plausible neighbors of the question while the actual evidence sits under a term the system has not seen yet. The fix is to make retrieval a loop rather than a call: retrieve, read, extract the bridge entity, issue a new query, repeat until the chain closes. That loop is exactly what agentic search does, which is why multi-hop is the workload that separates an agentic search system from a search endpoint with an LLM on top.

For someone building an AI product, the practical question is whether your queries are actually multi-hop, because the loop is expensive. Each hop adds latency and a full model call, and errors compound: a wrong bridge entity on hop one guarantees a wrong answer on hop three, with no signal to the user that anything went wrong. Support-document lookup and policy questions are usually single-hop and should stay single-hop. Competitive research, entity enrichment, and due-diligence questions are usually multi-hop, and running them through a single vector query produces confident answers assembled from the wrong documents. If you do need the loop, cap the hop count and log the intermediate queries, because an unbounded agent will happily spend forty tool calls on a question it was never going to answer.

The benchmarks buyers see quoted are measuring this loop rather than single-query relevance. BrowseComp is a multi-hop benchmark in the strictest sense: its questions were built by combining several identifying properties precisely so that no single query resolves them, and the scores vendors publish against it are, in effect, multi-hop retrieval scores. Exa, Tavily, and Parallel AI all position their APIs for agent loops that issue many queries rather than one, and Parallel's published BrowseComp harness runs with a tool-call budget rather than a single request. When a vendor quotes a research-benchmark number, it is describing how well their infrastructure supports this loop, not how good one query is.