Web data pricing in 2026: why a $0.30 page and a $5 query can cost the same

Three pricing changes arrived in April 2026 within fifteen days of each other. Tavily cut its free tier from 1,000 to 500 monthly searches, effective April 11. Firecrawl raised its per-credit price by 8% on the entry tier. Browserless launched a $99/month "AI Agents" plan targeting LangChain and CrewAI workloads. None of these was a headline event. Together they signal something: web data pricing is tightening, vendors are moving upmarket, and the window for cheap or free access until you scale is closing.

Comparing these prices is harder than it looks. The AI search API you're evaluating charges per query. The scraping API charges per request. The browser infrastructure vendor charges per session or per action. Each unit measures something different, and vendors choose their units deliberately – partly for accounting reasons, partly to make direct comparison difficult.

This post is a framework for normalizing to what actually matters: the cost per 1,000 usable outputs for your specific workload.

The unit problem

The most common mistake when evaluating web data costs is treating all "per-request" numbers as equivalent. They aren't.

A "$0.30 per 1,000 requests" price from a SERP scraper delivers raw search result pages – titles, URLs, snippets, raw HTML. That's exactly right for rank-tracking or keyword-research tooling. For an AI pipeline that needs clean text in an LLM context window, you'll parse that HTML, strip boilerplate, and likely lose some fraction of requests to anti-bot failures. The effective cost per usable output is higher than the list price suggests.

A "$5 per 1,000 queries" price from an AI search API delivers markdown-formatted content, pre-processed for LLM consumption, from an independent web index. The list price is 16× higher. The engineering effort to use the output is substantially lower. Whether it's "more expensive" depends entirely on what you're building.

Two hidden variables compound the unit problem. JS rendering – necessary for any modern JavaScript-heavy target – typically doubles or triples the per-request cost for scraping APIs. Success rates matter too: an API claiming $0.06/1K that succeeds only 60% on your actual targets costs $0.10/1K effective, plus engineer time spent on retries and error handling. Neither variable appears in a vendor's published pricing table.

What each category actually costs

Working from current list prices as of May 2026.

AI search APIs

AI search APIs return LLM-ready content. The unit is a query; the output is formatted for consumption without post-processing.

Brave Search API charges $5/1K on its Pro plan. It runs the only independent Western web index at scale – 40 billion pages indexed, no upstream dependency on Google or Bing. Exa's credit-based pricing lands at roughly $5–10/1K depending on which features a query invokes. Serper.dev, the cost leader for structured Google SERP access, charges $1/1K with volume discounts to $0.30/1K at the Pro tier. Perplexity Sonar departs from the per-query model and charges by token: roughly $1 per million input tokens for the base Sonar model, $3 per million for Sonar Pro, which means cost scales with the length of context being processed rather than with query count.

Tavily, acquired by Nebius Group in February 2026 for up to $400 million, now offers 500 free monthly queries post-April cut, with Pay-As-You-Go at $0.008/query or paid tiers starting at $30/month.

At $1–5/1K, AI search APIs look expensive against SERP scrapers on a pure price-per-request basis. For AI pipeline workloads where the output must land directly in an LLM context window, the post-processing cost and quality gap often justify the spread.

Web scraping APIs

Entry-tier pricing across the scraping API category is deceptively wide.

ScraperAPI's entry plan runs $49/month for 100,000 credits – $0.49/1K – dropping to $0.08/1K at the Business tier. ScrapingDog is $0.12/1K at entry, one of the cheapest serious options for non-adversarial targets. ZenRows charges $0.27/1K at entry, down to $0.06/1K at Enterprise volume, with a premium that reflects its specialization in bypassing Cloudflare and Akamai.

Firecrawl is the outlier. At $19/month for 3,000 credits – $6.33/1K list – it sits 50–100× above the cheapest scraping APIs on a per-request comparison. The difference is what a credit buys: the price includes JS rendering, markdown extraction, link normalization, and the option of structured output for LLM consumption. If your alternative to Firecrawl is ScraperAPI at Business tier pricing plus a markdown pipeline plus retry logic plus team engineering time, the gap narrows considerably.

The number buyers should use is not list price but effective cost: (list price ÷ success rate on your targets) + (engineering hours × hourly rate to fill the gaps). At anything above a few thousand pages per day, that second term often dominates the comparison.

Browser infrastructure

Browser infrastructure pricing is the hardest category to normalize because the unit is a session or an action, not a page.

Browserbase charges per-session and per-action. The total depends on session duration and how many interactions a task requires. Browserless, which launched its flat-rate $99/month AI Agents plan in April 2026, is currently alone among top-five providers in offering a flat monthly option. For teams running predictable, bounded agent tasks, a flat plan is meaningfully easier to budget than per-action billing that spikes with task complexity.

The cost comparison with scraping APIs doesn't translate per-page; it translates per-task-completed. A task requiring four page loads, a login, and a form submission on a JavaScript-heavy authenticated site is a different problem from fetching a static URL. Scraping APIs solve the second problem. Browser infrastructure solves the first. They aren't interchangeable, which is why comparing their per-unit prices produces misleading results.

The April 2026 pricing signal

Three moves in fifteen days is not coincidence. It is a market maturing.

Vendors move prices upmarket when the developer-subsidy phase of customer acquisition is over. The AI builders who started using these tools in 2024 for prototyping are now running production workloads. The cheap early access was a customer acquisition cost. As vendors compound enterprise contracts and shift their revenue mix toward high-volume buyers, the economics of subsidizing free users changes.

Tavily's free-tier cut is partly post-acquisition behavior. An AI cloud company that paid up to $400 million for a search API is not going to leave the generous developer subsidy intact when its primary customers are enterprise LLM deployments. This is expected and disclosed; it doesn't change the product's quality. But it is a signal that the product's audience has shifted upmarket and won't shift back.

Firecrawl's 8% entry-tier increase is smaller but directionally more interesting. Firecrawl built its brand entirely on developer adoption – 350,000+ developers, 48,000+ GitHub stars, a freemium model that let teams get to production without a procurement conversation. A price increase on the entry tier is evidence of pricing power: the product is valuable enough that users absorb rather than defect. That's useful signal for buyers evaluating vendor durability.

The Browserless AI Agents flat plan is different in character – it's not a price increase but a new tier aimed at a new buyer segment. It tells you that browser infrastructure vendors have identified agent workloads as a distinct, recurring revenue segment and are starting to structure pricing around it.

How to actually compare

Before signing a contract or upgrading a plan, three questions that don't appear in pricing tables.

What is the success rate on your actual targets? Most vendors publish success rates against their own benchmark suite, which underrepresents adversarial targets. For sites running Cloudflare Enterprise or Akamai Advanced Bot Protection, real-world success rates can be 20–40 percentage points below what a vendor quotes. Ask for references from customers scraping similar targets, or run your own benchmark on a sample before committing.

What does the vendor's pricing unit include versus what you build? A scraping API at $0.08/1K that returns raw HTML requires a markdown parser, a boilerplate stripper, and a retry handler. Firecrawl at $6.33/1K includes all three. The comparison is not $0.08 versus $6.33; it's (API price + build cost) versus (API price + build cost). For a small team, build cost at $0.08/1K can easily exceed the price difference.

What is the discount floor at your tier? Web data vendors at the $2,000+/month scale routinely offer 30–60% below list for annual commitments. The April price changes are all list-price changes; actual enterprise pricing has more inertia. If you're approaching that volume, the published price is a starting point, not a ceiling. Request a conversation before assuming the list price is what you'll pay.

The market in May 2026 offers more production-ready options than it did eighteen months ago, and the quality gap between tiers has compressed. Teams paying 2024 prices on 2024 vendor agreements are likely due for a renewal conversation. The tools they evaluated then may have repriced, restructured their free tiers, or been acquired – and the alternatives they dismissed may now be ready.

The April signal is not that web data is becoming unaffordable. It is that the free-ride era is ending and pricing power is concentrating with vendors whose products have become infrastructure. Choosing on list price alone in this environment is the wrong frame. The right frame is: what does this actually cost to run in production at my scale, and what happens to that number when the vendor's next pricing update arrives?