Three payment rails, one web: Cloudflare Pay Per Use vs Microsoft's marketplace vs RSL

On 1 July 2026 Cloudflare announced that Pay Per Crawl, the per-fetch toll it launched a year earlier, is evolving into something it calls Pay Per Use: publishers get paid when their content shows up in an AI answer, not when a bot pulls the page. That was the third answer to the same question in five months. Microsoft Advertising opened its Publisher Content Marketplace on 3 February 2026, and RSL Media launched on 12 May 2026 to extend the Really Simple Licensing standard from page content to a person's identity and likeness.

Coverage has treated each of these as a separate story. They are not. They are three incompatible designs for the same problem, and the design choice each one made determines what your extraction vendor will eventually be billed for and therefore what appears on your invoice. When we mapped the pay-per-crawl stack in June, one rail was live and the payment question looked like a plumbing problem. It has turned into a pricing-model question, and the three answers do not agree on the unit of account.

Rail one: Cloudflare, and the shift from fetch to citation

Pay Per Crawl metered the wrong thing and Cloudflare says so plainly. A page can be crawled once and cited thousands of times, or crawled every day and never used to answer anything. Fetch count is a proxy that tracks the underlying value poorly in both directions.

Pay Per Use replaces the proxy with the thing itself. Compensation triggers when content appears in an AI search result. Two named first partners show what that means in practice. Ceramic, whose founder Anna Patterson describes the model as payment that follows "the value the work delivers rather than the number of times a crawler happens to fetch it," pays opted-in publishers when their content surfaces in Ceramic's results and returns query, citation and ranking data back to the publisher. You.com runs the transactional variant: an agent pays on demand for one specific premium item, with no standing commitment.

Citation is the more defensible unit for the buyer as much as for the publisher. A fetch toll charges you for exploration, which is exactly what an agent doing multi-hop research does most of. A citation toll charges you for the small subset of retrieved documents that actually made it into an answer. For a grounding pipeline with a wide retrieval funnel and a narrow generation step, those two numbers can differ by an order of magnitude.

The catch is measurement. A fetch is a log line at the edge, countable by the party doing the charging. A citation is an event inside somebody else's inference pipeline, and the AI company reports it. Cloudflare sits in front of enough of the web to make the fetch side auditable. Nothing in the announcement makes the citation side auditable in the same way, which is why the first partners are two search companies that chose to instrument it rather than a general obligation on all crawlers.

Rail two: Microsoft, and the catalog approach

Microsoft's Publisher Content Marketplace makes a different bet. It treats the bilateral deal, rather than the metering, as the hard part. Every AI company negotiating separately with every publisher produces a small number of large licences and leaves everyone else out. The marketplace replaces that with a catalog. Publishers set their own licensing terms and pricing tiers, AI builders browse and license content for specific grounding scenarios, and payment is usage-based with reporting that shows publishers where their content earned its keep.

The launch partners are mostly large US news and magazine groups: the Associated Press, Vox Media, USA TODAY, Condé Nast, Business Insider and Hearst, with Yahoo joining as an early demand partner. Initial pilots ground Microsoft Copilot responses in licensed content.

For a buyer, this is the rail with the clearest cost structure and the narrowest coverage. Clearest, because a catalog with published tiers is something a procurement process can actually price. Narrowest, because it covers only what participating publishers have listed. It is a licensing venue, not a property of the web. A crawler that never opens the catalog is not blocked by anything; it simply does not have the licence.

Rail three: RSL, and the declaration model

RSL arrived first, in September 2025, run by the nonprofit RSL Collective co-founded by RSS co-creator Eckart Walther and former Ask.com chief executive Doug Leeds. It does not move money and it does not sit in the request path. It publishes machine-readable licence terms alongside robots.txt, so a crawler can read the conditions rather than infer them from a yes/no rule. The 1.0 specification credits Yahoo, Ziff Davis and O'Reilly Media among its contributors, and the standard's own promotion claims support from more than 1,500 publishers, brands and technology companies. That count is self-reported by the project, not audited by anyone, and should be read as a signal of momentum rather than a measure of enforcement.

RSL Media, launched 12 May 2026 as a public benefit nonprofit co-founded by chief executive Nikki Hexum with Cate Blanchett, Doug Leeds and Eckart Walther, extends the same declaration model past page content to an individual's works, identity, likeness, voice and characters. Venable's June 2026 analysis is blunt about what that rests on. Without statutory backing, it notes, bad-faith actors can decline to query the registry or work around it, and passage of the NO FAKES Act would "fundamentally alter the standard's enforceability calculus." The NO FAKES Act has not passed. Treat RSL Media as a registry whose force currently depends on AI developers choosing to check it.

The comparison

The rows that matter are the last two. Only one of these rails can stop a request. The other two describe obligations that a crawler has to opt into noticing, which is the same structural weakness llms.txt has had since it appeared.

The layer that would make them interoperable

All three are vendor or consortium artifacts, and none of them can express a preference the others understand. The IETF's AIPREF working group is the neutral candidate. Its charter is narrow and useful: a common vocabulary for expressing content-usage preferences, plus a standard way to attach that vocabulary to content, either embedded or in a robots.txt-shaped file, with rules for reconciling conflicting expressions. Both deliverables carry an August 2026 target for submission to the IESG as proposed standards. Working group milestones slip routinely, so read that as intent rather than a shipping date.

If it lands, the rails stop competing on the permission question and compete on settlement instead, which is where they actually differ. Cloudflare, Microsoft and RSL would all be reading the same declaration. Until then, a publisher who wants coverage has to configure three things, and a crawler that wants to behave has to check three places.

What this does to your cost line

The practical change is that grounding-quality content is acquiring a list price. For the last few years the cost of web data was extraction: your vendor's per-request fee plus the engineering time to keep parsers alive. Content itself was priced at zero, with legal risk standing in for a price. A catalog with publisher-set tiers, and a citation toll with named participants, put a number where the zero was.

That number is a new input to build versus buy. If a licensed feed of a publisher's archive costs less than the scraping budget you spend fighting that same publisher's bot defenses, the licensed feed wins on cost before you get to the risk argument. Linkup has been running that thesis since before the rails existed, licensing publisher content instead of crawling it. At the other end of the same spectrum, Common Crawl remains a free archive whose usability depends entirely on publishers continuing to allow it, and the direction of travel across all three rails is toward fewer of them doing that for free.

There is a coverage asymmetry to watch, too. If a vertical's major publishers sign exclusive or tiered terms on one rail, vendors on the other rails lose depth in that vertical without any visible failure. Retrieval keeps returning results. The results are just thinner, sourced from whoever did not sign. That is the failure mode worth adding to your RAG source evaluation, because it does not show up as an error rate.

Three questions to put to your extraction or search vendor before your next renewal. Which of these rails do you participate in, and under what terms. If you start paying tolls, is that cost passed through to me, absorbed, or handled by silently dropping the sources that charge. And what happens to your coverage in my verticals if the largest publishers there license exclusively somewhere I am not buying.

What this means if you build on web data

• The unit of account is now a vendor selection criterion. Fetch tolls punish wide retrieval; citation tolls punish nothing until an answer ships. If your pipeline retrieves fifty documents to cite three, those two models produce very different bills for identical output.
• Only one of the three rails can refuse a request. Cloudflare enforces at the edge. Microsoft's marketplace and RSL both describe obligations that a crawler has to choose to honor, which makes them contract and legal instruments rather than access controls.
• Ask about pass-through in writing, not about participation. A vendor that pays tolls and absorbs them has a margin problem that becomes your price increase later. A vendor that quietly drops charging sources has a coverage problem you will never see in an error log.
• Watch AIPREF, not the vendor announcements. A neutral permission vocabulary would turn three rival stacks into three settlement options over one shared declaration. The August 2026 IESG target is the date to check, with the usual caveat that IETF milestones move.