TollBit vs Cloudflare Pay Per Crawl: what is the difference?

Cloudflare's Pay Per Crawl is a toll gate at Cloudflare's edge: a site on Cloudflare sets a price per request, and a verified crawler pays it, passes free, or is blocked. TollBit is a bot paywall and agent site that runs on any CDN, including Cloudflare, with per-agent rate cards, a licensed index developers fetch from, analytics across 10,000+ sites, and a layer that changes what bots receive, not only whether they receive it.

Both answer HTTP 402 Payment Required. The difference is what sits around the 402.

What Cloudflare Pay Per Crawl does

As described at its launch in July 2025, Pay Per Crawl lets a Cloudflare customer choose, per crawler, one of three outcomes: allow, charge or block. The site owner sets a single flat price per request across the domain. A crawler registered with Cloudflare that signs its requests (Web Bot Auth) can present payment intent. Cloudflare returns 402 with the price when the crawler has not agreed to pay, serves the page when it has, and handles billing between crawler and site. It launched in private beta.

What it does well: it runs at the edge with no new vendor, the pricing is simple, and crawler identity is cryptographic rather than a user-agent string.

Where it stops: it is for Cloudflare customers only, the price is flat across the site rather than per page type or purpose, and a paying crawler receives the same page a human would. It is a toll on access, not a change to the content, sitemap or format.

What TollBit does

TollBit is the reverse-proxy origin for bot traffic on your own domain. Identified AI agents are routed to a tollbit.yourbrand.com subdomain by a rule in whatever CDN or WAF you already run (Akamai, Cloudflare, Fastly, CloudFront, DataDome, Imperva, HUMAN, WordPress VIP). Humans, Googlebot and Bingbot are never routed.

On that rail:

Bot paywall. 402 per agent and per path. One rate card can price a training bot, a retrieval bot and an AI search indexer differently for the same page. 2.6B+ AI bots were directed to it in H1 2026.
Licensed RAG. Developers search a licensed index, see the per-fetch price and fetch Markdown with licensing metadata. Paying is the default path, not a gate.
Agent site. What bots receive changes: slim server-rendered pages, agent-only pages, a bot-only sitemap, per-platform versions, and a citation fallback so a cited agent-only URL never 404s for a human.
Analytics and intelligence. Reads by agent, page, status and referral from your logs, and demand across 10,000+ sites.

Side by side

	Cloudflare Pay Per Crawl	TollBit
Where it runs	Cloudflare edge, Cloudflare customers	Your subdomain, behind any CDN or WAF
Payment signal	HTTP 402	HTTP 402
Price granularity	One flat price per request, per site (at launch), allow/charge/block per crawler	Per agent, per path, per purpose
Crawler identity	Web Bot Auth signatures, Cloudflare registration	User agent plus IP verification and request checks
What a paying bot receives	The ordinary page	Markdown or slim HTML, agent-only pages, bot-only sitemap
Developer access	Pay at the gate	Search, price, fetch from a licensed index
Analytics	Cloudflare's crawl dashboard for your zone	Reads, blocks, referrals per agent and page, plus network demand

Where each fits

Choose Pay Per Crawl if you are on Cloudflare, want one price for the whole site, and want a toll with no new vendor and no change to what crawlers get.

Choose TollBit if you are not on Cloudflare or run more than one CDN, want different prices per agent and path, want to be in the licensed index AI developers fetch from, want to change what bots read (and publish pages only they can see), or want analytics across your logs and the network. TollBit has run agent sites for about three years.

The two are not exclusive. Cloudflare's bot detection can be the rule that routes identified AI agents to TollBit, and sites on our network sit behind Cloudflare today.

Frequently asked questions

Can I use TollBit if my site is on Cloudflare? Yes. A Cloudflare rule matching AI user agents forwards them to your TollBit subdomain, and Cloudflare keeps serving humans and search engines as before.

Do both use HTTP 402? Yes. 402 Payment Required has been reserved in the HTTP specification since the 1990s, and both use it to quote a price before content is served. The pricing logic and what happens after payment differ.