What is licensed RAG?

Licensed RAG is retrieval-augmented generation that fetches a publisher's content with permission and pays for it at the moment of use. An AI application searches a licensed index, sees a price per page, fetches the page in a machine-ready format, and the publisher is paid for that fetch and sees it in its logs.

The difference from a training deal is the clock. Training is a one-off archive sale. Retrieval happens every time a person asks a question, so it is the recurring revenue line.

Retrieval, not training

AI applications use the web in two ways. Training bots (GPTBot, ClaudeBot, CCBot, Bytespider) ingest archives on a slow cycle to build a model. Retrieval bots (ChatGPT-User, Claude-User, Perplexity-User) fetch pages live because a person asked something right now, and the answer is composed from what those pages say.

Retrieval cannot be cached away. RAG and grounding load a page fully into a context window, and the context window does not persist between questions, so the same page is fetched again and again. Our logs across 10,000+ sites record more than 200M AI bot fetches a day, and the volume follows human questions, not crawl schedules. A page about an event spikes when people ask about the event.

That is why retrieval is where the money is for publishers. A training deal prices the back catalogue once. Retrieval prices every use of today's page, and today's pages are the ones fetched most.

What "licensed" adds

Unsanctioned retrieval already happens. In H1 2026 our network detected 22B+ AI bot scrapes, and 1.9B+ of them ignored robots.txt. Licensed RAG replaces that with an exchange both sides can account for:

The publisher sets the terms: which agents, which paths, what price per fetch, and what the agent receives (full text, a summary, or a refusal).
The AI application gets structured access instead of parsing a 1,201 KB HTML page to find the 41 KB of article inside it (the Forbes example from our page-weight comparison, 97% smaller).
Every response carries licensing metadata, so the application can show where a fact came from and the publisher can see who read what.
Both sides see the same record: agent, page, time, price.

How a developer fetches licensed content

The flow is three steps, all over ordinary HTTP.

Search. The developer queries the licensed index for the question the user asked and gets back candidate URLs from publishers that have opted in, with the metadata needed to pick the best source.
Price. Each URL carries the publisher's per-fetch rate. If the application requests the page directly without a license, the publisher's edge answers HTTP 402 Payment Required with the same rate attached, so the price is the same at either door.
Fetch. The application pays the rate and receives the page as Markdown or slim HTML with licensing metadata. Our Markdown versions ran 56% to 97.5% smaller than the HTML across ten test articles, which means fewer tokens per answer and more of the context window left for the content itself.

The developer never scrapes, never maintains a parser per publisher, and never wonders whether the fetch was allowed.

What a rate card looks like

Publishers price per fetch, and rates vary by content type, freshness and the agent's purpose. A typical rate card separates three kinds of visitor: free or low for AI search indexers that send people back, a per-fetch price for retrieval bots that compose answers, and a higher price or a refusal for training bots. The same page can carry different prices for different agents, because a fetch that powers a cited answer and a fetch that feeds a model are not the same transaction.

Where TollBit fits

TollBit runs licensed RAG for the 10,000+ sites on our network: the bot paywall that answers 402, the licensed index developers search, the Markdown rendering, and the analytics that show each publisher which agents fetched which pages. Associated Press, Gannett and TIME are among the publishers who have said publicly that they work with us. Developers start at our docs.

Frequently asked questions

Does licensed RAG replace licensing deals with AI companies? No. Bulk deals suit the largest archives and the closest partnerships. Per-fetch licensing covers live retrieval and every publisher the deal market never reaches, and the two coexist on the same site.

How much does a fetch cost? The publisher sets it. Across our network, rates range across orders of magnitude by content type and freshness, and there is no single market price by design.

Who benefits, the publisher or the AI application? Both. The application gets current, structured, attributable content it is allowed to use, and skips the parsing and the legal uncertainty of scraping. The publisher gets paid per use and a record of exactly what was taken.