The three kinds of AI visitors
AI visitors come in three classes, and each needs something different from your site. Simple bots and crawlers need pages they can read. Computer-use agents need pages they can use. Personal assistants need a way to log in, remember preferences and pay. Today nearly all AI traffic is the first class, the second is growing fast, and the third has arrived in single digits.
Treating them as one audience called "bots" is the mistake behind most AI strategies. You would not serve a search crawler, a shopper on a phone and a logged-in app user the same experience. The same is true here.
Simple bots and crawlers: are you readable?
This is the bulk of AI traffic. It includes training crawlers (GPTBot, ClaudeBot, CCBot, Bytespider), AI search indexers (OAI-SearchBot, Claude-SearchBot, PerplexityBot, meta-webindexer) and retrieval bots that fetch a page live because a user asked a question (ChatGPT-User, Claude-User, Perplexity-User).
They have three things in common. They do not run JavaScript. They read the whole page into a context window and pay for every token. And they come back constantly, because an AI application cannot cache the internet: retrieval re-fetches at answer time, and agents do not get tired. Our network of 10,000+ sites sees 200M+ AI bot fetches a day. On tollbit.com alone, 28 distinct agents made about 44K crawl attempts between September 15 and October 14, 2026.
What they need: server-rendered text, because a page that assembles itself in the browser is a page with a few dozen readable words. High availability, because blocked and failed fetches are answers written without you. Pre-computed answers, with the question as the heading and the facts in a table. And a bot-only sitemap, so indexers find the pages you want read rather than the ones you retired.
The question this class asks of your site is "Are you readable?"
Computer-use agents: are you usable?
The second class does not just read. It navigates, clicks, fills in forms and completes tasks: compare these three products, add the blue one to the cart, book the 7pm table, download the spec sheet. These are the research-and-task agents behind agent modes in the major assistants, Google-Agent, and a growing number of developer-built agents. The coding agent user-agent claude-code fetched our State of the Bots report about 1,350 times in a month, more than any other page on our site, which means developers are already piping web pages into agents that act.
These agents operate under a budget. Every step costs a screenshot, every screenshot costs vision tokens, and the bill lands on whoever runs the agent, increasingly your customer. A controlled benchmark across five test stores and 1,000 runs found that an agent-facing rendering cut time to completion by 24% to 35% and steps by up to 38%, with 100% completion and zero failed runs on the agent site.
What they need: token-efficient formats (slim HTML with forms and buttons intact, overlays suppressed, sticky elements static), high task completion (no CAPTCHA on a product page, no hover-only menu, no newsletter modal between the agent and the cart), and goal-oriented navigation from question to action in as few steps as possible.
The question this class asks is "Are you usable?"
Personal assistants: the groundwork
The third class acts for one named person. It holds their preferences, their logins and their payment method. Meta launched Muse, a personal assistant, in September 2026. Amazon's Amzn-User, Google-Agent and Manus-User appear in our logs today in single digits. In September 2026 Resy deactivated accounts whose personal assistant was hitting the site at roughly 200 requests an hour, which is what happens when a site meets this visitor without a plan for it.
What they will need is authentication built for an agent acting on a user's behalf, payments an agent can complete (HTTP 402 is the rail being laid for this), APIs for the actions a site already supports, and personalisation that respects the user's stated preferences. This is groundwork today. The sites that have the first two classes handled will be the ones ready when the third arrives in volume.
Where the traffic is today
| Class | On tollbit.com, Sep 15 to Oct 14 2026 | Largest agents |
|---|---|---|
| Simple bots and crawlers | Nearly all of about 44K attempts | meta-webindexer 16.3K, ChatGPT-User 10.5K, ShapBot 3.7K, Claude-User 3.2K, Amazonbot 2.9K |
| Computer-use agents | Growing, led by developer agents | claude-code, about 1,350 fetches of one report |
| Personal assistants | Single digits | Amzn-User, Google-Agent, Manus-User |
Readable first, usable second, and the groundwork for assistants laid along the way. That is the order the traffic arrives in.
Where TollBit fits
An agent site serves all three from one rail: server-rendered text for crawlers, slim interactive HTML for computer-use agents, and the 402 payment rail for assistants, with server logs showing which class read what.
Frequently asked questions
Which class should I build for first? The first. If crawlers and retrieval bots cannot read your pages, nothing downstream works, and this is where nearly all AI fetches come from today.
Are computer-use agents the same as scrapers? No. A scraper copies pages in bulk. A computer-use agent is completing a task a person asked for, on your site, and it fails or succeeds the way a customer does.
How do I tell the classes apart in my logs? By user agent. Training crawlers, indexers and retrieval bots declare themselves by name. Computer-use and personal agents increasingly declare themselves too (Google-Agent, Amzn-User, claude-code), though some still arrive looking like a browser.