NEWTollBit State of the AI Bots - Q1 2025

February 12, 2024

The Sinkhole is Opening, Publishers Are the Start

The way we consume information on the internet is undergoing radical change. AI-powered bots and assistants, capable of scraping and synthesizing information from the open web, are shifting how we find answers and learn about the world. While convenient for users, this spells impending doom for businesses built on the old foundation of web traffic.

Digital publishers are simply the first to sound the alarm. News headlines about websites suing AI companies and widespread industry layoffs aren't a recurring complaint against tech – this is an existential threat. It's a warning of what's to come for countless other industries.

Imagine you ask your AI assistant a question. Instead of showing search results, it instantly produces a tailored answer, drawing from multiple websites. You no longer need to click on websites themselves to get what you need. And that disintermediation is the problem. Websites rely on your traffic for advertising revenue. No traffic? No revenue. This isn’t theoretical - Arc Search does this.

If we let this model continue, it becomes economically unfeasible for publishers to keep producing original content. They've already experienced the pain of audience shifts - first to search engine ranking, then social media algorithms. The rise of AI agents might be the final knockout punch.

Websites could try to block these AI tools and play a never-ending cat-and-mouse game to distinguish a bot from a human. But with tools mimicking human behavior perfectly, that fight might already be lost; there are venture-backed web scrapers that guarantee that they cannot be distinguished from human visitors.

It's tempting to see this as just a "publisher problem." But this disruption cuts deeper, like a sinkhole slowly opening up, swallowing industries one at a time. As mainstream users increasingly adopt AI tools, other digital industries face devastation:

User-generated content sites see traffic wither. Need travel tips? No need to visit TripAdvisor. The assistant scrapes and summarizes reviews seamlessly.

Aggregators get bypassed. Ordering food? The question isn't "which app is cheapest," it's just "get me food." Your AI figures it out and there isn’t a need to go to Doordash.

The way we interact with the internet is fundamentally changing. The era of website-centric online business is dying. Publisher’s cries for help aren't pleas to preserve an outdated model; they're a dire warning about the unsustainable future we're heading towards which will bleed into other industries. The focus shouldn't be on playing whack-a-mole with AI; it's about reimagining how the Internet can operate to ensure creative, meaningful content remains valuable.

Written by Olivia Joslin and Josh Mayer

You may also be interested in

October 22, 2024

Announcing Our Series A

TollBit founders Olivia Joslin and Toshit Panigrahi today announced that the company has raised $24M led by Lightspeed in Series A funding for their platform, which has created a fluid payment system benefitting both AI companies and content creators. TollBit allows AI agents and applications to pay websites directly to use their content or data. TollBit empowers publishers and content owners to monitor AI bot traffic and put up a tollbooth to monetize usage. The platform makes it easy for publishers to onboard multiple AI and LLM partners in real-time. The first clients have begun to use the system, which will benefit content creators and make AI more useful and up-to-date.

Read more

February 2, 2024

The New Era of Internet

The Internet we've known, for all its promise of accessibility, often falls short of true information democracy. Finding accurate, in-depth knowledge frequently means navigating complex paywalls, biased search rankings, and siloed platforms built to capture our attention, not empower our minds. With the advent of AI assistants, there might be a chance to break down barriers and rewire how we access the collective knowledge humanity has gathered online.

Read more
New

July 1, 2025

How to Monetize AI Bot Traffic with Fastly AI Bot Management + TollBit

The conversation around AI and web content has reached a tipping point. As blocking becomes the predominant industry approach to AI bot traffic, a fundamental question emerges: what if there's a better approach than blocking? Our customers have been asking this question for months. Instead of treating all AI bot traffic as adversarial, they want the ability to monetize legitimate AI traffic while maintaining security against bad actors.

Read more