October 7, 2025
NLWeb 101: What Publishers and Site Owners Need to Know
We're moving into a world where AI agents will vastly outnumber human visitors on the Internet. When you look at it through that lens, you realize that everything about how content gets discovered and consumed is fundamentally changing.
Humans click links and browse. Agents need structured answers with citations. Humans can parse through search results. Agents need information delivered in formats they can immediately use. The infrastructure for this new reality is still being built, and publishers who position themselves correctly today will own distribution tomorrow.
What is NLWeb
NLWeb is an open protocol spearheaded by Microsoft that allows any website to support natural language interfaces powered by its own content.
At its simplest, NLWeb turns your site into something you can “talk to.” Instead of searching or clicking around, a reader could type a natural language question, “What’s the latest on tariffs?” and get an answer pulled directly from your site’s content.
Under the hood, NLWeb creates standardized endpoints (like /ask and /mcp) that work not just for humans but also for AI agents. It’s powered by your own content, not an external dataset.
Every NLWeb instance also acts as an MCP (Model Context Protocol) server, which basically means your content is structured in a way that AI agents can easily consume.
The Current Problem
Right now, when AI agents need information from your site, they have to:
- Scrape your entire website, hoping to find relevant content
- Parse through HTML, trying to separate article content from navigation
- Extract facts without proper attribution or context
- Hope they didn't miss important information scattered across multiple articles
Meanwhile, publishers are seeing:
- Referral traffic declining as search behavior changes
- AI-driven queries surging with no way to capture value
- Content being consumed without attribution or compensation
This creates friction for everyone. Agents get incomplete information. Publishers lose control over how their content is accessed and used.
Why publishers and site owners should care
You’ve probably already felt it: human visitors are slipping, but AI-driven queries are surging. The question isn’t if AI will change your audience’s behavior—it already has. The question is whether your content is prepared to meet them there.
Here’s what NLWeb makes possible:
For your human readers
- Faster answers, directly from your content - visitors can type questions in plain language and get instant responses from your content.
- Lower bounce rates - readers can more easily find what they came for.
- A modern, conversational experience - you add value to your site without needing to build a chatbot or custom AI.
For AI agents
- Structured access - instead of scraping, AI tools query your site through standardized endpoints.
- Transparency - you control what content is exposed and how.
- Future compatibility - NLWeb ensures your site is readable by the next generation of AI browsers and assistants.
For your business
- Visibility in AI ecosystems - Your articles are accessible not just in search engines, but in the growing ecosystem of AI chat tools and assistants.
- Monetization - every request from an AI agent can be billed, creating a new revenue stream.
- Control over access - you set the rules for how and when your content is accessed.
- Control over answers: Shape how your content is surfaced, ensuring AI agents deliver responses that reflect your reporting and context.
This is about making sure your content stays both discoverable and valuable in a world where the way people (and agents) search is changing.
Who NLWeb makes sense for
NLWeb is especially useful if you:
- Run a content-heavy site and want to make it more interactive for readers.
- See a rise in AI-driven traffic and want to monetize it, not just block it.
- Like the idea of a conversational interface but don’t want to build or train your own AI system.
How TollBit helps
We built TollBit for NLWeb to eliminate the technical complexity while giving publishers immediate access to this emerging protocol with a click of a button. Here's what you get:
One-Click Deployment:
- Fully hosted /ask and /mcp endpoints on your TollBit subdomain with a single click
- No engineering work required on your side
Smart Content Processing:
- Automatic semantic search across all your content
- AI-powered synthesis with accurate citations
- Support for both list queries (article discovery) and generate queries (synthesized answers)
Technical Foundation:
- RESTful API with standard NLWeb parameters (query, mode)
- JSON responses with structured citations and metadata
- MCP-compatible endpoints for agent integration
- Hosted infrastructure that scales with your needs
The entire technical stack is managed by TollBit, so you can focus on what matters: serving your audience and preparing for the agent economy.
NLWeb in Action: How Publishers Are Using It
Sometimes it helps to see it for yourself. Here’s a look at a publishers NLWeb instance:
Screenshot of how to enable NLWeb from your TollBit Dashboard.
Screenshot of an NLWeb instance completing a query grounded in the publishers content.
Getting Started
Step 1: Enable NLWeb – Log into your TollBit dashboard and enable NLWeb with one click. The system immediately begins processing your content for semantic search.
Step 2: Test Your Endpoints – Use the provided URLs to test both /ask and /mcp endpoints with sample queries from your content.
Step 3: Share with Stakeholders – Your NLWeb endpoints are immediately ready for integration by AI platforms and agent developers.
Check out our documentation to learn more.
Ready to get started? Visit https://app.tollbit.com/sign-up
Want a demo of NLWeb? https://app.tollbit.com/schedule-demo
You may also be interested in
December 16, 2025
The Imperva + TollBit Integration is Now Live
AI systems are consuming more website content than ever before. A majority of this scraping is happening without consent, attribution, or compensation — placing strain on infrastructure and diminishing the value of high quality content.
Read more
February 12, 2024
The Sinkhole is Opening, Publishers Are the Start
The way we consume information on the internet is undergoing radical change. AI-powered bots and assistants, capable of scraping and synthesizing information from the open web, are shifting how we find answers and learn about the world. While convenient for users, this spells impending doom for businesses built on the old foundation of web traffic.
Read more
September 15, 2025
TollBit Integration Now Available in WordPress VIP: Monitor, Manage & Monetize AI Traffic
AI bots and agents are changing the way information flows online. As AI tools like Perplexity become default entry points, websites are seeing their content scraped without consent, attribution, or compensation.
Read more