NLWeb: The Microsoft Protocol That Wants to Make Every Website Queryable by AI

NLWeb is an open-source protocol from Microsoft that turns any website into a conversational interface for AI agents. It leverages the Schema.org data you already have, exposes an /ask endpoint and an MCP server, and lets users query your content in natural language. For sites rich in structured data, it's the logical next step for your SEO investment.

For the past year, making your site visible to AI has been the hot topic: structured data, llms.txt, GEO. But all of that remains passive — you optimize your content and hope LLMs cite you. NLWeb changes the game. Instead of waiting for AI to crawl your site, you give it a direct entry point: an /ask endpoint that any agent can query in natural language. And this isn't a garage project. NLWeb was created by R.V. Guha — the man behind RSS, RDF, and Schema.org — and launched by Microsoft at Build 2025. Eventbrite, Shopify, TripAdvisor, and O'Reilly Media are among the early adopters. The message is clear: the structured web that Guha has been building for 25 years has finally found its native consumer — AI agents.

NLWeb in 2 minutes: what it is and how it works

NLWeb (Natural Language Web) is both an open protocol and an open-source reference implementation under MIT license. The protocol defines how a website exposes a conversational interface. The implementation provides the full technical pipeline to run it.

The principle is elegant. NLWeb consumes the Schema.org data your site already publishes — the JSON-LD tags in your pages, your RSS feeds, your Atom feeds. It converts them into vector embeddings (mathematical representations of meaning), stores them in a vector database, and exposes two endpoints:

/ask — a REST endpoint that any application can call with a natural language question. It returns structured results in Schema.org with relevance scores.

/mcp — an MCP server that AI agents (Claude, GPT, Copilot) can use directly. I detailed how MCP works in a dedicated article.

When a query arrives, NLWeb runs a multi-step pipeline: decontextualization (rewriting the question based on conversation history), relevance check (does this question concern this site?), semantic search in the vector database, then scoring and ranking of results through specialized LLM calls. The result: precise, sourced, structured answers — not a generic hallucination.

Why Schema.org is the key to everything

NLWeb's smartest architectural choice is not inventing a new data format. It builds on Schema.org — the structured data vocabulary that over 100 million websites already use. If you've followed the recommendations in my article on structured data, you already have the raw material.

Concretely, NLWeb ingests your Schema.org types — Article, Product, Event, Recipe, Place, PodcastEpisode — and transforms them into queryable objects. Since LLMs are natively trained on Schema.org vocabulary, they understand these types without additional configuration. Your BlogPosting with its author, datePublished, and mainEntityOfPage becomes directly exploitable.

This is where traditional SEO investment takes on a new dimension. Every Schema.org property you've carefully filled in — sameAs for identity, speakable for key passages, mentions for linked entities — becomes a signal that NLWeb leverages to answer questions more precisely. The markup you were doing for Google now directly serves AI agents.

That said, NLWeb works best with content structured as lists of items: recipes, products, events, podcast episodes, articles. It's less suited to long unstructured content pages — yet another reason to properly type your data.

NLWeb vs llms.txt vs MCP: who does what in the AI stack

blog.articles.nlweb-le-protocole-microsoft-qui-rend-chaque-site-interrogeable-par-l-ia.content.section3

Implementing NLWeb: from demo to production

The reference implementation is available on GitHub under MIT license. The quick start is fast: clone the repo, install Python dependencies, configure API keys (LLM + vector database), load your data via python -m tools.db_load [RSS_URL] [name], and start the server. In minutes, your site has a working /ask endpoint.

But — and this is a big "but" — the demo is not production. NLWeb makes roughly 50 LLM calls per query. Each question triggers dozens of specialized micro-calls: rewriting, relevance checking, scoring each result, final ranking. This explains the quality of responses, but also the cost and latency (several seconds per query). For a low-traffic site or internal use case, that's acceptable. For a high-traffic public site, serious optimization is needed.

The Cloudflare alternative deserves attention. Cloudflare AI Search offers a managed NLWeb integration (in public preview): they handle the vector store (AutoRAG), LLM routing, and expose the /ask and /mcp endpoints without you deploying any Python code. NLWeb becomes a configuration, not a deployment. This is probably the most pragmatic path for most sites.

On the vector database side, NLWeb supports Qdrant, Milvus, Azure AI Search, Elasticsearch, PostgreSQL, and Cloudflare AutoRAG. On the LLM side, OpenAI, Anthropic, Google Gemini, DeepSeek, and Hugging Face are supported. The architecture is modular — you choose your stack.

Should you adopt it now?

Honest answer: it depends on your site. NLWeb is particularly relevant if you have a site rich in structured content — a product catalog, a dense technical blog, a recipe database, an event directory. If your Schema.org is already well-populated (and if it isn't, start there), NLWeb can leverage it immediately.

It's also relevant if you want to position early on the agentic web. AI agents that query /ask endpoints are still rare, but the ecosystem is growing fast. Being ready when users arrive with their agents is a competitive advantage — exactly like having a mobile-first site before Google required it.

On the other hand, if your site is mostly long unstructured content (institutional pages, free-form text), NLWeb will add little. And if you don't yet have solid Schema.org, start there before thinking NLWeb — it's the foundation.

My recommendation: in 2026, the priority remains the foundation. Rich, well-typed Schema.org (Organization, BlogPosting, FAQPage, sameAs), a clean llms.txt file, an active RSS feed. This trifecta is what makes NLWeb possible — and already serves you for classic SEO and GEO. Then, if your site has structured content at scale, test NLWeb via Cloudflare AI Search or a local deployment. The entry cost is low, and the signal you send to AI agents is strong.

Our free SEO & GEO audit tool already checks the state of your structured data and llms.txt file. And if you want to go further — implement NLWeb, structure your data for AI, or define your GEO strategy — let's talk.