The Standard for AI Web Indexing & Agent Discovery
Just as robots.txt guides search engines and sitemap.xml outlines URL structures, llms.txt serves as the curated directory for AI agents. By providing a clean Markdown index of your core documentation, you ensure LLMs provide accurate, up-to-date citations about your product.
Specification Guidelines
- H1 Project Title: Starts with
# Project Name. - Blockquote Summary: 1-2 sentence high-level overview prefixed with
>. - Section Hierarchy: Group links under
## Documentationor## Optional. - Link Format: Bullet points using standard markdown:
- [Title](https://...): Short description.
Frequently Asked Questions
What is llms.txt and why should my website have one?
llms.txt is an emerging web standard (similar to robots.txt and sitemap.xml) designed to provide AI crawlers, LLMs, and autonomous agents with a clean, Markdown-formatted summary of your website, documentation, and API endpoints.
Where should I host the generated llms.txt file?
You should place llms.txt in your website's root public directory so that it is accessible at https://yourdomain.com/llms.txt. Optionally, you can also host llms-full.txt at https://yourdomain.com/llms-full.txt.
How does having an llms.txt help with Generative Engine Optimization (GEO)?
When AI search engines (like ChatGPT Search, Perplexity, and Google AI Overviews) crawl your website, llms.txt provides concise, high-signal markdown that AI models can ingest and cite without HTML parsing noise.
Does this tool validate formatting against the official standard?
Yes, our built-in compliance linter checks for required H1 title headers, blockquote summaries, list structures, and absolute HTTPS URLs.
What is the difference between llms.txt and llms-full.txt?
'llms.txt' is a compact table of contents linking to your most critical documentation pages. 'llms-full.txt' is an extended file that can concatenate complete documentation pages or detailed API specs in raw markdown for direct context window ingestion.
Does llms.txt replace robots.txt?
No. 'robots.txt' controls crawler access permissions and indexability. 'llms.txt' complements robots.txt by providing content guidance and markdown resources specifically curated for AI reasoning engines.