llms.txt is a markdown file placed at the root path (/llms.txt) to guide large language models (LLMs) through a website's core content and structure. Proposed as a new standard in 2024, it helps AI understand and cite your site more accurately. As AI search and GEO momentum accelerates, more companies are evaluating adoption.
How it differs from robots.txt
| Category | robots.txt | llms.txt |
|---|---|---|
| Target | Search crawler bots | AI / LLM |
| Purpose | Allow or block access | Guide and summarize core content |
| Format | Rule-based text | Markdown (human-readable too) |
| Location | Root path | Root path |
While robots.txt defines "where bots can and cannot go," llms.txt is closer to a brief that organizes "the key pages and descriptions AI should reference when understanding you." The two files are not alternatives - they are complementary and meant to coexist.
What it contains
llms.txt follows a fixed skeleton.
- H1 title: Site or service name
- Summary block: A one or two sentence intro wrapped in a blockquote
- Body description: One or two paragraphs on what the site covers
- Link sections: Grouped as
## Docs,## Products, etc., with each item in[Title](URL): short descriptionformat - Optional section: Separates lower-priority supporting links
Some sites also provide an llms-full.txt that consolidates the entire body into a single file. The goal is to let AI read the core text directly instead of crawling each page individually.
Practical implementation points
- Select important pages: Don't include every URL. Curate only documents tied to conversion and trust.
- Quality of description sentences: The one-line description next to each link drives how well AI understands it. Use clear copy aligned with search intent.
- Keep it current: Update the file when your site structure changes. Broken links erode trust.
- Pair with structured-data: llms.txt does not replace structured data or EEAT signals. The impact grows when you have them together.
Note
llms.txt is not yet a standard officially supported by every AI engine. It also does not directly guarantee exposure in search results or AI Overviews. That said, organizing content in a machine-readable form makes it a foundational step for improving GEO and AI citation share, and the cost of adoption is relatively low. 238lab handles llms.txt design alongside technical SEO and content structure audits.