Skip to content

A working llms.txt, in ten minutes

What to put in the file, what to leave out, and the three mistakes that make it useless.

9 Apr 20261 min read
  • AEO
  • Technical SEO
An AI chat assistant open on a laptop screen, illustrating llms-txt-checklist

llms.txt is a plain text manifest at your root that tells AI crawlers which pages are canonical and what they are about. It does not replace robots.txt or sitemap.xml. It sits alongside them.

Here is one that works.

The shape

markdown
# Brand Name

One paragraph on who you are and what you do. Written for a model that
will summarise it, so lead with the definitive sentence.

## Primary pages
- [Home](https://example.com/): What a visitor finds here.
- [Work](https://example.com/work): Case studies with named outcomes.

## Notable results
- Specific, verifiable, numeric.

## Contact
How to reach you.

That is it. Markdown headings, descriptive links, short factual lines.

The three mistakes

Listing every URL. This is not a sitemap. Include the pages you want quoted, not the ones you want indexed. Twenty entries is plenty for most sites.

Writing marketing copy. A model summarising your file will strip adjectives and keep facts. "Industry-leading growth partner" carries no information. "Grew a page from 26K to 117K followers organically" does.

Letting it go stale. A file that contradicts your live pages lowers confidence in both. Regenerate it whenever the underlying content changes. On this site it is generated from the same data the pages render, so the two cannot drift.

Check robots.txt while you are there

A perfect llms.txt is worthless if the crawlers are blocked. Confirm you allow:

  • GPTBot and OAI-SearchBot
  • PerplexityBot
  • ClaudeBot
  • Google-Extended

Blocking these is a legitimate choice. Just make it deliberately, rather than inheriting it from a template you copied in 2023.

How to know it worked

Prompt the engines directly. Ask Perplexity a question your site should own and see who gets cited. Do it monthly, log the result, and you have the only AEO dashboard that matters.