llms.txt: what it is, why it matters and how to implement it

In December 2024, Jeremy Howard, the creator of fast.ai, proposed a simple standard: an llms.txt file placed in a domain's root directory, telling AI systems about the structure and content of the site. The standard is supported by Anthropic (Claude), Perplexity and a dozen or so other AI tools. Implementation cost: a few minutes. Potential effect: better understanding of your site by AI systems.

What is llms.txt?

llms.txt is a Markdown-format text file placed at yourdomain.com/llms.txt. Its job is to convey key information about the site to AI systems in a structured, easily processed form. The analogy: robots.txt tells search engines what they may index; llms.txt tells AI systems what is worth knowing about your site.

The standard is backed by platforms including Reddit, Medium, Cloudflare, Akamai and Creative Commons. It is not mandatory - AI systems manage without it. But its presence signals that your company understands AI mechanisms and wants to shape actively what AI knows about it.

Note:

llms.txt: what it is, why it matters and how to implement it

Other newsrooms on this story

Related reading

Do You Need an llms.txt File?

llms.txt — what it is, what AI models use it, how to write one, and the…

llms.txt: what it is and how to add one

Complete llms.txt guide for 2026

llms.txt for AI Discoverability: Should You Add It?

I Added llms.txt to Four of My Websites - Here’s What I Learned