llms.txt — what it is, what AI models use it, how to write one, and the mistakes that quietly tank the whole exercise.

What is llms.txt?

llms.txt is a plain-text file you put at the root of your domain — at https://yoursite.com/llms.txt — that tells large language models what your site is about, which pages matter, and how to navigate it. It's written in Markdown. It's small. It exists for one reason: AI models have very limited context windows, and they can't read your whole site, so you give them a curated map instead.

The standard was proposed by Jeremy Howard of Answer.AI in September 2024, and over the following eighteen months it became the de facto convention for what's now called Generative Engine Optimization, or GEO — the practice of getting your site cited by ChatGPT, Claude, Perplexity, Gemini, and the other large language models that have started to replace the traditional search box.

If you've ever written a robots.txt or a sitemap.xml, you already understand the shape of llms.txt. It's the same idea — a small file at a well-known URL that gives automated systems structured hints about your site — except the audience is language models rather than search-engine crawlers, and the format is Markdown rather than text directives or XML.

Why it exists (the actual problem)