Guide

What is llms.txt?

A plain text file at the root of your site that tells an AI which of your pages are worth reading. Five minutes to write, and a longer answer than most pages give on whether it does anything.

Check your site for free

What the file actually is

One Markdown file, served at yourdomain.com/llms.txt. It opens with the name of the site, a sentence saying what the site is, and then a list of links with a short note on each. That is the whole specification. It was proposed in late 2024 as a way of handing a model a clean index instead of making it crawl a site and work out the shape for itself.

The reasoning behind it is sound. A model reading a modern website gets navigation, cookie banners, footers and related-post blocks along with the content, and it has a limited amount of room to read in. A short list of the pages that matter, in plain Markdown, removes all of that.

What goes in one

  • An H1 with the name of the site. One line, no tagline.
  • A blockquote under it saying what the business does, in a sentence.
  • Sections of links, each link followed by a colon and a short note.
  • An optional section at the end for pages a model can skip.

Keep it to the pages you would send a new customer. Twenty links that matter beat two hundred that do not, and a file listing your entire sitemap defeats its own purpose.

Does anything actually read it?

This is the part most pages about llms.txt skip, so here it is plainly. No major AI company has committed in writing to reading the file. Google has publicly compared it to the old keywords meta tag, which is not a compliment: that tag was abandoned because people filled it with whatever they wanted to be true.

We check for the file in our own scan, and we report what we find rather than claiming it earns you anything with an engine. A checker that scores you heavily on llms.txt is scoring you on a file nobody has promised to open.

So is it worth doing?

Yes, with your eyes open. It costs twenty minutes, it cannot be read as a ranking trick, and there is no downside to a correct one sitting at your root if adoption arrives. The argument for writing one has little to do with engines.

Deciding which twenty pages represent your business is the exercise. Most teams find that harder than expected, and the answer is useful whether or not a model ever fetches the file. If the list is difficult to write, the site is difficult to read, and that part is measurable today.

Where it sits against the things that do matter

Below all of them. Crawler access decides whether an engine reaches you at all. Whether your page says what your business does decides whether it can describe you. Both are read by engines today, by every engine, with no adoption question attached. llms.txt is a sensible thing to add after those two are right, and a strange thing to do first.

How to check the one you have

The llms.txt checker fetches yours, tells you whether it parses, and shows what a model would see. It is free, it asks for nothing, and it reads one file rather than scoring your whole site.

Questions about llms.txt

What is llms.txt?

A plain Markdown file at the root of a site that lists the pages worth reading and says in one line what each one is. The idea is that a model reaching for your site gets a short, clean index instead of crawling everything and guessing.

Where does llms.txt go?

At the root, as yourdomain.com/llms.txt, the same place robots.txt lives. Anywhere else and nothing will look for it.

Is llms.txt the same as robots.txt?

No. robots.txt says what a crawler may not take. llms.txt suggests what is worth taking. One is a rule that crawlers obey or ignore, the other is a recommendation with no enforcement behind it at all.

Does ChatGPT read llms.txt?

No engine has committed to it in writing, and we have seen no evidence of one fetching it. Treat any tool that scores you on llms.txt as scoring you on a file nobody has promised to open, including ours, which reports it and says so.

Do I need an llms.txt file?

Not in the sense that anything breaks without one. It takes twenty minutes, it cannot hurt you, and writing one forces you to decide which of your pages actually matter. That last part is worth more than the file.

The file is the easy part.

Check crawler access, structure and whether a model can say what your business does. Free, in under a minute.

Run the free check