SEO/November 2, 2026/6 min read

Does llms.txt Work? What the Data Says and When It Is Still Worth Adding

llms.txt is often sold as a quick win for AI search. Large log studies and Google's own guidance point the other way. Here is what the evidence shows, where the file still makes sense, and what to do first.

Bella Ng
Bella NgCo-founder, Growthtrait
Does llms.txt Work? What the Data Says and When It Is Still Worth Adding

If you have looked into AI search in the past year, someone has probably told you to add an llms.txt file. It is easy to see why the advice spreads: the file is simple, it takes little time to create, and it feels like a direct line to ChatGPT.

The evidence does not support that picture. Ahrefs studied 137,210 domains and found that 97% of the llms.txt files it saw received zero traffic in May 2026. Google states plainly that Google Search does not use these files.

This post covers what the file is, what the main studies found, what they cannot prove, and which kinds of sites should still publish one.

What Is llms.txt?

llms.txt is a Markdown file placed at the root of a website. Jeremy Howard proposed it in September 2024 as a way to give language models and agents a concise guide to a site, rather than leaving them to work through cluttered HTML.

It is a proposal, not an adopted requirement. The format is light: only an H1 with the name of the project or site is required. A typical file then adds a short summary and lists of links with notes. It also differs from robots.txt, which tells crawlers what they may access. An llms.txt file only suggests what is worth reading, and nothing obliges a system to look at it.

What Does the Evidence Show?

Three data sets come up most often, and they measure different things.

Ahrefs: most files are not requested

Ahrefs checked every domain in its Web Analytics product that received traffic in May 2026. About 28% published a file, and 97% of those files received zero traffic that month. Among the requests that did occur, 96% came from bots and 19.5% came from named AI tools, with GPTBot first and Claude-Code second.

Ahrefs flags two limits. Its customers skew more technical than the web at large, so 28% is an upper bound on adoption. And “fetched doesn't mean read”, so the study treats every consumption figure as a ceiling.

SE Ranking: no link with AI citations

An SE Ranking analysis of about 300,000 domains, reported by Search Engine Journal, found the file on 10.13% of domains and no relationship between having it and how often a domain was cited. SE Ranking's wording was that the file “doesn't seem to directly impact AI citation frequency. At least not yet.” This research dates from November 2025, so it is a snapshot rather than a final verdict.

Limy: very few bot requests

Limy, a GEO vendor, says it analyzed 515,382,577 AI bot events over 90 days and found 408 requests aimed at llms.txt. The direction matches Ahrefs, but treat this as supporting evidence only: the data comes from the brands Limy monitors, the post gives no formal method, and the company sells services in this market.

What Does Google Say About llms.txt?

On this question there is no need to infer anything. Google's guide to AI optimization says you do not need to create new machine readable files, AI text files, markup or Markdown to appear in Google Search, including its generative AI features, “as Google Search itself doesn't use them.”

The same page says creating one is fine for other services that use it, and that doing so “will neither harm nor help” your visibility or rankings in Google Search. So for Google, the file is neutral. This statement covers Google Search only. It does not tell you how other AI companies treat the file.

What Can These Studies Not Prove?

A fair reading has to include the gaps:

  • Correlation is limited: finding no link across many sites does not rule out a benefit for one particular kind of site.
  • Server logs show requests, not influence: a fetch does not prove the content shaped an answer.
  • Quality was not tested: Ahrefs did not check files against the specification, so weak files may pull down the average.
  • Behavior can change: agents and coding tools are moving quickly, and a low request rate today may not last.

Even so, the burden of proof sits with the claim. Nobody has shown that adding this file improves AI citations for a typical business site, so it should not be sold as a visibility tactic.

When Is llms.txt Still Worth Adding?

The Ahrefs finding that Claude-Code was the second most common named requester hints at the real use case: tools acting on behalf of developers. Stripe, for example, publishes a long llms.txt for its documentation. The file makes most sense when an agent has a job to do on your site and needs to find the right page.

  • Add it if you have developer documentation, an API or a software product people integrate with.
  • Consider it if you have a large help center that agents may need to navigate.
  • Treat it as optional if you run a marketing site or online store and it takes under an hour to create and maintain.
  • Skip it if you are buying it as a paid AI visibility service, or if the time would come out of higher priority fixes.

If you do publish one, keep it curated. Link to your most useful pages, describe each one in a few words, and update it when URLs change. A stale file with broken links helps nobody.

What Should You Do Before Adding llms.txt?

Most sites have more valuable work to finish first:

  • Make sure AI crawlers you want can reach your pages, by reviewing robots.txt and any firewall rules.
  • Put important content in crawlable HTML, not only behind scripts or logins.
  • State key facts plainly, including what you do, who it is for, prices and policies.
  • Build evidence that others can cite, such as original data and named experts.
  • Earn mentions on the third-party pages that AI engines already cite, which AI visibility tools can help you find.

The Short Answer

llms.txt does not appear to be an AI search ranking lever. Google Search ignores it, most files receive no requests, and a large correlation study found no citation benefit. It is a small, reasonable addition for documentation heavy products, and a low priority for almost everyone else.

If you want to know which AI search changes are worth your time, our SEO and AI search service starts with that question. Contact us and we can review your site together.

Frequently asked questions

Does llms.txt help with AI search rankings?

There is no good evidence that it does. A study of about 300,000 domains reported by Search Engine Journal found no relationship between having the file and AI citation frequency, and Google says Google Search does not use it.

Does Google use llms.txt?

No. Google's AI optimization guide says Google Search does not use llms.txt or similar AI text files, and that creating one will neither harm nor help your visibility in Google Search.

Does ChatGPT read llms.txt?

OpenAI's crawler does request some files. Ahrefs found GPTBot was the top named AI requester, but it also stresses that a fetch does not mean the file was read or used, so requests alone do not prove any effect on answers.

Should my website have an llms.txt file?

Add one if you have developer documentation, an API or a large help center that agents need to navigate. For most marketing sites and online stores it is optional and should come after crawlability and content work.

Need help with this?

Growthtrait can help you put this into practice. Let's talk about your goals.

Contact us