On this page
Both of these are true at once: Google says its Search systems do not use llms.txt, and Anthropic recommends publishing one and ships a file for its own documentation. They are not contradicting each other — they are talking about different audiences. One is a search surface and the other is an agent reading documentation.
Which means the honest answer to "should I publish one" is a question back: who do you think is going to read it?
What does Google say?
That you do not need it.
Its guidance on optimising for generative AI features states plainly that you do not need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search. Published commentary from the Search team has been more direct still — that the file is not used and that the bots were not even requesting it.
Publishing one does not harm you either. It is simply not a lever on that surface.
So why do the model companies publish them?
Because an agent reading your documentation is a different problem from a search engine indexing your site.
When someone points a coding agent at a documentation site, the agent has to work out what is there. A hand-maintained index — here are the pages, here is one line about each — is genuinely more useful to that agent than making it crawl a navigation menu rendered in JavaScript.
That is the use Anthropic recommends it for, in guidance aimed at people writing docs that agents consume, and it ships one for its own. OpenAI uses the file in agent contexts as well. None of that is a claim about ranking.
| Audience | Does llms.txt matter? |
|---|---|
| Google Search, including AI Overviews | no, per Google's own guidance |
| An answer engine citing you | no published evidence either way |
| A coding agent pointed at your docs | yes, this is the actual use case |
| Your own tooling | occasionally, as a cheap site index |
The whole list
67 tools, what each costs and when you actually need it. Updated twice a year.
Is there any evidence it affects citation?
Not that stands up.
Published analyses across large numbers of domains put adoption somewhere around a tenth of sites and report no measurable relationship between having the file and how often a site gets cited. That is observational rather than experimental, so it does not prove absence — but it is the opposite of the confident claims made for the file.
The pattern is familiar to anyone who was around for earlier meta tags: a proposal appears, it is plausible, an industry forms around it, and the question of whether anything consumes it goes unasked for about eighteen months.
Should you publish one anyway?
If you can name the reader, yes. If you cannot, it is busywork with a maintenance cost.
The case for:
- It takes minutes to generate if your site is built from data you already have.
- It is a decent machine-readable index for your own tooling.
- If the agent use grows, you are already there.
The case against:
- It is another file that can go stale, and a stale index is worse than none.
- It encourages the belief that the problem is machine-readability when the problem is usually that nobody has linked to you.
What makes a bad one?
Hand-maintenance.
The failure mode is a file written once, listing twelve pages, on a site that now has a hundred and sixty. The one audience that genuinely reads these files takes them literally, so a wrong index is actively misleading in a way a stale sitemap is not.
If you publish one, generate it from the same source that builds your pages, so it cannot drift. On this site it comes out of the build step along with the sitemap, and it lists what actually exists because it is built from the same data.
What would change this answer?
One of three things, and it is worth knowing which to watch for:
- A provider documenting it as an input to a search or answer surface, in its own documentation rather than in someone's blog post.
- Server logs showing the file being requested at meaningful volume by named agents. This is the cheapest signal and you can check it yourself.
- A controlled comparison rather than a correlation across domains.
Until one of those, the right posture is: publish it if it is free to maintain, claim nothing for it, and spend the attention on something with evidence behind it. How do you get cited by AI answers? is where that attention should probably go instead.
Where this came from
Google's guide to optimising for generative AI features in Search, and Anthropic's published guidance on writing documentation for agents, checked 14 October 2026. Adoption figures are from third-party analyses across large domain samples and are cited as directional, not as measurement.
This is an area where the honest answer changes. The date at the top of this page is there so you can tell how stale it is when you find it.
Tools in this piece
| Tool | What it does | Price |
|---|---|---|
| Vercel | Hosting. Puts a project live on a real address with one command. | Free tier |
Every tool has its own page with the price, who should skip it and what to check before paying. Some links are affiliate links.
Questions people ask
Does llms.txt help your Google ranking?
No. Google's guidance states that you do not need to create AI text files to appear in Google Search, and that Google Search does not use llms.txt. Publishing one neither helps nor harms visibility there.
Why do Anthropic and OpenAI publish llms.txt files then?
For agents rather than for search. Anthropic recommends the file in its guidance for writing documentation that agents consume and ships one for its own docs. OpenAI uses it in an agent context too. That is a developer-tooling use, not a ranking signal.
Should you publish an llms.txt?
Only if you can name who reads it. For a documentation site whose users point coding agents at it, yes. For a blog hoping to rank better in AI answers, there is no evidence it does anything, and the provider with the largest answer surface says it ignores the file.
Does it cost anything to publish one?
A few minutes to generate and the ongoing cost of it going stale. A file that contradicts your site is worse than no file, because the one audience that does read it is the one taking it literally.
Is there data on whether it works?
Published analyses across large numbers of domains report adoption around 10% and no measurable relationship between having the file and how often a site is cited. Treat that as directional rather than definitive — it is not a controlled experiment.
What does this site do?
It publishes one, automatically generated from the same data that builds the pages, and claims nothing for it in search. It exists so that an agent pointed at the site gets a clean index instead of scraping navigation.