llms.txt: What It Is, and Why No Engine Honours It Yet

We used to describe llms.txt on this page as “a quick, high-leverage GEO win.” That was wrong, and this post is the correction.

Here is the honest 2026 position: llms.txt is cheap, harmless, and — on the current public evidence — ignored by every major AI engine.

What it is

llms.txt is a proposed convention: a single Markdown file at your domain root (/llms.txt) giving AI engines a clean, curated map of your most important pages. A short summary, then hand-picked links with one-line descriptions.

The idea is genuinely good. Models have to infer what your business does from messy HTML; a curated summary removes the guesswork. It’s robots.txt’s helpful cousin — where robots.txt says what crawlers can’t do, llms.txt says what your site is.

The problem: nobody reads it

The idea being good is not the same as it working. What the primary sources actually say:

• Google explicitly rejects it. Google’s own AI-optimization guidance names llms.txt and states it is not used. Google has been direct that there is no special file that gets you into AI Overviews.
• OpenAI’s crawler documentation covers robots.txt, user-agent tokens, and IP ranges. It never mentions consuming llms.txt from third-party sites.
• Anthropic’s and Perplexity’s crawler docs likewise document robots.txt-based controls and say nothing about reading llms.txt from sites they crawl.
• Vendor claims that “Perplexity supports llms.txt” trace back to uncited marketing copy, not to Perplexity’s documentation.

There’s an irony worth naming: Google, OpenAI, Anthropic and Perplexity all publish an llms.txt for their own docs. Publishing one is not the same as consuming one. That distinction is where most of the confusion in this category comes from.

No engine documents honouring llms.txt on third-party sites. None. That’s the state of the evidence.

So should you ship it?

Sure — with clear eyes.

Reasons to ship it: it costs an hour; it’s a plain Markdown file; it can’t hurt you; and if an engine ever adopts it, you’re already there. Writing one is also a genuinely useful exercise — being forced to summarise your business in five sentences and pick your twenty best pages surfaces real positioning problems.

Reasons not to oversell it: it will not, today, get you cited. Anyone selling llms.txt as an AI-visibility lever is selling you a file, not a result.

We ship one on this site. We keep it current. We do not pretend it’s doing anything for our visibility right now.

What actually works instead

The documented lever in the same neighbourhood is per-engine crawler access control — and it’s the one nearly everyone gets wrong.

Each major engine runs several separately-blockable agents that split different jobs: training, live retrieval, and search indexing. Blocking “AI bots” with one blanket rule is a common misconfiguration that silently removes you from the citable surface — you stop being quotable in the live answer, not just out of training data.

This is unglamorous and it is real, primary-source-documented, and free:

1. Audit which agents you’re blocking, per engine, and know what each one does.
2. Allow the retrieval and search agents if you want to be cited in live answers.
3. Decide separately about training agents — that’s a business call, not a technical default.
4. Then check your server logs to confirm what’s actually reaching you.

Get that right before you spend a minute on llms.txt.

Why we corrected this post

We sell honest measurement. Publishing “llms.txt is a high-leverage win” while the primary sources say otherwise would make us exactly the kind of vendor we tell clients to avoid. The evidence changed our mind; the page changed with it.

See our evidence page for the full list of what we think is supported, contested, and unproven — including claims that cut against our own commercial interest.