Is llms.txt an official standard?

Not yet. It is an emerging community convention proposed in 2024, similar to how robots.txt started as a widely-adopted informal standard before it was formalized. No major AI engine has officially documented it as a required or confirmed ranking signal.

Will llms.txt help me rank in ChatGPT or Gemini?

There is no confirmed public evidence that ChatGPT reads llms.txt or that it directly causes citation gains, and Google has not confirmed it affects Gemini or AI Overviews. Some crawlers have shown early awareness of the convention. Adding the file costs almost nothing, so it is worth including as part of a broader AEO program, but do not expect it to move your citation share on its own.

What is the difference between llms.txt and robots.txt?

robots.txt tells crawlers what they are NOT allowed to access, so it is restrictive. llms.txt is additive: it points AI models toward the content you most want them to read and understand. They serve opposite purposes and can coexist.

What is the difference between llms.txt and sitemap.xml?

A sitemap lists every indexable URL for completeness, often thousands of them. llms.txt is curated: a short, human-readable Markdown file that highlights only your most important pages. A sitemap inventories; llms.txt recommends.

How long is llms.txt and how many URLs should it list?

Keep it short. Aim for the 10 to 20 URLs that most completely answer your buyers' questions. If you are tempted to list every page on the site, you are treating it like a sitemap, which defeats the purpose.

Does adding llms.txt have any downside?

Almost none. It does not interfere with robots.txt, your sitemap, or your SEO, and it takes under an hour to create. The only real risk is letting it go stale so it points to pages that no longer exist, which is easily avoided by updating it when you restructure content.

What Is llms.txt (and Does It Actually Work)?

The Short Answer

llms.txt is a plain-text, Markdown-formatted file placed at the root of your domain that maps your most important content for AI language models. Think of it as a lightweight guide that tells an AI crawler: here is what we do, here are our key pages, here is the content most worth reading. The convention is modeled in spirit on robots.txt and sitemap.xml, but where those files restrict or inventory, llms.txt curates.

Does it actually work? Honestly, the evidence is early and mixed. It is not a confirmed ranking factor for any major engine today. Some crawlers have begun reading it, the cost of adding one is minimal, and the downside is close to zero. So the practical answer is: add it as one small part of a broader Answer Engine Optimization program, but do not expect it to move your citation share by itself. For the full picture of what drives citations, see what is Answer Engine Optimization.

What Is llms.txt

The llms.txt convention was proposed in 2024 as a way for site owners to give AI models a cleaner, more intentional signal about what matters on a site. A standard sitemap.xml can list thousands of URLs. An AI model crawling a large site faces a lot of noise: PDFs, login pages, duplicate content, outdated posts, thin category pages. llms.txt cuts through that noise by pointing the model to the pages you actually want it to understand.

The file lives at yourdomain.com/llms.txt, the same root level as robots.txt. It is written in Markdown, which makes it easy for both humans and models to read. The content is intentionally terse: a short description of the site, then a structured list of the most important URLs with brief labels.

Where It Sits and What It Points To

The whole point of llms.txt is location and curation. It lives at a single, predictable path so any crawler that supports the convention knows exactly where to look, and it points outward to the handful of pages you most want understood. The diagram below shows how it fits alongside the two files it is often confused with.

llms.txt in context. It lives at the domain root next to robots.txt and sitemap.xml, but does a different job: it curates a short list of the pages you most want AI models to read, rather than restricting or inventorying everything.

What Goes In It

The file follows a consistent Markdown structure. Here is a minimal working example:

# AEO Labs

> AEO Labs helps B2B brands earn citations inside AI-generated answers from ChatGPT, Perplexity, Gemini, and Google AI Overviews.

## Services

- [Answer Engine Optimization](https://aeolabs.ai/services/answer-engine-optimization): Full AEO program including content, structured data, and citation building.
- [AI Visibility Audit](https://aeolabs.ai/services/ai-visibility-audit): Baseline measurement of citation share across major AI engines.
- [Digital PR and Citations](https://aeolabs.ai/services/digital-pr-and-citations): Earning the third-party citations AI engines rely on.

## Key Articles

- [What Is AEO](https://aeolabs.ai/blog/what-is-answer-engine-optimization): Definition, how it works, and how it differs from SEO.
- [How to Rank in ChatGPT](https://aeolabs.ai/blog/how-to-rank-in-chatgpt): Tactical breakdown of the levers that drive AI citation.

## Optional

- [AI Search Glossary](https://aeolabs.ai/blog/ai-search-glossary): Definitions of the terms used across our content.

The format is deliberately simple. A short H1 with your brand name, a blockquote with a one-sentence description, then H2 sections grouping your most important URLs with labels. The ## Optional section at the bottom is a convention many adopters use for supporting content you want available but consider lower priority, so a model with limited budget knows what to read first.

The principle is curation, not comprehensiveness. If you are tempted to list every page on your site, you are doing it wrong. Aim for the 10 to 20 URLs that most completely answer the questions your buyers ask. Pair this with schema markup for AI search for a more complete technical signal layer.

llms.txt vs the files it gets confused with

The single most common point of confusion is how llms.txt differs from the two other root files it sits beside. They are not interchangeable, and llms.txt does not replace either one. Here is the distinction at a glance:

File	Purpose	Format	Typical length	Who it is for
robots.txt	Restrict what crawlers may access	Plain text directives	A few lines	All crawlers
sitemap.xml	Inventory every indexable URL	XML	Thousands of URLs	Search crawlers
llms.txt	Curate the most important pages	Markdown	10 to 20 URLs	AI models

You still need robots.txt to control access and a sitemap for full indexation. llms.txt is additive: it does a job neither of the other two does, which is to say "of everything on this site, these are the pages worth understanding."

Does It Actually Work?

Honestly: the evidence is early and mixed, and you should not expect llms.txt alone to transform your citation share.

What is reasonably clear: independent researchers have observed some crawlers checking for the file, and it is fair to assume AI systems built around real-time web retrieval are the most likely to make use of it in the short term. Publishing one signals that you take AI content accessibility seriously.

What is NOT confirmed: there is no public documentation from OpenAI that ChatGPT reads llms.txt during retrieval, and Google has not confirmed that llms.txt affects Gemini or AI Overviews. No major engine has adopted it as an official spec. Anyone telling you llms.txt is a guaranteed path to AI citations is overstating what the evidence supports.

So why bother? Because the downside is close to zero and the real driver of AI visibility sits elsewhere. When we classified every source AI engines cited for live client brands over a 90-day window, a brand's own website was a tiny fraction of what got quoted. That reframes what a root-level file like llms.txt can and cannot do for you.

2-6%

Share of the sources AI engines cite that belong to the brand's own website, measured across live brands in three unrelated verticals. The other 94 percent or more comes from third-party sites, which is why an on-site file like llms.txt is a minor lever, not the main one.

Source: AEO Labs, aggregated AI citation tracking, 90-day window, 2026

Read that alongside llms.txt honestly. A file that helps a model understand your own site can only ever influence the 2 to 6 percent of citations that come from your own domain. It does nothing for the third-party editorial, community, and reference sources that make up the overwhelming majority of what AI actually quotes. That is not a reason to skip it. It is a reason to keep it in proportion.

The practical assessment is this: adding a llms.txt file takes less than an hour. It does not hurt your robots.txt rules, your sitemap, or your SEO. If even one major engine reads it and uses it to better understand your site, the return is positive. AEO Labs publishes its own llms.txt at aeolabs.ai/llms.txt for exactly this reason: it costs almost nothing and closes an unnecessary gap.

How to Add One

Adding llms.txt is a short, repeatable process. Follow these five steps.

Create the file. Open a text editor and write the Markdown structure described above. Start with your brand name as an H1, a one-sentence description in a blockquote, then H2-grouped URL lists.
Curate your most important URLs. Focus on service pages, key articles, and any page that directly answers a question your buyer might ask an AI. List the 10 to 20 pages that best represent what you do. Do not list pages that are thin, outdated, or behind a login.
Place it at your root domain. The file must be accessible at yourdomain.com/llms.txt. If your site uses a CMS, this usually means adding it to your public or static folder. If it is served by a platform like Webflow or Shopify, check whether you can add custom files at the root; some platforms require a workaround via a redirect rule.
Verify it is accessible. Open a browser and navigate directly to yourdomain.com/llms.txt. If you see the raw file contents, it is live. Make sure no robots.txt rule blocks AI crawlers from reading it.
Keep it current. Update the file whenever you publish major new content or restructure your services. It does not need daily maintenance, but an outdated llms.txt that still points to pages that no longer exist does more harm than good.

Common mistakes

A few patterns undercut llms.txt before it can do anything useful.

Treating it like a sitemap and dumping every URL in. The value is curation. A wall of links is noise, which is the exact problem llms.txt is supposed to solve.

Pointing it at weak pages. If you list thin or outdated content, you are directing a model's attention to your worst material. List only pages you would be happy to be quoted from.

Letting it go stale. An llms.txt that links to dead pages signals neglect and can mislead a crawler. Refresh it whenever you restructure.

Expecting it to replace the real work. llms.txt cannot manufacture the third-party citations that decide most AI answers. If you add the file and stop there, you have done the easy 5 percent and skipped the 95 percent that matters.

Watch: how AI answer engines actually work

Before you spend time on any single technical signal, it helps to understand how AI answer engines decide what to surface. This short explainer from Ahrefs covers what Answer Engine Optimization is and why it matters.

Should You Bother?

Yes, with the right expectations. llms.txt is not a shortcut to AI visibility and it is not a substitute for the content and citation work that actually drives citation share. But it is a fast, low-cost technical signal that aligns with how AI content accessibility is likely to evolve.

The brands that show up consistently in AI answers are the ones that have built every layer of the technical foundation: clean crawlability, structured data, answer-first content formatting, and signals like llms.txt that make their most important content easy to find. Think of it as one tile in a larger floor. It does not make the floor by itself, but leaving it out is an unnecessary gap. The bigger levers are covered in how to rank in ChatGPT and delivered by our Answer Engine Optimization service.

Key Takeaways

llms.txt is a curated, Markdown-formatted file at your domain root that points AI models to the pages you most want them to read.
It differs from robots.txt (which restricts access) and sitemap.xml (which inventories every URL) by curating a short list of your most important content.
It is an emerging convention, not a confirmed ranking factor. No major engine has documented it as an official signal, so do not overclaim its impact.
The downside is near zero and it takes under an hour, which is the whole case for adding it.
Because your own site is only 2 to 6 percent of the sources AI cites, an on-site file like llms.txt is a minor lever. The real work is third-party citations.
Curate ruthlessly, point only at strong pages, and keep it current, or it does more harm than good.

Where to Start

If you are thinking about llms.txt, you are probably also thinking about the broader question of your AI visibility. Add the file this week; it is an hour well spent. Then put your real effort where the citations actually come from. Our AI visibility audit measures your citation share today and identifies the gaps, technical and otherwise, that are keeping you out of AI answers. AEO Labs builds the full program from there, and if you are still mapping the landscape, start with what Answer Engine Optimization is or the tactical how to rank in ChatGPT playbook.